Speaking evaluations made simple

This is a very complicated subject. It is not easy to conduct a speaking test but I will go over just a few things about it and touch on them lightly. There are many ways to do speaking tests. I have studied them, tried some of them and have settled on this way. It is similar to the way I was trained as an IELTS examiner with a few differences.

QUESTIONS

Design three levels of questions.

(1) Easy questions which are answered with straight factual answers. "Where are you from?" "How long have you been here?" "What did you do yesterday?" "What do you like to do on the weekends?" These questions make little demand on the student and only very low-level students will have problems with these.

(2) Moderately difficult questions demand more from the student. These are questions asking a student to describe a city or restaurant, relate the story of a movie recently seen or a book recently read. "Tell me about your last holiday? "Describe your best friend."

(3) Difficult questions are those that require the student to give an opinion and justify their opinion with reasons. "Should students be required to wear school uniforms? Why?" "Should smoking be banned in all buildings? Why?"

Be aware that some questions are not only difficult to discuss in English, sometimes they are just plain difficult to discuss at all. I once designed a question, "If you had two weeks to live, what would you do?" This question was so deep that the students became extremely thoughtful in trying to give their answers to the point that it interfered with any attempts to show fluency. Questions do not need to be so deep.

Although the question may be difficult at times, to understand the question should be simple. Remember, this is a speaking test, not a listening test. For example, "Given the opportunity to go on a round-the-world cruise or participate in a scientific exploration in Africa, which do you think could potentially be more beneficial for your career development?" Many low and mid level students would not be able to understand that question and therefore would not be able to speak on it. Make sure your questions are easily understandable.

I like to let the students ask each other the questions. This way I can focus on listening and evaluating. But I do not allow the students to prepare for the questions except for perhaps just a couple minutes before the interview.

Look me in the eye? In western countries we have no problem looking into people's eyes when speaking to them but this is something that Asians do not do. Therefore, when you conduct the speaking test with Asian students it is best to not try to look deeply into their eyes or to hold their gaze. Look elsewhere, shift your eyes around or even just focus on your band descriptors or rubric.

EVALUATION

You can use a rubric or band descriptor to measure the student's level such as the IELTS band desciptors or the Common European Framework.

You will notice in the IELTS descriptors that at Band 4 it says:

"Is able to talk about familiar topics but can only convey basic meaning on unfamiliar topics and makes frequent errors in word choice. Rarely attempts paraphrase."

and then at Band 5 it says:

"Manages to talk about familiar and unfamiliar topics but uses vocabulary with limited flexibility. Attempts to use paraphrase but with mixed success."

That is why it is important to design your interview questions with easy, moderate and difficult topics so that the student will have to try to produce a full range of English at different challenging levels to respond accurately. The English of many students will begin to break down at the higher levels and this will allow you to see the limit of their English.

I put the band descriptors and all the students names on an Excel spreadsheet. I give the student a score for each rating catagory (Fluency and coherence, Lexical resource, Grammatical range and accuracy)
and the program averages it out into a final band score. Depending on the situation I will add formulas to work that score into a grade, average all the scores to compare one group with another or other things. Click on the picture (above) to see it enlarged.

ACCURACY

The more realistic the task is, talking naturally about a topic the student may actually need to discuss rather than some sort of T/F or multiple choice, the more difficult it is to test. So this sort of test will always be subjective, affected by your personal judgment of the student's performance.

One thing that helps is to be sure to base your judgment as closely as possible on the rubric or band descriptors you are using. You should never compare students to each other. This will lead you off the track. Always compare to your chosen rubric.

I always record my test interviews. A couple days later I will listen to some of the interviews and rescore them without looking at the score I gave the first time. If there is a strong correlation then that is good. If you find that you are scoring much differently the second time then you need to try to understand why and may even need to rescore all your interviews. It happens that you can be in a certain mood that will cause you to score differently. (Another good reason to record is to contribue to a record of the student's progress.)

IELTS research has even shown that male interviewers will sometimes give attractive females a slightly higher score which leads to inaccuracy. If the interviewer is tired, sleepy, hungry or if the interviewer has scored several high level students in a row and suddenly gets a low level student it can affect his accuracy. To run an effective test you need to be aware of all of these things and try to guard against them effecting your judgement.