Evaluating speaking and validity

A teacher, who has his students do a class presentation as the sole English speaking evaluation task, defended his approach by saying: "I guess this just shows that there is more ways than one to skin a cat - the principles of public speaking are things that we can use every day in our lives and, if it helps to build up the confidence in the students to use their English, then it seems as good a technique to use as many of the others."

Indeed, there is more than one way to accomplish a task. But there are also good ways and bad ways, effective ways and ineffective ways.

I can't say for sure but I'll make a guess that you haven't taken a TESOL course or had any formal training to be an English teacher. I'll admit my formal training is skimpy but I study as much as I can.

Brown & Yule explained some of the challenges of speaking. "...it should also be borne in mind that even native speakers of English find that a straight description is easier, in some sense, than telling a story and, in turn, that telling a story is easier than a justification of an opinion....This is a rather general guide to the level of difficulty. Naturally, a short narrative involving a single character and only two or three events may be easier than a lengthy description covering many details and relationships."

Here is a list of examples of speaking that Brown & Yule point out:


1. Static relationships

a) Describing an object or photograph
b) Instructing someone how to draw a diagram
c) Instructing someone how to assemble a piece of equipment
d) Describing/instructing how a number of objects are to be arranged
e) Giving route directions

2. Dynamic relationships

a) Story telling
b) Giving an eye-witness account

3. Abstract relationships

a) Opinion-expressing
b) Justifying a course of action


If you are familiar with the IELTS speaking test you can see these elements in the test. First the examiner asks some basic information or a description of something. Then to tell a story of something like a holiday you took. Then finally he will ask for some opinions on some topics.

He works from easier tasks to more difficult tasks. Students should begin having difficulty at some point during the test by which the examiner can see the student's level of competence.

Of course, as Karen pointed out, there are different kinds of tests and what we are talking about are not IELTS English competency tests but more like a pass/fail type test.

But my point is that there are a lot of different demands English speaking tasks make and it's important to match the task to the need and to achieve validity.

Brown explains: "The general concept of validity was traditionally defined as 'the degree to which a test measures what it claims, or purports, to be measuring' (Brown, 1996, p. 231).

"Validity was traditionally subdivided into three categories: content, criterion-related, and construct validity (see Brown 1996, pp. 231-249). Content validity includes any validity strategies that focus on the content of the test. To demonstrate content validity, testers investigate the degree to which a test is a representative sample of the content of whatever objectives or specifications the test was originally designed to measure."

So a test of General English would not be valid for Business English students. A Public Speaking test would not be valid for General English students. Would it be good for General English students to be competent in Public Speaking. Probably. Does the ability to speak publicly show competence in a general English speaking situation? No. Could the specific anxiety generated by speaking in public interfere with General English production? Absolutely.

General English requires the ability to negotiate meaning, explaining, understanding when they are not being clear to the listener and using appropriate strategies (circumlocution or defining terms), turn-taking, etc.

To this the teacher questioned: "Why??? What support do you have for this statement. In my years in China, the correlation is pretty strong, good english communicators give good speeches. This seems natural."

I meant invalid as far as content validity is concerned. Are you familiar with content validity?

As explained by Underhill the question is:

"It is relevant? Do the items or tasks in he test match what the test as a whole is supposed to assess? Where the objectives of the programme are set out in detail, for example in a syllabus that lists skills or functions, then the content validity can be assessed by comparing the kind of language generated in the test against the syllabus. The questions then is whether the test produces a good sample of the contest of the syllabus."

Using a public speech as a way to test general English is like giving a driving test on a motorcycle instead of a car.


Brown & Yule (1983) Teaching the Spoken Language, Cambridge University Press. Brown http://www.jalt.org/test/bro_8.htm
There is a lot of good information about testing at: http://www.jalt.org/test/news-title.htm