Tony Lee - Shengda College, Zhengzhou, Henan, China
My marking of the first half semester exam last year was a disaster of sorts. Totally inexperienced, I was far more subjective than I should have been but the kids seemed satisfied with their placings. The marks were not a problem because another techer with zero grading experience of any sort scored the whole class between 90 and 100 so to make it fair we had to do the same.
Next exam I used a 'proper' grading rubric and slavishly referred to it while listening to each member of the groups of three. It still resulted in a bunched up result - spread of 10%.out of a possible spread of 40%. Changed the rubric again and this last lot of results was just as closely bunched.
I know that you can turn them into any spread you like but that is a bad way of correcting my own testing "incompetance". My problem is I lack the killer instinct of a practiced examiner I find it impossible to nit-pick to the extent required and then ruthelessly apply the marking scale.
As others have commented -- perhaps with oral it does not matter quite so much if the ability to speak is what we are testing because provided the students are given at least a couple of minutes to prepare they all do a pretty good job of speaking (I see them in 3's and each student has a different topic to talk on and they have had 10 minutes to prepare. After each has spoken I get involved in a general discussion on some safe personal topic and gauge how well they cope).
The problem is I do not really test their ability to cope with a typical staff meeting or questioning from an audience or even a job interview situation. I have had interview sessions in the classroom and the typical result is that neither side has a clue what the other is saying.they are so busy worrying about working out what they are going to say that they have nothing to spare for listening what the other side is saying -- a situation many native speakers also find themselves in. No answers from me I'm afraid - just the same old problems.
Our limited experience with failures here and a bit of insider info from the Chinese teachers leads me to the conclusion that there are no real failures, just those who just scrape through.