Educational Disengagement: Undermining Academic Quality at a Chinese University

By Dick Tibbets - University of Macau, Macau, China

This is a fascinating study and so much rings true that I go along with all that I've managed to read so far.

On the Chinese side there is the view of education as the ingesting of information and lack of emphasis on the synthesis of information to create and advance. There is the xenophobia that assists the belief that one can teach a neutral English that allows learners to absorb information but not evaluate any of the values and attitudes associated with that information. There is the view of teaching as a job that is done between certain times with little thought of being more than a figurehead in front of the class and there is the inability of many teachers, in spite of official statements of aims, to move beyond the stage of reading the textbook to their students - pure information transmission that makes no attempt to involve learners and, when students already have the textbook, makes no attempt to come to terms with a new reality. This isn't a problem of Chinese teachers alone; I've come across plenty of Western presenters at conferences who read from the handouts they have given their audience while displaying the same words with Powerpoint on the screen behind them. But experience in Chinese schools and colleges lead me to believe that "teacher holds the book" is a very common scene in the classroom.

The article also brings to the fore the inability of the Chinese administration to evaluate teaching except in terms that have little to do with learning and more to do with time keeping. This causes problems with foreign teachers especially because they tend to fall outside the criteria used by administrators to judge teachers and there is in consequence a bewilderment among the Chinese when it turns out that inadequate teachers have been employed but the administration is unsure even how to judge their inadequacy. The reaction described in the article is typically xenophobic - a shrug of the shoulders
followed by "well, they aren't Chinese".

I read the potted descriptions of some foreign teachers. I may have missed some but of those I read none had much in the way of TEFL qualifications or TEFL experience. The teachers described had various degrees of enthusiasm for their work and various amounts of previous classroom experience but, in the absence of any real syllabus or teaching aims, they lacked the knowledge to design and implement effective courses. I'd say the university desperately needed experienced TEFL teachers with post graduate qualifications both theoretical and practical. A team with a few TEFL MAs and DELTAS coupled with at least 10 years solid TEFL experience for each member might be able to put togetehr an effective program, though the administration might well then swipe it aside as the administration would be unable to comprehend such a program.

I will read the whole thing more carefully because I want to find signs for hope for the future. Many of the views and attitudes quoted in the article were identical to those expressed by Chinese emperors, diplomats and officials over the last three or four centuries and I believe it is these attitudes that changed China from a from an innovative civilisation with a technology well in advance of the West, a country that came within a whisker of starting an industrial revolution centuries before Britain and Europe, into a country where thinking and change are seen as risky occupations. There must be a way forward but so often I see Chinese in authority struggling to keep the status quo and effectively managing to turn the future into the past.


http://www.agelastos.com/disengagement/326-362.pdf

The 'home page' for the complete text can be found at:
http://www.agelastos.com/disengagement

The Common European Framework for testing and teaching

By Jennifer Wallace
Anhui Gongye Daxue, Ma'anshan, China

Lots of us are trying to develop tests appropriate for the situations we're teaching in. One document I'd recommend, because I've found it enormously helpful, is the Council of Europe Frameowrk, which is on the Internet, as a downloadable pdf file (for which you need to have Adobe Acrobat Reader on your machine). I like the document for several reasons.

The work behind it is the work of a large number of experts across Europe, who've developed one framework to cover the teaching (and testing) of any of the languages taught and used in Europe - which of course includes a variety of non-European languages. In other words, the whole thing is language independent. I understand it to be very much a reflection of the most up to date understanding we have of measuring language performance. The particular document in question is the latest version, the result of many revisions.

The document addresses the fundamental questions in all this, and looks at every dimension conceivable - so I can use it as a basis for testing speaking, listening, reading, anything. It looks at things on general levels and on detailed specific levels - so you can home in on the level that is relevant for you at the moment.

Because this framework is as comprehensive as it is, it's let me think up a variety of activities for the form of my tests, activities that reflect the students experiences and what they've done in a course. But at the same time it's kept me very much on track, enabling me to see clearly what level our target it.

Because it's not language-specific, you can test yourself (there's one section on self-testing) for your Chinese to see how this sort of approach works.

Someone also commented about examiners' ability not to be swayed - well, I think what allows me to be more objective is using a number of scales and criteria when I test. For example, this semester my college end-of-first-year students will get some marks for pronunciation (because we've done quite a bit of pronunciation work on their Oral English classes), some marks for fluency, some marks for grammar, some marks for vocabulary/lexis and some marks for coherence. I'm also thinking about including some marks for how they deal with problems - repair work, asking for help, paraphasing, miming, using fillers to gain thinking time and to fill a silence, and the suchlike - what's called strategic competence. My criteria for vocab/lexis and grammar will not be whether they demonstrate use of anything in particular, but in how effective they are at communicating successfully - do their errors interfere with communication, or hinder it, or render it impossible! This is because I teach college English majors - I think testing for specific aspects of these dimensions is the responsibility of other teachers in other classes. but at the same time, my students do realise that I consider grammar and lexis to be seriously important.

As regards a quick test, my experience, and the experience of other testing large numbers quickly for summer schools (in UK language schools), is that in an informal chat of around 5 minutes, grading only on a 5 pint scale (with very easy to understand scoring 5) is a remarkably effective tool in the hands of a native speaker. Even on the most mundane of topics (your home town, your family), it sorts the lower from the higher from the in betweens. I did this at the beginning of this year with my 225 new students, and on subsequent reflection, having taught them now for 2 semesters, remarkably few of my initial assessments were wrong, and none were way off. What's interesting is looking back at their subsequent development! The value for me is how much respect I have for the students who got a low rating at the beginning who would only now get a middle rating - but wow, what progress! In each band, I can see students who have really made big efforts and made progress, and I can also see students who've made almost no progress. Of those, a small number are not interested in the effort it entails (basketball etc is more important), but I also have one or two who I realise are making efforts but little progress. I think that initial testing and placement has really helped me, and I plan to do it for future Oral English classes.

One thing I did was use the test results to make groups according to level, and that's been very successful as well.

On testing oral English

By George

To accurately test my students, I give them oral exams which are recorded on tape. These exams have two parts. The first part is Q&A covering things we have covered in class. They almost always have a memorized response for the basic questions. I tend to ignore these. I focus on their responses to the followup questions. For example, I've told them that we might discuss their grandparents, soI might ask

"Are your grandparents alive?" "How many children did they have?" How many boys and how many girls.? "Do you know your aunt's and uncles?" "O.K let's talk about your youngest aunt" Here is where they begin to breakdown because they didn't think to prepare for a discussion about their youngest aunt. I've also begun ba asking about a favorite middle-school teacher and them focus on the teacher they liked the least. Once I gotten to the real subject I'll begin with what is the persons name, age etc. and gradually lead to more complex questions. Then I start looking for syntactic, grammatical and vocabulary failure. In many cases the exam has ended in 2 or 3 minutes and some have gone as long as 30 or 40 minutes. In all cases I use subjects they are familiar with. Family, School, Friends and Hometowns. If I knew more about sports I would dwell on that. I have been know to ask a student to explain what a mid-fielder, a striker or a goalie does if they play those positions in football or the role of Guards, the Center or Forwards in basketball. I've even asked guitar playing students to explain how to play a particular son. In short they give me a guitar lesson.

To test for middle school, determine what is grade appropriate and start from there.

Again, start simple and progress to the complex. At what level do they abandon an answer or the topic entirely. The second part is a short oral reading which incorporates most of the english phonemes. I sometimes give the samples to practice with but they get a new reading for the exam. The must read cold.

Also, I've just begun developing a set of reading passages tha will begin at about fifth or sixth grade level for native speakers using Flesch-Kincaide RGL measures and which become progressively more advanced. This way I can determine the level at which they begin to break down, identified by their rate of word abandonment. In the first year I will be mainly concerned with phonetic identification and production. As we progress, stress and intonation will become more of a factor.

Oral exams can be quantified, but I don't like using them as the basis for a grade. I tell the school that grades shoud be considered as a report of a student's speaking level and how much they have improved. In my classes, the only one's who acutally fail are those who only show up for exams and the rare film. Those who come to class but aren't there count as absent. Our school weeds them out pretty quick. Last term eight of my students flunked out including two who were pretty good english speakers. Six were expelled for cheating on Chinese teacher's exams.

A speaking test examiner teaches speaking

By Jennifer Wallace - Anhui Gongye Daxue, Ma’anshan City, Anhui Province, China

When I came to teach here, although I‘d been a speaking test examiner for more than 10 years (for UCLES exams) I‘d actually never had to set an oral English exam before. I’d taught always in situations where the students were either taking no exam or were working towards an external exam. So if I did have to set tests, they were very much on the mock-exam model. I’d never taught a modern language within a university/college setting where this was the student’s main subject (major). Although I had a short training specific to coming to this post in China, provided by the NGO who sponsor my post, I came with some sort of assumption that there would be a syllabus, there would be designated attainment targets (although not necessarily expressed in that way). Well, you all know the reality here. I was timetabled for first year Oral English classes who were provided with one of the ORAL ENGLISH WORKSHOP series of books. If anything, I found that was worse than arriving to nothing. It implied someone somewhere thought the content of this course book was what my students should be mastering.

Anyway, after a semester of muddling along and getting some sort of impression about what might be possible, I realised that the lowest of the UCLES EFL exams I’d been a speaking test examiner for was probably within the reach of everyone in the class. I’d been warned about the tradition of everyone in the class passing the exams. Remember I’d done those UCLES tests for years. I could remember the type of tasks set in the exam, and I produced a parallel. Those UCLES tests are taken in pairs, but I chose to give each student an individual exam - partly as a public relations exercise about oral exams within my department. I was interlocuter as well as
assessor. So I recorded all the exams and marked them from the tapes. I was right in that all my students were capable of attaining that first level in the UCLES hierarchy, which means that in a grander scheme of things they had all achieved the Council of Europe Basic User level. The descriptors for this (in summary) are:

Can understand sentences and frequently used expressions related to areas of most immediate relevance (e.g. very personal and family information, shopping, local geography, environment).

Can communicate in simple and routine tasks requiring a simple and direct exchange of information on familiar and routine matters.

Can describe in simple terms aspects of his/her background, immediate environment and matters of immediate need.

At the end of the first semester all my students could do that - although a good number could only just do it with a very sympathetic interlocutor. Others walked through it. Which gave me a good spread of marks. And on that basis I decided to model the exam at the end of the second semester on the next level up (which I’d also done examining work for). By that stage my classes had included a fair amount of group work, and so the exam was done in small, randomly selected groups of 4, not including me. This year I’ve done lots more group work, but am actually planning to give one-to-one exams at this stage instead - partly for comparison.

So my decisions were based on a combination of what was within my own capabilities as well as the students. I’m not an expert on language testing. My only teacher training is a CELTA, and in the past when I’ve had to devise and construct college tests it was done under the supervision of a very experienced head of department. But also I’m not into re-inventing the wheel. The Council of Europe stuff - which relates to ALL the languages taught in Europe (and that includes teaching non-European languages) - is the result of mega-input from experts over heaven knows how many years now. I feel I’d be deluding myself if I thought I could devise any better sort of structure to work within - so I’m using it. I do also like it - I find it clear and easy to get my head around.