Doubts about the effectiveness of grammar teaching

By Scott Miles

Some grammar teaching advocates referred to the Norris & Ortega survey of the effectiveness of explicit grammar instruction, quoting the abstract:

"[T]he data indicated that focused L2 instruction results in large target-oriented gains, that explicit types of instruction are more effective than implicit types, and that Focus on Form and Focus on Forms interventions result in equivalent and large effects."

Krashen has written about this in his Explorations in Language Acquisition and Use book. Some of the main problems:

1. The bulk of the reviewed studies only test declarative or 'learned' knowledge (multiple choice questions, find the mistakes, etc.) rather than any measure of procedural use (able to use the grammar in unrehearsed speaking or writing).

We all know that students can be taught for a grammar test. I teach at one of the top universities in Korea and thus my freshman students are among the top 2% in the whole country. They have all aced (or nearly aced) the English portion of the entrance exam which has a grammar component. Yet they cannot use the grammar very well in their speaking or writing. Language teaching isn't just about test preparation. If our teaching does not affect students' actual performance, then we haven't done them much good.

2. The bulk of the studies included in the survey do not have delayed post tests.

Students may remember the instructed grammar for a test, but forget it weeks or months later. Studies with delayed post tests generally show a drop in knowledge and usage, and it is not uncommon to see all gains disappear after a few months. If the knowledge doesn't stick, then can we say the instruction was that useful?

3. Few comparison groups had anywhere near sufficient comprehensible input.

Some studies compared explicit instruction groups to those that simply had nothing (neither grammar instruction nor sufficient comprehensible input). Others had comparison groups with just a few hours of comprehensible input.

Studies which do not address these issues are simply not that useful in regards to the debate on explicit vs. implicit grammar approaches.

There is just a handful of studies covered in the the Ortega-Norris survey which do not have the problems listed above. Krashen reviews those studies in detail in his book and he makes a fairly strong argument that Norris and Ortega's conclusions are overstated.

Having followed this current TESL-L online debate over the past few months, I wonder how many people have actually looked at the studies which compare programs with explicit grammar teaching and those which just provide comprehensible input. Grammar teaching (or non-teaching) is a big issue in our field and I think it is worth taking the time to look into it directly rather than just rely on the conclusions of other scholars.

I'd like to post on a few studies (starting with this post) which compare explicit instruction with a comprehension-based learning group. If nothing else, I just want to show that this whole issue is not as cut and dried as some people would like to believe.

The Harley (1989) study which Norris and Ortega include in their review is one of the very few studies which does not have the problems noted above.

Harley compared to groups that were a part of a French immersion program in Canada. The experimental group had 12 hours of work with passe compose and imparfait over 8 weeks. The comparison group simply continued their immersion program with no explicit focus on these grammar items.

Here are the results:

Interview Test:....Pre test..Post test.....Delayed Post test (3 months) Experimental.........42% ......57%................ 63%
Comparison.......... 44.5%.... 48%................ 60%

Considering that 12 hours were spent on 2 grammar forms, and that the questions in the interview specifically cued those grammar forms, it is no surprise that the students would recall their grammar instruction and use it in the interview. Nonetheless, the scores are still not that impressive and with the delayed test the immersion group has closed the gap (there were no statistically significant differences on scores at the delayed test).

Harley (and presumably Norris and Ortega) look at these results as a victory for explicit instruction. I look at this and think that this is not a very good return for 12 hours of valuable class time. Normal classrooms cannot devote 12 hours for just two grammar points and again, the differences between the groups are no longer statistically significant after 3 months. What was really gained? And note that the immersion only group is progressing along fairly well despite not having any explicit instruction.

There were two other tests in Harley's study as well:

Cloze:...........Pre test..Post test.....Delayed Post test
experimental:.....63%......67%..................68%
comparison........64%......64%..................67%

Again, statistically significant gains that are shown on the immediate post test were lost on the delayed post test, as the comparison group closes the gap simply by continuing their immersion program.

Composition......Pre test..Post test.....Delayed Post test
experimental.....2.85..........3.19................3.34
comparison.......2.92..........3.13................3.06

The students' writing was rated on a 5 point scale for grammatical accuracy. Neither the post or the delayed post scores showed statistically significant differences between the two groups. Again. the 12 hours of grammar instruction did not deliver much to get excited about.

Furthermore , in the speaking and cloze tests these small gains seem to be disappearing, so where is the support for the idea that the instructed students are at any advantage even in the long run (the often proclaimed idea that explicit grammar instruction helps students attain the form more quickly)?

There is another issue that is often overlooked in these studies. Hours devoted to grammar instruction and practice do little to benefit other areas of language acquisition. Sure, the students in Harley's study might have picked up a little vocabulary or grammar incidentally while they were focusing on the passe compose and imparfait, but most likely not a whole lot. The question is, what did the comparison group get for that 12 hours of extra input in which they were exposed to much more language? The research results above show that they were slowly but surely developing the target grammar forms despite no explicit instruction, and thus assuredly they were also developing many other grammar forms as well. For vocabulary learning, they most likely received a lot more vocabulary exposure during that 12 hours than the grammar focused group, meaning that their vocabulary was probably developing more effectively as well. And of course, their listening and reading skills were also most likely benefited more from that 12 hours of input in comparison to the grammar group.

So I think one could make a strong case that in the sum total of language acquisition among these two groups, the input only group actually came out well ahead.

Of course, this is just one study and there are others that should be discussed.

Reference:
Harley, B. 1989. "Functional Grammar in French Immersion: A Classroom Experiment." Applied Linguistics 10:331-59 Norris, J. & Ortega, L. Effectiveness of L2 Instruction: A Research Synthesis and Quantitative Meta-analysis. Language Learning 50:3, September 2000, pp. 417-528.