The Talking Pineapple Nonsense: Why Reading Comprehension Tests Suck

From eighth-grade classrooms in New York comes the incredible story of a talking pineapple, the protagonist of an utterly incomprehensible reading-comprehension question on a standardized test.  Teachers, students, parents, and administrators alike admitted they had no idea what the question was supposed to be asking.  Even Ken Jennings was brought in to discern whether the question was possible, and his answer was a resounding "no."  Soon after people from all across the country started mocking it, the pineapple question was stricken from the test.  That was a good move, but the controversy has brought to light a systemic weakness of reading-comprehension questions on standardized tests: many of them are wholly subjective, yet they demand test-takers to shoehorn in objective answers.

Math and science questions have right answers.  You can't argue that if 2x + 3 = 7 then x does not equal 2.  There's no debate that the Earth's mantle lies between its crust and its core.  Most English questions have right answers, too.  Regardless of context, "stingy" is a pretty good antonym for "generous".  But in reading comprehension--and in the larger field of literary criticism--things are not so black and white.  Good literary critique is all about formulating an idea then finding evidence from the work to support it.  It's why people today can read Crime and Punishment or Moby-Dick and have new discussions about them, interpret them in different ways, find personal meaning in them.  If subjective personal meaning and novel interpretation make language education a valid discipline in the first place, then why are we testing proficiency in that discipline with ostensibly objective multiple-choice questions?

I noticed it as an eighth-grader taking these tests.  I was fine with questions like "what did this character mean?" or "in what order do these events take place?".  Questions that asked me to interpret how a character was feeling or his motivation for doing something always felt a little hazy.  But by far the worst were the questions that asked me to straight-up speculate.  Talking Pineapple has one of the worst I've ever seen: "What would have happened if the animals had decided to cheer for the hare?"  We don't know for sure, because that's not what happened.  The author, the test-writer, and the test-taker might all have very different answers to that question, none of which would necessarily be "wrong" as long as their answer made sense based on the rest of the story.

But the "what would have happened" question asks the test-taker to give an objective answer based on a hypothetical situation.  There is no way that any answer to this question should ever be scored as "correct" as x = 2, or as "incorrect" as x = 5.  Yet that's exactly what multiple-choice questions on reading-comprehension tests do.  And although Talking Pineapple gave one of the most public, and egregious, examples of this forced objectivity, it's been a mainstay of standardized tests for decades.

It seems obvious that in a field where reasoned argument is a critical skill, we should be testing students' abilities to make reasoned arguments.  A binary, "either you're entirely correct, or you're entirely incorrect" bubble on a Scantron sheet doesn't test that ability at all.  (In the worst case, it tests the ability to guess an answer at random out of four or five.)  So why not allow test-takers to make their arguments?  What would be wrong with asking "What would have happened if the animals had decided to cheer for the hare?  Pick an answer A-D and explain your answer."  Then, don't score the answer at all, but score how well that answer was supported from textual evidence.

Of course, the reason that's not done is expedience: it takes a lot more effort to score thousands of short-answer essays than thousands of multiple-choice bubbles.  But these are high-stakes tests.  If they're going to be used to determine whether a student is allowed to go to high school, or whether a teacher is performing adequately, then both the students and the teachers deserve a little extra effort from the scorers.  Regardless of the stakes of the test, though, putting questions like these in any sort of exam shows a complete lack of understanding of the very discipline it's claiming to test.  Reducing literary comprehension and critique to a series of multiple-choice questions isn't just a simplification; it's antithetical to the skills that classes about language education should be teaching.

Currently listening: "Blue," First Aid Kit