From Library Babel Fish, April 26, 2012

The other day, I was nonplussed to read a recap of a study that found human and robot graders fared equally well in assessing the work of student writers. The robo-graders, according to the study, do as good a job as humans at assessing clarity and sentence structure. While my initial reaction was “huh?” it’s important to note that this study only compared processes for scoring standardized tests. It has nothing at all to do with what happens in the classroom when students are learning to write. In fact, it really has nothing to do with teaching or learning, only testing, and testing the wrong things at that.

Les Perelman, who directs the writing program at MIT, told a New York Times reporter he  is not impressed by the robotic approach. He has tried out one of the robo-graders. It’s worth noting that only one company was willing to let him actually test the product – we’re supposed to trust machines to examine our students’ writing but we’re not allowed to examine the machine. You can read one of his nonsensical but high-scoring essays linked from the Times story. Apparently making sense is not important, nor is building an argument with truthful information, so long as the sentence structure is correct; extra points for using a big word when a simple word would do. Since using words to express meaning is not on the test, writing instruction will go backwards, to being grammar drills and five-paragraph exercises in vapidity. Pearson, a textbook company that markets a robo-grader, disagrees, saying essentially that any student clever enough to game the system deserves to score well. Education is learning how to follow rules, to play the game. Truth and meaning are irrelevant and so don’t count.

This depresses me so much I hardly know what to say.

Fortunately, Marc Bosquet said it for me in a thought-provoking essay at the Chronicle. He argues that too often we teach as if we are schooling robots to perform mechanically. Take the five-paragraph essay in which a student plucks lines of text from three sources  as “evidence” to support an “argument” about which the writer cares not a jot. He writes, “the forces of standardization, bureaucratic control, and high-stakes assessment are steadily shrinking the zone in which free teaching and learning can take place.”

Commenting on the troubling findings of the Citation Project, he argues that the way we teach undergraduates to use sources fails to engage them with scholarly ideas because they  encourage “quote farming.” Students don’t have to read or understand a source to use it for an argument. They simply have to mash up a random assortment of context-free material and document it.  We are teaching students to argue the way lazy journalists do: “quote-farming, argument from authority, false binarism, fake objectivity.”

So much of what academic libraries do is in support of that very practice. We train students to mine a database efficiently for five scholarly articles. We even have a database that provides sources that give “both sides” of popular if insipid paper topics. We help students perform the work they are asked to perform, even if we suspect the five scholarly sources that first semester student has earnestly gathered are beyond his comprehension. Comprehension isn’t the point. Organizing an essay and documenting sources is.

For many academic librarians, the only classroom they ever get into to help students understand how to use the library and the wider world of information is the first year composition classroom, where what students are doing bears little resemblance to authentic inquiry. Yet it’s where students develop habits that last the rest of their college careers.  Copy, paste, cite. Done.

I realize that many gifted and hardworking teachers go to great lengths to start students off on a different foot. But too often even teachers who know the limits of the mechanical research paper feel compelled to help students write that way as a survival skill, because students will be getting those assignments in other classes.

Textbook companies are ready to jump in and help mechanize the process. Historiann reports that Pearson offered her some cool cash to gather up some student writing that they can use to refine their algorithms. How scummy is that, paying professors to assign writing that they will turn over to a textbook company for cash?

And it gets worse. I recently I saw on Mashable that Cengage has bought a clever software product that lets you harvest quotes from sources for which citations are automatically created, and dump it all into a screen where you rearrange and stitch them together without actually, you know, having to read those boring papers. Even better, it will add your quotations to a database from which other students can easily harvest material on popular paper topics! That sounds like a gizmo designed to violate of our campus honor code, but it’s actually a clever mechanical reproduction of the kind of writing our students do routinely. Not long ago Cengage got into a wrangle with another company that helps students harvest notes from sources, including Cengage textbooks. Cengage claimed that taking notes from their textbooks was copyright infringement. If you take notes, they argued, you are creating a derivative work. A feature of Cengage’s own notetaking machine is that it will draw students to juicy quotes found in Cengage databases, providing a kickback to the publisher of the quoted material.

So if you’ve reached a point in the semester when you have a stack of papers to grade, take comfort. Soon we will not only have robo-graders, we’ll have robo-writers, with students guided toward previously-digested quotes that earn companies extra cash.

If that makes you queasy, think about your assignments that ask students to find sources and write about them. What are you hoping students will learn? Are they learning it? Is there a way to make the process less mechanical? Think about what habits our assignments actually instill, the habits that machines can so easily emulate. Maybe there’s another way.