Monday, June 18, 2012

Look It Up, Look It Up: The Open-Book Test

Remember what teachers would often say when they were unable to answer your question? “Why don't you look it up.” If there is one skill that pays rewards far beyond the school years, it is the ability to find answers on one’s own to questions that arise, when and as they arise.

I used to tell my students, “There isn’t any question you can come up with that you cannot find an answer to, or at least a good approximation of an answer to it, in the library.” In the past fifteen years, I changed the location of those answers to the Internet, since nearly everything in written form is rapidly being digitized. In the past year, I have been saying to my students, “I look up most answers to questions I have on my pocket computer.” And then I hold up my smartphone. “Look it up” is how we all educate ourselves beyond the school years.

Why do we have to memorize so much in school? Because that’s how schools have been run for millennia. A score or grade must be produced and memorization is said to be the key to that score or grade. But is it? The world does not work that way. Yes, memory is used in most jobs in the real world, but it is the memory of habit built up over weeks, months, or years of experience. When memory fails, the world says, “look it up.”

William Glasser makes this point in his 1969 book Schools Without Failure. The closed-book, memory-measuring examination, he states, is based on the “fallacy that knowledge remembered is better than knowledge looked up” (p. 72, italics omitted). “I would hate,” Glasser continues, “to drive over a bridge, work in a building, or fly in an airplane designed by engineers who depended only upon memory.” Good surgeons halt surgery to look up key steps in a procedure; the not-so-good ones—products apparently of “too many closed-book tests,” Glasser observes—rely on memory, sometimes “to the extreme detriment of the patient.”

The world says “look it up.” School says “memorize it,” even though decay rates after testing, without further use of the material, are exceptionally high. So why not use open-book testing in the schools? This is precisely what Glasser advocates, to teach students how “to use reference material quickly and efficiently.” It is what I have been doing over the past sixteen months . . . and no ivory-towered walls have come a-tumblin’ down.

In three different courses I have given ten open-book, open-notes exams (both midterm and final). Average scores have been higher than with the closed-book versions, but the grade distributions are not unusual; advanced study is still required to be successful at taking an open-book exam. My students certainly prefer open-book testing, though all or nearly all have never been exposed to it before and, as in all test-taking, a certain strategy has to be learned. I told my students to go through the test initially as if it were a closed-book exam, then go back to look up specific answers lest they run out of time looking up an answer to one question and find themselves unable to finish the test.

As Glasser concludes, “Faced with a problem in life, we marshal all of the facts we can; we don’t rely on our memories unless we have to.” Open-book tests teach us “to give thought to necessary reference material, and to utilize facts to solve problems, develop concepts, and explore issues.  Closed-book tests defeat all of these objectives.”

Then there is the artificial and false nature of testing. As I have written before (Montessori, Dewey, and Capitalism, p. 158):

Testing is a contrived situation that seldom corresponds to the reality it is supposed to represent. Supermarket shoppers in one study performed arithmetic calculations far more accurately in the store than on a formal test. . . . And one boy, considered the dumbest in his class, was discovered by his teacher to be a paid scorekeeper in a bowling alley, simultaneously tracking the progress of two teams of four players each. The teacher promptly created word problems, requiring students to calculate scores for games of bowling. The boy could not do the problems.

Time to inject education with a little reality.


4 comments :

jkenny313 said...

Can't agree. There is much that should be internalized as a framework of learning, like the years when the Civil War was fought. You don't look that up. You should know it. I used to go out with an intelligent woman who thought it was fought "sometime in the 1700's !" I think most exams should concentrate on writing, writing and writing...composing one's thoughts on a question. But I rant...

Robbie G said...

I wish I had more context so I could know whether this reply is just and accurate (e.g., what the exact natures of the teaching and tests in these classes are), but I have always thought open-book testing was the most glaring example of what a mockery my public school education was.

The skill of looking up information is a straightforward matter of common sense, requiring at most a brief modicum of formal training, certainly not a multi-year educational program. If I am to spend a great deal of time and money (through tuition and/or tax dollars) on school, my teacher better try his absolute best and summon all the creativity in his power to help me *retain* the knowledge I'm supposed to be acquiring. Otherwise, why waste those hours and years of my youth repeatedly practicing a skill I can master in a week? Why not stay home and just consult wikipedia when I have a question?

Knowledge that's mine, knowledge I've retained and integrated, is a hundred times more valuable than "knowledge" I possess in the sense that I have access to a computer which allows me to look it up. The reason is that only retained knowledge allows for intellectual growth, because (1) abstract conclusions require a variety of concrete facts from which they can be drawn; if I do not have those facts in my possession I cannot draw the conclusions, and if I forget the facts after forming the conclusions they become floating abstractions, and (2) if I continually look things up and then forget them without resistance (on the premise that memorization is old-fashioned mumbo-jumbo with no application to reality), I will in 2016 be exactly where I am in 2012. I will stagnate. I will not be able to carry on an intelligent conversation, or achieve a more factually-grounded sense of reality, or acquire an ever-expanding abstract understanding of the world. I will be just like Snookie on Jersey Shore and all the other lazy dolts I went to school with who LOVED open-book and open-note tests because it served their end of getting as high grades as possible while having to expend as little effort on learning as possible, and who are adept at using the internet, but who spend their lives as complete ignoramuses and contribute to running our society further and further into the ground.

Jerry Kirkpatrick said...

Thank you Robbie for your thoughtful comments.

My context in this post is the lecture courses I teach at my state-run university in which I have to give tests and grades, neither of which I would give if I had my druthers. Glasser’s context was Watts, where he worked with children of broken families and products of abuse and neglect. In addition, he spent many years working with incarcerated women, also products of abuse and neglect, in the Ventura School for Girls, but he also worked with privileged, middle-class children and adolescents and came to the same conclusion about testing.

Be careful about the words “common sense.” As I say in my book on education, the phrase “too often means conformity to the ideas and values of whoever is describing something as common sense.” That the government should not be involved in education is to me transparently obvious common sense, but the opposite to most people in the world today is just as obvious. And the skill of looking up information? I’m still learning how to do it. As I said in my post, that is how we educate ourselves beyond the school years. Most people never try, yet the world’s knowledge is in their pockets (their smartphones).

Your anger at your teachers, which I share with you, is slightly misplaced. To use a marketing expression, I many times wanted to throw the education products my teachers were selling back in their faces and demand the return of my money. I didn’t do that, of course, because I knew that the my grade point average would suffer. This just means that the anger should be directed at the government for the guns they allow the teachers to brandish. There will always be good teachers and bad teachers. Government involvement in education gives the bad teachers power over you.

Integrated knowledge, as you say, is best remembered, but that can come from reading a book (“looking it up”) as well as from a good lecturer. The integration has be done by the learner, not the teacher, and the driver of integration is interest. If you’re not interested in something, you won’t remember it. I took two American history courses in junior and senior high school. Not until the college course, when I was interested in the subject, did I retain anything. My best retention of the American Civil War resulted when I visited the Gettysburg battle ground. The “history of place”—the ultimate field trip—is the best teacher of history.

Then there is the finance class I took in my master’s program. The professor assigned about eight chapters that he said nothing about in class. I read the chapters, took extensive notes on them, and then aced the exam. Today, I haven’t got a clue what was in those chapters, except perhaps something about capital budgeting, which to say anything more about I would have to look up!

Robbie G said...

I remembered this post today when I came across this article by George Reisman as I browsed through his blog archives. Thought it interesting that you and he link to each other's blogs and have such differing opinions on this matter (not that there's anything wrong with that). Reisman writes:

"...In college and graduate courses, this approach is expressed in the phenomenon of the “open-book examination,” in which satisfactory performance is supposedly demonstrated by the ability to use a book as a source of information, proving once again that the student knows how to find the information when he needs it.

"With little exaggeration, the whole of contemporary education can be described as a process of encumbering the student’s mind with as little knowledge as possible. The place for knowledge, it seems to believe, is in external sources—books and libraries—which the student knows how to use when necessary. Its job, its proponents believe, is not to teach the students knowledge but “how to acquire knowledge”—not to teach them facts and principles, which, it holds, quickly become “obsolete,” but to teach them “how to learn.” Its job, its proponents openly declare, is not to teach geography, history, mathematics, science, or any other subject, including reading and writing, but to teach “Johnny”—to teach Johnny how he can allegedly go about learning the facts and principles it declares are not important enough to teach and which it thus gives no incentive to learn and provides the student with no means of learning.

"The results of this type of education are visible in the hordes of students who, despite years of schooling, have learned virtually nothing, and who are least of all capable of thinking critically and solving problems. When such students read a newspaper, for example, they cannot read it in the light of a knowledge of history or economics— they do not know history or economics; history and economics are out there in the history and economics books, which, they were taught, they can “look up, if they need to.” They cannot even read it in the light of elementary arithmetic, for they have little or no internally automated habits of doing arithmetic. Having little or no knowledge of the elementary facts of history and geography, they have no way even of relating one event to another in terms of time and place.

"Such students, and, of course, the adults such students become, are chronically in the position in which to be able to use the knowledge they need to use, they would first have to go out and acquire it. Not only would they have to look up relevant facts, which they already should know, and now may have no way even of knowing they need to know, but they would first have to read and understand books dealing with abstract principles, and to understand those books, they would first have to read other such books, and so on. In short, they would first have to acquire the education they already should have had."