"... is preferred by the students, does not seem to decrease learning and retention, and decreases anxiety levels."
Says a new study, which you can read here (PDF). I found that via The Chronicle of Higher Education, but you can't get to that article without a subscription.
I do open-book exams, by the way, and for precisely the reason stated above.
ADDED: If you don't want the test to be a test of who's most susceptible to anxiety, you've got to go with open book, right? It's not that simple. Knowing it's open book may help students feel calm before the exam and save them from devoting study time to ensuring memorization, but there's still the experience of seeing the exam question and knowing you're pressed for time. There's that big book to consult. Are you leafing through it, wondering where the answers are, while others are already writing?
December 12, 2012
Subscribe to:
Post Comments (Atom)
Posts
75 comments:
Do 'em both. Open book to force research of harder questions and closed book to see just what they have retained.
Urban legend from when I was in Math at Berkeley, ages ago:
To get your doctorate, you had to pass an oral exam, which was (at least for me) pretty painful. A bunch of professors fire questions at you, generally complicated things you had to try and work out on the spot. Well, there was this one old guy who always asked the exact same question: "What is Stirling's Formula for the factorial?" [http://en.wikipedia.org/wiki/Stirling's_approximation, for those of you who didn't take the Berkeley exam.] Everyone knew he would ask it, so everyone looked up the stupid formula the day before the exam.
Well, there's always one guy who won't go along... This student decided he wouldn't put up with this kind of stupidity in a high-level math program. Didn't look it up.
So comes the day of the exam, and he's answering all these questions, and sure enough, this old guy asks him, "What is Stirling's Approximation for the factorial?" He shoots back, "I don't know, sir, but I know where to look it up when I need it!" Answers the progressor, "Son, you need it now."
Horse Shit!
I had a stats class in college where the formulas were provided; the prof said that in real life, you can always look the formulas up, the test is whether or not you can apply them correctly.
That always made sense to me. Being able to find and use the information is far more important than being able to recall it from memory. Of course, I was always pretty good at memorization, so closed book tests were really better for me (when compared to the rest of the class).
Open book exams?! What we would've done for an open book exam in nursing school. How many bones in the body? Name them, starting at the head.
For what I teach it hardly matters. Physics is a skill. You can't do it if you didn't practice it, open book or not.
I am tempted, but open-book exams might give my students one more reason to not read the book before lecture like they are required to.
They can consult their books in class, of course, when we do practice problems, and they do. It doesn't help. They never know what chapter we are in, they do not know what the index does, and they can't understand what they read in there anyway, since the sentences are more complicated than noun-verb-noun.
Why is reducing student anxiety a necessary component of testing? Because there's no pressure or anxiety in the real world?
"How many bones in the body? Name them, starting at the head."
I had to do that for health class when I was in junior high school. It's not that hard.
My students often say "In real life you can look it up."
I reply with, suppose you have a customer who wants you to do some contract work for him. He wants an idea of what it costs, how long it will take, what equipment you need, what the local regulations are. And you tell him you'll look it up and let him know later... you're not getting hired.
Try remembering all the Supreme Court cases about due process and equal protection — not just the doctrine, but the variations in the cases and the reasons the justices have given and the way legislatures and academics have responded, etc. etc.
Or suppose every time I got a question from the student I said I would need to look it up. Would I inspire confidence that I know my subject well enough to teach it?
I've talked myself right out of ever giving open-book anything. There is a bare minimum of knowledge they need in their minds to be considered competent in whatever it is.
If you have a job where you can look everything up, that job is getting offshored to India, where they can look it up too for less money.
@Ann:Try remembering all the Supreme Court cases about due process and equal protection — not just the doctrine, but the variations in the cases and the reasons the justices have given and the way legislatures and academics have responded, etc. etc.
You should know the general gist and what works or people are the right ones to go to, at the minimum. No one says you have to memorize every detail of what you ever might need to know.
But there needs to be a basic minimum in their heads that sticks, or they didn't learn anything.
"Why is reducing student anxiety a necessary component of testing? Because there's no pressure or anxiety in the real world?"
If what you want to test is the ability to deal with stress, then a closed-book exam would be one way to do it.
Another way is to let people have the book, then give them a challenging question where having the book is a false security. You jumped out of the plane with a parachute, but then the parachute doesn't open. That's even more stressful.
I swear I did every single thing wrong in my entire school/learning life. I learned to do math even though there are calculators. I learned to spell even though there are dictionaries (and now spell check).
I chose a major that led to employment opportunities. I paid back my student loans. And now I find out that studying was a sucker play too. All of them, now considered useless endeavors. I should have been out smoking pot!
"You should know the general gist and what works or people are the right ones to go to, at the minimum. No one says you have to memorize every detail of what you ever might need to know. But there needs to be a basic minimum in their heads that sticks, or they didn't learn anything."
That applies to an open-book test. If nothing is in your head, how could you possibly write a good essay in 3 hours, when you are graded on a curve with intense competitors?
@MikeR:Answers the progressor, "Son, you need it now."
Awesome. If my experience was typical, what he would have accepted, if you didn't know it, was a competent attempt to figure it out, or approximate it.
The bare minimum you ought to know is that the log of a factorial is a sum, and you ought to know how to sum a series at least. If you got tangled in the weeds he'd give you a hint, if you hadn't mouthed off.
Well maybe some of those junior high students went directly to nursing school...maybe even med school.
There needs to be a lot in your head or you won't be able to understand the question and know how to start putting together an answer, even where you can look things up.
(For law school exams, which I've been writing and grading for 25+ years.)
@Ann:If nothing is in your head, how could you possibly write a good essay in 3 hours, when you are graded on a curve with intense competitors?
Different university environment. There are no physics majors at my university, and the students are the opposite of competitive--if you curve none of them will do anything, because then they all get a C. "D is for diploma", as they say.
The world don't move to the beat of just one drum. What might be right for you, might not be right for some.
Forgive me, Ann, it's very near the end of the semester and I've been rolling that stone uphill for months, and it just keeps rolling right back down.
I hope you know what a blessing bright, motivated students are, and appreciate it, not all of us are so lucky.
I took a physics class on mechanics and for the midterm we all asked if it was open book or could we bring a cheatsheet. The prof said he would give us all the information we would need. On the day of the exam he walks in and writes in big letters on the board F=ma (force=mass x acceleration) and said that was the only equation we would need. He was right
Of course I know that knowledge is not the only thing that should be tested, and different types of tests will reveal different things about students. But sometimes knowledge is also worth testing, and having.
@dix:On the day of the exam he walks in and writes in big letters on the board F=ma (force=mass x acceleration) and said that was the only equation we would need. He was right.
Most new physics students are formula-hunters. They see something in seconds and something in meters and an acceleration and they look for the formula that has those things.
Then they get to force problems, which cannot be done by formula. It's a skill, like if someone said "Play this sheet music on the clarinet". If you didn't practice, you can't do it, with or without the book.
More than open books, I would have loved to be able to type answers on essay exams. Handwriting has always been slow and painful for me.
I do provide formulas on exams, since they usually don't know what to do with them it doesn't matter.
My worst test taking experience was an open book exam in seminary.
I'm good at tests--which was actually my downfall in high school, as I knew I could do well on tests so never did homework.
But that test was horrendous. He was asking for very, very specific information based on finding an exact phrase in the 300 page book, with no cues other than a half sentence quote. Meaning, we had to thumb through the book trying to find where the quote was.
Very stressful, as knowing the overall information helped not at all.
Weird thing, even though I've very good at taking tests, I don't like them and don't like giving them.
No tests at all in my courses. I do assign a lot of research and reflective essay writing--which are basically open book tests they can do at home and type on a computer so that the result is legible.
The trouble with tests, in my opinion, is they test short term memory not long term understanding.
@Paddy O:The trouble with tests, in my opinion, is they test short term memory not long term understanding.
I think a test should require a student to do something with what they know. Then it does not test short-term memory, though it still might not test understanding as well as you'd like.
I tell my students they need to practice for an hour or two every day all semester in order to do well on the exams, they still try to cram and they don't get anything right, and they are surprised every time.
Curiously, in theology and Biblical studies a whole lot of modern scholarship focused on why we shouldn't trust the main text.
Talk about a stressful open book exam. There's a test, you can use the book, but the test is to show why you shouldn't use the book.
Yeah, Gabriel, I've also taught Math for Poets type courses. I try to stress to them the basic rule in any math course: If you do lots of problems of each type, till you can do them easily, then you know it. Until then you don't. Do them until you start to get bored 'cause now they're easy. There is no other way to learn any math-y subject.
Most students would never believe this, however I tried. I had little choice but to give lots of homework problems and grade them too, and make homework grade a large part of the ultimate grade - my desperate attempt to get them to find out how to do them, and actually do them.
"I think a test should require a student to do something with what they know."
Very good point. I think this is true, but the trick is to "do something" besides recite memorized knowledge.
This is also why I've not entirely ruled out tests, just that I haven't seen a good way to test "doing something with what they know" in my field.
Or, I might just be rationalizing not wanting to put in the effort to write a good test and then grade it.
Ann Althouse said...
"How many bones in the body? Name them, starting at the head."
I had to do that for health class when I was in junior high school. It's not that hard.
Not that hard?
I Hate Every Bone In Your Body But Mine
In one ear and out the other
Told myself this girl is trouble, yeah
Everyone that looks at her
Is tragically struck in love
Wait a minute...
This chick's got me addicted
I hate her so even though I wish she was my girl
I hate every bone in your body but mine
I can't wait till I can hate you tonight
Never thought I'd understand
but she's too good just for one man, yeah
Round and round and round she goes;
Who she's with don't wanna know
A good way to learn law is a series of questions with multiple choice answers followed by the Prof giving the right answer and a discussion of why it's right.
Hopefully the experience will stick at exam time.
At least that is best method for the Rules of Evidence. Evidence is an area where the Judges seem to know less than a good lawyer who needs to give them an education similar to that experience.
@MikeR:I had little choice but to give lots of homework problems and grade them too, and make homework grade a large part of the ultimate grade - my desperate attempt to get them to find out how to do them, and actually do them.
Thanks to Yahoo Answers and Google this no longer works. I ran up against the five-minute barrier. They would try for about five minutes, google the answer, and then say, I think I know how to do that and move on.
So I stopped giving any credit whatsoever for homework. So they stopped doing it. So their final grade is much less easily argued about--I simply point to their homework that they didn't do, and say since they didn't practice how could they expect to do well?
It shifts the burden of proof, you see. They used to say, I did all my work, you made the test too hard. Now, they didn't do ANY work.
"How many bones in the body? Name them, starting at the head."
It varies. 206 is an average.
I was taking a course in fluid mechanics under T. K. Sherwood, one of the greats at MIT. He announced that there would be a quiz on Friday. Someone asked if it would be open book. He said that it would be closed book. A chorus of groans went up. He said, "Some day some of the best engineering you will ever do may be on the corner of a wind-blown piece of equipment out in the middle of nowhere, on the back of an old envelope. And it will be closed book."
One test a pipefitter/plumber must pass is fitting a rolling offset with welded pipe fittings.
http://www.americanpipefitter.com/Cal17,_Pipefitter_Calculator,_P.html
One of the tools needed is a "wrap-around", which has all the formulas printed on it. You either know how to use them, or not.
http://www.pces.uk.com/products/pipetools/wraparound/
I've heard the argument that "google" ought to be allowed too. If all you are required to do is regurgitate facts, who needs schooling? Grab it from google.
To me the distinction is between quantity and quality of information. In a statistics or elementary ordinary differential equations, the formulas are so distilled that they don't have much relationship to their derivation, so you don't gain anything much by memorizing them. In those classes I allow the use of a "cheat sheet." On the other hand in classes like advanced calculus or abstract algebra I have closed book exams and explicitly ask for statements of basic definitions. In elementary abstract analysis and algebra the primary skill students learn in the rigorous manipulation of the structure of basic definitions, and I feel memorization is one path to internalization of those structures.
"I don't know it, but I know where to look it up" has been around for a long time, and I think most people now recognize that it's validity is limited. I think Alhouse's distinction between doctrine and case detail is the right one here. My guess is that in law open book is the way to go because the quantity of detail swamps doctrinal concepts. There's no way to master doctrine in a two-hour test, so there is no need to force them to memorize case details in order to force them to learn the doctrine.
@MikeR: Ask yourself why you give homework? Because they need to practice to learn.
So why do you give them points for homework? To motivate them to practice so they learn.
But the students do not, in many cases, want to learn. They want to acquire points. You are trying to motivate them by having them imitate what a good student would do. They want to get points, and will do the minimum thing that gets them the points.
Give them points only for demonstrating learning. Half of them will never bother to do anything. The other half will start studying, but half of that half will start to late for it to do any good. If you can weather that storm of DWFs, and irate parents (in my case), you can actually see people learn ad remember things.
@MikeR. and Gabriel Hanna - I've graded my last homework from a major textbook this semester. Next semester I'm assigning homework, going over answers every Friday, and then giving a quiz directly from the homework on every Monday.
Closed-book, of course.
And, of course, we want to prepare them for a career in law which will be without stress or anxiety. Good that they're entering into a profession which will never call upon them to think on their feet. As a trial lawyer of 30 years' experience, I can say that situation has never arisen.
I taught as an adjunct for twenty years. Open book, closed book...it just didn't matter. The curve was always the same.
Name these on a junior high school test
Name them, starting at the head.
Toe bone connected to the foot bone
Foot bone connected to the leg bone
Leg bone connected to the knee bone...
The trick is to sing it backwards!!
"And you tell him you'll look it up and let him know later... you're not getting hired."
On the contrary, I wouldn't hire someone who didn't bother to double-check his assumptions. It only takes a few minutes to run the real numbers on your phone, after all.
Anyway, I like open book tests. I have trouble remembering my own cellphone number, but I'll somehow remember that the production numbers for the 1965 model year are in the book with the blue cover, under the second major heading on page 12 of Appendix III.
As a first semester undergrad I made the strategic error of scheduling an 8A M-W-F class, a literature survey. Ont he Thursday night before the final class I went out with my German class and put on a horrible, epic drunk, which rendered me unable to attend the final 8A class the next morning. Consequently, when I arrived at the final the following week, I was the only member of the class who did not know it was an open book test. Everyone else had all their books with them, and I had a #2 pencil. Did not turn out well.
I'm not sure how this is relevant to your post. I just enjoy telling the story.
@Inga: For a medical professional inspiring confidence is possibly the most important thing, and knowing a great mass of information so well you don't need to look anything up is probably the best thing.
I am not sure Ann was trying to minimize that, I think she was pointing out that it was just memorization. Well, I agree with you that it happens to be important, and doctors and nurses need to learn it so well they never forget it, and that's a bit more than was expected in junior high school.
I use the strategy Lyssa says she liked from her college stats class (I teach Mathematics at a state university). I give the pertinent formulas, such as a brief integral table for calculus-related courses. But I don't give open-book exams, because students mostly use math textbooks for just looking at example solutions, and I don't want them to be able to do that. Know the methods and how to apply them -- I don't care if you remember the derivative of the inverse secant function.
Actually, modern undergraduate textbooks are good for little *other* than looking at example solutions anymore. But that's a different topic.
@Bryan C: I wouldn't hire someone who didn't bother to double-check his assumptions. It only takes a few minutes to run the real numbers on your phone...
No one is saying "never double check".
My wife's aunt wanted some remodeling done. One contractor said, sure sure, we can probably do that, I'll go back and check on that.
The other said, this is a converted pole barn and local regulations will not allow us to do what you want done to it.
Was the first one out to fleece us, or did he just not know?
@madashell: The curve was always the same.
Actually my curve changed a lot when I switched to grading only on demonstrated learning. Instead of a bell curve, or a bimodal curve, I got an exponential curve--the one that governs rare events...
@MarkNielsen:I don't care if you remember the derivative of the inverse secant function.
I never remember it, I remember how to work it out if I need it. Takes a few lines and doesn't clutter my brain up.
There was only ever one thing I set out to memorize in my physics career, and that was the formula for a radiating dipole. I knew there would be a prelim question on it, there was every year. And it was hard to remember the assumptions needed to derive it. Everything else I just counted on being able to work out if I needed it.
Strelnikov said And, of course, we want to prepare them for a career in law which will be without stress or anxiety. Good that they're entering into a profession which will never call upon them to think on their feet. As a trial lawyer of 30 years' experience, I can say that situation has never arisen.
Stress, anxiety, and thinking on your feet - definitely (for a trial lawyer), but one of the best things that you can learn for motion practice or trial is how to create good notes that you can refer to as you present in court.
I've found open book exams are easy only if you really know the material. If you've been lazy or had trouble with it, you'll have problems.
I had a take home final in chemistry in V Form and aced it, partly because I was able to use my Dad's old chemistry books from his days at Cornell.
Inga said...
Open book exams?! What we would've done for an open book exam in nursing school. How many bones in the body? Name them, starting at the head.
The Blonde hated them, she said it just gave the instructor the chance to make the questions that much more difficult.
Open case books are worse than useless in a law school exam. If you haven't extracted the relevant info and put it in context in an outline yet, you're sunk. And if you have, you won't need to crack the case book.
Well, I see that others are still struggling with what used to be the most painful thing for me about teaching classes for unmotivated students. That weekly quiz on the HW sounds like a good idea; anyone tried that? Lots of little questions, not too much time, hopefully very easy to grade. They can do it if and only if they've spent the time the week before doing the homework.
@BobR and MikeR: I tried the quizzes over reading and homework. My students just wrote off the points and complained to the chair that the class average was too low and therefore I had made things too difficult.
My brief teaching career was at an institution with students whose motivation was much like those Gabe Hanna depicts. Being an "old school" political scientist all my exams were essay exams as had been the case for me during my undergrad days. Silly me. It got to the point that I was giving credit for three consecutive grammatically-correct sentences--never mind whether they were even marginally on point. I always wondered: just who were the English profs who gave these kids passing grades in the first place by the time I got them?
@virgil: just who were the English profs who gave these kids passing grades in the first place by the time I got them?
First, not kids. Adults. We should not infantilize them, this is part of their problem.
Secondly, no one has a tougher row to hoe than those first English profs they see in college. Trust me, they are doing all that any one could do in their situation. I do not blame them.
My students can't usually add fractions or do algebra, yet I have to teach them calculus-based physics. They had algebra and calculus in college, and I'd hate to see what they were like before then.
I always hated tests. Always.
Gabriel,
"There are no physics majors at my university, and the students are the opposite of competitive--if you curve none of them will do anything, because then they all get a C. "D is for diploma", as they say."
"My students can't usually add fractions or do algebra, yet I have to teach them calculus-based physics"
Ow.
I have to ask--forgive me if this is too personal--what heinous crime did you commit in a previous life? That just seems like way too thankless of a task...
I had zero test anxiety and hated open book tests because I figured they were taking away my advantage.
"I had a stats class in college where the formulas were provided; the prof said that in real life, you can always look the formulas up, the test is whether or not you can apply them correctly. "
I take it you went to a different school than Michael Mann.
"Life is an open-book exam."
Every engineering exam I took in college was open-book. This does not make them easy; it makes them a different kind of exam. The hardest one was open-book with no time limit.
Toughest exam I took was for an advance "Theories Of Personality" class. Ir was a "take home exam". THE ONE QUESTION: Develop your own theory of personality; Defend it".
Some of the graduate students in that class failed it.
Open-book exams are cruel. The best exams are those in which you're only allowed a pencil. In those you can be 100% certain you can pass using just what an ordinary human can be expected to fit into his brain. You won't be savaged because you haven't indexed your book well enough, your fingers can't fly fast enough or your visual cortex scan dense paragraphs for key words fast enough, or the book is poorly written, or anything else out of your control or having zero to do with integrated mastery.
By contrast the ability to look up some minor fact or other is poor recompense. Unless, of course, the exam is so stupid that jotting down mere data correctly constitutes the bulk of it. I guess such exams do exist, mores the pity.
My hardest and best course in college used open book exams. It was an organic chemistry class for chemical engineers. I took it to avoid the rote memorization required in the pre-med organic chemistry course. I think I learned a better appreciation of organic chemistry in the open book class. The professor would make up molecules. We had to predict the outcome of their reaction based on the princples of organic chemistry. It was not an easy course, but I learned a lot more than I would have learned in the premed class.
I give open-book law exams for all the reasons stated by Prof. Althouse. It would surprise me, however, if my students have any time to look anything up. The possibility that they might be able to do so seems to be enough to ease some anxiety.
Gabriel Hanna, I feel your pain. I've been trying all sorts of things to increase learning. This year I'm giving 10-point open notes quizzes on each section of each chapter. The students get to use their notes. While we are covering the material, I tell them, "This will be on the notes quiz for this section." Most of the students respond well to the hints and appreciate the "easy" points, but there's still a fair percentage who routinely fail the little quizzes.
My most recent test was open book, on the periodic table of elements. Since they needed a reference table and their book has one, I just let it be an open book test. It was not a hard test -- gave them the atomic number of an element and had them list out all the information about it. I wanted to see if they could read the table, which we had practiced in class 2 or 3 times. No surprise, they mostly couldn't, even when reminded they could use the whole book, that is, look up terms they didn't know.
The kids that don't care won't work no matter what, the students who do appreciated being able to use the book.
I second Lyssa's 8:00 a.m. comment: Life is open book, mostly.
But every so often, there's a life-or-death pop quiz.
Reality always ahs this pesky habit of living in between the binary alternatives our minds create
Post a Comment