August 25, 2009

"To say you can’t improve scores is to say you can’t improve students, and I disagree with that."

Said Stanley Kaplan, dead now, at 90.
He began by preparing students for the New York State Regents exams. But when a student showed up in 1946 asking for help on the Scholastic Aptitude Test (as the SAT was then called), he saw an opportunity. And when students later sought his help on medical school exams, he signed them on, too.

For decades his services remained local, marketed to Roman Catholic schools and to yeshivas. Most students arrived by word of mouth. But he gradually began to attract students from around the country....

Despite his growing success, Mr. Kaplan faced resistance from the College Board, which continued to assert that gains from test-preparation courses were minimal. Opposition was so strong, Mr. Kaplan recalled, that some students felt a need to register under false names, like Jane Doe and Albert Einstein.

In Philadelphia, the University of Pennsylvania student newspaper refused to run his advertisements, and the university denied his requests to hang posters and rent rooms for his courses. He called the opposition “elitist” and distributed T-shirts on campus. Students flocked to his classes.


traditionalguy said...

Why are Educators threatened by a wide distribution of their knowledge monopolies? Job security must be a basic human instinct.

Balfegor said...

For decades his services remained local, marketed to Roman Catholic schools and to yeshivas.

. . .

That experience made him a champion of standardized tests when others attacked them. If there had been a medical school admissions test, he said, he could have shown the medical schools that he was the equal of students from private universities.

I think a lot of people do not realise what a leveling effect standardised tests have -- unlike most admissions methods, there's no hidden ball moving where the test-taker can't see it. Instead, there's a test, you prepare, you do well, and you get your foot in the door of elite institutions that would have rejected you without a second glace a generation ago. It's a huge leg up to people who are "outsiders" to those elite institutions (e.g. Jews and poor Catholics, in the early and mid-20th century; Asians in the later 20th century).

John Lynch said...

I like the tests because there is an honesty to them. A or D? B or C? That's it. It's you versus the test, and everyone else taking it.

Test taking IS a skill. It can be learned. There's nothing wrong with learning it, and anyone can. People who refuse to learn this skill are just as ignorant as those who refuse to go to class. You're just ceding your future to others.

Balfegor said...

Test taking IS a skill. It can be learned. There's nothing wrong with learning it, and anyone can.

Up to a point, I suppose -- I think there's a lot less "skill" involved, and more acclimation to the testing mode. There are, after all, people who seize up and perform poorly during high pressure examinations, and doing hundreds of drills and simulated tests (which I understand Kaplan to involve) is a good way of getting over that anxiety. Even if his testing "skills" weren't directly useful (and I'm inclined to think they're not particularly valuable, although I would, since I have never taken a Kaplan course) the placebo effect is still probably worth the money people pay -- to some test-takers, at least. Gives them confidence.

Big Mike said...

John Lynch is dead right. Just convincing people to read the instructions carefully and not to waste too much time on any single question is worth 10 - 20 points. I don't think you can turn an 1100 into a 1600, but you can turn a 1300 into a 1400, and that will move a student up one whole tier in their choice of colleges.

(Pardon -- I'm still thinking in terms of the old 1600 = perfect. My kids got into college before they added the essay.)

Bissage said...

It was upon the advice of a law professor that I signed up for Kaplan. The classes were a waste of time. Trying to read the outlines induced obsessive thoughts about running the head of my penis over a cheese grater.

So I ended up taking the practice exams at my leisure and thinking hard about the right and wrong answers. Passed first time.

Thank you Mr. Kaplan. I’m glad you got rich.

I hope you were happy, too, which is better, after all is said and done.

Balfegor said...

Trying to read the outlines induced obsessive thoughts about running the head of my penis over a cheese grater.

I did Bar-Bri. For the outlines, I actually just read them into MP3s at the beginning of the summer and listened to them through headphones -- easier than reading them over and over, and I could do it with my eyes closed, lying in bed in the heat of summer. I don't read notes and things into Mp3s much now (very little occasion for it), but I've found I can learn specific material much more effectively that way, through aural repetition, rather than through drills or reading. Did it for an amateur play once too -- someone dropped out, if I recall correctly, so I had to learn his part in a day or two before our full rehearsals started. Audio works extremely well, I find.

Joan said...

Balfegor, you're an unusual person. Most students retain only 10% of what they hear, which is why straight lecture is the least effective method of instruction (at least in elementary school).

Kaplan's methods work because they de-mystify the test. Walking into such a crucial test cold can make a huge difference in your score.

Don't forget that standardized tests are very good at keeping people out, too. We're dealing with the local school system's use of the CogAT test for honors placement -- years of consistent A-level work and high scores on other standardized tests mean nothing here; if you don't score in the 90+ percentile on the CogAT, you can't get into the Honors program. It's something I should have looked into more thoroughly before I moved him into his new school -- live and learn.

Balfegor said...

Balfegor, you're an unusual person. Most students retain only 10% of what they hear, which is why straight lecture is the least effective method of instruction (at least in elementary school).

Fair enough. I always put those things on endless repeat, though, so it's very different from the lecture context.

Don't forget that standardized tests are very good at keeping people out, too.

True. But that's true of any admissions method -- that's exactly what they do. Most other admissions methods are much more heavily biased towards insiders, and tend to be directly reflective of the prejudices of the people doing the picking. A bad test day is problematic, obviously (I have an uncle who had a gap year to retake the Seoul National entrance exam), but there's a tradeoff between constraining the prejudices of the admissions officers and minimising bad luck. Bad luck can be overcome if you get a retake, but you can't really get around admissions officers who don't like your kind, or think they've already got too many of you.

Big Mike said...

@Joan, there are several different learning styles, and if K-12 education was handled intelligently in this country then students would be helped to identify their best learning style and maybe even separate the students according to learning style and develop class plans accordingly.

Instead every students gets treated as though there is only one "correct" learning style. So kids, especially boys, drop out.

Big Mike said...

@Balfegor, very insightful. But note that they never seem to have too many power forwards or guys who can run fast holding a football under their arm.

fivewheels said...

As someone who did well on tests, I sometimes struggled to keep myself from viewing those who rail against testing as whiners who didn't do so well. I even went through a noblesse oblige period when out of misplaced sympathy I tried to be on the side that said tests don't really tell you how well someone knows a subject, it's all just test-taking skill -- but reality is hard to deny for long.

So if you want to help the people who don't do well, you have two options.

1) Rig the system. Introduce subjective factors so you can boost the "performance" of those you wish to help simply by declaring it to be so. "Sure, she was only in the 40th percentile on knowledge, but she's 95th percentile in Leadership and Overcoming Adversity! And as we all know, no Asians are ever leaders or ever experienced adversity, so they can go to the back of the line."

2) Help them do better on the tests. This is the way Kaplan chose, and I respect it.

Joan said...

Big Mike, I'm in a teacher certification program right now. Instruction in the different types of learners began with the very first class. The "methods" courses make a very big deal of engaging all types of learners. It turns out that everyone learns better if you minimize lecture and use hands-on, relatable techniques. If you handed in lesson plans consisting solely of lectures followed by tests, you'd fail the course.

Balfegor, of course you are right. I'm just annoyed on behalf of my kid. ;)

John said...

My son just passed his final medical board. He had tried to do it before by himself and almost passed but failed.

He went to Kaplan and spent several months with them working his ass off10-14 hours a day 5-6 days a week. He kept wanting to take the exam (it is given monthly) they kept telling him "No, you are not ready yet"

Finally they said "Go for it" and he did very well indeed acing several sections.

I am a big believer in Kaplan.

John Henry

hdhouse said...

My high school forbid the taking of anything like an SAT prep. No good student I know would lower themselves to taking a course on preping for a test..any test.

I'm pretty much of the school that AP classes are just a horrible idea and "no child left behind" is an abomination.

Just where I come from.

Synova said...

"Instead every students gets treated as though there is only one "correct" learning style. So kids, especially boys, drop out."

Teachers are supposed to teach each lesson to a variety of learning styles so no one gets missed. This, of course, means that the material is covered at least three times and most of the class is usually disengaged and bored.

"It turns out that everyone learns better if you minimize lecture and use hands-on, relatable techniques."

Not really. Boys do better with hands on (usually) and girls do better with group co-operative tasks (supposedly) but a large slice of the boys and a good number of the girls would rather that they got a verbal description and a read through and were left alone to write the answers on a piece of paper that they, and no one else, got credit for. The bright "stimulating" educational decorations in a classroom are a horror for any child who is ADD or ADHD (and ADD flows under the radar without the disruptive H). The noise and motion of the "hands on" learning is the same.

"Relatable" is great. I just sent a letter to my daughter's math teacher that in order to relate math effectively to her life the teacher should concentrate on the usefulness for the language of Math for an insufferable pedant. ;-) Okay, so I didn't say that my child is an insufferable pedant, but close enough and it's true that the idea that with Math she could be RIGHT and exact down to the minutia made her eyes light up. (She hates math.)

What, really, are the chances that this sort of abstract relation of math to a child's life is going to get any attention next to the more concrete and pointless things like using math to buy something?

I honestly think that the more that teachers are taught to try to reach everyone the more students are missed. Now even the "good students" are missed.

The old way favored students who were suited to sitting in chairs and learning from books. Those who were better with hands-on went on to hands-on sorts of careers rather than sitting in chairs and reading books sorts of careers.

Now we're trying to turn every child into a college student.

Synova said...

I'd also say that, yes, taking tests is a skill that can be learned.

The "old way" taught kids to take tests, multiple choice!, and all that. The new way? Not so much, I think. How does cooperative learning and projects, no matter how great it sounds to learn in a way that is useful to an office work environment, translate to writing an essay correctly or taking a huge, vastly important, multiple choice test?

Chip Ahoy said...

All those test prep workbooks are the best puzzle books ever. Better even than NYT crossword Sunday compendiums. I recommend them all, for fun.

Big Mike said...

@Joan, I had a son who was coded in elementary school, and one of the things I noted up and down the line with his teachers was that if you fed the right key words they could quote chapter and verse for learning disabled children, but with a child right in front of them they could not connect the dots. His problem was that he wasn't retarded, he was in fact quite bright. But he had fine motor control problems so severe that he couldn't write in cursive, he needed extra time to do an exam, and he had to do fill-in-the-bubble standardized tests by dictating the answers to a proctor. But he wasn't retarded so the teachers had a problem recognizing the situation even though, as I said, he fit his profile perfectly.

So, pardon me, but as far as I'm concerned those teacher certification programs are a waste of time and effort, and whoever teaches them must be worthless on the best day of their respective lives. Apparently one passes the certification programs by memorizing and vomiting back a few sentences verbatim in response to a key word or phrase. But they can't translate any of that to the child in front of them.

I hope you turn out better than the alleged "educators" my wife and I dealt with. But pardon me if I look down on your profession. If it's not about the kids, then what's the point?

John Lynch said...

I retain things I hear very well, so straight lecture is the absolute best way for me to learn.

Soooo, when I started having courses that changed that, I wasn't happy. What's good for the majority of students is not good for the minority who are exceptional. The old school teaching methods are not for white males (I'm not one anyway) but for people who are serious about learning.

I ended up being a mailman, so this isn't one of those posts about how smart I am. But I did get a 1400 SAT back in 1990, and I did spend a lot of time in dumbed down public school courses with low standards. "AP" courses were for smarter than average upper middle class types who wanted to go to a good college. They were college prep, not great in their own right. I failed AP English class from non attendance and got a 5 on the test. Not my smartest move ever. Remember, I ended up as a mailman. I like it.

If you make your teaching methods for the mean, you'll get the mean. The whole system is set up for the above average student who shows up and does their homework. It's not set up for the students that are bored out of their minds because the material is something they read on their own years ago. Nor is it set up for the students whose background does not include reading or education as goals. A lot more effort is put into that, however.

If you push kids with higher expectations, you'll get better results. My favorite teachers were the hard ones. I'd get an "A" in classes everyone else hated, but get a "C" or "D" in an easy course. It wasn't easy for me to sit through.

Sure, just because a class was bad for me doesn't mean we should change everything. I was only one difficult student. Except that the education system says it will educate everyone, and it certainly doesn't. It punishes everyone too far away from the mean, in either direction.

Joan said...

Well, Big Mike & Synova, I don't blame you for looking down on teachers and being disgusted with a lot of the "teaching" that's going on these days. My daughter's 5th-grade teacher insisted yesterday that "chose" is the correct spelling of "choose" -- repeatedly, apparently not able to tell the difference between the two. At least last year, one of her teachers was at least as bright as she is; this year, it appears to be a complete rout.

That said, I hope to be the kind of teacher that parents respect and kids remember. The program I'm in is not U of Chicago based, it's at a community college and focuses on the work of teaching, not all that philosophical crap. Synova, when I said lessons need to be relatable, I meant to real life: your daughter's fondness for math may increase if she's reminded that it's her money, she ought to know how to spend it best as well as how to count her change. And I detest group projects because, frankly, they suck in all ways. It's impossible to tell who did what, and grades never reflect effort or ability accurately. There's a lot that's wrong with pedagogy today but there are people who are trying to make it better, too.

wv: xenzool - I don't know about you, but my kids' zool doesn't make me feel in the least bit xen.

Big Mike said...

@Synova, right about now I'd be happy if your daughter and her classmats wind up understanding that a trillion is a million millions. That would put them one up on the President, at least.

Seven Machos said...

This topic is near and dear to my heart. The idea that standardized test score cannot be raised is an absurd myth, and I thank Stanley Kaplan for starting the process that punctured it (though it sadly still survives).

What's more interesting to me is the persistent hue and cry that standardized test scores are not going up, thanks to our schools which must need more money. Just read an article about this in the WSJ.

People (especially WSJ writers), listen: these tests are graded on a bell curve, and the scores are fitted to that curve. Do you understand what that means? It means that the scoring system is rigged so that scores will never go up collectively.

Hence, we have a strange social situation. (1) Enduring belief that individual scores cannot be raise. (2) Rigged scoring system that ensures that collective scores fit a preconceived bell curve. (3) Belief that collective scores should go up.

That's crazy.

Big Mike said...

@Joan, the point about group projects is not only that it cuts the number of projects to grade by a factor of three or four, but by carefully arranging the groups you can legitimately pass somebody who should be kept back by sticking them with a group of brighter kids, and moreover you can reward your pets by doing the same thing.

I've heard plenty of agitation for more group projects on grounds that kids exiting college today allegedly don't know how to function on teams. But I have led teams and managed teams of teams on large programs, and a team of programmers or a team of engineers is nothing like a group project.