A time-pressure test isn't really detecting how quickly you can answer questions. Time-pressure can interfere with your concentration and create static that makes you slower than you'd be if you took the test without a time limit. Some people worry about the time and some people don't. Some people hate to be rushed and others find a deadline motivating. Why systematically disadvantage those who are inclined to be careful and systematic and favor those who take risks and shortcuts?
September 21, 2023
"For decades, educators have seen speed as a marker of aptitude or mastery.... But a race against the clock doesn’t measure knowledge or intelligence."
"It assesses the much narrower skill of how well students reason under stress. As a result, timed tests underestimate the capabilities of countless students.
New evidence shows that although smarter people are faster at solving easy problems, they’re actually slower to finish difficult ones. They’re well aware that haste makes waste, and they don’t want to sacrifice accuracy for speed.... Although it pays to be quick, it also pays to be determined, disciplined and dependable. Strangely, though, the tests that define students’ grades and help determine their educational and professional fates... evaluate students as if they’re applying to join a bomb squad or appear on 'Jeopardy.' Time pressure rewards students who think fast and shallow — and punishes those who think slow and deep...."
Subscribe to:
Post Comments (Atom)
Posts
98 comments:
Wow kids on speed but no tyranny of the clock for tests. Weird times.
Oh, OK. Because IRL nobody ever has to "reason under stress." They just savor the flavor of the tobacco in their Sherlock Holmes pipe, and go back to Baker Street and think on things, and the next morning they rouse Watson with a breakthrough idea.
What utter fatuity. The whole point is not about getting a "perfect" read on intelligence, it's about how well you do against others in a reasonable environment of a reasonable cognitive test. It might be "counting backwards from 100 while doing push-ups." It might be "running differential equations to 22 decimal places while wearing headphones blasting Led Zeppelin."
The test hardly matters --if it's a reasonably relevant test of cognitive capacity-- because what matters is that EVERY CANDIDATE TAKE THE SAME TEST UNDER THE SAME CONDITIONS.
Sheesh.
"although smarter people are faster at solving easy problems, they’re actually slower to finish difficult one"
So no prob on the old SAT, which is mostly easy problems, leaving plenty of time for the few hard ones. Anyway, since I was too lazy to delve into the article links, how much more predictive is the new math SAT for, say, freshman calculus performance and overall grades?
Much shorter tests of cognitive functioning have been shown to have enormous predictive value, both for other dimensions of cognitive ability and for life competency. Anyone know of comparisons between, say, Wechsler and the SAT?
SAT to explore Parkinson’s law.
This was a joke, right?
If not, please list a career that DOESN'T have stress.
New evidence shows that although smarter people are faster at solving easy problems, they’re [the smarter people, that is] actually slower to finish difficult ones
So it appears that giving a time-sensitive test will still suss out the smarter people; it can just do so in a shorter time.
The SAT is the Scholastic Aptitude Test. I've always understood that to mean that it was attempting to measure a student's general ability to address a variety of types of problems. If it tracks accurately with how the student does in college - and it does, and if it also tracks well with how a "smart person" solves both simple and complicated problems - and the research to which they're referring seems to suggest that it does, then what's the problem?
Is this just a churn operation for test prep companies?
Time-pressure can interfere with your concentration and create static that makes you slower than you'd be if you without a time limit.
Sorta like life in the real world, Eh?
Being able to come up with correct answers within a reasonable amount of time is an extremely valuable skill. It forces you to develop the ability not to overthink, and to understand which information should be considered and which should not be, and the relative weights to assign to the information that you consider. On the other hand, taking a long time to figure something out is often not very useful. As there is no single definition of "intelligence" that everybody agrees on, measuring a related skill that clearly affects real-world success is very useful.
My sense is that time management is largely culturally determined, and that certain cultures are worse at it than others. (This is related to Malcolm Gladwell's explanation of how rice farming has strengthened math skills among the Chinese.) Could it be that relaxing time constraints is intended to improve the scores of groups that historially have not performed well on the SAT?
And furthermore - until I see some compelling and replicable research to show that working to a deadline is not sufficiently valuable in the real world to want to see a measure of it in a standardized predictive test, I'm going to have to rely on my personal experience.
Which is that working to a deadline is a valuable skill in pretty much every field of endeavor.
Because in the real world, speed (and the ability to work quickly under pressure) is often important?
I always thought large capacity standardized tests were timed in part because a shorter test time makes cheating more difficult.
I’m slow and deep… like a sub.
Run Silent Run Deep (1958)
You got a lot of guts sir… with other peoples lives
Because taking risks and using shortcuts can sometimes be a good thing.
Have you never heard os paralysis by analysis?
Vicki Hearne suggests blacks are slower to believe things, which works against reading comprehension tests.
Nevertheless the existing tests do correlate with success, which is what they're meant to predict, no matter what real intelligence is changed to.
Peterson that IQ is the most solidly verified metric in all of psychology.
Gee, does that apply to all the other timed exams out there? The GRE, the LSAT, the Bar exams, insurance, the CPA and EA and nurses and teachers and MD step exams...
Gonna make scheduling hard for Prometric and the other proctoring sites.
I'll chalk this up to another instance of our testing parameters aren't giving us the results we want so we'll change the parameters to see if we start getting the results we want. I'm assuming this is a result of the recent Supreme Court case, and now institutions want to find ways to get the racial demographics they want in admissions without running afoul of new standards. I'm assuming one of the practical effects of timing tests is similar to adding 'tricky' questions that are phrased in negative or sometimes double negative terms in that they give a wider distribution of scores. So removing time limits, might narrow the curve a bit. More so than regular exams and finals, isn't the main purpose of the SAT to try to rank order students by aptitude? I would guess if there is change in the test score distributions, it is going to be more centered in the middle and lower end of the distributions. I find it hard to imagine there are many students who are getting 1500 on the SAT who are now scoring 1600 because they have more time. Though I can imagine more students who are getting 900 on the SAT who are now getting 1000.
Time pressure is particularly problematic for people at the high-functioning end of the autism spectrum, and most especially so if it forces them into areas in which they are not neurologically capable of functioning even at neurotypical speed. For example, dysgraphia, one expression of ASD in some people.
Merely writing, by itself, requires intense concentration and effort. Simultaneously demanding rapid performance or production often triggers overwhelm or 'freeze' reactions. ADA might have something to say about refusing to accommodate such a disability feature.
"the tests that define students’ grades and help determine their educational and professional fates... " (emphasis added)
Yeah, but they don't anymore, do they?
Huh? So the new SAT is measuring determination, discipline and dependability? That seems implausible, that there can be a standardized test that measures those things.
My company merely lowered its (timed) CE exam pass rate to 70 from 80.
The whole world is dumbing down.
"Careful and systematic" always always requires factoring time constraints into the solution. Life is like that.
Sigh. This reveals wishful thinking but not much more. Change the SAT anyway you like...the same groups and individuals are extremely likely to bubble to the top. The new scores will likely have a 0.9+ correlation with the old ones (i.e., practically the same).
Background
IQ testing began well over 100 years ago, but really got rolling with WWI. The military wanted to assess the potential of draftees, and created short tests to place unknown farm boys in support roles, strategy, combat, or whatnot. Physical tests evaluated strength, shooting tests evaluated shooting, and IQ tests evaluated logical reasoning ability. For whatever reason they correlated very, very well with potential academic performance. If one scored high on a validated IQ test then they'll generally be capable of scoring at the top of any given class for essays, reports, and exams. This is because IQ is based on bell curves (aka normal distributions aka standard distributions), and by definition it must place all test takers into high, middle, and low clusters. This means there will always be a winners-vs-mediocre-vs-losers competition.
The 1960s civil rights era sparked a huge backlash against IQ tests, famously citing an item requiring knowledge of the meaning of a "regatta." Only the wealthy had yachts or any clue about boat races. So, IQ tests were renamed as skills, abilities, and experience tests, and changed to real-world tasks based on real job assignments. Furthermore, the language component was removed so that even those without any writing ability could be given validated picture-based IQ tests.
Drum roll please: Same outcomes! Same outcomes! Same outcomes!
By the 1990s even the fiercest scientific critics of IQ tests admitted that some groups perform differently than other groups. The outcomes remained with methodological changes they required. Biological males tend to have a broader distribution than females, with more extremely high and extremely low scores. The 'geniuses' are always outliers, and thereby more often males. Racial group differences were always found too, and they remain in the same order with every IQ or skills test that I know of.
Time stress: This is a modest factor but will perhaps shift some people a few points up or down. BUT BUT BUT, most everyone does best with "moderate" stimulation and stress. Too little and they become mentally sleepy, and with too much stress they panic and lose focus. So...provide enough time for 90% of the test takers to complete all of the test items and the outcomes will be similar as if providing unlimited time. Some people can never dunk a basketball or hit a home run no matter how many tries they are given. Some people can never complete complex logic problems, nor can they ever write a coherent paragraph. Cold facts of nature.
Quick and dirty sample references
Yerkes-Dodson bell curve on stress and performance: https://www.healthline.com/health/yerkes-dodson-law#stress-performance-bell-curve
https://www.britannica.com/science/intelligence-test
https://www.brookings.edu/articles/the-black-white-test-score-gap-why-it-persists-and-what-can-be-done/
https://www.nature.com/articles/nature04966
https://iqtestprep.com/minimum-iq-for-military/
https://www.verywellmind.com/what-is-test-anxiety-2795368
Sigh. Mother Nature is brutal, but don't deny Mother Nature.
"Time-pressure can interfere with your concentration and create static that makes you slower than you'd be if you took the test without a time limit."
Slow is smooth. Smooth is fast.
Old news.
OK...mea culpa. My NYT paywall reading trick didn't work this time.
What game is being changed?
And why does it matter if institutions are scrapping the score anyway?
All they want now is a strong hint in your essay that you are either black, Hispanic, gay, or trans...
Since pretty much everything the Lefties (like the College Board) do has an intended beneficiary, usually one of their pet-cause groups, you have to ask, who do think benefits from this change? Who shows up better on the sort of test described here, instead of the traditional SAT? (Where the advice always was, if you don't very quickly know the answer, move on, keep moving, don't stop!!!)
I expect that the groups that do poorly on the faster-faster-faster tests, will also do poorly on the stop-and-smell-the-roses tests, too. Individuals will vary in which is better for them, but it's unlikely that there will be demographic groups that will excel on one or the other style of test, but not both.
Maybe they're just so frustrated that they can't seem to solve the testing underperformance of certain groups, they're just going to try anything, and see what works. Roll the dice!
Because human flourishing prizes those who who take risks and shortcuts.
- Rafe
New evidence shows that although smarter people are faster at solving easy problems, they’re actually slower to finish difficult ones.
Reminder: SATs are for High School students, and the questions are, for the most part, comparatively easy.
The 18 year old brain is a much different machine than a 40 year old one. More high speed and intuitive, so connected that you solve toy school problems quickly without any conscious deliberation. At 40 you deal with complexity and careful reasoning is appropriate, and at 40 the cost of making a mistake is higher so you are more careful for your reputation.
In Stem class tests if you have to think about a problem before doing it for more than 10 seconds you are probably screwed. Homework is different and it can be a longer deep dive because you have time.
One of my profs in math told me if you study the average person should be able to answer 80% of the questions by rote, and the last 20% require insight, to find the A people, to get them into grad school.
A race condition measures intelligence, recall, and confidence, which may or not be an aptitude required under all circumstances.
How well I reason under the stress of a time constraint depends on how much I've mastered the reasoning and how much aptitude I have for the reasoning in question.
Give me world enough and time, and I can probably figure out the Pythagorean theorem from first principles. But if you're trying to test whether my innate talent for such things has allowed me to already internalize the concept and all its attendant reasoning, you don't need to give me an hour to answer the question. Five minutes will do.
One possible answer to your final question:
If the shortcut is not actually a risk, but is a well-grounded consequence of the systematic rules (I’m thinking about algebra shortcuts that arise out of proven theorems), then the use of the shortcut demonstrates a deeper understanding of the material.
(On the other hand, I’d prefer an essay question asking the student to explain the algebra shortcut. That would REALLY separate the wheat from the chaff.)
They're still hoping to find a way to bring black and Hispanic scores up closer to white and Asian. Otherwise, they'll lose more market share to Woke schools dropping their SAT requirement to avoid more anti-"diversity" lawsuits.
I had extra time left to reexamine difficult questions, but it didn't do me any good.
I don't really remember much about bar exam questions or law school exam questions (I graduated from LS in 1968 and took and passed my first bar exam that year; I took (and passed) my last bar exam in the '90's). You were frequently told regarding exams that you were supposed to "spot the issue", and this was particularly true when exams had to be machine-graded. It IS important to spot the issue, and if you have command of the subject matter it shouldn't take long.
But having spotted the issue, then in the real world you have to figure out an approach to that issue that is relevant (helpful) to your client, or agency, or court. To do it right, that certainly requires research that you can't do during an exam, as well as a lot of deep (time-consuming in some cases) thinking.
Issue spotting is a useful tool for a lawyer, even though it is only ONE tool, but it may be the only thing that can be tested for in an exam setting.
At some point, they will simply make the SAT a take home test that you don't have to turn in for a month.
Timed tests do a few things:
Provide a bound on difficulty and number of questions.
Test the ability to work and decide under pressure.
Prevent perfectionists from spending hours trying to... perfect... their answers.
Provide an indication of subject mastery in terms of ability to comprehend and solve problems expeditiously.
Where one testing style is disadvantageous to one group, another will be disadvantageous to another group. To let everyone preform at their best, there would need to be several available options to choose from: timed, untamed, open book, take home, live interview, etc...
Whether this change is good or bad really just depends on what the end users of the data from the tests are looking for since it will subtly change what is being tested.
Time is money. The SAT is designed to identify the moneymakers.
If you remove the time element, the SAT has little or no value as a tool for evaluating the ability to achieve.
So it will be replaced by something that evaluates "the much narrower skill" of how to "reason under stress."
That is what it is all about.
I don't want to hire someone who takes 4 hours to do what someone else can do in 2. Because it costs me money.
Having taken the SAT 40 years ago when it was probably a tougher test, I doubt the there is anything in today's test that requires long, deep thinking (there certainly wasn't 40 years ago)- it isn't like you are being asked to prove the intermediate value theorem or write a term paper on T.S. Eliot.
Why don't we just add 200 free points to all non-Asian or non-white minorities and get it over with rather than all this bullshit effort to find a version of the test where every ethnicity scores exactly the same. Or, just award everyone a 1600 and call it a day?
How many more times will they dumb down the SAT's? Why bother, the leftist runs schools ignore them in the name if DIE anyway.
I think this is an excellent development. No one in my family has an issue with the current time limits, but we've talked about how arbitrary they seem and how there's no way you're getting an accurate ability measure for most test takers with so much time pressure in place.
In fact, I remember thinking that when I took the SAT in high school. I finished, but I thought the required speed was silly. You had to enforce a no reflection rule in your mind. That seemed pointless. Why would we purposely select for the ability to do that? There are so few other contexts where this is useful, and none that I can think of are tied to SAT performance.
I've been taking tests since first grade in 1948 and have no memory of an open ended time limit.
You either know the info being tested or you don't! I don't remember whether or not I was first to finish SAT's in mid-60's (I'd dropped out of hi school during Sr. year of '59) but, most were still seated when I finished (800 on verbal and 760 whatever the other was named). Was usually first to complete finals in college and always first when taking professional exams (never came close to failing one). That said, last month took DMV knowledge exam for DL renewal and barely made it by the "skin on my teeth". Had to work that phrase in as haven't seen it used in a long, very long, time.
I don't think it will make much difference. If you don't know, you don't know. More time won't help.
Why systematically disadvantage those who are inclined to be careful and systematic and favor those who take risks and shortcuts?
Ya' know, never thought from this perspective, and you're right. Always thought it was good to put people under a little pressure. But now am asking Are we testing for performance under pressure or are we testing for intellect?
MISTER BIALYSTOK I CANNOT FUNCTION UNDER THESE CONDITIONS!
You make me extremely nervous.
I do not need to take these kinds of tests any longer, but I well remember my approach. First, go through the test quickly, and answer those questions that are obvious, or very easy. Second, go through the open questions, and devote a set period of time to answering each. Then, answer the tough questions. Leave any time left for intensive review of EVERY question. I always did well.
In my engineering tests in college, some tests were untimed because we were solving complex problems over hours or days, while other tests were timed because we were being tested on knowledge of basic facts, and we either knew them or we didn't. So a hybrid approach of timed and untimed sounds reasonable.
They would not be changing the test if they were getting the results they prefer from the current test.
Everything else said to explain/excuse the change is pretextual.
In simpler words, smart people do better on timed tests.
They also do better on untimed tests.
And in real world decision making.
And in any kind of emergency decision making needs to be quick, as in now, not minutes from now while you ponder the options.
I recall one time at a facility where I was the person in charge of the high voltage distribution system and emergency generators. Arcing and sparking on pole on the utility side of the generator. Stopped quickly to tell the maintenance supervisor- my nominal supervisor- to inform everyone we were going on generator power in less then a minute. He said, "Whoa, we have to call and get permission..." I didn't pause as I told him _ "I didn't ask you for permission- I told you what I was going to do." and then went and did it. So of course the facility started a disciplinary action against me. Which went to the real head honchos. Who asked the facility- "Why was anyone thinking they could interfere in his doing his job properly?"
The facility level people didn't like me before that- they liked me less after. Operations and maintenance are often combined- but operations personnel have a totally different mindset. Doing things right by the equipment- like boilers and generators and high voltage switchgear- is more important then feelings. The equipment can kill you without any remorse if you treat it wrong.
Making decisions quickly, without consultation and permission, when others dither makes them think you think you think you're smarter then them. Which more then likely- is true. Both thinking you're smarter then them and that you are smarter then them- or at the very least more knowledgeable in that field.
Oh, sure. The psychologists studying cognitive ability in the last century had it ALL wrong. There were no imbeciles, idiots, or morons. There were just people who needed more time to take the test! It's all clear now. The people who actually APPEAR to be stupid are, in fact, the ones who should be in charge. Leave the highly intelligent people out! This augurs well for Joe Biden and others below the mid-point of the bell curve.
Yeah, OK. This will just give one more way for affluent parents to privilege their kids over the tatterdemalions living on the wrong side of the tracks.
I call bullshit. Time limits measure cognitive efficacy, a type of mental productivity. There is such a thing as being slow-witted, and it's not a good thing.
By similar reasoning, we shouldn't expect students to learn Physics 1 in one year, when a deeper, more subtly profound mastery can be achieved through zen meditations on the true meanings of F=MA. This is just more excuse making to try to give advantages to rich kids with an allergy to No. 2 pencils.
The SAT was used because it was effective. It was a good predictor of success in college. We have been hearing this for decades about how not everyone is a good test taker and I will agree that there are applicants that would do well in college despite not doing well on the SAT. The problem is when something is very good at some task, we generally do not discard it because it is not perfect, unless we find something better. They have provided no replacement for it, because the point is to not provide a replacement for it. They want to be able to discriminate and if the objectivity is stripped out that is so much easier.
Of course, at this point most colleges have been so dumbed down and the schools are so motivated to push everyone through whether they deserve it or not, that I'm not sure there is any useful predictor anymore. It may be a sign of intelligence to refuse to go to college.
They're not being entirely intellectually honest here. Psychology researchers have long known that reducing the amount of time in which a task must be done does an excellent job of pushing people to the limits of their performance. If the purpose of the test is to determine APTITUDE, then everyone (except maybe the top 1%) needs to get at least some questions wrong. So you can either make the test much, much longer, or the questions much harder, or you can speed it up. Psychologists claim that the results will be the same regardless, and they seem to have evidence to support that contention.
But we're using the SAT for things that it was never meant for*, anyway, so further screwing it up isn't going to make much of a difference.
*The SAT was intended to be a screening tool to find smart and talented kids (hence, APTITUDE) who had not be recognized in a little schoolhouse in the middle of Iowa, the slums of Baltimore, etc. It should never have been used as a MINIMUM for collge admission. There are plenty of very smart, talented people (i.e., with high aptitude) who will have a bad day and test poorly--I've taught many of them over the years--but there are very few people who score a 1500 by accident.
It's a real stretch to claim that time pressure during tests explains why, over the centuries, very few women have achieved world-class status as mathematicians.
As savants like Srinivasa Ramanujam show, you don't even need an advanced education to be a math genius. And how do you explain Newton, Leibnitz, the Bernoullis and other math titans who INVENTED whole classes of mathematics?
But the idea that males are innately better at advanced mathematics than females is now cancellation material.
The next "good idea" to achieve "equity" will be to limit test times for males, but give females an extra 50%. Just to make it fair.
Prof posted at 3PM and here it is 730 already. And I was afraid that I would not be fast enough with an apposite comment.
Now I need time to think about it. I guess I'll just say that "SAT been very good to me."
Just make it Pass/Fail and never Fail anybody. That's the idea, isn't it?
The problem isn't specific to SAT scores. Lieutenants' exams etc. have the same trouble. Any cognitive test. There's a reason that they all correlate, that they're measuring the same thing.
Game changer?
Probably for blacks who will now be acing the SATs. Right?
More BS. The elimination of standards. In real life, there are time limits.
There were many comments on this story in the first few hours after publishing, dominantly negative. The comment section was then entirely deleted, not just shut down. This is rare for the NYTs, I think. I’d love to know the inside story.
My own view is that processing speed is one aspect of intelligence. We should just go for transparency. Report how a test taker did and how long it too them to do it.
The SAT was used because it was effective. It was a good predictor of success in college.
It is a good predictor of first year success in college.
Lots of kids drop out after their first year because they cannot manage the work. (BTW, the workload puts a lot of time pressure on them).
SAT scores do not predict success after that point because a lot of the less bright kids are gone and other attributes (e.g., conscientiousness) become better predictors.
They also basically had to do this to have a valid test because now anyone with money and the willingness to game the system can get extra time.
Some Genius said...
There are so few other contexts where this is useful
ABSOLUTELY! There are so few other contexts where this is useful, except just about EVERY THING that ANYONE has done, from when they were born until the day they die.
He who hesitates is LOST
although smarter people are faster at solving easy problems, they’re actually slower to finish difficult ones. They’re well aware that haste makes waste, and they don’t want to sacrifice accuracy for speed.... Although it pays to be quick, it also pays to be determined, disciplined and dependable.
So, they're smarter.
Time pressure rewards students who think fast and shallow — and punishes those who think slow and deep....
Which, of course, explains why the smart kids get better scores.
This smells oxymoronic.
Althouse:
"A time-pressure test isn't really detecting how quickly you can answer questions. Time-pressure can interfere with your concentration and create static that makes you slower than you'd be if you took the test without a time limit. Some people worry about the time and some people don't. Some people hate to be rushed and others find a deadline motivating. Why systematically disadvantage those who are inclined to be careful and systematic and favor those who take risks and shortcuts?"
All of that is pretty much how you determine how quickly one can answer questions.
Those who are inclined to be careful also tend to get more answers right. Or does getting wrong answers not impact "finishing the test"?
I'd rather have a slow 95% than a fast 60%.
Umm, I thought colleges were doing away with the SAT & ACT placement tests. You know, ‘cause EQUITY for minorities.
Next up, freshman year is too short a time. Take 4 years as a freshman.
Finally, on a serious note- this won't accomplish what the test designers are trying to do- smart people will make better use of the extra time, dumb people will still not be able to answer the questions in 5 minutes or 20 minutes.
Don't subscribe so I can't go further but nothing I read mentioned that those that finish quickest (aside from the few who just slash and dash) are typically the smart ones.
Person dealing with RL who couldn't finish the timed test:
"Look around. Do you see anything you could fashion into a rudimentary lathe?"
Oso Negro said...
"This augurs well for Joe Biden and others below the mid-point of the bell curve."
Only took two weeks of seeing him as POS to flatten that curve.
Welcome to the wonderful world of Maytaaaaaaaaag Warshing Machines.
That no one mentioned coping with dyslexia is astonishing to me
“I don't really remember much about bar exam questions or law school exam questions (I graduated from LS in 1968 and took and passed my first bar exam that year; I took (and passed) my last bar exam in the '90's). You were frequently told regarding exams that you were supposed to "spot the issue", and this was particularly true when exams had to be machine-graded. It IS important to spot the issue, and if you have command of the subject matter it shouldn't take long.”
When I sat for the bar the first time, in 1990, there was two parts: multiple guess MBE and essay test, given on successive days. 2x3 1/2 hour sessions. Essay portion was your issue spotting. The MBE tended to test legal concepts more. I got lucky -my first bar admission was in CO, and they would automatically pass you, if you scored high enough on the MBE. So, I hit that level consistently about a week out. Stopped practicing, several days out, had a bunch of nuts ready for energy, so that I wouldn’t fall asleep with too many carbs, took the MBE, and came close to pegging it - despite leaving the testing more than a half an hour early for each session.
Time pressure is a motivator in testing (assuming the test taker is committed to doing their absolute best). First you need to understand it's important to move along quickly instead of agonizing over the "too hard" questions. You also have to trust yourself - when you're not 100 percent about the answer, your first pick is ususally the right one. Keep moving. Even the hard stuff will yield a bit if you eliminate an obvious wrong choice and watch out for "traps" that invite hasty mistakes. Even then, your first "guess" is probably the best guess. If you finish the test in less than half the time allowed, STOP. If you go over and over your answers, you're going to change a few right ones to wrong ones.
If you think abot a question long enough, you can come up with many ways to rationalize a wrong answer being the right answer. A lot of what's wrong with our culture now, is acceptance and promotion of novel ideas that fly in the face of reality.
Smart people are smart, regardless of test outcomes.
Smart people went (past tense) to college/university because smart people were capable of pushing the bounds of human knowledge, understanding and capability. They need to be encouraged to do so, but the need to be identified.
It seems that’s not important anymore.
Remember being praised as a "quick study"?
I do.
A "slow study"?
Not so much.
I wonder if when the dumber kids finish the harder questions more quickly, they get the right answer as often, seems like an important detail.
In 1970, the SAT was largely an IQ test, you can't drill a young person into perfection on an IQ test, but by 2000, the IQ testing aspects were gone, and affluent and motivated parents could, in fact, drill their children to achieving near perfect, to even perfect scores.
Go back to an IQ test, if you want to provide a route for less privileged kids out of poverty, anyway. Or drop the test, if you have some racist idea that says that the *wrong* poor kids will succeed under IQ testing. A lot of time factor problems could be overcome, and kids would do better in math in general, if complete rote mastery of the multiplication tables were required in elementary school. Not having achieved that burdens a student with a useless but real cognitive load in solving math problems the rest of his life.
The NCLEX (nursing exam) uses Computerized Adaptive Testing. This is probably the future. Less about time, more about assessing what you know.
Maynard said...
The SAT was used because it was effective. It was a good predictor of success in college.
It is a good predictor of first year success in college.
Lots of kids drop out after their first year because they cannot manage the work.
SAT scores do not predict success after that point
huh? So, you're saying that "after the first year", the kids with bad scores (that dropped out), REENTER college, and perform well? REALLY? because THAT is what you JUST said.
What I explain to people taking tests. The process is to determine the stuff you are supposed to have learned.
I go through a test answering all the questions I know. This allows me to get an idea of what the test writer is focused on. Lots of time, questions are answered by other information provided in the questions.
Then go back through with this gained insight and attack the questions I did not know. Then its a process of eliminating wrong options and choosing the most reasonable guess. Lots of 50/50 guesses get pushed one way, by just understanding what is important to the test writer.
Time is really not much of a factor. You have studied the material, you knew the test was comming, you are supposed to know the material. Taking all to take the test will not magical infuse you with knowledge you did not already have.
My high school shop teacher hit the sweet spot.
His semester test ran over 300 questions. The questions were weighted by importance of grasping the material covered.
The grade was determined by the number of answers correctly answered. So you could get a perfect score by finding and answering the 15 highest ranked, and 20 of the next highest ranked and 40 of the lowest ranked.
This a great way to determine if you learned the material. It is not used much because writing the test is a huge lift for the teacher.
Oh please, people with higher iqs do the tests faster than those with lower.
Time pressure also sorts those who know the subject matter from those who have to sit and try to pull answers out of various bodily orifices.
Tests in College are all timed. When, say, a doctor is operating on you, and things go awry, you want someone to think on their feet very quickly.
It's like the SAT is trying to make itself meaningless, which is okay I guess. That way, Colleges can base admittance on something else.
"although smarter people are faster at solving easy problems, they’re actually slower to finish difficult ones." This sounds like BS to me.
Difficult problems are usually difficult because they are complex, which means you need to solve sequential easy problems to get the answer to the whole question. To the extent that t a problem is tractable to a test taker, the smart person will solve simple and complex problems faster than a less-smart person.
In any case, even in the unlikely case that the claim is true--it should all even-out. The smart persons will zip through the early easy problems and get bogged down in the later, hard questions. The not-smart people will lag early on and then gain in the latter questions, where they supposedly have an advantage.
So, the rationale for change is that cramming or "shared responsibility" will compensate for lack of preparation. If you lack intelligence, study. If you lack skill, practice. If there are deficits of each, then the fault lies with the individual, parents, teachers, and system. Jury rigging is a short-lived solution with forward-looking consequences.
Why systematically disadvantage those who are inclined to be careful and systematic and favor those who take risks and shortcuts?
Well, first - that sentence is constructed ("systematically") as if to suggest that the test's purpose is to give risk- and shortcut-takers an advantage. If you meant that it incidentally has that effect, I don't think that was clear.
Second. Why make someone trying out for a play act from the stage without a mike? Why make an aspiring chef make a Bearnaise or chop an onion? Because the skill and knowledge being tested is important to the endeavor, even if you are not inclined to perform unamplified or to prepare anything but empanadas*. Because an inclination can be pushed against, even overcome, and that determination is also a valuable skill.
I have never had, nor heard of anyone's having, an entry-level job in which time isn't a factor. In mid-career, sometimes you have more deadline control, but you are definitely not without deadlines. Once you're quite senior in your job, maybe you'll finally be able to take all the time you want to solve a problem - though I doubt that too; my husband is a C-suite inhabitant and is still beholden to the board. Entrepreneurs? They have even harsher deadlines than corporate people. Artists? Depends on whether they want to eat.
Good Lord, I had to work to deadlines as a mother.
Time management is not the only skill the SAT tests, but it's such an important one that to do away with it - especially with what test prep coaching enables you to do even when you have no idea of the answer - seems to eliminate or militate against pretty much all the value of the test.
*(My neighbor, a Cordon Bleu-trained chef, sometimes complains that all her catering customers ever seem to want her to make are her grandfather's empanadas. I've had many other things she's made - it's not because of any lack of skill on her part! People just love empanadas around here.)
My problem here is that fast thinking is not really a requirement for college, or law school. In our family of five boys, the 3 odd numbered boys had it, and the 2 even numbered ones did not. My next brother scored 100 pts lower on his math SATs than the odd numbered brothers. We went to the same college, as undergraduates, and were both math majors. He was considered one of two math prodigies in the department (the other ended up as a top national science advisor to Biden, after having spent much of her career teaching at MIT). Frustrating, because my math SATs suggested that I would be better there. I wasn’t. He is slow and methodical, but very thorough. He has more graduate degrees than I, and did well as an engineer and then patent attorney. The joke is that Einstein would probably have failed the math SAT for similar reasons - his thought process was slow and deep.
My father had two mentors when he started his career as an attorney. One jumped to the conclusion. The other slowly ground out the solution. The former was mostly correct, the latter almost always. He made the better teacher, exposing his thought process, step by step - esp for my father, from whom my next brother inherited his slow and thorough process of thought. The three with fast thinking got it from our mother.
There are places where quick thinking was helpful in law. Best time I had in 30 years in law was sitting solo in a week long jury trial. Few things have gotten my adrenaline pumping like that. When you are on your feet, you are on, in my case, my brain was on overdrive. Someone like David Begley is probably well beyond that, given his practice, but I loved it. The other was in patent committee meetings at a major semiconductor company. Two patent attorneys and maybe a dozen top PhDs would compete to understand the inventions being presented to us. Going into the meetings “hot” (having read the patent disclosures first) was considered cheating. Took me six months, before I was accepted by the group. Other patent attorneys tried, but no others succeeded. Half a day, once a moth, or intense mental challenge. Thing is, that slow and thorough is just fine in patent law.
Plenty of attorneys do just fine, thinking more slowly and methodically. Our father and my brother did. His best friend and his son, our estate attorneys did too. There is a place for slow and methodical in the practice of law. I always wondered if that was why Justice Thomas didn’t participate (and compete) very much in oral arguments. Yet, in my view, his opinions were some of the clearest, and most forceful, as a result.
That said, my partner’s thought process is lightning fast, faster than mine - indeed probably faster than anyone she has ever met, except for her late father. Which is part of why we may still be together - she never bores me, and I can sometimes outthink her. Our closest non family friends are similar.
How well does the SAT score correlate to success in college?
That should be the primary issue.
If it correlates well, keep it as is.
If we MUST drop time constraints where dose THAT end? Bring this back by Tuesday, no later than Thursday...maybe Friday.) do a sample and check that correlation.
If it tracks better, fine by me. If it lets in more of the preferred group, but leads to worse results in college... kill it with fire.
The Crack Emcee said...
That no one mentioned coping with dyslexia is astonishing to me
Okay. I'll mention it. A problem that seems to be confined to those who have learned reading by hieroglyphics- look/sa, sight reading, or whatever the term of the day is.
Doesn't seem to affect anyone who has learned to read by phonics.
Bart Hall (Kansas, USA) said...
Time pressure is particularly problematic for people at the high-functioning end of the autism spectrum,
Not certain how you define the high functioning end.
When I was young there was no spectrum. Either you were autistic, or weren't. If you were what is now called "high functioning" you were told to get your act together. Hi! Welcome to my world. Every online test I've taken indicates if the diagnosis "Asperger's Syndrome" had existed when I was young I'd have been diagnosed with it. The military for a very short while wouldn't take anyone in who was diagnosed as being "on the spectrum". And very shortly thereafter rescinded it as they were suffering a severe shortage of people qualified for highly technical skills. The Navy nuclear field, among others, is filled with people on the high functioning end. Electronics technicians in all the services. The higher the ASVAB score required- the ASVAB being a timed test, the more likely the specialty has high functioning autists dominating it.
"Why systematically disadvantage those who are inclined to be careful and systematic and favor those who take risks and shortcuts?"
Works for me. Close the fucking public schools already.
Ah, the persistent test that is based on "the test of lower order thinking for the lower orders" to quote the creator of the multiple choice test after realizing what an abomination it is. But multiple choice and time-testing eases the workload on the teacher/proctor. And that is, after all, what education as it is in the US is all about.
Many have written and commented on the fact that education, more accurate to call it schooling, incentivizes not real learning but quick response and getting a good grade on the test.
See Paul Graham's Dec 2019 essay 'The Lesson to Unlearn'
And here is a 10 yr old video of Joseph Epstein relating a story of his now deceased department head cousin about what was realized by the cousin's poor performance in seminars. They mistakenly use "talk show" but it is game show education that we reward, not real learning.
https://youtu.be/JF2eJSHKKd0?t=1045
MadisonMan said...
Tests in College are all timed. When, say, a doctor is operating on you, and things go awry, you want someone to think on their feet very quickly.
But what of the claim that college is not suppose to teach you job skills but is about intrinsic education?
What of this refrain from more than a century ago?
"A man does not come to college to learn to earn a living; he comes to college to learn live!"
How does quick response teach one how to live?
Why systematically disadvantage those who are inclined to be careful and systematic and favor those who take risks and shortcuts?
My recollection is that the SAT has already been offered without time limits (or greatly relaxed time limits) as an accommodation to disabled students. Hence the use of fake disability diagnoses to secure a time advantage:
In what's become known as the Operation Varsity Blues scandal, some parents were instructed to apply for extra time for their students, "including by having the children purport to have learning disabilities" to get the documentation needed for the SAT and ACT, according to court documents.
The real systematic disadvantage is actually to normal students who took the tests as designed. The test administrators have neither the capacity nor the will to ferret out this kind of cheating -- inevitably, such an attempt will catch a lot of innocents who really do have disabilities. So relaxing time pressure just evens the playing field again.
The whole thing is kind of a joke. The SAT has no meaningful time constraints. The ACT, on the other hand, has a difficulty factor strongly linked to time constraints.
As usual, you can count on the NYT to completely misunderstand the purpose of the thing they are "reporting" on.
If someone tells you that the SAT measures intelligence, they are too stupid for you to care about their opinion.
The SAT has a role in predicting aptitude for college. Nothing more, nothing less. It doesn't measure intelligence or knowledge. It doesn't really predict anything about success later in life, except how those scores correlate to college success.
What has happened, of course, is that most of modern college is now useless and too easy to bother testing aptitude for.
Post a Comment