September 26, 2011

U.S. News decides — after a 2 year study — that it's not going to change the way it does its rankings.

"That's what the head of the rankings said here Friday at a session of the annual meeting of the National Association for College Admissions Counseling after a special committee from the association presented its final report examining the rankings."

ADDED: The committee had argued against using standardized test scores as the primary indication of student quality. Soft variables matter too, and U.S. News is rewarding the schools that mostly ignore those soft variables. The schools that resist U.S. News — as I well know! — take a hit in the rankings, while other schools scramble over them in the rankings by simply picking the highest test scores they can get. And with every passing year and climb in the rankings, those schools can get higher and higher scores, creating an upward spiral, as the U.S. News-resistant schools sink lower.
The committee’s report takes issue with the way the publication styles the rankings as the “Best Colleges” because the weights assigned to different metrics are essentially arbitrary.... One of their main concerns is that, given the diversity of higher education institutions, having a list of the “best” is impossible.

Robert Morse, who oversees the rankings for U.S. News, said .. [a]s long as colleges and universities continue to weight test scores and class ranking as a crucial component of admissions criteria... it is hypocritical for institutions to ask U.S. News not to do the same. 
That's lumping all schools together. Some emphasize scores more than others. Plus, it's a vicious circle. Who will stop first? If some schools backed off on test scores — as indeed, some schools already do — other schools would see an enhanced opportunity, snatch up the relatively high scorers that higher-ranked schools passed over, and the next ranking would show the results, punishing the school that tried to back off and redoubling the incentive to admit aggressively on test scores.

But what is the alternative? Other metrics of quality are worse, and there's no way to tell people to stop relying on the rankings or prevent U.S. News from continuing to process its imperfect information and publish it.

37 comments:

Fred4Pres said...

University of Wisconsin dissed again.

Badgers get no respect, no respect at all!

Scott M said...

It's not a big deal. The way college costs are going, it's going to be like looking at a Skymall catalog for most of us.

Fred4Pres said...

Schools in Illinois laugh and sneer "suckas!" at Wisconsin.

It is really quite shameful.

Maguro said...

U.S. News' rationale seems perfectly valid to me. The schools themselves place a heavy emphasis on standardized test scores during the admissions process (within racial groups, at least), so it's natural and appropriate for the rankings to do the same.

Shouting Thomas said...

When does the bubble burst?

Gov. Walker has ended one kickback scheme... the deal that the Democratic Party had with WI public employees to scam the taxpayers.

Who's going to end the higher education kickback scheme?

You know, ever higher student loan burdens that allow administrators to siphon off public funds with no cost limits. The scam that allows administrators to play around with propaganda crap like Diversity and Climate. And, allows the poobah of D&C to call out his own goon squad to protect his racket.

The higher education system is ripping off the taxpayers big time. Who's going to challenge the system? Who's going to bring down this Ponzi scheme?

EDH said...

At what point does this quest to perfect the ranking of schools, students, clerkships, judges and firms reveal an unhealthy obsession within the profession?

Are those on the inside so blind to what this looks like to those on the outside?

Scott M said...

Who's going to bring down this Ponzi scheme?

Don't say the "P" word or you'll be vilified in the press.

WV - "goams" - the exposed legs on a pinup of a WWII-era female gnome

Betsy said...

I'm rather astonished at this, actually -- that the NACAC would presume to prescribe to U.S. News changes in its proprietary product (and basically the only thing it does these days). If it's flawed, ignore it!

As far as college costs go, the only way they'll come down, or slow their growth, at least, is when they can't sell their product -- and that shows no sign of happening. The government keeps increasing student aid, families keep going into debt (and I don't believe the official figures of average debt, since that doesn't seem to count parental supplemental loans, let alone "unofficial" debt like a second mortgage or a personal loan when you've maxed out on the official programs), and universities keep filling their freshman classes. Only if a university can't get enough revenues will they consider controlling their expenses.

sorepaw said...
This comment has been removed by the author.
sorepaw said...
This comment has been removed by the author.
E.M. Davis said...

Always with the rankings with you ...

Shouting Thomas said...

There is another route to breaking the higher education kickback scam... virtual education.

Sooner or later, some clever entrepreneur will find a way to prove that his online certification program is just as (if not more effective) than the classroom version. Some clever employer will discover that there's no point in paying for a status certification from an Ivy League university.

You can see clearly how Althouse would respond. She's a genius. Does it really take a genius to provide legal education? Does a lawyer really need to be a genius?

The answer to both questions is "No."

Sooner or later, this will be noticed.

Fred4Pres said...

Ann, rankings do not matter. Are your students succeeding after they graduate? That is what matters.

Focus on the real things and not stupid rankings.

Fred4Pres said...

Shouting Thomas, there has to be a more affordable way to get an education.

Henry said...

From now on I want college rankings to be done by the IPCC based on surface station data and computer modeling. After all, as an applying student, you don't care how a college ranks now. You care how it ranks five or six years from now, when you're looking for a job.

Likewise U.S. News can take over the climate change reporting gig based on the SAT scores of all the scientists getting grants, cross-indexed by a survey of their self-importance.

Triangle Man said...

Don't say the "P" word or you'll be vilified in the press.

It's a racist smear against Italian-Americans. Like the Sopranos.

Ann Althouse said...

"Ann, rankings do not matter. Are your students succeeding after they graduate? That is what matters. Focus on the real things and not stupid rankings."

Don't you realize it's all connected? Employers look at credentials and the value of the degree is affected by the rankings. And then surveys are taken of employers to produce numbers that represent the schools' reputation and that number is fed back into the rankings.

Paddy O said...

This is what democracy looks like.

Fred4Pres said...

Wow, you sound like Justice Thomas who got bitter over his Yale degree not having much value because employers assumed he got it because he was black. He put a 50 cent sticker on it as a protest.

I agree reputation of the school matters to students, but cream also rises to the top.

Pogo said...

Students are still going on to college because there are no other options. No jobs or careers to start, so might as well park at a school and try to get some marketable skills, hoping the market rights by the time they graduate.

Yet recent graduates stack up like cordwood, unemployed or working as baristas and department store clerks. Their student loan payments are crushing them.

These form the first years of a lost generation. The longer Obama's Depression lasts, the less likely they will ever have careers or start new businesses.

So ranking schools that train you for minimum wage jobs for $80-160K seems a little odd to me.

Fred4Pres said...

Ann, you point out an inherent bias in the rankings. Those schools clustered in the top tier have an interest in not getting bumped out (and will cheat and lie to stay there). And their alumni feel the same way. So they will vote to maintain their school, regardless of objective data.

Which is why I take all such rankings with a very big grain of salt.

Smilin' Jack said...

What does it matter which students they admit? They all come out as lawyers. Just like, whatever you eat, it all comes out the same.

NYTNewYorker said...

Oooh, female gnomes, sooo maneuverable.

Pogo said...

Popularity contests are unchanged by adding statistics.

Pogo said...

How do I love thee, let me rank the ways.

Big Mike said...

But what is the alternative? Other metrics of quality are worse ...

Thank you, Professor! That's the whole point in a nutshell, isn't it?

clint said...

The free market provides an excellent solution to this issue.

Someone should publish another list of colleges -- perhaps ranking them with many *different* metrics, so that each reader can decide what he or she is looking for in a college?

You could publish a list of colleges based on the 10-year income of its graduates, or on success in different fields. You could even divide the income "surplus" by tuition, to rate colleges the way you'd rate investments.

You could rank schools based on gender balance, or ethnic profile, or student-faculty ratios, or on the percentage of students participating on NCAA teams. You could note the percentage of graduates earning Rhodes and Marshall scholarships, the fraction taking a year abroad, the size of the ROTC presence on campus, or the quality of the meals and dormitories.

If there's a demand for that kind of information...

Steve Koch said...

In general, an objective approach is preferable to a subjective approach because it is less susceptible to political corruption by our politicized lefty academics. The decision by US News isn't a step forward but it avoids a step backward.

MikeR said...

Lot of nonsense about how the bubble will never break. Of course it will. People were saying the same stuff about the housing bubble five years ago. The more you keep pumping it up, the steeper the crash will be.

Eventually, online certifications are going to decimate these places.

Carol_Herman said...

Administrators informing a magazine of their ranking! It's a wonder the same "columns" don't appear in Vogue.

But Vogue is the unrealistic clothing set ... for anorexic people.

You think you're gonna find the world's next best physicist in the rankings of US Blew Da' News and World Report? REALLY?

I'd prefer BARGAINS!

If I was gonna compete with this magazine, I'd come up with the general profile of everyone in the middle of the pool. None of the outliers that get into Harvahd's Crimson.

Then you'd pick Texas. Which is big. Florida. Which is big. And, California. Because everybody loves Mediterranean weather. Which is what Southern California officers.

Then, as a component, (outside of Stanford and Berkeley), I'd list all the schools up the northern coast. Through Oregon, and Washington State.

And, for added interest, I'd throw in a few Canadian ones. Some people don't mind cold weather. And, think of the SKIING! Which would also give you Maine and Vermont. (All their slopes are made of ice.)

Oh, yeah. Back from around 1955 ... if your male kid didn't want to go to college. And, the cost of college was less than $6,000 ... you could give him a year or two in Europe. Living out of a small duffle bag. With two pairs of underpants. A toothbrush. A roll of toilet paper. And, an extra pair of corduroy pants. Europe could be had at $5 a day.

When back, and needing info for a resume ... nobody checked out the European schools. So you could say you attended one of them. Or not. Depending on your preferences.

Today? The same amount of money ... and you could set up your kid in business of some sort. They work, then, 7 days a week. And, whether they fail or succeed ... they learn about the realities of the marketplace.

If you need to buy a magazine to convince yourself there's a school our there "just for you," ... you'd be better off buying one of the hats the models are wearing in Vogue Magazine. Waste your money. See if I care.

Lucius said...

I've no problem with conceeding that University of Wisconsin-Madison is an elite school (as these things go-- even if it has claimed a No. 1 Party school trophy or two, has it not?).

But what does U Wis-M want to compete with, Harvard?

I've no problem with the idea that a bright person can go there & leave with as fine, or finer, a mental training than they might have had at Harvard.

But Harvard is still "Harvard." And driving up the average SATS of the people you admit has a way of looking 'elite' in the way that people want to be 'elite.'

In a contest to be the Harvard of state schools, how much weight does USNews finally have? It's probably more a question of: most people have no idea on this fundamentally subjective question, so turning to a US News ranking is 'fun' for them.

If Timmy was born on the northern plains, it still makes better sense to send him to Madison than to Penn.

Oligonicella said...

People lie to polls. This is especially true if they have a bias or if the poll annoys them. And polls have become really friggin' annoying.

I lie on them constantly just to skew their results and I know others that do the same thing.

Chip S. said...

True: The weights US News attaches to its assorted indicators are arbitrary.

Non sequitur: Therefore the weight attached to test scores is too high.

It's very difficult to construct a valid syllogism with only one premise.

William said...

Only Euclid looked on beauty bare. The desire to a have a precisely accurate measure of rankings is of a piece with the pursuit of a just and fair admissions process. I suppose it's a good thing for lawyers to believe that there can be equity in these things and that if you just discuss the process long enough then that that equity can be reached.....Sadly there are too many cynics in the world who believe that such fetishism distracts from the ultimate goal of procreation. For example, I would much prefer to go to law school with Dan Quayle than with someone who scored 100 points higher on the LSAT. It can only help your job prospects if you knew the son of one of the state's wealthiest men. Such prospects could be further enhanced, if you volunteered to caddy for him on weekends. You have to know how to play the game......Does William & Mary have a law school? There should be a law school where 90% of the admissions are based on family wealth and connections. The other 10% can be for bright kids who want to suck up to rich kids. I'm sure such a school would be a fabulous success. It probably already exists but people like me are not supposed to know of its existence.

Peter said...

“Other metrics of quality are worse”

So, how does one measure the quality of a metric?

A basic measure would be, “how well does it predict whether the student will graduate?” I’d guess that ACT/SAT scores combined with high school class rank does a pretty good job here.

The main deficiency is, there are many with high ACT/SAT scores and top grades who will not distinguish themselves after graduation. BUT, does anyone actually have metric(s) that can do this better than the objective measures?

Or is it back to the same old holistic “I know quality when I see it!” nonsense (plus covert quota-mongering, of course)?

Dustin said...

The US News rankings are stupid.

What matters is how likely you are to get a great job after law school. That is the one thing that matters.

Academic respect is meaningless. Money spent per student is meaningless.

Libraries are cool. I love libraries. But legal research is done on laptops if you know what the hell you are doing. And mostly you learn that from the Lexis or Westlaw reps.

Law school is really research and writing school. It is the ideal school for correspondence and commuters, but one must have high standards to build a brand for a school's alumni.

So the schools with last year's best will get next year's best. And the firms will hire the brand that maintains this.

US News is a cheat. By making money so relevant, schools can buy next year's best students, many of which have a harder time getting a great job than they expected. After a few years, the brand has actually improved a bit.

How many schools boast 95%+ hiring rates with $90k+ salaries?

Caveat emptor, law students. You are buying a ferrari and usually driving away in an Acura.

gregq said...

No, it's a virtuous circle, because judging people based on the color of their skin, rather than the content of their character, is evil, but that's what schools are actually doing when they're feeding us the "considering the whole student" line of BS.

You're a school. What should matter, the only thing that should matter, is how good of a student the person was, and how good of a student you think the person will be. (If you're a state funded school, then it's reasonable to care about whether the person is from your state, and therefore one of the people funding the school.)

University Admissions officers can not be trusted. They are, as a group, a bunch of racist pigs (calling your racism "affirmative action" doesn't make it the slightest bit less racist). Grades and test scores, yes. "Diversity" no.