[T]he Annapolis Group, a loose association of liberal arts colleges... released a statement that said a majority of the 80 presidents attending had “expressed their intent not to participate in the annual U.S. News survey.”...Link via my colleague Bill Whitford, who emailed our faculty discussion list. I quote his comment with permission:
Brian Kelly, the editor of U.S. News, ... said more than 50 percent of the presidents, provosts and admission deans who were sent the annual survey of colleges’ reputations continued to fill it out. “We think the vast majority of presidents and academics are still supporting the survey,” he said....
Many presidents who favor no longer participating in the U.S. News rankings said they expected the magazine to be able to continue to produce its annual rankings because much of the data on things like admission and graduation rates are publicly available. Colleges report most of that data to the federal Department of Education.
When will the law schools show similar courage? Perhaps it is not possible for law schools to frustrate U.S. News with a non-cooperation strategy. But there is widespread agreement that the rankings have not been good for legal education, yet we (i.e., legal educators as a group) continue not to do anything about it. Obviously this is not a problem for any one school to tackle by itself; it requires collective action.But it doesn't look as though we could end the U.S. News rankings, only undermine the parts of it that rely on our participation. My school has always done especially well on the "academic reputation" factor, so I tend to worry about this. Wouldn't the hard numbers dominate even more, thus benefiting the schools that, for example, premise admissions on LSAT scores instead of the individual's entire profile? On the other hand, I tend to think academic reputation is a very unreliable thing. Most people who fill out the U.S. News survey -- I filled it out this year -- know almost nothing or nothing about most of the schools. And their ideas about the reputations of various schools are unavoidably infected with knowing where the schools are listed in the damned U.S. News rankings.
ADDED: Bill emails (and gives permission to publish):
My concern is that there is no collective effort (by legal academia) to delegitimize the U.S. New rankings. Instead law schools all play the game, trying to manipulate their individual ranking, with harmful consequences to legal education generally.I think schools have been complaining all along, but also trying to take advantage. Even if you could get everyone to act collectively and say they wouldn't participate by sending info anymore, they would still do things to compete in the statistics that U.S. News procures from other sources. The game would go on, and we'd get burned if we didn't pay attention to it (not that it should trump all other concerns).
The list of deleterious consequences is long, but I'll start with the shift from need to "merit" (meaning high LSAT scores) in the distribution of financial aid, which has happened at virtually all law schools. Another is the publication, at considerable expense, of glossy puff literature designed to raise the reputation of the school and its faculty with other legal academics and judges -- something we academics get all the time in our faculty mailboxes.Yeah, the schools are contributing to global warming with all that printing and mailing. Let's rank the schools in the order of harm to the environment done by the "law porn" they keep sending out.
There is a question of how to act effectively. Perhaps refusing to participate in the academic reputation surveys is not the right tactic. That can be debated. But doing nothing, which is what is happening, is not the right response either. We need a discussion first on the need to do something -- and here I think there is a substantial majority of legal academics coming to the view that the rankings have had negative effects on legal education -- and second on how to act collectively in an effective way.I feel like we've been having this discussion for 20 years. The only thing new is this tactic of not participating. If it's a bad tactic, best avoided, then nothing significant is new. Life goes on, with imperfect data and endless complaining about it. I don't think much of the collective action idea, because I don't trust the law schools not to do what they can to compete for status (or criticize other schools for competing).
I note that the U.S. News rankings have given applicants some useful information to inform their school choice. That is a good effect. But the overall effect of the rankings has been to reduce of the quality of legal education generally (as measured by the standards I prefer). I would like to find some way to continue the information function with the accompanying deleterious consequences.This is the usual idea of creating alternate rankings, but everyone who'd be involved in this new system self-interested and will argue for whatever helps their school. At least U.S. News is a neutral arbiter.
Posts
16 comments:
Could law schools simply refuse to give out any data whatsoever to any public source--e.g., median LSAT, student-faculty ratio, etc.--or are certain disclosures mandated by AALS or another sanctioning body (U.S. DoE, etc.)?
I think the ABA collects data, which is how U.S. News caught some schools lying to them a while back.
The problem with U.S. News is that it holds itself out as some authoritative list, which is bullshit, because there is simply no way that Harvard is better than Yale this year, but that Stanford is better than both next year, or, frankly, that there is any quality difference among Harvard, Stanford, and Yale.
However, information wants to be free and, in a free society, there's not a lot that anyone can do about rankings. Americans love their rankings. And I'm an American...
Um, your school was tied for 22nd with three other schools, in the most recent "faculty quality rankings" on Leiter's rankings site.
Ah, Wisconsin: another top 40 school that thinks it is the top 20!
U.S. News will continue to publish rankings if law schools refuse to disclose information. The rankings will just be somewhat less accurate.
Pretty much any ordinal ranking of any category of things will be oversimplified and some people will place undue emphasis on these imperfect rankings when making important decisions. But, like it or not, there's a huge demand for oversimplified rankings. And I think the argument against rankings is actually not very strong in the context of law school comparisons. The law school experience is incredibly standardized and thus pretty amenable to accurate comparisons and ranking, as compared to, say the "liberal arts college experience" or most any other discipline.
Sure, ABC law school might have a famous professor whose worth is not reflected in the rankings, but that professor may not still be on the faculty by the time you get there or they may be visiting another school the semester you want to take that class. XYZ law school might have a great clinic or specialization that doesn't affect its rank, but if you really want a law school with that specialization, the rankings won't hide that information from you. In fact, they separately rank schools with the best programs in particular fields. The average law school applicant probably doesn't have strong feelings about a particular field they want to study and, if she does, that preference may change once she gets exposure to more areas of the law.
Other obvious considerations don't seem to be too distorted by the rankings either, so long as you keep them in mind. Like, if you know you definitely want to practice in Arizona, an Arizona school may be a better choice than the rankings suggest.
The real concern with overreliance on rankings is that deans will distort their admissions criteria or other academic policies in order to game their place in the rankings. I suspect some law school administrators engage in this or feel pressure to engage in it, feel guilty about it, but instead of simply refusing to let that influence them, they project their guilt on U.S. News.
I feel that this ranking process is one of the factors that is causing difficulty with the quality of student produced. As a professor at a medical school, I know that there is a major emphysis on having the students score well on various national test. This would seem fine but the cirruculum is thus geared to make sure that the student preforms well on the exam and not necessarily know a thing about how to care for a patient.
As we see the students at several levels of their training, it becomes apparent that those that are able to take a test are not necessarily the best for actually applying that knowledge.
This is taken to another level when you assess faculty on students perceptions. They fill out a evaluation form and this only reflects their interaction with the personality of the teachers. If a teacher covers too much material in too short of a time, this is rate poor. Yet there is only so much time to cover topics. So someone comes out looking bad. If you do something really bad like demanding that the students read, then your evaluation pluments.
As with all old farts, I have to interject that this is not what we did in my day. When I went to medical school, we were told that 30% would fail in the first year and be shipped to Vietnam as medics. This really happened, and it seems to have worked as far as making us know our subjects. Now students are not failed at all beyond a certain level. If they are, this looks bad on the rating.
I assume that Prof. Whitford doesn't teach antitrust.
When I went to medical school, we were told that 30% would fail in the first year and be shipped to Vietnam as medics. This really happened, and it seems to have worked as far as making us know our subjects.
ah, the powers of enlightened self-interest :)
The ABA in its inappropriate (see VC postings) accreditation function, acting as a agent of the Dept of Ed, collects nearly all of the metrics. FOIA will get it, USNWR will publish the ratings with or without cooperation. Schools that don't cooperate be perceived as trying to hide something.
The boycott will fail
And now at least I find my school certified in Tier I-AA rather than "Um, yes. Never knew anyone from there, but I hear it’s a good school."
No, unless schools are Tier I-A, they cannot help themselves; they cannot suppress their desire to crack that real Top Tier.
Re the four tiers in US News, I remember a snobbish mot by Paul Fussell (in his book, "Class") to the effect that once upon a time there were 25 colleges in America & now there are still only 25 colleges in America.
The world does engage in ranking law schools, which is why SM is correct in saying that it’s bulls**t”, but I'm taking take a minute or two from looking at the projected respective final standings in the football BCS & the NFL to continue playing this law school ranking game.
Before US News, rankings were governed by some elitists' gut feelings. (They'd say "educated guesses".) Now they are governed by a wider universe of elitists' gut feelings, such feelings now presented with a “spurious air of scientific precision”. And now at least I find my school certified in Tier I-AA rather than "Um, yes. Never really knew anyone from there, but I hear it’s a good school."
Perhaps it would help to channel Orwell: Regardless of whether we use US News, anecdotal ramblings, “educated” guesses, or some other “expert ranking” re "who's who in law schools", or, more to the point here, "who can we look down on, my dears?", we should consider the possibility that in the legal world, some of the US News' Top Tier are more Top Tier than others!
Specific points: Perhaps I’m buying into the myth, but the five Ivies (+ NYU, Mich, Virginia, Chicago, &, more recent, Stanford & possibly one or two others) are perceived to be (to use athletic terms for want of any better) the I-A of Tier One, while the remaining schools in the Tier One-top 50, including, alas, my Alma Mater, are perceived to be in I-AA. My “perception” (feeeeelings, oh, oh, oh, feeeeelings) is based on clerkships & Wall Street firm hiring & the expressed perception of friends & acquaintances, in or out of the profession. Of course, such perception is not scientific & can & should easily be dismissed by any serious persons as anecdotal, but then the USNews’ rating system, while presented with, as I noted, its spurious air of scientific precision, is, IMHO, not much better.
Interestingly, if I recall correctly from previous years, & I admit memory is dangerous,
(1) there’s not that much difference in US News’ rating points between schools ranked, say 25, & 35, or ones ranked 35 & 45 & a school ranked 30 in 2005 might have moved up to 25 in 2006 & back to 30 in 2007, but there sure is a big point difference between, say #5 & #25,
(2) the schools in both USNews & my anecdotal Top 15 or so (sweet 16?) seem always to remain in such Top 15, though their rank within that group may change (&, of course, people I know outside the profession just throw out “top” names in no particular order rather than 1,2,3,4, etc., which is SM’s point also), and
(3) most of the “I-A” schools score incrementally higher in the peer assessment & lawyers/judges assessment columns than the I-AA ones. (OK, we don’t know how representative these “assessors” are; are most from “I-A” schools? Is there incest in the system? Enquiring Minds Want To Know®.)
This is a long note from someone who doesn’t “know” anything, but I’m aided by the fact that no one else seems to “know” any more.
For most colleges, performance data would be far more useful.
I would want to know not test scores but placement data (especially where and by whom) and professional exam pass rates.
If you think US News is a neutral arbiter, then I don't know what to tell you.
Some of the information reported by schools is patently false. Anecdotally, I know enough people who graduated from top-ten law schools without having jobs lined up to render their 99/100% placement claims false.
Wade,
I agree; placement data from the schools would be worthless. No, I want to know if I send my kid to law school here, will she get a job ...in law? Where? With whom?
Right now, those data are closely held, passed along by word of mouth only.
When I applied to law school, I paid careful attention to the US News rankings. Among other things, they were the only place that I could get much real information. Simple things like "Am I wasting my time to apply to Harvard with my GPA/LSAT?" that are very difficult to figure out by looking at websites and law school porn.
Likewise, they all certainly shade their career information - but at least USNews tells me what they say. I can discount it and try to guess how much they shade the numbers, but without USNews I am pretty much left to make up my own numbers. I suspect that my fabrications would turn out to be even less true then the numbers in USNews.
The thing about all those glossy brochures is that they all say the same thing. "Law school here is great!" without actually telling me why it's great there as opposed to somewhere else.
If law schools want to reduce the importance of the rankings, make them obsolete. Make a web site that actually tells prospective students something useful about the school - and include the numbers! Send me a brochure that tells me something substantive about how Law School X is different from the others. Because believe it or not, I learned more about the differences between law schools from USNews then I did from the websites, applications packages and other recruitment materials that the law schools themselves published.
Those schools that are objecting to the US News rankings, do they similarly plan to stop awarding grades to their students?
The "use by date" of this post has by now expired, but I couldn't help adding "something I saw in the window" of my computer, from Variety:
"The American Film Institute ...returned with a new version of its "100 Greatest Movies of All Time" ... and eight original titles remained in the top 10."
Exactly. Sounds just like what I noted with USNews on Law School ranking, which ranking, unlike the film poll, is presented with a "spurious air of scientific precision":
"...the schools in both USNews & my anecdotal Top 15 or so (sweet 16?) seem always to remain in such Top 15, though their rank within that group may change...."
Post a Comment