Althouse: "Proof That Typos Are Racist."

April 18, 2014

"Proof That Typos Are Racist."

"The partner evaluators found an average of 2.9 spelling and grammar errors for the ‘Caucasian’ authors and 5.8 such errors for the ‘African American’ authors."

53 comments:

Anonymous said...: I was reading on the internet about this Jewish high school student who is continuously tweeting to get Kate Upton to go to the prom with him. I thought it was cute, so I hit the "Kike" button.

Wait.

Damned racist Auto-Correct, it was the "Like" button -- the "Like" button!

This reminds me of the misfortunate that happened when I typed that "Tigger" was my favorite childhood book character. "Tigger" with a "T" -- a "T" --Damn it Auto-Correct!

Don't even get me going about using the word "Maggot"....; 4/18/14, 8:32 AM
tim maguire said...: Superficially damning, but in typical Elie Mystal fashion, the results are exaggerated so he can cry "RACISM!"

Small sample size, but diverse--lots of women and minorities--raise questions about both the generalizability of the results and the motivations of the graders. Also, the people scoring fewer errors by whites are not the same people scoring more errors by blacks.

Pretty significant problems.; 4/18/14, 8:34 AM
David said...: How many partners evaluated? Did they have a sample for which the race of the associate was not known? Why did these partners seemingly miss the majority of the "errors?" Did this study mean anything?

And finally, since these were actual errors that required correction, aren't they doing the black associates a favor by pointing them out? Allowing someone to get by with sloppy work is not a favor.

An vats wrong wit tipos anyvay ?; 4/18/14, 8:35 AM
chickelit said...: From the comments there:

HammeredSidway • 21 hours ago
I wonder how many partners thought it was weird that someone told them whether or not the person writing the memo was black...

Yeah, I wondered that too.; 4/18/14, 8:38 AM
Lewis Wetzel said...: No wonder the only lawyer job Obama could get in Chicago was working for a corrupt law practice.
Racism, racism, racism!; 4/18/14, 8:57 AM
Michael said...: A subject failing to agree with a verb is not a typo.; 4/18/14, 8:58 AM
CWJ said...: As David suggested, the lede could just as easily have been law partners on average miss vast majority of spelling and gramatical errors.; 4/18/14, 8:59 AM
PB said...: Interesting the article doesn't mention the race of "law firm partners". It seems the assumption is that all are white. This is a racist assumption.; 4/18/14, 9:00 AM
Illuninati said...: tim maguire nailed it. The story does not provide enough information to draw any conclusions.; 4/18/14, 9:20 AM
rhhardin said...: Bayes' theorem.

If blacks make more non-typo mistakes than whites, then you attribute more non-typo mistakes to blacks than to whites.

The race you're told is information that you can use to make the best guess.; 4/18/14, 9:22 AM
exhelodrvr1 said...: Do black proofreaders find more typos in "black" memos than "white" memos?; 4/18/14, 9:42 AM
The Godfather said...: I spent almost 40 years in a big DC law firm. I cannot imagine being asked, as a partner, to evaluate a memo by an alleged but anonymous associate, about whom I am told only his/her race. But if I were to imagine that such a thing happened, I and every other evaluator would refuse to participate, or if we did participate we would figure that the only purpose of such an exercise would be to determine if we were racist, so we would bend over backwards to grade the "black" memos better than the "white" ones.

Because that's not what allegedly happened in this study, I am very suspicious.; 4/18/14, 9:44 AM
Lewis Wetzel said...: PB Reader, if you follow the links to the source paper (http://www.nextions.com/wp-content/files_mf/13972237592014040114WritteninBlackandWhiteYPS.pdf)
The methodology is crap. The kind of "survey" that a group of high school students might come up with. For all we know, the white male reviewers gave the "Afro-American" author the highest grades and the female and minority reviewers gave the "Afro-American" author the worst grades.; 4/18/14, 9:49 AM
Jane the Actuary said...: Just to second "chickenlittle", it would be pretty odd to be told "this was written by a black man" without a pretty good back-story to the study. If, on the other hand, the authors were identified as Miles O'Brien and J'amal Jackson, that would at least be less subtle.; 4/18/14, 9:57 AM
jacksonjay said...: As the law prof said a few weeks back, "Everyone knows you can't trust a lawyer!"; 4/18/14, 9:59 AM
Michael K said...: Not worth commenting on.; 4/18/14, 10:17 AM
William said...: Suppose I did a study that shows black lawyers make more grammatical mistakes than white lawyers. What are my chances of getting it published versus getting this study published. Also what are my chances of getting my next study funded versus the author of this study.; 4/18/14, 10:19 AM
Andy Freeman said...: Maybe this is AA strikes again.; 4/18/14, 10:28 AM
Ann Althouse said...: If you are among those here who are wondering how the information about race was conveyed to the evaluators of the memo, click through and read about the study. There is an answer in the text, so you don't really have to wonder.

Also, you people who are wondering… what race are you. I need more data for racial analysis. Is that the kind of question white people are more likely to ask?; 4/18/14, 10:28 AM
Ann Althouse said...: That is to say, what I'm seeing here is that when there is some kind of study showing that there is racism, white people reflexively think that there's something wrong with the study.

Maybe there is, but why don't you even feel you need to pursue your skepticism and figure out what if anything is wrong with the study.

Why do you just comment and say this isn't worth paying attention to or that there must be something wrong with the study?

Can we get some introspection here?; 4/18/14, 10:32 AM
exhelodrvr1 said...: Ann,
It's not very smart to try to be introspective about the results of a survey when, on the surface, the survey appears to be very poorly designed. At a minimum, there is not nearly enough information provided.; 4/18/14, 10:44 AM
Matt Sablan said...: "Also, the people scoring fewer errors by whites are not the same people scoring more errors by blacks."

-- That makes this utterly useless. I'll catch more errors than a lot of people, solely because my job is an editor. If there more editors in one group, then they may catch more edits unrelated to the author.; 4/18/14, 10:48 AM
Howard said...: The short answer is Motivated Reasoning where this example shows how it works for lefty bias, hopefully to help the teabaggers understand the concept.; 4/18/14, 10:49 AM
chickelit said...: Can we get some introspection here?

Stick to your lawyerin' and I'll stick to sciencin'; 4/18/14, 10:57 AM
Jupiter said...: Althouse is right about this one. If you read the paper (not all that long), most of these questions are answered. Interestingly, the results were the same across races and sexes (words have gender, people have sex). That is, the attorneys were not all white males, but the race and sex of the attorney rating the paper did not affect the results.

This may be because racial expectations are cultural, not personal, and people of all races share the dominant American expectation that blacks are lazy incompetents who skate by on affirmative action. Which expectation, in turn, is simply a reasonable inference from the well-known fact that affirmative action places a heavy thumb on the scales at all universities and large organizations. The chances that an educated black person you encounter at work has not been repeatedly the beneficiary of affirmative action are essentially zero.; 4/18/14, 11:00 AM
Left Bank of the Charles said...: The problem with dealing with averages is it only takes a small percentage of virulent racists to skew the averages. It's the old problem of mean versus median.

That creates the impression that the mean white person is racist. In fact, it may only be the outliers who are racist, and the typical white person may not be racist at all.

Nonetheless, the long tail effect of those racist outliers could be quite pernicious, particularly for minorities who encounter large numbers of white people in their day-to-day lives.

That puts white people in the position of the Muslim community. Only a small percentage are extremists, but the community hears the term Muslim extremists as tarring the whole community. That's not helpful in fighting terrorism, as it rhetorically puts the whole community on the side of the extremists.

Likewise, speaking rhetorically of white racism has the danger of putting whites as a group on the side of the racists. Indeed, the word racism itself has come to do that.

Another trigger term is African American, which has embedded meaning. It's hard to say what the effect of that trigger is. Some people who are not racist might see that and subconsciously think, this is the one that matters so I better pay closer attention on this one.

It would be interesting to conduct this experiment with a memo that was excellent instead of sloppy. Would giving an excellent memo extra attention result in the memo labelled African American being rated higher or lower than the one labelled Caucasian?; 4/18/14, 11:04 AM
The Crack Emcee said...: "I need to email this study to David Lat, Bryan Garner, my mom, Matt Levine, and Partner Emeritus, from my fake, white-person, @post.harvard.edu account, so they take it seriously…."; 4/18/14, 11:06 AM
chickelit said...: Can we get some introspection here?

Nothing in the story resonates with my 12 years working in Big Law alongside black co-workers.

As piece of science, the worst aspect is the lack of a meaningful control - evaluations wherein the author's racial identity was neutral or not disclosed.; 4/18/14, 11:09 AM
Illuninati said...: Althouse said:

"Can we get some introspection here?"

Introspection implies the need for self examination. Those who were participants in that study and judged people more harshly based on their skin color, have reason to spend time examining their inner self. If the implication is that there is still racism in this country, the answer is obviously. If the implication is that those prejudiced lawyers are representative of all white people, that in itself would be a racist statement.; 4/18/14, 11:22 AM
n.n said...: The study begins with a biased premise, which can be understood as its own confirmation bias.

23 were women, 37 were men, 21 were racial/ethnic minorities, and
39 were Caucasian

The outcome of this study reveals a near universal truth. Men and women with darker skin are perceived to have toiled in conditions where education was not forthcoming. This was true in most societies, where ruling and intellectual classes were separated from working classes by virtue of physical conditions and luxury time.

What should be taken from this study, is not that bias exists. It does, by virtue of nature. But that cues which were once historically and geographically accurate are less meaningful in first-world integrated societies.; 4/18/14, 11:31 AM
Howard said...: We know this study is bullshit. Blacks are well respected and honored in all of Tom Clancy's novels. This is exactly how it works in real life too. Until all those thugs and welfare Moms stop taking all the good jobs from us white folks, then we can talk about the so-called racialist problem.; 4/18/14, 11:34 AM
Michael K said...: "Can we get some introspection here?"

OK. I introspected and decided this wasn't worth my time. If there is any validity to a "study" like this it is the low expectations in an era of affirmative action.

As for typos:

From a two star review of my book "Why would it be helpful for young physicians to spell the name of the famous immunologist "Paul Erlich" instead of "Paul Ehrlich" (p. 171-172)?
I can see you do not care..."

Nobody cares but a few OCD types. This appeared after the book has been out ten years with nobody noticing. Fortunately, lots of people are still buying it in spite of the OCD types. Legal memos may be more concerned with spelling.

"OCD" mean Obsessive-Compulsive Disorder."; 4/18/14, 11:36 AM
Fen said...: "...when law firm partners read identical memos, the partners who believed the author was white were much more forgiving than the partners who thought the author was black."

I think its due to the assumption that the minority lawyers were hired not because of their ability, but to fill racial quotas.

Hence the extra scrutiny - they already come to the table with the idea that their black associates are subpar lawyers, so they are keyed to actively *look* for mistakes.; 4/18/14, 12:01 PM
William said...: There are useful lessons to be extrapolated from this study. If a white teacher notes the grammatical errors of a black student, then that teacher is probably being racist. Likewise, if a black student is corrected by a white teacher, his immediate assumption should be that the teacher is a bigot. Such useful lessons will help us all get along better.; 4/18/14, 12:10 PM
Lewis Wetzel said...: Point one: this is no more "social science" than a powerpoint presentation at a mandatory workplace sex discrimination training session.

Point two: "There was no significant correlation between a partner’s race/ethnicity and
the differentiated patterns of errors found between the two memos. There
was also no significant correlation between a partner’s gender and the
differentiated patterns of errors found between the two memos."

Point three: Do black litigation associates write less well than Caucasian litigation associates? note that the Above the Law writer admits to having issues with grammar and identifies himself as a black man and a lawyer.; 4/18/14, 12:13 PM
Anonymous said...: They could just use a stereotypical white or black name to get the point across in a more subtle manner. Are Aaliyah Washington's typos seen as more egregious than Brett Olson's? Probably.

Women often change our names when we get married. If the name is substantially different, we can see a difference in how we are perceived on paper. Never mind race, a vowel is enough to do the trick.

People are biased. They will send their minds on a pattern search to confirm that bias.; 4/18/14, 12:29 PM
MaxedOutMama said...: You commenters who are saying that the study proves nothing should read the study:
http://www.nextions.com/wp-content/files_mf/13972237592014040114WritteninBlackandWhiteYPS.pdf

The origin of the bias in treatment may not be racism, though. This from the study:
"There was no significant correlation between a partner’s race/ethnicity and the differentiated patterns of errors found between the two memos. There was also no significant correlation between a partner’s gender and the differentiated patterns of errors found between the two memos. We did findthat female partners generally found more errors and wrote longer narratives than the male partners."

Only 53 partners out of 60 completed the exercise even after a deadline extension. 29 of the "C" papers were returned vs 24 of the "AA" papers (there was only one paper used in this study).

I'd say the study was about as well-designed as this type of study can be.

Regardless of the source, surely there are implications! To assess the implications for AA lawyer performance, perhaps this Nextions white paper might be of use:
http://www.nextions.com/wp-content/files_mf/1364847971_magicfields__attach_1_1.pdf

Human beings aren't machines, and only confirmation bias would make most people believe they are.

Even if the origin of the differential treatment is a belief (conscious or unconscious) that affirmative action has led to less well-qualified AA lawyers, and that such individuals need "further help" to develop properly, it's likely that this sort of feedback could affect workplace performance.

I've got to say that I would find it weirder than hell to receive anything for evaluation that was labeled by race, so that may have had an influence.

Anyway, if any of you are actually interested, here's a link to the Nextions papers. The one on political incorrectness is pretty interesting.

You could argue that there is bias in this, because Nextions has an interest in finding such patterns in order to make sales of their services. But I think the differential pattern demonstrated is strong enough to generate a healthy suspicion that this is a real phenomenon. I'd like to see it replicated by a skeptic.; 4/18/14, 12:32 PM
Anonymous said...: They could just use a stereotypical white or black name to get the point across in a more subtle manner. Are Aaliyah Washington's typos seen as more egregious than Brett Olson's? Probably.

But by letting a second thing vary they'd be introducing an element of ambiguity: are law firm partners biased against blacks, or just against people with peculiar names? (Though it's possible that they may have accidentally done something similar in the study as designed; I wonder how many of the raters assumed that the "Caucasian" Thomas Meyer was Jewish.); 4/18/14, 12:39 PM
tim maguire said...: Prof, I'm a white guy. I had already read it on Above the Law and the standard for commenting on an Elie Mystal article is you have to insult Elie. That and because I had my doubts about the study methodology, I went to the source article, I did the research. What I found is that this study doesn't prove much. Which is what I expected. Confirmation bias.; 4/18/14, 12:50 PM
ken in tx said...: I went to school with blacks starting in 1964, through public school, college, military training, industrial training, graduate school, and post grad. No body corrects blacks on their grammar. As a result, they don't know that they are making mistakes. I discussed this with a professor once when we were supposed to be critiquing a graduate level presentation. I pointed out some mispronunciations. He refused to include my critique in the evaluation. It is the soft prejudice of lower expectations. Hence, we have corpseman for corpsman, and aircrafts for aircraft. Nobody ever tells them.; 4/18/14, 1:27 PM
n.n said...: tim maguire:

Perhaps that's what they were hoping to demonstrate. That bias, while naturally derived, is independently expressed by individuals. That there is no conclusive bias which can be attributed to classes of individuals, where the bias is not an intrinsic feature of the class. If anything, this study challenges the validity of the methods and practices of the incorporated civil rights movements.

That said, I think there are historically and geographically meaningful cues, whose validity is lapsing in the modern era. Especially in first-world integrated societies.; 4/18/14, 1:38 PM
CWJ said...: FWIW, I found the findings troubling. What bothered me was not so much the racial difference in the number of errors found, as the racial difference in the overall scoring of the paper; the 3.2 rating of the "black" paper versus the 4.1 rating of tbe "white" paper. I realize that those are average scores. But with them being so close to 3 and 4 respectively, I bet they represent a whole lot of 3s for the "black" paper and a whole lot of 4s for the "white" paper.

But when I finally clicked past the Above The Law piece to the study itself, I found that portion of the evidence to be the most thinly presented. After bringing up the racial difference in reviewer comments and qualitative assessments, only 6 examples (3 each) are presented. Surely, with such a small sample size, a more complete listing of the actual evidence had it existed would not have been that burdensome. Nonetheless, the 3 versus 4 out of 5 scoring is pretty troubling on its face.

Still, its a small number of data points in a research study conducted by a firm whose business apparantly depends upon finding racial differences of this sort. So caution is in order.

For example, if I were writing an article purporting to show that people were less willing to go on record expressing an opinion about black people, I'd emphasize how the nonresponse rate for the "white" papers was only 3% compared to a whopping 20% for the "black" papers. The actual numeric difference is only 5, but it sounds much more dramatic than anything the authors actually wrote in their report.; 4/18/14, 1:44 PM
jacksonjay said...: No body corrects blacks on their grammar. As a result, they don't know that they are making mistakes.

Micro-aggression!; 4/18/14, 1:49 PM
Anonymous said...: Whah, whah, whah. Oh, gee, reviewers found more actual mistakes by "black" writers. I'm sure this could not have anything to do with "affirmative action", and the forced knowledge that because of such racist programs, credentials "earned" by people who identify as "black" may not have the same value as ones earned by people who didn't have the support of racist programs.

If the reviewers have found "false" problems", you could try to put together a valid complaint. But reviewers finding problems that actually are there?

Piss off.; 4/18/14, 4:09 PM
Anonymous said...: Ann Althouse said...

That is to say, what I'm seeing here is that when there is some kind of study showing that there is racism, white people reflexively think that there's something wrong with the study.

No, Ann, what this is is a "study" claiming that "everyone is racist", that "there are no valid standards", and that the political opinions of the study authors: "we must have more 'Affirmative Action', because everyone is racist and tries to keep blacks down" are the one true valid opinions.

IOW, this study reeks of being tendentious political crap, and those of us who disagree with the politics of the study authors are responding by looking for the flaws we're pretty sure we'll find, since leftists "scientists" are well know for putting together crap and selling it as gold.

It's the basic idea behind the scientific method: you are going to lie to yourself, and fool yourself, when doing research where you have a preferred answer. You expose your research to review by people who have just as much of a vested interest in tearing your ideas down, as you have in building them up.

The problem with the "social sciences" is that they are all a bunch of leftists, and most of them are political whores first, not scientists. So you can not expect them to impose honest peer review on anything that has a left wing result.

Thus we have to do it for them.; 4/18/14, 4:26 PM
Anonymous said...: Having read the article, I'm utterly unimpressed:

1: "Of the 41 edits and/or comments on formatting, 11 were for “Caucasian” Thomas Meyer’s memo in comparison to 29 for “African American” Thomas Meyer’s memo."

11 + 29 = 40, not 41. I'm supposed to trust an article from people who can't do basic math?

2: There are no "p values" anywhere in the article. If you haven't even bothered to figure out how likely your results were to come about by chance,

3: "There was no significant correlation between a partner’s race/ethnicity and the differentiated patterns of errors found between the two memos. There was also no significant correlation between a partner’s gender and the differentiated patterns of errors found between the two memos. We did find that female partners generally found more errors and wrote longer narratives than the male partners."
4: We undertook this study with the hypothesis that unconscious confirmation bias in a supervising lawyer’s assessment of legal writing would result in a more negative rating if that writing was submitted by an African American lawyer in comparison to the same submission by a Caucasian lawyer. In order to create a study where we could control for enough variables to truly see the impact of confirmation bias, we did not study the potential variances that can be caused due to the intersection of race/ethnicity, gender, generational differences and other such salient identities

Each of those sentences in #3 requires a p value, none is given. Were there more female partners reviewing the "African American" than the "white"? Not discussed. That would in and of itself have a significant effect on the scores.

But the most damning part is #4. They established a hypothesis, and then refused to examine the data in any way that might invalidate their hypothesis. They had very small sample sizes, and did not provide any p values to give their results meaning.; 4/18/14, 5:29 PM
pst314 said...: Ann Althouse "Maybe there is, but why don't you even feel you need to pursue your skepticism and figure out what if anything is wrong with the study."

Maybe because after many decades of utter bullshit studies and grossly dishonest academic moral hectoring, some of us don't feel the need to read yet another one.; 4/18/14, 5:45 PM
pst314 said...: chickenlittle "Stick to your lawyerin' and I'll stick to sciencin'."

Heh.; 4/18/14, 5:47 PM
Anonymous said...: MaxedOutMama said...

You commenters who are saying that the study proves nothing should read the study:
http://www.nextions.com/wp-content/files_mf/13972237592014040114WritteninBlackandWhiteYPS.pdf

I read it. It's crap.

"There was no significant correlation between a partner’s race/ethnicity and the differentiated patterns of errors found between the two memos. There was also no significant correlation between a partner’s gender and the differentiated patterns of errors found between the two memos. We did find that female partners generally found more errors and wrote longer narratives than the male partners."

Only 53 partners out of 60 completed the exercise even after a deadline extension. 29 of the "C" papers were returned vs 24 of the "AA" papers (there was only one paper used in this study).

I'd say the study was about as well-designed as this type of study can be.

What are the p values of the results? For that matter, where is the R data set for their results, so we can provide independent statistical analysis, and look at the factors they deliberately ignored because they might get in the way of the authors' pre-determined results?

The data isn't available, because it doesn't support the study.; 4/18/14, 5:55 PM
rhhardin said...: Introspection, Bayes simplified.

There are two authors.

X makes few grammar errors and is a good speller.

Y makes lots of grammar errors and is a poor speller.

You're presented with a sentence with an error in it. Is it a typo or ignorance?

If you're told it's author X, the best guess is it's a typo.

If you're told it's author Y, the best guess is it's ignorance.

If you want to make the best guess, the guess with the fewest mistakes, you use the information you're given.

This is not to be stamped out. It's how you live without being mugged.

What you want is being open to new information, not ignoring what's true.; 4/18/14, 6:21 PM
rhhardin said...: What's his face at the WSJ points out an Althouse grammar mistake today.

I make the assumption that Althouse knows his rule and thinks it's stupid.

Bayes theorem in operation.; 4/18/14, 6:23 PM
MadisonMan said...: I would find it totally bizarre if someone came up to me and said "Read this, it was written by a (insert race here)"

And I echo the observations made above: Why did they miss all the errors?; 4/19/14, 5:57 AM
stlcdr said...: If you don't find this sentence racist, you aren't trying hard enough.; 4/19/14, 7:35 AM