This myth probably originated with an old military study in which scientists put subjects in arctic survival suits (but no hats) and measured their heat loss in extremely cold temperatures. Because it was the only part of the subjects’ bodies that was exposed to the cold, they lost the most heat through their heads. Experts say, however, that had this experiment been performed with subjects wearing only swimsuits, they would not have lost more than 10% of their body heat through their heads.Now, now... if we're going to be scientific, we'll need to factor in the cold stares you get when you go out without your pants. I mean, take off the swimsuits! Accuracy will require nudity.
That link is from from a 2 part article debunking myths — here's part 2 — that was just linked on Freakonomics.
By the way, it was 19° here in Madison yesterday — the same temperature it was on New Year's Eve in New York City when Ryan Seacrest was whining about the cold while wearing a hugely puffy down jacket and earmuffs — and I saw a young man on State Street who was wearing only a T-shirt, shorts, and moccasins. Not even socks. He wasn't shivering or huddling against the cold, just walking along briskly, talking with his friends.
I see guys like that all the time in Madison, and my theory is that — like people who sleep naked — they just don't want their limbs encumbered in the slightest, and they are walking from one indoor place to another and willing to put up with a little discomfort during the relocation.