"The phrase well-rounded derives from phrenology..."

I'm reading that in "Rebel Souls: Walt Whitman and America's First Bohemians," by Justin Martin:

Whitman haunted the Phrenological Cabinet of Fowler & Wells.... On July 16, 1849, Whitman paid three dollars to have his skull read by Lorenzo Fowler. On a scale from 1 to 7, Whitman rated an exemplary 6.5 on such traits as benevolence, self-esteem, and firmness. He received one of his lowest marks for acquisitiveness, the pursuit of money and material gain. The low rating sat fine with Whitman, struck him almost as a veiled compliment. In his report, Lorenzo noted, “Size of head large . . . a certain reckless swing of animal will, too unmindful, probably, of the conviction of others.” He added, “You are yourself at all times.” Overall, Fowler painted a flattering picture of Whitman, casting him as a well-rounded modern man. (The phrase well-rounded derives from phrenology and is based on the notion that an actualized person has a nicely shaped head, without any distortive bumps.) The results greatly pleased Whitman.

Is that really true? I'm seeing in the OED that "well-rounded" to refer to visible objects like trees and horse's hooves is very old, but in reference to a person's character it does go back only to the mid-19th century when phrenology was was the rage. The oldest quote comes from Henry Wadsworth Longfellow — "There was something so complete and well-rounded in his life." Longfellow was loathed by Whitman, enough so that — according to the above-linked book — Whitman once insulted a man by calling him "a young Longfellow." Whitman's crowd understood:

The jibe was brief, pithy, and a direct hit: where Poe was this crowd’s patron saint, Longfellow was its bête noire. As a sentimental poet, Longfellow was anathema to many in the Pfaff’s set. They snidely referred to him as “Longwindedfellow.” 

By the way, here's how Whitman (in 1856) insulted the President of the United States: "The President eats dirt and excrement for his daily meals, likes it, and tries to force it on The States."

Here's an article in The Atlantic — "The Shape of Your Head and the Shape of Your Mind" — that reinforces the claim that the term "well-rounded" comes from phrenology:

The national obsession with head size and shape... infected daily conversation. Many modern phrases trace their roots to phrenology, including “highbrow” and “lowbrow,” “well rounded,” and “shrink” (as in “shrinking” certain undesirable qualities). “Getting your head examined” also has phrenological roots. Though generally considered an insult today, in the past, it was just what most people wanted.....

ADDED: Pfaff's:

"Pfaff’s was the Andy Warhol factory, the Studio 54, the Algonquin Round Table all rolled into one."