March 3, 2006

BuzzMetrics.

Here's a WaPo article on the sophisticated software companies are using to track public opinion as it is expressed on the internet:
To capture the chatter, Nielsen BuzzMetrics, a giant in the industry, uses software that collects hundreds of thousands of comments a day. The technology can scan for specific companies, products, brands, people -- anything searchable. It can slice data into a range of categories to quantify the number of times a subject was discussed online, the individuals who mentioned it and the communities where it appeared.
Examples of knowledge acquired by this amazing new technology? People are going to be wanting more snacks in the future, and they prefer "American Idol" to the Olympics.

Or do you find this technology ominous?

Talk Left speculates that the government is using these techniques too. Is that bad? Is ascertaining the drift of the on-line conversation be any more threatening than than taking political polls? This process of aggregating large numbers of statements to read the general opinion is quite different from monitoring an individual. But in any case, the individuals generating the statements are writing in a completely open public space. We can't say the government is invading our privacy when we are inviting the entire world to read us. Yet maybe something terrible is happening, and we blithe bloggers will live to regret it.

15 comments:

Hoots said...

Technology is a wonderful thing. I remember our first TV when I was about ten years old. Only one or two channels had a picture, and even those were often full of snow. I remember thinking how television was better than radio because you could not only hear static, you sould also see it.

Today's technology extends the same phenomenon to public opinion. Thanks to comment threads mob behavior is no longer restricted to streets and fields. We get to participate from the comfort of our keyboards.

Likewise, corporations (just like people, you know, except bigger) are no longer limited to squeezing out the competition down the street, they can now do battle with the whole world as a playing field.

I have a flash of that scene in King Kong where the prehistoric monster is fighting a contemporary monster.

MadisonMan said...

Or do you find this technology ominous?

I expect it. But what if bloggers suddenly start throwing in words like Charmin Bath Tissue or Cheetos -- will that skew the Starbucks (Or as we call it, the church of St. Arbucks) demographics?

Osama Bin Laden.

Gaius Arbo said...

It's funny how people get massively worked up about the government invading their privacy but completely ignore the corporations who are doing much, much more privacy invasion every day.

Inquiring minds want to know about the properties of izlloy!

Joan said...

This is akin to the methods supermarkets use to spot buying trends among their customers. I know people who won't sign up for the "club cards", but all they are basically doing is opting out of the sale prices. There's just so much data out there, it's rather incomprehensible.

I used to work for the software company that wrote the database engine and applications that did all that data slicing and dicing, many many years ago. (There's probably more than one company with similar offerings now.) Multi-dimensional analysis is not as easy as it sounds. And while you certainly can use the software to apply filters and find everyone who bought a specific product within a certain time period, or everyone who commented on a particular issue, the idea that someone would secretly do something like that for nefarious purposes remains absurd to me. People don't realize how tedious and expensive this kind of analysis is. It's not the kind of thing an individual or even fringe group would easily have access to -- at least, it didn't used to be.

Anonymous said...

We can't say the government is invading our privacy when we are inviting the entire world to read us.

How do you feel about your email being read by the government? Your email is most likely transmitted in unencrypted cleartext from the time it leaves your computer to the time it arrives at its destination. Along the way, passes through an untold number of other computers and devices, all in the clear, and none under your control.

You have agreed to allow your email to travel unencrypted through these very public routes. Do you feel you have an expectation of privacy from the government regarding your email?

Your cellphone and home wireless calls (used to be and may still be) transmitted in the clear from transmitter to receiver. Do you feel you have an expectation of privacy from the government regarding those calls?

The times you leave your home and the routes you take to work are observable by all, do you feel you have an expectation of privacy from the government regarding this information?

The addresses of the USPS letters that you send and receive, do you have an expectation of privacy from the government regarding this information?

The heat signature of the house you reside in as transmitted through its walls, windows, and ceilings, do you feel you have an expectation of privacy from the government regarding this information?

The trash, its contents, its weight, your patterns as you throw it away, do you feel you have an expectation of privacy from the government regarding this information?

Your passage through streets and how it may be observed by cameras, do you feel you have an expectation of privacy from the government regarding this information?

How do you supposed George Washington, Benjamin Franklin, John Hancock and the rest of our rebellious forefathers would have answered?

Anonymous said...

If you publish your blog under the Creative Commons, Attribution-NonCommercial-ShareAlike 2.5 license

Should buzzmetrics scan it? Should the government scan it?

Oh, how would you feel about toilets that monitor your urine to detect health conditions? This was once a Larry Niven science fiction device, but are now a real device. Would it be okay for a commercial operator to place those toilets in her building (perhaps to reduce her liability?) Would it be okay for the government to take those records and scan them and analyze them?

Are you sure we can't say that our privacy is being invaded when information and patterns about our everyday existence becomes digitizable and then becomes scanned?

MadisonMan said...

...and there are far fewer government employees out there with nefarious impulses than we would be led to believe.

This begs two questions. What do you base that statement on, and how many nefarious government employees is too many, in your opinion?

I agree it is inefficient for the government to monitor citizenry. Why, then, should they do it?

Anonymous said...

If the South Dakota law abortion law makes plan b illegal, if parental notification laws further restrict abortion, wouldn't it be a reasonable step for schools to install such toilets that could monitor chemicals, anti-biotics, hormones? Or not just schools, but say, any government building?

When price drops, what happens to demand?

If Griswold were overturned, what better way these days for the government to monitor contraception use than to install toilets and other sensors in public places? Perhaps cameras outside pharmacies.

What are computers but devices that transform the "highly inefficient" into the "cost effective?"

No tinfoil hats required, all cameras, smart toilets, and computers are all very 2005 and 2006. Along with warrantless wiretapping based on the AUMF and the emplacement of a Supreme Court Justice that writes a letter thanking an anti-abortion group for their help in getting him his seat.

Unknown said...

Buzzmetrics sounds like a lot of empty...buzz. The kind of narrow focus results in, for instance, the insipid, timid political rhetoric of today. FDR didn't have buzzmetrics, and he moved a population (yes, sometimes to rage, but he moved them).

I worked for the government for many years. Believe me, they are not capable of making any coherent use out of this mass of information they are supposedly secretly gathering any more than they are capable now of finding the guys who stole your TV, despite their having left fingerprints all over your house.

Terrorists, OTOH, with this information worries me greatly. All it takes is one lucky strike.

MadisonMan said...

sure, one nefarious government employee is too many, but that's why it's so frigging huge.

I ponder how J. Edgar Hoover would work in the computer age.

As for accepting snooping to counter terrorist threats, well that's been discussed earlier re: the "Patriot" Act. I am willing to live with a very low-level threat and be unmonitored. Others here prefer more government snooping for (what I believe to be) a false sense of security.

Anonymous said...

Your marital status (Lines 1-5), number of dependents (Line 6), income from wages, taxes, and tips (Line 7), interest (Line 8), and dividends (Line 9).... do you feel you have an expectation of privacy from the government regarding this information?

How they use that? Absolutely. Should congressman, senators, or random bureaucrats be able to pull that information up? No way.

Should specific authorized representatives of the IRS be able to use that information to determine the tax? Sadly, yes.

Anonymous said...

I can't believe anybody would actually believe that our government ENFORCE contraception use?

I didn't say enforce, I said monitor ... Perhaps you have never heard of Griswold, Plan B, or South Dakota's anti-choice bill.

SippicanCottage said...
This comment has been removed by the author.
Anonymous said...

"Privacy from the government", then, means only that the government agent who's spying on you has the proper credentials?

If by credentials you mean warrants or legal authorization than I guess yes. I would think that Franklin would be for warrants and statutes, but I am not the expert.

If you want to argue that the income tax itself neccessitates an unconstitutional invasion of our privacy you're on your own.

Smilin' Jack said...

quxxo said...

No tinfoil hats required, all cameras, smart toilets, and computers are all very 2005 and 2006.


More like 1995-96. We've come a long way since then. Didn't you read Ann's post last month about the government's tiny robotic spy insects? There's one behind you right now, recording every move, every word, transmitting it all to Washington, even reading over your shoulder as you type your comments on this blog. I'd hate to be you when Karl Rove gets around to reading the report on you.