Althouse: IBM's computer "Watson" beats Ken Jennings on "Jeopardy."

January 15, 2011

IBM's computer "Watson" beats Ken Jennings on "Jeopardy."

Interesting. I think the really unfair part of the matchup was at the buzzer-skill level.

29 comments:

Crimso said...: "What we have done is advance artificial intelligence by miles."

Bullshit. He prefaces this by saying advancements in AI are measured in inches. Did Watson learn these facts on its own, or were they supplied for it in some huge database? Can this far-advanced form of AI drive a car or hunt a deer? How about wipe its own ass? I'll be miles (rather than inches) more impressed when they come up with a computer that can beat the world's best at Advanced Squad Leader.; 1/15/11, 9:13 AM
shake-and-bake said...: "I'll take product placement for $1000, Alex."; 1/15/11, 9:17 AM
Fred4Pres said...: That's nothing: My kid's abacus, without someone using it, can probably beat more than a few political pundits.; 1/15/11, 9:40 AM
Anonymous said...: My brother is a past Jeopardy champ and played the computer earlier this year during development.; 1/15/11, 9:53 AM
Unknown said...: The buzzer thing is important. If you ever saw one of those celeb games where one was a pro athlete, the speed getting to that buzzer can be night and day.

Crimso is also right in that Watson had a db loaded with trivia while the other two were weighted down with memories, emotions, and stuff.; 1/15/11, 9:54 AM
The Crack Emcee said...: edutcher,

Crimso is also right in that Watson had a db loaded with trivia while the other two were weighted down with memories, emotions, and stuff.

That doesn't bother me - that's what computers are for - they just better not try to convince me Watson "knows" what he's spitting out.

I know: it's just a different way of saying the same thing, but still,...; 1/15/11, 10:41 AM
The Crack Emcee said...: Fred4Pres,

That's nothing: My kid's abacus, without someone using it, can probably beat more than a few political pundits.

Who is Wolf Blitzer for $100, Fred?; 1/15/11, 10:43 AM
The Drill SGT said...: edutcher said...
The buzzer thing is important. If you ever saw one of those celeb games where one was a pro athlete, the speed getting to that buzzer can be night and day.

Put a geek in between the computer and the buzzer. make the geek punch the buzzer when he sees a light; 1/15/11, 10:49 AM
Joan said...: The type of technology powering Watson, which is named after IBM founder Thomas J. Watson, tends to double every 18 to 24 months, though, making it only a matter of time before it catches up with human capabilities, Kelly said.

My best laugh of the day so far. I love it when reporters try to explain something that they themselves don't understand. Semiconductors ≠ power supplies, and Watson's capabilities are as much software-based as they are hardware-limited.; 1/15/11, 11:12 AM
rhhardin said...: Artificial Intelligence has been on the verge of a major breakthrough since 1955, which is three generations of older but wiser male graduate students.; 1/15/11, 11:16 AM
Bruce Hayden said...: There has to be a Jeopardy site somewhere, so the easy way to do this would be to hook up some sort of wireless connection between this computer and a van outside, that had a satellite link to the Internet. Then when the question is asked, Watson hits the buzzer and sends off the query, confident that most of the time, by the time the humans get around to asking the computer what its answer was, it would have gotten the response from the Internet. And, maybe with Google and Wikipedia, they wouldn't even need a Jeopardy site.; 1/15/11, 11:22 AM
Bruce Hayden said...: AI is coming, just a lot more slowly than we all thought it would be. Indeed, HAL (i.e. IBM-1), was in the movie partially titled 2001, from 1968. We have a ways to go until we get there.

But I think that we will get there. There a lot of different projects going on now, approaching it from different points of view, and developing different parts of a while. We have artificial vision, walking, driving in traffic, voice recognition, simulation of a brain, etc.

I, probably along with many others here, have been following this for a long time. I have belonged to the IEEE for several decades now, and there has been a constant stream of different approaches and directions in its Spectrum general (engineer) articles over that time, with few issues not including something or another in this area.

I would not be surprised if we saw an article on how IBM (i.e. HAL+1) pulled this off. We shall see.; 1/15/11, 11:34 AM
Crimso said...: But how will we know when we get true AI? Especially when so many supposedly brilliant people seem to have failing NI ("natural"). What's the best case if we do achieve it? The worst would be SkyNet. No, check that, the worst would be that described in Frank Herbert's "Destination: Void."; 1/15/11, 12:46 PM
Crimso said...: And I shudder to think what true AI will do when it starts hearing the hate spewed by Palin, Beck, and Limbaugh.; 1/15/11, 12:51 PM
Crimso said...: And I'd dearly love if IBM would devote these kinds of resources to besting climatologists rather than trivialists and chess players.; 1/15/11, 12:54 PM
Unknown said...: Where to start...

Watson does not use large databases to give answers. That's a losing strategy since the domain is hopelessly large.

There can be questions about fish, ballet, wainscoting, colors, etc. Do you think there exists a database that covers all human knowledge and experience?

You can build a database of the twelve months. What if the answer is Ramadan?

How do you build a database to differentiate between the roman number at the end of Pope Paul VI and the letter at the end of Malcolm X.

Also, IBM provides weather prediction supercomputers to the NWS and in other countries too. Look at http://www.research.ibm.com/weather/DT.html

Watson is not connected to the Internet. It is totally self contained.

And it is great product placement. You think Google,HP or Oracle wouldn't do this if they had anything close? None of them are scoffing at this.; 1/15/11, 1:46 PM
Crimso said...: I scoff at it because I can beat it at everything with the possible exception of Jeopardy. I actually do think it is both valuable and interesting work, but "advance artificial intelligence by miles" is a lot less impressive when you consider that we're working on a scale of light years. I once had a device called Simon, and it eventually beat me every time. My handheld calculator can figure square roots much faster than I can, but I don't find it all that amazing.

Now if Simon could routinely beat me, calculate square roots, drive a car, write a novel (even a poor one would suffice), cook a decent meal, teach people regulation of metabolic pathways, and reproduce, well that would be a very small start. It's like the argument about whether a cat or a dog is smarter. I recall a study some years ago that concluded dogs were more intelligent, but cats were better at what they do. Is the goal a computer that is more intelligent (and therefore more versatile because it can actually learn), or a computer that is better at one specific thing (which to me does not equate with intelligence).

Designing computers that can be the best in the world at what they do is easy, depending upon the task assigned. Tasks requiring true intelligence (can't define it but know it when I see it) are way beyond anything that's happening now. Too often scientists and engineers convince themselves they've achieved something grand when the reality is far more modest. I know this because I have graduate degrees in both fields and have worked in both. I would dearly love to see true AI in my lifetime (the philosophical and religious implications alone would be incredibly fascinating to explore), but I am highly doubtful that I will (I'm 46).; 1/15/11, 2:21 PM
Crimso said...: "How do you build a database to differentiate between the roman number at the end of Pope Paul VI and the letter at the end of Malcolm X."

Sexual intercourse.; 1/15/11, 2:24 PM
Crimso said...: "Also, IBM provides weather prediction supercomputers to the NWS and in other countries too."

Will these supercomputers be teaching and doing research in metereology and climatology, or will we still have to rely on human intelligence for that? Both on the sending and receiving ends.; 1/15/11, 2:39 PM
Gentleman Farmer said...: "Watson does not use large databases to give answers. That's a losing strategy since the domain is hopelessly large."

We must be using words differently: if "Watson" does not draw on a body of stored information, then where do the answers come from? Prayer?

All you're seeing here is pretty cool fuzzy logic language interpretation bolted on to a lot of brute force computing power. It's possible that this sort of thing will eventually pass Batshit Crazy Alan Turing's test, but that will have more to do with the limitations of human perception than actual machine intelligence.

When a computer starts giving me useful answers for why girls behave as they do, THAT will be intelligence.; 1/15/11, 2:53 PM
Robt C said...: Here's a link to the IBM website about Watson. Might help get through all the conjecture.
http://www-03.ibm.com/innovation/us/watson/what-is-watson/index.html; 1/15/11, 2:59 PM
Crimso said...: A more direct and concise explanation was found on Wikipedia (admittedly not always reliable, but the following statement had a citation to support it):

"The IBM team provided Watson with millions of documents, including dictionaries, encyclopedias and other reference material that it could use to build its knowledge, such as bibles, novels and plays. Rather than relying on a single algorithm, Watson uses hundreds of algorithms simultaneously to find the correct path to the answer. The more algorithms that independently arrive at the same answer, the more likely Watson is to be correct. Once Watson comes up with a small number of potential solutions, it is able to check against its database to ascertain if the solution made sense. In a sequence of 20 mock games, human participants were able to use the six to eight seconds it takes to read the clue to decide whether to buzz in with the correct answer. During that time, Watson is also able to evaluate the answer and determine if it is sufficiently confident in the result to buzz in."; 1/15/11, 3:17 PM
Bruce Hayden said...: Here's a link to the IBM website about Watson. Might help get through all the conjecture.

Not really. IBM hype hasn't changed much in the last 35 years since I started dealing with them. I get that they think that Jeopardy is harder than search, but I remain unconvinced. While sometimes Google goes off into the weeds, much more often than not, it gets me what I want.

So, I would think that it might be a better challenge to have a search engine company go up against a computer manufacturer (and apparently now, primarily a service provider), with a human in there as a control, than against two humans.

Do we really care that IBM has put all their disk drives in a relatively small box, in comparison with the acres of them that Google uses? Yes, we know that IBM produces good hardware and good database software. But so what?; 1/15/11, 3:22 PM
Bruce Hayden said...: BTW, I am cynical about IBM for a number of reasons. They named the box apparently after the father-son team who really put the company on the map. But the father's time in the dirty-tricks department at NCR left its imprint at the company.

Add to that, that IBM is no longer an American company, except maybe legally. Not when it has more employees in one city in India than in the entire U.S.; 1/15/11, 3:25 PM
jsled said...: Watson did have to press a physical button to answer, presumably the same as the other contenders. http://www.popsci.com/technology/article/2011-01/preview-battle-jeopardy-all-stars-take-ibms-robot-watson

@Crimso it might still be a "miles" advance in AI and not able to wipe its own ass … AI is just at that point. That being said, I don't rule out the possibility of "Strong AI"; at the end of the day. we are all just biochemical machines.

@stan by definition it is a database of facts that it uses to answer the questions. how else do you think it stores such pieces of information? it might not a common Codd-model RDBMS, but it is some sort of "data base". I'd imagine most knowledge representation is graph-based.

@BruceHayden what are you talking about seriously I don't even…; 1/15/11, 6:56 PM
The Dude said...: I thought the movie Malcolm IX was better than the next one in the series.; 1/15/11, 8:50 PM
Known Unknown said...: About a year back, they had a challenge Watson game/quiz on the IBM website. Perhaps it was a different, less-developed Watson, but I was able to answer more questions correctly.; 1/15/11, 10:57 PM
Steven said...: @rhhardin I think the "we've been promised AI for decades and we still don't have it" crowd is speaking tautologically; their definition of AI excludes anything that researchers know how to do. Before computers could play a decent game of chess, algorithmic chess was considered AI. Vision, knowledge representation, natural language processing -- all AI. None of those three are played out, as chess is, but your guys from 1955 would be pretty damn impressed with the Kinect. Or with Watson.

I would have added roughly what @jsled wrote to @Crismo, if he hadn't preempted me. I do want to agree, though, that IBM's PR guy may be overstating things somewhat. This is probably against the PR guy code of ethics. It is, at least, unheard of. I'm not sure who the proper authorities on that are, but I hope you file a complaint.

It will take a long time before robots in our world behave like robots in Isaac Asimov's world. (Though not as long as it will take for humans in our world to behave like humans in his world. But I digress).; 1/16/11, 12:20 AM
SingularSam said...: All the people who vilify this computer as some lame excuse for AI need to understand this: you're right. Nonetheless, it is AI. Those without a sense of history will of course view it as lame. People said the same thing when a computer beat the world chess champion. But now voice recognition? Now the ability to understand complex word play? What will computers be like in ten years? Hmmmm, not so lame now is it?; 2/11/11, 3:30 AM