Tuesday, February 15, 2011

JEOPARDY! VERSUS WATSON

It's an age old question: Who is smarter? The human(s) who invented the computer, or the computer that can outhink the human(s) that invented it?

During an exhibition match to be shown over the next three episodes of the syndicated game show Jeopardy!, February 14-16, 2011; there might be an answer to that question.

Within the basic two game grand championship format of the program, where the scores from the first game are added to the scores of the second after that one is completed to decide the overall winner, the contestants are:
Ken Jennings: Jeopardy's longevity champion, who competed in 74 straight games before being defeated. Between his original run and championship tournament appearances since, he has won over $2.5 million US Dollars to date.
Brad Rutter: The overall highest prize winning player (to date) in Jeopardy! history, with $3.25 million US Dollars so far between his original run and championship tournaments since.
WATSON:  The computer system, named after IBM's founder Thomas John Watson, Senior and not Alexander Graham Bell's assistant Thomas Augustus Watson. Created to think and respond verbally to questions like any human being, it is the company's latest explorations in the field of artificial intelligence. While IBM has high hopes for its success in the competition and any future technological developments, Watson still needs to be told the questions through standard electronic methods, although such input is closely monitored so it is only presented the clues as they appear to any other player.

Because of the unit's overall size, requiring ten banks of computer memory storage terminals maintained within an environmentally controlled room, the experimental/exhibition match is being conducted at the IBM laboratory in Yorktown Heights, New York where Watson was developed. The computer itself is represented in a mock up of the Jeopardy! studio by an avatar on a display panel, with a color scheme designed to show its thought process. Confidence/correctness is in green, uncertainty in yellow, and red represents being incorrect/not knowing/not having an answer in time.
It has been interesting to see Watson in action, for a display at the bottom of the screen shows the results of its thought process to each question, whether or not it attempts a response during the three seconds allowed players before host Alex Trebek reveals the answer and moves on to the next clue. By means of an electronically controlled mechanism, Watson can physically push a buzzer to attempt ringing in to respond in time like any other competitor.

The contestants are playing for both various charities, and in the humans' case, themselves.
Jennings will split his winnings with Village Reach, an organization that helps with health care in under resourced areas around the world.
Rutter is sharing his winnings with the Lancaster County Community Foundation of Pennsylvania.
On behalf of Watson, IBM is splitting whatever it wins between World Vision, a relief and development organization, and the World Community Grid, an organization that provides computer resources to public and non-profit organizations.

While Jeopardy! was chosen and approached for its cooperation because the syndicate game show represents a true challenge to human intelligence, this is not the first time IBM has pitted one of their computer developments against human skills. In 1997, a computer named "Deep Blue", programmed with every known bit of knowledge of masters past and present, won a chess tournament, although someone had to physically move the pieces on the board for it and some players complained that it was just finding the proper moves from reference material and not truly "thinking".

The Free Choice E-zine covered this event as each episode aired, and this post was updated accordingly.
NOTE: ALL SCORES ARE IN US DOLLARS.

GAME ONE>ROUND/EPISODE ONE: FEBRUARY 14
Because of all the background and introductory material that needed to be presented, only the opening round of the first game was completed in Episode 1.
Watson had a good early lead, maybe because Jennings and Rutter were amazed watching it in action. But then the computer stumbled on a question in the category "Name that Decade" because, not having any audio input processors, it repeated an already given incorrect response instead of its second choice, which was the correct answer.
As the humans came from behind, at the end of the first Jeopardy! round, Brad and Watson were tied for first place at $5000 while Ken was third at $2000.

GAME ONE>ROUND/EPISODE TWO: FEBRUARY 15
I have heard some criticism that these first two days of play were not true Jeopardy! but more akin to an IBM infomercial. I will grant you that with the rest of the background/documentary material presented, only the Double and Final Jeopardy rounds of the first game were completed in this installment.
There were a couple of instances whereupon either all three contestants got the answer wrong, or else did not know the proper response.
At the end of the traditional Double Jeopardy round, Ken Jennings was in third place with $2400.
Brad Rutter was in second place with $5400, and Watson was in the lead with $36, 681 How the computer managed to find all the Daily Doubles between the two rounds in the first game, let alone why it chose the odd amounts it did in betting, is debatable.
The Final Jeopardy category was U.S. CITIES. Both Jennings and Rutter responded correctly, while Watson answered "What is Toronto?????" with the multiple question marks representing the fact that somehow it was confused about either some part of or the overall question itself: "This major U.S. city has one airport named for a World War 2 hero, and another named for a famous battle site of that war."
The correct response was "What is Chicago?", with O'Hare and Midway being the respective airports.
Apparently Watson had too many options to choose from to give the correct response, but even a properly programed computer should have known that Toronto is a city in Canada, not the United States. And why its bet was such an odd figure ($947) is debatable, any Jeopardy! player with that big a lead would never risk a major amount, no matter what the category was.
In any event, at the end of the first game, the scores stood at:
Ken Jennings: $4800, third place
Brad Rutter: $10400, second place
Watson: $35734, first place
The final game of the two game exhibition event will be played in the traditional half hour Jeopardy! format tomorrow, with the contestants starting fresh at $0 and the Game One totals added in to whatever their Game Two final scores are to determine the overall winner.


EPISODE THREE/GAME TWO: FEBRUARY 16
Ken Jennings took and early lead and kept it to win the Jeopardy, finding the Daily Double and successfully doubling his score. Both Jennings and Brad Rutter rallied to have Watson finish third in the first half.
However Watson came back in the Double Jeopardy round, finding both Daily Doubles, but only getting the second one correct.
At the end of regulation play, the score was:
Ken Jennings, $18200
Watson, $23440 and
Brad Rutter $5600, with the Final Jeopardy category being "19th Century Novelists" and the correct respone of "Who is Bram Stroker?"
While all three contestants answered correctly; for whatever reason, Kennings only risked $1000 to up his score to $19200.
Rutter successfully doubled his winnings to $11600, and Watson, with an odd bet of $17793, won the game with $41413.
Adding in the scores from Game One, Watson won with a combined score of $77147 and its charities splitting a $1 million US Dollar purse.
Ken Jennings came in second with a combined total of $24000 and splits $300000 with his charity.
Brad Rutter finished third with $21600 and splits $200000 with his charity.

CONCLUSION
There is no doubt that computers are wonderful things. IBM's Blue Gene helped map the human genome of DNA.
Yet artifical intelligence still has a long way to go if it is ever to understand organic intelligence.
If Jennings had not answered before Watson, the computer would have given an incorrect response during the "Mouse-tery" category with "What is Pinky and the Brain?" instead of just "What is the Brain?" since the clue asked for the specific character that Maurice LaMarche voiced and not the series.
(And for those curious, Rob Paulsen voiced Pinky. Narf!)

In the end, Watson is a great computer and a fantastic search engine. But it is the etheral qualities that compose human life which makes us special.
The computer still does not truly 'think', yet.
But someday, even if it has to be held back in the Yorktown Heights facility because of its size; it would be interesting to see a rematch between Watson and Jeopardy!
Perhaps a true test of Watson would be to start from the beginning of a traditional Tournament of Champions competition as a contestant like everyone else and see if it can work its way up to the final rounds.
In any event, I for one would truly like to see a rematch, someday.

2 comments:

George said...

What is the watson technology helping the world do now (since Deep Blue actually had a significant contribution to science)? Or is just google faster now?

Lee Houston, Junior said...

Actually George, that is a very good question. But IBM tends to keep their developments and progress on such matters "close to the vest" as they say, until the company is ready to go public with the information.
But when IBM does make an announcement, we will report it!
Lee Houston, Junior
Editor-In-Chief: The Free Choice E-zine