Can you believe Watson got the question about Slovenia wrong on Jeopardy?


Slovenia made it to the spotlight again, for the first time after the soccer world cup (when Slovenia was trending topic on Twitter and top search on Google). This time, it happened because IBM's supercomputer Watson competed against human champions in the famous TV show Jeopardy. IBM's computers are known to destroy people in various challenges, Deep Blue beat the world champion Garry Kasparov in a chess tournament in 1997. But chess is simple for computers to play, because it is pure logic and mathematics – the capability of a player is determined by the number of operations and actions it can calculate in advance. But a quiz is a totally different story, where the biggest challenge is semantics – understanding the meaning of words.

In 1950, Alan Turing, one of the greatest pioneers of computing introduced the Turing test, a methodology that could separate humans from computers using a set of questions, some of them formed in such a way computer wouldn't be able to understand and answer them. There are many questions which can't be answered with pure logic, the one I remember from high school goes something like this:

"Jack attended Sally's party, bring a doll. What was the present?"

The catch is in the connection between party – (birthday) – present – doll, which can't be noticed without abstract thinking humans are capable of. And today's computers still face the same problem - even though Watson dominated Jeopardy, it failed miserably on the following question about Slovenia:

"As of 2010, Croatia & Macedonia are candidates but this is the only former Yugoslav republic in the EU"

Watson's computing capabilities and knowledge banks are huge, but a question and an answer so obvious to humans presented a huge problem. Watson surely knows which countries are EU members, but it obviously didn't understand the question, thinking it was asked about which country would be next to start negotiating for EU membership, answering Serbia. The right answer was, of course, Slovenia.

The video is also fascinating from the cultural point of view – and extremely creepy. Those who have watched (or read) "2001: A Space Odyssey" may have experienced a slight shiver and carefully waited if Watson would say it: "Hello Dave". Others might have enjoyed this science fiction presentation, but besides Watson's obvious advantage in being the fastest to answer the question, it's clear that computers are still far away from being intelligent. And hopefully they will stay that way.

Like what you read? Now tell the world!
Vote on Hacker News

A few more things you might find interesting:

written 9.3.2011 8:33 CET on chronolog
2338 views   •   2 likes   •   Like   •   

Connect with :