The World’s First Supercomputer to Compete at Jeopardy!
Back in the mid-1990s, IBM was a joke in the supercomputing field, and Cray and Silicon Graphics were building the truly innovative and powerful machines and cheap x86-Linux clusters were just starting to get traction. IBM’s Power2 and PowerPC processors were 32-bit chips with nothing much impressive about them and its 64-bit PowerPC 620 and 630 parts were utter failures and would not come out for several years. In the past decade, IBM has put its system engineers, scientists working at IBM research facilities around the globe, and numerous supercomputing experts from government and academic labs to build a portfolio of different parallel computing platforms, including the massively parallel BlueGene to the hybrid x64-Cell blade architecture embodied in the “Roadrunner” to giant clusters of its commercial Power Systems such as the future “Blue Waters” Power7 monster.
Now, IBM is no longer content to merely crush the spirits of chess masters like Garry Kasparov thanks to the Deep Blue parallel supercomputing (in 1996, it lost, but it won a six-game match by one game — two wins for Deep Blue, one for Kasparov, and three draws — the next year) and have started working with US television quiz show Jeopardy! to create a supercomputer that will undoubtedly answer questions as accurately as ever.
The Blue Gene supercomputer, dubbed simply as a “Question Answering” system, is named Watson. The Watson QA system software has been in development for the past two years and is based on open source code created by IBM’s Software Group called Unstructured Information Management Architecture (UIMA), which is available as an Apache project. Designers believe it will have the speed and “understanding” necessary to research, buzz in, and then answer questions fast enough to compete on the popular game show.
To win on the long-running show, contestants need good knowledge across a range of topics, and must quickly recognise unusual and unexpected patterns in that information. But more than that, they must do so in response to a question that might be posed in a confusing format.
That makes IBM’s challenge a difficult one, says Tristan Cazenave, an AI expert at the University of Paris VIII. “In contrast to chess and go, Jeopardy! is not a closed world,” he says. “It is based on natural human language understanding, which is very difficult for computers, so it is a real artificial challenge.”
So Watson’s success depends as much on its ability to understand and respond to the subtleties of human language as it does on the extent of its knowledge database. “We’re trying to get the computer to deal with natural language more effectively,” says David Ferrucci, Watson project lead and the principal researcher working on the Watson QA software and the iron that will support it. “Since Jeopardy is such a large domain, it is like we are trying to get the computer to study. Of course, the challenge is that the game has such a broad domain and people play with such confidence”.
The Watson QA system is based on various forms of textual data (books and other types of authoritative data) that will be pumped into the system and organized based on the general Jeopardy question categories. Most of the system is programmed in Java, according to Ferrucci, but there is a smattering of C++ where performance is critical and Prolog is also used for some of the rules relating to textual analysis.
Just like the Deep Blue chess playing supercomputer stored a gazillion possible chess games and moves and sorted through those for the best possible moves based on where it was in a real game, the Watson QA system is being fed with likely statements Jeopardy game host Alex Trebek will make and the possible questions relating to those statements. (In some bases, past Jeopardy statements are being pumped into the machine, in fact). The UIMA framework is what is used to do deep analytics on the raw data ahead of time, and it is also used to parse the statements to “understand” them.
The exact configuration of the Watson QA super has not yet been determined, and it is not clear if the machine will be equipped with electronic ears to hear and speakers to talk as it responds. The machine will be put in the Jeopardy studios to take on people live in 2010 and it will be disconnected from the Internet - no cheating with Google allowed by humans or computers.
The final question is Will Watson QA win at Jeopardy? “We’ll see,” says Ferrucci with a laugh. “We certainly wouldn’t be in the game if we didn’t think we had a good chance.”
Source: Gizmodo.com; Newscientist.com;