IBM's Watson computer takes the Jeopardy! challenge
Typing a query into an Internet search engine may give you thousands of results, but how do you know which answer contains the most meaningful information? For now at least, the answer is that you don't. The computer can't tell you that—yet. David Ferrucci, principal investigator for the DeepQA project at IBM, is leading the development of a computing system called "Watson" that is intended to propel computer-based question answering technology to a point where it clearly and consistently rivals the best human performance. In the latest IBM grand challenge, Watson will face the ultimate test of its capabilities when it competes against human beings in an upcoming episode of the TV game show Jeopardy! Regardless of whether Watson wins or loses, Ferrucci says Watson technology may eventually drive business intelligence, analytics and information management by helping decision makers find the precise information they need from the mountains of data they produce.
Read excerpts from the interview below and listen to the podcast for the full interview.
ForwardView: Tell us a little bit about your role at IBM. When did IBM's Watson project first begin, and what has your involvement been?
David Ferrucci: I'm a research staff member at IBM and the leader of the Semantic Analysis and Integration Department. My team focuses on analyzing natural language text to extract useful knowledge. Around 2004 there was a lot of interest in finding the next grand challenge for IBM. IBM had built Deep Blue, which beat a grand champion at the game of chess, but what would be next? Charles Lickel, an IBM vice president, had witnessed all the excitement around the winning streak of Ken Jennings and wondered if it was even feasible for a computer to take on Jeopardy! I had great interest in the area of artificial intelligence, question answering and other aspects of automated reasoning, but it was not until 2006 that the opportunity came my way to take on the challenge of Jeopardy! I suggested that I get a few people together and we do a feasibility study. The result looked like while it was really, really hard, but possible and just too good of an opportunity to advance the science to pass up. IBM backed the plan to take it on and we were able to kick off the project in 2007.
ForwardView: How is Watson different from Deep Blue, the IBM supercomputer that beat chess grandmaster Garry Kasparov in 1997?
David Ferrucci: When you think about chess, it can be cast as this well defined mathematical problem. Every piece on the board, every rule, every move is perfectly defined. It's unambiguous. There's no room for interpretation or uncertainly about what the nature of the problem is and how it works. Deep Blue of course needed to be a powerful enough computer to search enough moves in advance that it could figure out what's the next best move to make to win the game. But there was no ambiguity or external context or uncertainty to deal with. The problem was finite and well defined. Human language is entirely another story. To get precise about answering questions you need to do a better job of understanding the language, which is full of ambiguity and nuance.
ForwardView: Talk about the technology that makes Watson possible. What is Watson made of?
David Ferrucci: There's no single formula that makes a computer understand and answer natural language questions. It's really this huge combination of smaller algorithms that look at the data from many different perspectives, and they consider all sorts of possibilities and all sorts of evidence. Watson brings all these various algorithms together to judge that evidence with respect to every one of its possibly hundreds or thousands of different answers to decide which one is most likely the correct answer, and ultimately computes a confidence in that. And if that confidence is above a threshold, then Watson says, "Hey I want to answer this question. I want to buzz in and take the risk."
ForwardView: How has the field of question-answering systems progressed over the years? What makes Watson different from previous efforts in the field, or even Internet search engines like Google?
David Ferrucci: There's this extreme precision required to not just deliver documents that contain the words in your query, but to better interpret the question and deliver the precise and correct answer. What's interesting about Jeopardy! as a driver of technology is that there are no points for second or third place. You can't just throw something up there and say, "Hey you might like this." So Watson produces very accurate confidences-not only does it actually pinpoint the precise answer, it produces a confidence in the answer by deeply considering many different dimensions of evidence. It's reading the content that it has available to it much more deeply to try to understand that natural language content well enough to say, "I think I really have the right answer here-I'm 80 percent sure. I'm going to take the risk and buzz in for that."
ForwardView: Tell us about the Jeopardy! Challenge. What are your goals for Watson to compete on this TV game show?
David Ferrucci: While the driving objective is to play to win, we don't expect to win every game. There are just too many variables in the core language challenges themselves. There's also the luck of the draw. When Watson plays Jeopardy! it's completely self-contained. So not only does it have challenges in understanding the question and understanding the content it's reading and relating the two, there are just answers it may not have anywhere. So it really depends on what comes up, how it's phrased, what information Watson has access to and how well it can understand it. In the end, we expect to demonstrate that we can compete with the best humans by understanding and answering these Jeopardy! questions and in using confidence estimations (what we know and don't know) to manage our risk.
ForwardView: What is it about "intended meaning" that makes human language so hard for computers to understand?
David Ferrucci: Because we're so steeped in human experience and human cognition, we're so used to using words and using language, we're so used to making assumptions about the real implications of our words, we don't realize how hard it is to throw a computer into that mix that has no way to ground any of these terms. For a computer, there is no connection from words to human experience and human cognition. The words are just symbols to the computer. How does it know what they really mean? So I think that's what the challenge is when we say "intended meaning."
ForwardView: How do you see the Watson technology evolving and being used in other fields in the next 5 to 10 years? What are some other possible applications of this technology?
David Ferrucci: Whether it be in healthcare, medicine, technical support, finance, or government, the ability to sift through and more deeply analyze all this information and deliver it on an as-needed basis I think can really be dramatic. When you think about the Star Trek series, Captain Picard or Captain Kirk just speaking to the computer and the computer immediately has a sense of what's the context, what is he asking me about, what are my follow-up questions, how to behave as an information-seeking tool that helps this person get at what they need rapidly through a natural language dialogue in their terms. So the user doesn't have to figure out what the right query is. Rather the user can just start dialoguing with the computer in its own terms, in natural language, and that, I think, can ultimately have huge impact on business and society.