2011 in review: A computer named Watson
Delaney Turner 270003RQ8K Delaney.Turner@ca.ibm.com | | Tags:  ibmwatson information-insights ibm100
0 Comments | 4,974 Visits
In February, we saw the future of analytics.
By October, that future had arrived.
In the months between, an IBM computer system named “Watson” would best top Jeopardy! champions Ken Jennings and Brad Rutter. Slate Magazine would name Watson Principal Investigator David Ferrucci one of America’s 25 best real-world problem solvers. And for documenting Watson’s long road to reality , the web site Strategy + Business would name Steven Baker’s Final Jeopardy! one of the three best business books of 2011 (The top book was Kevin Kelly’s What Technology Wants, which I wrote about here).
Like Deep Blue in 1997, IBM took on Watson as its latest Grand Challenge – the ambitious research projects that push science in ways not thought possible. Should they succeed, as did Deep Blue, they reveal new insights into the power of computing, showcase the expertise of IBM Researchers and open new avenues of business and technological innovation.
For this Grand Challenge, IBM chose the scientific field of Question and Answering (“QA”). In building Watson, IBM researchers were to build a computing system that rivals a human’s ability to answer questions posed in natural language with speed, accuracy and confidence.
It wouldn’t be easy. And it would be would be tested on prime-time TV.
The Jeopardy! format provided the ultimate test of Watson's abilities. As anyone who’s watched the show could tell you, the game’s clues involve analyzing subtle meaning, irony, riddles and other complexities of language. Humans are good at this, computers are not. Plus, the questions could touch on any conceivable subject, and Watson was limited to what its creators put into its memory.
Building Watson involved asking many additional challenging questions: What semantic technologies would be needed to understand unstructured data? How do you build a system based on information as opposed to transactions? How do you build a system that can learn instead of being programmed?
Four years and 27 PhDs
To build Watson, IBM brought together 27 PhDs from various streams of research – natural language, parallel processing, big data systems, system architectures. Few team members had ever worked together before.
Over a period of four years, they fed Watson mountains of information, including text from the World Book Encyclopedia, Wikipedia and books from Project Gutenberg. All told, Watson held the equivalent of about one million books worth of information. The team also wrote (and rewrote) the algorithms that let Watson break down each question into its key components and assess its confidence in each of the potential answers. To power Watson, they chose a cluster of Power 750™ computers—ten racks holding 90 servers, for a total of 2880 processor cores running DeepQA software and storage. Then, through 55 sparring matches with former Jeopardy! champions, they tested and tweaked, tested and tweaked, tested and tweaked.
New frontiers of analytics
I wrote about Watson a lot. Specifically, I was interested in what it meant for business analytics. There’s something beautiful – and no doubt valuable to a business – about data that comes in neat and tidy rows. Sadly, little about business is either neat or tidy. Now, a full 80 percent of an organization’s data is unstructured; volumes are growing at an exponential rate. Organizations need to understand not only at what their internal systems are telling them, but what their customers, partners and the market is telling them, too. The problem is that their computing systems aren’t set up to handle this new reality. If Watson could interpret the twisted logic of a Daily Double and answer with confidence, the analytical possibilities for interpreting and unlocking the business value of unstructured data would be endless. If you could use language, you could use Watson.
Like a knife through butter
The Jeopardy! Challenge aired over two days starting on February 14 and for those two days I marveled as Watson ran through the categories like a hot knife through butter. By the end of the first day it had accumulated $35,734 to Rutter’s $10,400 and Jennings’ $4,800. By the end of the game, Watson had racked up $77,147, besting Jennings' $24,000 and Rutter’s $21,60. IBM donated the $1M grand prize to charity, with equal donations to World Vision and the World Community Grid. Likewise, Jennings and Rutter donated half of their winnings ($300,000 and $200,000, respectively) to charities of their choice..
Afterwards, Jennings would observe: [T]here's no shame in losing to silicon [...] After all, I don't have 2,880 processor cores and 15 terabytes of reference works at my disposal—nor can I buzz in with perfect timing whenever I know an answer. My puny human brain, just a few bucks worth of water, salts, and proteins, hung in there just fine against a jillion-dollar supercomputer. Jennings was also surprised to learn that he was, in fact, the actual inspiration for project:
Watching you on Jeopardy! is what inspired the whole project," one IBM engineer told me, consolingly. "And we looked at your games over and over, your style of play. There's a lot of you in Watson." I understood then why the engineers wanted to beat me so badly: To them, I wasn't the good guy, playing for the human race. That was Watson's role, as a symbol and product of human innovation and ingenuity."
Putting Watson to work
The lights had barely dimmed on the studio before discussions turned to commercial applications for Watson. In keeping with its Grand Challenge, IBM chose healthcare, and announced IBM Content and Predictive Analytics for Healthcare at last October’s Information On Demand (IOD). At the press briefing, newly appointed IBM GM of Watson Solutions Manoj Saxena said IBM chose healthcare first because of its ability to make a direct improvement in peoples’ lives: “Watson has tremendous potential for applications that improve the efficiency of care and reduce wait times for diagnosis and treatment by enabling clinicians with access to the best clinical data the moment they need it."
Consider the stats:
Watson will transform for the better the way healthcare is administered, delivered and paid for, said Saxena. Watson's ability to analyze the meaning and context of human language, and quickly process vast amounts of information to suggest options targeted to a patient's circumstances, can assist physicians and nurses, in identifying the most likely diagnosis and treatment options for their patients.
It’s been fascinating for me to follow Watson’s progress from game show contestant to business solution. Watson adds another important dimension to the interplay between technology and humanity. In focusing on healthcare out of the gate, we saw the mission of IBM reflected once again – not only to make a profit, but to make a difference. On a smarter planet, Watson may soon be an indispensable asset to the medical profession, making a positive difference in the lives of thousands of patients.
Tweet this post using #ibmwatson or #ibmsoftware and follow IBM Watson on Facebook