A crowdsourced collection of big data challenges and analytics opportunities was on the agenda for the Technical Unconference at Information On Demand. Here’s a sampling of what transpired.
“Businesses, place your bets”
“Business is all about placing bets on what you think will happen next,” said IBM Big Data Evangelist James Kobielus
. He laid out some of the criteria that IT must satisfy and the capabilities they must embrace to turn the odds in their favor. For example:
- Engage Customers as Individuals: Organizations must use every tool at their disposal to drive personalization capabilities into every touch point and form a bond with each customer. Big data and social analytics can help organizations go beyond a 360-degree view of their customers (knowing who they are, what they’ve bought and what they’re worth in the long-term) to a 720-degree view that reveals their undeclared needs and opportunities to satisfy them. “When you develop this capacity, customers will stay with you because you make them happy,” he said. “Listen carefully, then lavishly respond. Intimacy is everything.”
- Go faster: Kobielus placed a premium on the speed and agility of your analytics operations, right down to the code. “Python and PHP didn’t get popular because they were better. They got popular because they’re faster.” Speed and agility are key IT attributes in a volatile market marked by an every-tightening spiral of shorter cycles, faster responses and urgent decisions. Successful organizations are powerful, flexible and faster, he said. They’re production-driven, business-focused and results-oriented. Users will choose sub-par tools if they help them get work done more quickly.”
- Automate your analytics: Predictive models are the backbone of the agile enterprise, Kobielus said. But with so many people needing them and with more and more decisions being automated, how does IT respond? “Automate your analytics,” he said. “Automating your modeling function is the only way to respond at scale.” This automation must also include governance capabilities to refresh the models or replace them with next-best challengers when outcomes start to slide
The two people you’ll meet in every big data project
Tom Deutsch, IBM Big Data Program Manager, gave an overview of the two personality types you’re bound to run into in every big data or analytics project.
"This Changes Everything” Guy: prone to hyperbole and susceptible to the new and shiny. You’ll know him when you hear things like this:
- “Everything older than six months is crap."
- “We’ll figure out the information and metadata lineage later.”
- “Breaking down silos takes too long. We’ll just create a new one."
“This will never work” Guy: obsessed with process and addicted to exactitude. His telltale expressions include:
- “All this new stuff is crap.”
- “We need to get all the governance and metadata right before we start.”
- “I don’t trust anything without a schema.”
These opposing dispositions aren’t going to go away, Deutsch said. Further, each has a valuable part to play on any project. Your job is to find roles for them where they don’t manifest these behaviors. There is a happy, successful and profitable medium between utter chaos and complete control.
On the “irony” of NoSQL
RedMonk analyst Steven o’Grady assuaged fears in the room that emerging NoSQL technologies would not, in fact, render their hard-earned and well-honed SQL skills obsolete. “A lot of people thought NoSQL would replace the relational database. They got scared that they would be replaced.” The truth, said O’Grady, is more sophisticated than that. “The SQL skills that you’ve developed are just as relevant in NoSQL as in SQL,” he said.
O’Grady said another misconception is that NoSQL is a blanket, one-size-fits-all term. Rather, he said, “there are lots of NoSQL tools that have nothing to do with each other. It’s a matter of knowing what each tool is good at and picking the right one for the job.” These new tools provide IT with considerably more options - and opportunities – to build the right solution.
“Before NoSQL, the answer to every question was to run an SQL query against a relational database,” said O’Grady. “Now, you can choose the right tool for each function.”
"We all have our horror stories"
The frustration floodgates opened wide when participants explained how user demands for faster analytics run straight into the persistent (and persistently thorny) issues of data governance and SLAs.
Here’s a sampling (business users, take note: they’re trying to drive better outcomes, too):
- “They’re distinct factions that don’t like to work together.”
- “People always want to go off and do their own stuff.”
- “Users don’t care how clean the data is. They just want all of it so they can do their own ETL. Taking time to clean it frustrates the bejeebus out of them.”
- “It’s difficult to establish service levels in the cloud when a build takes one hour one day and four days the next. My users won’t even wait an extra 30 seconds.”
- “I can teach smart people how to think, but it takes time for them to learn the business.”
- “I kept rubbing my eyes and saying ‘I don’t see the value’.”
- “The approach is generating revenue for us and I don’t want to stop it, but I can’t build a product out of something that only one person knows.”
- “I found a massive Microsoft SQL installation running on someone’s desktop that was running an entire shop floor. We didn’t know it was there until it crashed.”
- “The data becomes accepted, then embraced, then we find out it’s wrong.”
- “You can’t dictate use models when users are in control.”