IBM System x and Big Data
Interview with Phil Horwitz, IBM Senior Engineer,
Workload-optimized Systems and Big Data Technical Lead
Q: What is big data?
Phil: Basically, it’s data that comes from everywhere: from posts to social media sites, digital pictures and videos posted online, cell phone GPS signals and transaction records of online purchases. These are just a few examples. Companies are overwhelmed by this data. It’s important because it represents an opportunity for companies to gain insight from this all this data that can help their business be more agile, and to help them answer questions that in the past were beyond reach. Until recently, there was no practical way to harvest this opportunity. We are seeing that companies really get the importance of big data. I read an IDC article not too long ago that said the market for big data technology and services is expected to be more than $16B by 2015. That means businesses are investing in big data technology. I should mention that the big data hardware infrastructure is primarily on System x servers.
Q: How does your work contribute to advancements in big data capabilities?
Phil: I’m in a new IBM group called the Systems Optimization Competency Center (SOCC). It’s a development organization staffed with technologists who have diverse skills and expertise in hardware, software and services. Collectively, the team has decades of experience working with clients on data warehousing, OLTP and analytics, which gives us a great understanding of client data centers and workloads. Our team has this unique client-workload perspective, and we are able to collaborate across IBM on solution design, architecture, prototyping, experiments and proof of concepts. Our goal in the SOCC is to produce integrated and optimized solutions that target specific client workloads to create balanced offerings with leadership components and capabilities. We do this by driving improvements across a wide spectrum of optimization characteristics.
Right now, our team is aggressively pursuing big data optimization opportunities on System x. We are driving initiatives with the big data teams in hardware/software organizations and IBM Research to create optimized solutions for this environment. These initiatives will optimize performance, scalability and consumability, and recommend management components to simplify monitoring and deployment as well. We are also doing work to measure and analyze our storage and network subsystems to provide the best possible solutions for System x big data workloads.
Q: Can you give me some examples of what clients are doing in the big data space?
Phil: We are working with System x clients in all industries to help them create value from their data and analytics in areas like sentiment analysis, fraud detection, financial processing and social media to name a few. Retail is another interesting area. We have a large retailer that is using data from e-mail, call centers, kiosks, point of sale and online chats from their web site for better inventory planning, supply chain optimization, real estate planning and customer relationship management among other things. So basically, they have a better understanding of what products they should stock, how much of each product to stock, how much space should be allocated to each product category, where to locate stores, how to grow their customer base and even how they can more accurately target the right customers to make their marketing campaigns more effective. Another example would be a large insurance company we are working with who is now doing enhanced risk analysis for loans and insurance polices to set pricing more competitively and bringing in law enforcement and weather and catastrophe data to do analysis on high-hit areas.
Q: What resources are available for clients wanting to deploy big data solutions on System x hardware?
Phil: I would definitely check out the IBM big data web page to get a view of the big picture and take advantage of all the great resources there. You can also read more about what other clients are doing, which I personally always find very interesting. Specifically to System x, I would recommend this System x and software analytics page. You’ll find information about System x solutions for SAP HANA. Also, we recently just released System x reference architectures for IBM InfoSphere Streams, which handles unstructured data in motion, IBM BigInsights for structured and unstructured data and other Apache Hadoop implementations like Cloudera CDH. Reference architecture solutions are built based on IBM expertise and give you a standardized blueprint of IBM hardware, software and services that make the solution easy to order and implement.
Connect with Phil on LinkedIn.