How does an organization generate new insights from big data in a timely and cost-effective manner? That is a core challenge facing many organizations today.
IBM® InfoSphere® BigInsights™, a core component in the IBM big data platform, can help address that challenge. BigInsights builds on open-source Hadoop with innovative capabilities that help organizations cost-effectively analyze large volumes of data.
Recent enhancements in BigInsights are designed to address some key performance and data management challenges inherent in Hadoop.
New in BigInsights 2.1 is a feature called Adaptive MapReduce that can be enabled at install time to deploy an alternative Hadoop scheduler. Basically Adaptive MapReduce is a single-tenant version of Platform Symphony. Using the Adaptive MapReduce scheduler, customers can see dramatic performance gains for some types of workloads. There is a compelling short video on YouTube that shows the relative scheduling performance of Adaptive MapReduce compared to the open-source Hadoop scheduler here - http://www.youtube.com/watch?v=JtfaXycB0sM. Customers that need additional capabilities that Symphony offers such as resource sharing, SLA controls or reporting can optionally license IBM Platform Symphony on their BigInsights cluster.
BigInsights 2.1 Enterprise Edition includes IBM's General Parallel File System (GPFS) with a File Placement Optimizer (FPO) feature that gives users the option to deploy an enterprise-class POSIX file system in place of open-source HDFS. This feature makes the file system easier to manage and much more flexible since data can be shared by a more diverse set of applications. Users also eliminate the need to employ specialized Hadoop commands because the distributed file system can be manipulated using standard OS level commands and utilities. Also, the file system is far more reliable since it removes the reliance on the namenode, a single-point of failure in HDFS, by striping metadata across multiple disks in the cluster.
Unlike HDFS—which is optimized for large-block I/O—GPFS is flexible enough to support a variety of different access patterns, including applications with small or medium-size blocks as well as write-intensive applications. As a result, GPFS can provide better performance across a wider range of applications.
For a more detailed look at these specific enhancements to IBM BigInsights 2.1, and to learn how customers can use BigInsights with other optional products from IBM Platform Computing, download our new white paper here: