Infrastructure Optimization: The Key to Big Data Success
Brittany Gotschall 2700050P02 firstname.lastname@example.org | 2013-04-26 10:47:40.0 | 0 Comments | 2,007 Visits
Today, no one person or business department can absorb and analyze all the data that's generated within midsize and large enterprises. In fact, the variety of data from multiple sources is giving IT organizations something to think about: engineering data, healthcare data, transportation/logistics data, and — most noticeable — social media data generated by Web sites and mobile devices. So, new approaches must be developed to gather the multistructured data, to store the data, and to analyze the data in a timely way.
Two key infrastructure competencies found in leading organizations that are able to gain competitive advantage from business analytics are optimization and resiliency.
Optimization is the process of improving system performance by configuring hardware for the specific task in mind. The amount of processing power on the system can be tuned to match a specific workload — such as determining the ideal number of cores and sockets to be housed on the system. Operating systems can be adjusted to run faster on server hardware, and those adjustments can be tested, most often by vendors or IT specialists, to ensure faster processing. Storage and memory can be matched to the specific hardware configuration to ensure that the system is balanced — meaning that there is enough memory and storage to match the processing power aboard the server.
Compute power is important to system optimization, but it isn't the only consideration. Server I/O, network capacity, and storage characteristics all play important roles in supporting overall processing capacity for analytics workloads. Wherever large amounts of data must be moved from other storage resources, the amount of built-in server I/O capacity to move data can become a limiting factor. So, the number of I/O ports (e.g., PCIe interconnects) and the types of server I/O connections selected are key to this ability to expand the server capacity on the datacenter rack. These considerations must be understood at the outset of developing a deployment plan. (Read this IDC paper for a fuller discussion on matching analytics workloads with infrastructure and client deployments examples.)
The next installment will discuss the importance of infrastructure resiliency to analytic applications.