With the ebb and flow of today’s fickle economy, the demand for better informed decisions has never been stronger. Usage of business analytics and the underlying data warehousing foundation is growing at an astounding rate. In response, revolutionary technologies are emerging that make advanced analytics easy and affordable for organizations of any size. Businesses must address these challenges and work to achieve on-demand access to insight. Some of these challenges include bottlenecks in capturing and loading operational data that slow the ability for businesses to react in a timely manner as well as performance overheads in handling heavy workloads and complex queries for analytics processing. If they succeed in addressing the challenges, businesses can target smaller customer segments and communicate with them about their individual needs and wants, while driving new market opportunities within the current business landscape. They can identify and capitalize on even the smallest trends, attaining competitive advantages normally only realized by more flexible and dynamic smaller businesses. They can detect small behavior patterns that can have a significant influence and impact on the business in terms of revenue, expenses, and growth. Most important, they can build competitive strategies around data-driven insights and, in the end, generate impressive business results.
InfoSphere Warehouse V10.1 can help you to reduce the complexity of data warehouse management, accelerate business decisions, ease the effort and expense of deploying new functionality, and take advantage of performance and compression improvements to help lower your overall data warehouse cost and increase warehouse management efficiency. InfoSphere Warehouse Advanced Enterprise Edition V10.1 includes the additional tooling functionality to manage large scale warehouse deployments with additional performance, administration, backup, unload, recovery, and development tooling. In addition, all InfoSphere Warehouse Advanced Editions in the InfoSphere Warehouse Packs deliver the industry models and IBM Cognos reports to easily deploy a warehouse environment for customer, market and campaign, and supply chain insight.
Some of the highlights from this release:
- Continuous Data Ingest enables you to transparently load data from external sources into InfoSphere Warehouse databases without downtime, while currently performing real-time business analysis and decision making.
- Faster performance for BI style workloads with the addition of zigzag join and other optimizations v10 helps to significantly reduce the time for complex multi-dimension business queries. Enhanced query joins and optimizer enhancements help increase query performance of other analytic queries and reduce the need for additional indexes.
- Adaptive compression, the latest addition to DB2's industry-leading database compression technology, can further help reduce your storage costs while at the same time help to improve performance (especially for large I/O bound warehouse applications and query workloads).
- Multi-temperature storage helps businesses to leverage different types of storage to maximize performance at the lowest possible cost with integrated management of storage groups to migrate data to the appropriate class of storage.
- Row-and-Column Access Control provides you with easy and flexible rule and role definitions to manage and control data accesses designed to help enhance security and simplify application development. These security features provide you with a robust and flexible set of rules and access controls to manage and help secure data accesses that help reduce your security risks.
- PL/SQL compatibility features helping to reduce costs and time to market. SPARQL support enables new types of "no SQL" applications development with Resource Definition Framework (RDF) data stores.
- Time Travel Query is integrated into DB2 10 and InfoSphere Warehouse allowing for easier and faster time-based (historical trend based) analytics applications. Allows for developers to more easily develop and deploy historical based analytic solutions.
Looking a little closer at all the new features of InfoSphere Warehouse v10.1.Optimized storage utilization
Multi-Temperature Data Management allows for grouping of hot, warm, and cold data on varying storage systems. You can now use different storage technologies to help optimize performance at the lowest possible cost. Storage can be allocated depending on data temperature, such as placing hot data on the fastest and most expensive Solid State Disks (SSDs), while other data can reside on less expensive storage systems. Multi-Temperature Data Management is integrated into the DB2 Workload Manager to more easily prioritize queries, not only on "who" submitted the work or "what" type of work it is, but also on "where" the data resides. For faster performance, queries touching only hot data can be assigned higher prioritization than queries touching cold data. More information can be found in the article: InfoSphere Warehouse 10 and DB2 10: Optimizing your storage investmentFurther storage savings with improved compression
Significant enhancements to the industry-leading compression technologies of InfoSphere Warehouse come with the new Adaptive Compression technology, which can help further reduce your storage needs. The enhancements are designed to deliver efficient compression of high amounts of new and changing data. The improved compression ratio further helps reduce storage needs and allows for more data in memory, therefore, helping to increase performance. The new approach used in Adaptive Compression also reduces the need for table reorganization. As a consequence, overall maintenance of compressed data is reduced, helping you achieve additional cost savings. More information can be found in the article: Reduce Your Storage Cost With InfoSphere Warehouse 10 and DB2 10 Easier business trend analysis and application development
Time Travel Query, based on the SQL 2011 standard, is integrated into the database engine and can help provide significant performance and manageability advantages. Time Travel Query provides the infrastructure for time-aware analytics to reduce the operational complexity of collecting and analyzing time-based data. For both new and existing applications, Time Travel Query accelerates dynamic analysis of business trends and changes over time. Time Travel Query provides the flexibility to audit for change with system-controlled temporal data, and to analyze all kinds of temporal data including forward-looking data when using user-controlled temporal data. More information can be found in the article: Time Machine for Data!SPARQL and RDF datastore functions
InfoSphere Warehouse V10.1 can now be used as the Resource Definition Framework (RDF) datastore with built-in query processor and SPARQL support. With this support, you will be able to build highly scalable and reliable semantic data applications on the InfoSphere Warehouse platform. Application developers will have a flexible and expressive API to develop or be able to prototype applications rapidly where the schema is changing or where there is a wide spectrum of disparate entity types to be stored with evolving relationships between them. SPARQL query features, with open source Jena API, are supported. Utilities to create, load, and reorganize.Enhanced security features
Row and Column Access Control provides easy and flexible control over security access. With Row and Column Access Control, you can define the security policies that are optimal for your environment by creating permissions for rows and masks for columns based on rules defined for users, groups, and roles. This very fine granularity is designed to allow you to easily control access by roles such as departmental, supervisory, database administrators (DBAs), system administrators, and others, and allow for full separation of duty between security officers and DBAs. More information can be found in the article: DB2 10 Takes Security and Ease of Use to the Next Level Performance enhancements
InfoSphere Warehouse V10.1 adds many enhancements to help further increase the performance of queries. Zigzag join is now available and is designed to deliver sub-second response time on complex multidimensional queries. Other join methods are also enhanced for memory usage efficiency, early predicate evaluation, support of expressions in predicates, as well as optimizer costing and query plan optimization. Additionally, you may now set the INTRA_PARALLELISM variable dynamically to optimize performance on multicore systems for specific applications or workloads. More information can be found in the article: Faster Query Performance with InfoSphere Warehouse 10 and DB2 10
and InfoSphere Warehouse 10 and DB2 10: Multi-core Parallelism Continuous Data Ingest
Continuous Data Ingest allows you to add data to the data warehouse through two mechanisms: pipes and files. Using files to add data into the data warehouse allows for external sources to seamlessly ingest data in the warehouse efficiently and effectively. This gives the warehouse the ability to continuously receive data from operational external data sources in a more timely and active fashion. This seamless operation is due to the fact that it does not make a difference if the files were created by an external application or by DB2. For operational data warehouse environments, it provides for a nearly real-time responsiveness to the changing and dynamic business environment. More information can be found in the article: High speed, online, continuous data ingestAdvanced tools for managing your data warehouses
Updates to InfoSphere Optim Tools help enhance the value of key InfoSphere Warehouse features including Time Travel Query, Row and Column Access Control, and Multi-Temperature Data Management. InfoSphere Optim Tools may allow you to more easily deploy, develop, and administer InfoSphere Warehouse and take advantage of the new features of V10.1 in empowering organizations to make active, timely, and informed decisions as business events occur.
Using InfoSphere Optim tools, developers, DBAs, or security administrators can:
- Design, develop, and deploy temporal tables, develop and validate SQL access, and view row history.
- Use Data Studio and InfoSphere Data Architect for collaboration when using security features, and, therefore, enable security administrators to define, view, and deploy fine grained access control, and DBAs to analyze impact of new security controls and update routines accordingly.
- Use IBM Data Studio to monitor multiple standby availability and alert availability risks.
- Use Data Studio and Optim Configuration Manager to get recommendations for adaptive compression use, deploy compression, and validate compression savings.
- Use Optim Configuration Manager and Optim Performance Manager to enhance value of multi-temperature technology delivery by analyzing storage group growth and contents, enabling Workload Manager configurations to prioritize queries against high performing storage, and automating period roll-off for range partitioned tables to help lower storage costs.
- Analyze access plan improvements and regressions based on new optimization strategies and get actionable recommendations for improvements. Optim Query Workload Tuner is now the one-stop-shop for query improvement recommendations subsuming all DB2 Design Advisor functions under its broader value-add capability.
- Use Optim High Performance Unload to unload, extract, and repartition data in InfoSphere Warehouse V10.1 databases. It helps to work with large quantities of data with less effort and faster results.
- Use DB2 Merge Backup to help create the fastest possible InfoSphere Warehouse backups for your production system so that you have full current backups available for shorter restore times. DB2 Merge Backup combines incremental and delta backups with the latest full backup to create a new and current full InfoSphere Warehouse backup.
- Use DB2 Recovery Expert to recover faster and with greater granularity than what is offered with traditional database recovery. The log analysis features enable organizations to monitor changes to a database and gives the DBA the capability to quickly restore or correct erroneous data using less resources and reducing data outages.
Additionally, Data Studio and Optim Query Workload Tuner are now tightly integrated to provide an improved startup experience. Users can now enable Optim Query Workload Tuner functionality with a simple license activation. Data Studio, which includes a rich set of developer and database administrator productivity tools, is the replacement of the legacy DB2 Control Center tools. Data Studio is available for all editions of InfoSphere Warehouse at no extra charge.
Over the coming days we will start to dig in deeper into the technology behind these new features and how to apply them to solve the complex warehousing challenges facing business and customers today.