By Anthony Sofia, z/OS Software Development
The goal for zEDC Express was to augment the existing System z compression technologies to add a method of low latency standard compliant compression to allow for additional use case coverage. A couple of examples include
Cross-platform communication of compressed data – Current System z hardware assisted compression does not provide cross-platform compatible file formats. The zEDC Express works with the DEFLATE file format which has implementations on most platforms and architectures.
Compression of first-shift data – There are cases where compressing data during transactional workloads is not acceptable due to elapsed time and CPU consumption. The high throughput of the zEDC Express can provide the ability to compress this data.
Designing interfaces that allow software to efficiently use hardware resources can be challenging. How do you plug into existing infrastructure? What changes will new users need to undergo to see value? How will new use cases grow organically from the initial interfaces?
These are all issues we encountered with the design for the zEnterprise Data Compression offering. It was clear that the way we allowed for access to the new compression technology would guide the adoption rate of it by both IBM software as well as ISVs. The first step to making these decisions was to survey existing application interfaces that provide compression. Two very quickly stuck out: the java.util.zip package and the zlib Open Source library. Both of these provide cross-platform compression based on the DEFLATE file specification. As it turns out the java.util.zip support uses zlib under the covers. This made zlib a natural point to provide access.
So a decision was made. The zlib library would be enabled to transparently use the zEDC Express. This would allow for any applications that use zlib today to compatibly re-link their applications and be available to use zEDC Express. Also by maintaining the existing zlib interface new zlib use cases could start being developed in advance of the zEDC Express enabled zlib. The zlib implementation would also be used to provide a Java runtime which would use zEDC Express via the java.util.zip classes!
There was one caveat in that there is a start and stop cost associated with using the zEDC Express. The solution is to use buffers large enough where those costs are amortized. To prevent an application from using the zEDC Express for these streams logic exists in zlib to identify and route those streams to the software based compression/decompression implementation.
Even after this work it was identified that there was still a set of use cases where a lower level interface would be required. For authorized callers (for example z/OS BSAM/QSAM) it was unnecessary from a design standpoint to use the functions provided by zlib. To handle this set of use cases an authorized API is available which provides a more basic interface than zlib.
This duel interface approach was challenging to develop but the benefits of it will be seen over time as more products take advantage of the new zEDC Express capabilities.
Learn more about zEnterprise Data Compression: http://www-03.ibm.com/systems/z/hardware/features/
Anthony Sofia is a Software Engineer in the System z development organization in Poughkeepsie and was the development lead for the z Enterprise Data Compression software support. Anthony has worked in System Z development for 10 years and has experience in several areas of z/OS and System z.