Each of the LSPR workloads is run according to a specific set of rules, to assure that results can be compared with other measurements of the same workload. Neither changes to setup or operation, nor unique tuning activities are done to favor any processor. Some of the measurement methodology concerns include:
- Assure adequate configuration (storage, channels, DASD)
- Distribution of system data
- Distribution of program libraries
- Distribution of data files (datasets) and databases
- Files and databases restored to pristine state
- Logon end-users (terminals or clients), or load job queue
- Determine measurement period to obtain a repeatable sample of work
- Adjust activity to realize target processor utilization level
- Assure steady-state has been achieved
- Capture appropriate performance monitor data
- Capture operator console logs
- Verify that no hardware errors occurred
- Verify measurement data against acceptance criteria
- Construct detailed measurement reports
When a suitable testing environment is not available, analysis is used to address these methodology concerns.
As stated earlier, on-line workloads require some type of terminal or client simulator to generate the workload of an end-user community. There are a variety of products available that can serve this purpose, including IBM Teleprocessing Network Simulator (TPNS). Products like TPNS generally run out-board on a stand-alone processor that is connected to the system being benchmarked with normal teleprocessing hardware. Alternatively, TPNS could be run on the host processor (in-board) along with the benchmark workload itself. LSPR will choose between an in-board and out-board network simulator based on the functionality required and the processing overhead associated with the simulation. Generally, out-board network simulators are used.