IBM PureData System for Analytics, Version 7.1

Load continuation

If you enable load continuation with the allowReplay option, or set the session variable LOAD_REPLAY_REGION to true, the system ensures that a simple load that uses external tables continues after the system is paused and resumed. You do not have to stop and resubmit the load.

If no value is specified for the allowReplay option, or the option setting is 0, the system defaults to the Postgres default setting. If the setting is a valid non-zero number, it specifies the number of allowable restarts.

When you enable load continuation, the system holds the records to be sent to the SPU in the replay region in host memory. After the system sends the data in this region to the SPUs, it does a partial commit that forces all the unwritten data to the disks and allows the system to reuse the data buffers of the reload region. If a SPU reboots or resets, the system rolls back to the last partial commit, and reprocesses and resends the data.

Note: This option has a performance impact which depends on the speed of the incoming data. In addition, system memory is used for the data buffering that enables loads to be continued. When the buffer memory is exhausted, new loads will pend until needed memory becomes available.

Load continuation cannot operate on any table that has one or more materialized views in an active state. Before enabling load continuation, suspend the associated materialized views. You can suspend active materialized views either through the NzAdmin tool or by issuing the ALTER VIEWS command. Sample syntax for ALTER VIEWS follows.

ALTER VIEWS ON <table> MATERIALIZE SUSPEND

After loading is completed, you can update and activate the materialized views for the table. Sample syntax follows.

ALTER VIEWS ON <table> MATERIALIZE REFRESH