IC76984: TIVOLI STORAGE MANAGER SERVER CAN CAUSE CLIENT BACKUP SESSIONS T O STAGNATE IN A RUN STATE.

IBM Tivoli Storage Manager (TSM) server V6.2.3.x interim fix downloads

APAR status

• Closed as program error.

Error description

• Tivoli Storage Manager client backup sessions may stagnate in
  a RUN state on the Server.  For example, QUERY SESSION can
  report sessions similar to the following:
  .
  Sess   Comm.  Sess  Wait Bytes  Bytes    Sess Platform  Client
  Number Method State Time Sent   Recvd    Type           Name
  ------ ------ ----- ---- ------ -------  ---- --------  ------
  12,345 Tcp/Ip Run   0 S  34.2 M  73.2 K  Node  WinNT    NODE1
  12,346 Tcp/Ip Run   0 S  29.1 M   6.0 K  Node  WinNT    NODE2
  12,347 Tcp/Ip Run   0 S  29.3 M 244.8 K  Node  WinNT    NODE3
  12,348 Tcp/Ip Run   0 S  29.5 M  17.3 K  Node  WinNT    NODE4
  12,349 Tcp/Ip Run   0 S  716      2.0 G  Node  WinNT    NODE5
  .
  It's important to monitor the QUERY SESSION output over a
  period of time.  This APAR is applicable when the session
  remains in a RUN "Sess State" for long periods of time (hours),
  but the "Wait Time" remains 0 seconds and the "Bytes Sent" and
  "Bytes Recvd" categories do not increase over that same time
  period.
  .
  This behavior can prevent client backups from completing since
  the backup sessions are stalled. The client backup sessions may
  eventually timeout with the following messages in the Server
  activity log (or they may have to be manually cancelled via
  CANCEL SESSION from the server if they are not progressing):
  06/01/2011 01:02:03 ANR0538I A resource waiter has been aborted
  06/01/2011 01:02:03 ANR0444W Protocol error on session 12,345
                      for node NODE1 (WinNT) - out-of-sequence
                      verb (type StreamMarker) received.
                      (SESSION: 12345)
  .
  Customer/L2 Diagnostics:
  This issue is caused when a cache storing fetched database
  information becomes too large to provide a performance benefit
  during client backup operations.
  .
  Collection of the following diagnostic output at 10 minute
  intervals during the backup window can aide in Support
  diagnostics:
  .
    QUERY SESSION
    SHOW THREADS
    SHOW TXNT
    SHOW FETCHCACHE
  .
  QUERY SESSION can be used to determine if threads remain in a
  RUN state for a significant period of time without moving data.
  SHOW THREADS and SHOW TXNT can be used to determine what the
  backup sessions are doing and against what database tables.
  The SHOW FETCHCACHE output can report statistics that are
  useful in determining if the cache is providing a performance
  benefit or hindrance.
  .
  Initial Impact:
  High
  .
  Tivoli Storage Manager Versions Affected:
  All 6.1 and 6.2 Tivoli Storage Manager Servers.
  .
  Additional Keywords:
  ZZ61 ZZ62 TSM BACKUP HANG RUN ANR0538I ANR0444W HUNG FETCH
  CACHE SESSION QUERY FETCHCACHE
  .
  

Local fix

• Disabling fetch caching can provide temporary relief for this
  problem until the official fix is available.  Contact Support
  to determine if this is an applicable approach.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED: All Tivoli Storage Manager server users.     *
  ****************************************************************
  * PROBLEM DESCRIPTION: See error description.                  *
  ****************************************************************
  * RECOMMENDATION: Apply fixing level when available. This      *
  *                 problem is currently projected to be fixed   *
  *                 in levels 6.2.3.10, and 6.2.4. Note   *
  *                 that this is subject to change at the        *
  *                 discretion of IBM.                           *
  ****************************************************************
  *
  

Problem conclusion

• This problem was fixed.
  Affected platforms:  AIX, HP-UX, Solaris, Linux, and Windows.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels

• R61A PSY

     UP

• R61H PSY

     UP

• R61L PSY

     UP

• R61S PSY

     UP

• R61W PSY

     UP

• R62A PSY

     UP

• R62H PSY

     UP

• R62L PSY

     UP

• R62S PSY

     UP

• R62W PSY

     UP