IC64192: TIVOLI STORAGE MANAGER SERVER INSTANCE BECOMES UNRESPONSIVE WHEN SECOND INSTANCE STARTED WITH ENVIRONMENT OF FIRST INSTANCE.

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • When a second Tivoli Storage Manager server instance is started
    using the environment of an already running first instance, the
    first instance becomes unresponsive or may crash. When this
    happens, the already running server instance may not accept
    anymore connections and will stop doing any work.
    Whether the Tivoli Storage Manager server instance becomes
    unresponsive or crashes depends on the work being done by the
    instance at the time the second instance is started.
    
    Customer/L2 Diagnostics:
    Below is an example of conditions leading to this issue.
    
    1. First instance is started. For example :
    
    [root@mylinux ~]# cd /home/tsminst1
    [root@mylinux ~]# source /home/tsminst1/sqllib/db2profile
    [root@mylinux tsminst1]# /opt/tivoli/tsm/server/bin/dsmserv
    
    2. Second instance is started using same profile as first one.
       For example :
    
    [root@mylinux ~]# cd /home/tsminst2
    [root@mylinux ~]# source /home/tsminst1/sqllib/db2profile
    [root@mylinux tsminst2]# /opt/tivoli/tsm/server/bin/dsmserv
    
    3. The second instance correctly fails to start with the
       following error :
        ANR0227S Incorrect database opened.  Server cannot start.
       This is the expected result.
    
    4. Following this, the first instance becomes unresponsive or
       crashes. This is the unexpected result.
    
    In the case where the TSM server crashes, the following messages
    are logged in the server activity log :
    
    ANR0162W Supplemental database diagnostic information:
     -1:40003:-1224 ([IBM][CLI Driver] SQL1224N  The database
     manager is not able to accept new requests, has
      terminated all requests in progress, or has terminated
     your particular request due to an error or a force
     interrupt.  SQLSTATE=55032).
    ANR0130E dbieval.c(825): Server LOG space exhausted.
    
    The call stack of the core file produced by the server crash
    shows :
    
    #1  0x000000365c431ae0 in abort () from /lib64/libc.so.6
    #2  0x0000000000c63506 in PsAbortServer ()
    #3  0x0000000000c5dfe5 in pkAbort ()
    #4  0x0000000000787b9c in RdbHangMonitorThread ()
    #5  0x0000000000c5df0b in StartThread ()
    #6  0x000000365d0062e7 in start_thread () from
          /lib64/libpthread.so.0
    #7  0x000000365c4ce3bd in clone () from /lib64/libc.so.6
    
    In the case where the TSM server becomes unresponsive, the
    following messages are logged at the console of the first
    instance running in the foreground :
    
    ANR0130E dbieval.c(825): Server LOG space exhausted.
    ANR0162W Supplemental database diagnostic information:
     -1:40003:-1224
    ([IBM][CLI Driver] SQL1224N  The database manager is not able to
    accept new requests, has terminated all requests in progress, or
    has terminated your particular request due to an error or a
    force interrupt.  SQLSTATE=55032).
    
    Other error messages also logged, but not limited to, are :
    
    ANR0106E cscmdsch.c(546): Unexpected error 5 fetching row in
     table "Schedule.Pending".
    ANR9999D_4073936223 CsCmdSchedulerThread(cscmdsch.c:308)
     Thread<nn0>: Invalid recovery criteria (4) in the central
     scheduler - the task is terminating.
    ANR9999D Thread<nn> issued message 9999 from:
    ANR9999D Thread<nn>  0x0000000000bee072 OutDiagToCons+0x0x142
    ANR9999D Thread<nn>  0x0000000000bf0bf4 outDiagfExt+0x0x194
    ANR9999D Thread<nn>  0x00000000006f1ea4
                                       CsCmdSchedulerThread+0x0x174
    ANR9999D Thread<nn>  0x0000000000c5df0b StartThread+0x0x7b
    ANR9999D Thread<nn>  0x000000365d0062e7 *UNKNOWN*
    ANR9999D Thread<nn>  0x000000365c4ce3bd *UNKNOWN*
    
    ANR2010W Server is out of LOG space in adding entries to the
     Activity Log. Console messages will not be logged until LOG
     space is available.
    ANR0171I dbieval.c(812): Error detected on 0:8, database in
     evaluation mode.
    ANR0101E bfdedup.c(8038): Error 2157 opening table
     "BF.Dereferenced.Chunks".
    ANR0152I Database manager successfully started.
    ANR9999D_0645605689 tbOpenX(tbtbl.c:4248) Thread<5>:
     Failure participating on transaction.
    ANR9999D Thread<nn> issued message 9999 from:
    ANR9999D Thread<nn>  0x0000000000bee072 OutDiagToCons+0x0x142
    ANR9999D Thread<nn>  0x0000000000bf0bf4 outDiagfExt+0x0x194
    ANR9999D Thread<nn>  0x000000000071ad89 tbOpenX+0x0x699
    ANR9999D Thread<nn>  0x0000000000561629
                                        AdmActivityLogThread+0x0x2d9
    ANR9999D Thread<nn>  0x0000000000c5df0b StartThread+0x0x7b
    ANR9999D Thread<nn>  0x000000365d0062e7 *UNKNOWN*
    ANR9999D Thread<nn>  0x000000365c4ce3bd *UNKNOWN*
    ANR0101E admactlg.c(2836): Error 16 opening table
      "Activity.Log".
    ANR2009E Activity log process has stopped - database error.
    ANR2006E Activity log process was not started, the default
     output stream cannot be opened.
    DbiEvalSQLOutcome: Handle SQL_NULL_HANDLE.
    DbiEvalSQLOutcome: SQLGetDiagRec returned -2.
    DbiEvalSQLOutcome: SQL returned -1.
    ANR0101E bfdedup.c(7965): Error 11 opening table
     "BF.Dereferenced.Chunks".
    ANR0171I dbieval.c(812): Error detected on 0:4, database in
     evaluation mode.
    
    TSM Versions Affected:
    TSM 6.1 servers on all supported platforms.
    
    Additional Keywords:
    zz61 multiple servers same machine
    

Local fix

  • Start the TSM server instance with the correct environment.
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED: All Tivoli Storage Manager server V6 users.  *
    ****************************************************************
    * PROBLEM DESCRIPTION: See error description.                  *
    ****************************************************************
    * RECOMMENDATION: Apply fixing level when available. This      *
    *                 problem is fixed in level 6.1.3.             *
    ****************************************************************
    *
    

Problem conclusion

  • This problem was fixed.
    
    Affected platforms:  AIX, HP-UX, Sun Solaris, and Linux.
    

Temporary fix

Comments

APAR Information

  • APAR number

    IC64192

  • Reported component name

    TSM SERVER

  • Reported component ID

    5698ISMSV

  • Reported release

    61A

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2009-10-27

  • Closed date

    2010-01-04

  • Last modified date

    2010-01-04

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    TSM SERVER

  • Fixed component ID

    5698ISMSV

Applicable component levels

  • R61A PSY

       UP

  • R61H PSY

       UP

  • R61L PSY

       UP

  • R61S PSY

       UP



Rate this page:

(0 users)Average rating

Add comments

Document information


More support for:

Tivoli Storage Manager

Software version:

61A

Reference #:

IC64192

Modified date:

2010-01-04

Translate my page

Machine Translation

Content navigation