Informational tokens are DB2 v9.5.0.1, s080328, MI00225, Fix Pack 1.

Operating System Information:

OSName:   Linux

NodeName: server1.mydomain.net
Version:  2
Release:  6
Machine:  x86_64

Received the following message when I tried to initialise HADR and trigger the Remote Catchup on a existing HADR set up .This was due to an unplanned outage.

1)the initial observation was that after a short period of remote catchup the db2 instance would shut down
An attempt at connecting would return

“Unable to attach to database manager.  Please ensure db2start has been run.”

An interorgation on db2diag.log saw :

ADM0503C  An unexpected internal processing error has occurred.  ALL
DB2 PROCESSES ASSOCIATED WITH THIS INSTANCE HAVE BEEN SHUTDOWN.

 

Looking earlier on db2diag,log :

ZRC=0x870F0009=-2029060087=SQLO_EOF “the data does not exist”

DIA8506C Unexpected end of file was reached.

 

Error in agent servicing application with INBOUND SEQUENCE NUMBER:
2)I then tried a full back up onto TSM

db2 backup database mydb use tsm
SQL1224N  The database manager is not able to accept new requests, has
terminated all requests in progress, or has terminated your particular request
due to an error or a force interrupt.  SQLSTATE=00000

3)I updated the diaglevel to 4 to allow greater granularity of messages

db2 update dbm cfg using DIAGLEVEL 4

the issued the same command :

db2 backup database mydb use tsm

The following information was gatehered on crash :

sqloEDUCodeTrapHandler  created a signal 11 – which pointed to

Signal #11 (SIGSEGV): si_addr is 0x0000202AA8BEDD90, si_code is 0x00000001 (SEGV_MAPERR:Address not mapped to object.)

vas_socket_send_w_timeout + 0x00fd  (/opt/quest/lib64/libvas.so.4)

4)I executed a “ps aux | less quest”  this returned :

root 11372  0.0  0.0 12536 3192 ?     Ss   Jul26   0:38 /opt/quest/sbin/vasd – /var/opt/quest/vas/vasd/.vasd.pid
root 11425  0.0  0.0 20168 2444 ?     Ss   Jul26   0:00 /opt/quest/sbin/vasgpd -p /var/opt/quest/vgp/vasgpd/.vasgpd.pid
daemon   11527  0.0  0.0 12700 3464 ?     S    Jul26   0:03 /opt/quest/sbin/vasd -p /var/opt/quest/vas/vasd/.vasd.pid

I issued a : kill on the 3 processes (11372, 11425,11527)

5) I issued the following , which were not working earlier:

db2 backup database mydb use tsm   (this was OK -failing earlier)

db2 backup database mydb   (this was OK -failing earlier)
6)HADR reinitialisation – which was causing a problem on remote catchup earlier is now OK