Pages

Monday, March 21, 2022

Golden Gate - No Replication, REPLICAT is broken, (Timeout waiting for message)

================
Issue
================

In Golden Gate
There is no error, but data is not replicating for one of tables defined for P flow
Checking EXTRACT  EXT_P_01 - on server A - OK
Checking EXTRACT  DPM_P_01 - on server A - OK 
Checking EXTRACT  DPM_P_01 - on server B -  not OK 
Need to fix REP_P_01

================
Details
================
GGSCI (my_host) 24>  INFO ALL
Program     Status      Group       Lag at Chkpt  Time Since Chkpt
MANAGER     RUNNING
EXTRACT     RUNNING     DPM_I_01    00:00:00      00:00:00
EXTRACT     RUNNING     DPM_P_01    00:00:00      00:00:01
EXTRACT     RUNNING     DPM_S_01    00:00:00      00:00:01
EXTRACT     RUNNING     EXT_I_01    00:00:01      00:00:00
EXTRACT     RUNNING     EXT_P_01    00:00:01      00:00:03
EXTRACT     RUNNING     EXT_S_01    00:00:02      00:00:09
REPLICAT    RUNNING     REP_I_01    00:00:06      00:00:08
REPLICAT    RUNNING     REP_P_01    00:00:00      00:00:03
REPLICAT    RUNNING     REP_S_01    00:00:00      00:00:05


GGSCI (my_host) 26> SEND REP_P_01 STATS

Sending STATS request to REPLICAT REP_P_01 ...

2022-03-21 14:09:38  ERROR   OGG-15163  There was a problem sending a message to REPLICAT REP_P_01 (Timeout waiting for message).


Option A. - Lets try to restart REPLICAT using the skip transactoin option

GGSCI (my_host) 2> KILL REPLICAT REP_P_01

Sending KILL request to MANAGER ...
Killed process (12600) for REPLICAT REP_P_01

GGSCI (my_host) 3> START REPLICAT REP_P_01 SKIPTRANSACTION

Sending START request to MANAGER ...
REPLICAT REP_P_01 starting

GGSCI (my_host) 6> INFO REP_P_01

REPLICAT   REP_P_01  Last Started 2022-03-21 14:07   Status RUNNING
INTEGRATED
Checkpoint Lag       149:06:01 (updated 00:04:25 ago)
Process ID           13155
Log Read Checkpoint  File /software/ogg/191/dirdat/01/in/ep000000000
                     2022-03-15 08:56:41.588988  RBA 26182419


GGSCI (my_host) 22> SEND REP_P_01 STATS

Sending STATS request to REPLICAT REP_P_01 ...

2022-03-21 14:09:38  ERROR   OGG-15163  There was a problem sending a message to REPLICAT REP_P_01 (Timeout waiting for message).

Option B. - Lets try to restart REPLICAT using next file
Since this was a test system, I did not care for lost transactions.




oracle@my_host:/software/ogg/191/dirdat/01/in>% ls -ltr | grep ep
-rw-r----- 1 oracle dba 499999201 Mar 21 13:43 ep000000000
-rw-r----- 1 oracle dba 499999472 Mar 21 13:43 ep000000001
-rw-r----- 1 oracle dba 342178884 Mar 21 14:03 ep000000002


GGSCI (my_host) 2> KILL REPLICAT REP_P_01

Sending KILL request to MANAGER ...
Killed process (12600) for REPLICAT REP_P_01



GGSCI (my_host) 139> ALTER REPLICAT REP_P_01, EXTSEQNO 2 EXTRBA 0

2022-03-21 14:26:00  INFO    OGG-06594  Replicat REP_P_01 has been altered. Even the start up position might be updated, duplicate suppression remains active in next startup. To override duplicate suppression, start REP_P_01 with NOFILTERDUPTRANSACTIONS option.

REPLICAT (Integrated) altered.


GGSCI (my_host) 140> START REP_P_01

Sending START request to MANAGER ...
REPLICAT REP_P_01 starting


GGSCI (my_host) 158> INFO REP_P_01

REPLICAT   REP_P_01  Last Started 2022-03-21 14:26   Status RUNNING
INTEGRATED
Checkpoint Lag       00:00:00 (updated 00:00:04 ago)
Process ID           15038
Log Read Checkpoint  File /software/ogg/191/dirdat/01/in/ep000000002
                     First Record  RBA 0

 
GGSCI (my_host) 161> INFO REP_P_01

REPLICAT   REP_P_01  Last Started 2022-03-21 14:26   Status RUNNING
INTEGRATED
Checkpoint Lag       20:55:05 (updated 00:00:00 ago)
Process ID           15038
Log Read Checkpoint  File /software/ogg/191/dirdat/01/in/ep000000002
                     2022-03-20 17:31:39.998169  RBA 57324931


GGSCI (my_host) 175>  INFO ALL

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING
EXTRACT     RUNNING     DPM_I_01    00:00:00      00:00:08
EXTRACT     RUNNING     DPM_P_01    00:00:00      00:00:07
EXTRACT     RUNNING     DPM_S_01    00:00:00      00:00:01
EXTRACT     RUNNING     EXT_I_01    00:00:01      00:00:04
EXTRACT     RUNNING     EXT_P_01    00:00:00      00:00:03
EXTRACT     RUNNING     EXT_S_01    00:00:01      00:00:04
REPLICAT    RUNNING     REP_I_01    00:00:00      00:00:04
REPLICAT    RUNNING     REP_P_01    14:02:42      00:00:08
REPLICAT    RUNNING     REP_S_01    00:00:00      00:00:06

GGSCI (my_host) 176>  INFO ALL

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING
EXTRACT     RUNNING     DPM_I_01    00:00:00      00:00:01
EXTRACT     RUNNING     DPM_P_01    00:00:00      00:00:00
EXTRACT     RUNNING     DPM_S_01    00:00:00      00:00:04
EXTRACT     RUNNING     EXT_I_01    00:00:01      00:00:07
EXTRACT     RUNNING     EXT_P_01    00:00:00      00:00:06
EXTRACT     RUNNING     EXT_S_01    00:00:01      00:00:07
REPLICAT    RUNNING     REP_I_01    00:00:00      00:00:07
REPLICAT    RUNNING     REP_P_01    13:03:38      00:00:01
REPLICAT    RUNNING     REP_S_01    00:00:00      00:00:09

GGSCI (my_host) 177> INFO ALL

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING
EXTRACT     RUNNING     DPM_I_01    00:00:00      00:00:07
EXTRACT     RUNNING     DPM_P_01    00:00:00      00:00:07
EXTRACT     RUNNING     DPM_S_01    00:00:00      00:00:01
EXTRACT     RUNNING     EXT_I_01    00:00:03      00:00:04
EXTRACT     RUNNING     EXT_P_01    00:00:02      00:00:08
EXTRACT     RUNNING     EXT_S_01    00:00:02      00:00:08
REPLICAT    RUNNING     REP_I_01    00:00:00      00:00:05
REPLICAT    RUNNING     REP_P_01    00:00:06      00:00:02
REPLICAT    RUNNING     REP_S_01    00:00:00      00:00:06

GGSCI (my_host) 180> SEND REP_P_01 STATS

Sending STATS request to REPLICAT REP_P_01 ...

Start of Statistics at 2022-03-21 14:41:55.


Now, The backlog was cleared and REPLICAT REP_P_01 is running as expected
And SEND <REPLICAT> STATS command is returning a valid response.

No comments:

Post a Comment