SQL Server generate Reports Error "Culture is not supported"

================
General
================

SQL server has build in monitoring Reports.

Instance => Reports => Standard Reports => Performance Dashboard

It should look like this. From here can go to different findings.

================
Problem
================

When Running this Reports, there is an error:

"Culture is not supported"

================
Solution
================

The error is due to Locale compatibility issues.

Change Region For,at Settings to English (United States) 

Control Panel => Settings => Region -> Formats -> Format => Change from whatever value is now to English (United States) => Restart SQL Server management Studio

Replicat is showing: "No active replication maps."

======================
General
======================
The issue:

Data not replicating from server_39 to server_38
Replicat is showing: "No active replication maps."

======================
Details
======================
MANAGER     RUNNING
EXTRACT     RUNNING     DPM_I_01    00:00:00      00:00:09
EXTRACT     RUNNING     DPM_P_01    00:00:00      00:00:04
EXTRACT     RUNNING     DPM_S_01    00:00:03      00:00:09
EXTRACT     RUNNING     EXT_I_01    00:00:01      00:00:09
EXTRACT     RUNNING     EXT_P_01    00:00:02      00:00:04
EXTRACT     RUNNING     EXT_S_01    00:00:01      00:00:09
REPLICAT    RUNNING     REP_I_01    00:00:00      00:00:01
REPLICAT    RUNNING     REP_P_01    00:00:00      00:00:00
REPLICAT    RUNNING     REP_S_01    00:00:04      00:00:03
=================================================================
Checking the EXTRACT on server 39, looks OK:
SEND EXTRACT EXT_I_01 STATS
Extracting from MY_USER.IPN_INVALIDATE_TRIGGER to MY_USER.IPN_INVALIDATE_TRIGGER:
*** Hourly statistics since 2020-11-17 11:00:00 ***
        Total inserts                                      1.00
        Total updates                                      0.00
        Total deletes                                      1.00
        Total upserts                                      0.00
        Total discards                                     0.00
        Total operations                                   2.00

But on server 38, the REPLICAT is not getting any input: 
GGSCI (server_38) 2> SEND REPLICAT REP_I_01 STATS
Sending STATS request to REPLICAT REP_I_01 ...
No active replication maps.

check input files on 38
Nothing is coming for process for REPICAT REP_I_01
-rw-r----- 1 oracle dba        0 Oct 29 09:48 ei000000000
-rw-r----- 1 oracle dba    20633 Nov 16 14:25 disc01i.txt
-rw-r----- 1 oracle dba    58431 Nov 16 14:25 disc01s.txt
-rw-r----- 1 oracle dba    13947 Nov 16 14:25 disc01p.txt
-rw-r----- 1 oracle dba 21743643 Nov 17 11:47 es000000003
-rw-r----- 1 oracle dba 21219127 Nov 17 11:47 ep000000003

GGSCI (server_38) 5> INFO REP_I_01
REPLICAT   REP_I_01  Last Started 2020-11-17 11:55   Status RUNNING
Checkpoint Lag       00:00:00 (updated 00:00:08 ago)
Process ID           15605
Log Read Checkpoint  File /software/ogg/191/dirdat/01/in/ei000000001
                     First Record  RBA 0

REPLICAT is looking for flie /software/ogg/191/dirdat/01/in/ei000000001 
But this file does not exists.
But this file does not exists.
Checking datapump on 39 server

GGSCI (server_39) 6> SEND EXTRACT DPM_I_01 STATS
Sending STATS request to EXTRACT DPM_I_01 ...
No active extraction maps.
The Extract definition:
ADD EXTRACT DPM_I_01 EXTTRAILSOURCE $GG_HOME/dirdat/01/out/ei
ADD RMTTRAIL $GG_HOME/dirdat/01/in/ei EXTRACT DPM_I_01
START EXTRACT DPM_I_01
Seems that the Datapump on server 39 is not getting input. 
Thus nothing is sent to server 38.

DROP + CREATE Datapump - Does not work
To drop + recreate, can create do following steps, but it does not help:
To drop EXTRACT 
DBLOGIN USERID OGG, PASSWORD YYY
STOP EXTRACT  DPM_I_01
DELETE EXTRACT DPM_I_01
ADD EXTRACT DPM_I_01 EXTTRAILSOURCE /software/ogg/191/dirdat/01/out/ei
ADD RMTTRAIL /software/ogg/191/dirdat/01/in/ei EXTRACT DPM_I_01
GGSCI (server_39 as ogg@orainst) 10> INFO DPM_I_01
EXTRACT    DPM_I_01  Last Started 2020-11-17 12:50   Status RUNNING
Checkpoint Lag       00:00:00 (updated 00:00:08 ago)
Process ID           18315
Log Read Checkpoint  File /software/ogg/191/dirdat/01/out/ei000000000
                     First Record  RBA 0
======================
Solution
======================
ETROLLOVER to Extract + Update to Datapump
DBLOGIN USERID XXX, PASSWORD YYY
ALTER EXTRACT EXT_I_01, ETROLLOVER
ALTER EXTRACT DPM_I_01, EXTSEQNO 10, EXTRBA 0  
GGSCI (server_39 as ogg@orainst) 4> STOP EXTRACT EXT_I_01
Sending STOP request to EXTRACT EXT_I_01 ...
Request processed.
GGSCI (server_39 as ogg@orainst) 6> ALTER EXTRACT EXT_I_01, ETROLLOVER
2020-11-17 14:21:01  INFO    OGG-01520  Rollover performed.  
For each affected output trail of Version 10 or higher format, 
after starting the source extract, 
issue ALTER EXTSEQNO for that trail's reader 
(either pump EXTRACT or REPLICAT) to move the reader's scan to the new trail file;  
it will not happen automatically.
EXTRACT altered.
GGSCI (server_39 as ogg@orainst) 12> INFO EXT_I_01 DETAIL
EXTRACT    EXT_I_01  Initialized   2020-11-16 14:26   Status RUNNING
Checkpoint Lag       00:00:03 (updated 00:02:50 ago)
Process ID           27583
Log Read Checkpoint  Oracle Integrated Redo Logs
                     2020-11-17 14:20:37
                     SCN 2.949429219 (9539363811)
  Target Extract Trails:
  Trail Name                                       Seqno        RBA     Max MB Trail Type
  /software/ogg/191/dirdat/01/out/ei                2506          0        500 EXTTRAIL
 
  GGSCI (server_39 as ogg@orainst) 14> STOP EXTRACT DPM_I_01
  GGSCI (server_39 as ogg@orainst) 15> ALTER EXTRACT DPM_I_01, EXTSEQNO 2506 EXTRBA 0
EXTRACT altered.
GGSCI (server_39 as ogg@orainst) 17> START DPM_I_01
Sending START request to MANAGER ...
EXTRACT DPM_I_01 starting
GGSCI (server_39 as ogg@orainst) 18> INFO DPM_I_01
EXTRACT    DPM_I_01  Last Started 2020-11-17 14:24   Status RUNNING
Checkpoint Lag       00:00:00 (updated 00:00:03 ago)
Process ID           27776
Log Read Checkpoint  File /software/ogg/191/dirdat/01/out/ei000002506
                     First Record  RBA 0
Now Datapump is advancing.
GGSCI (server_39 as ogg@orainst) 19> INFO DPM_I_01
EXTRACT    DPM_I_01  Last Started 2020-11-17 14:24   Status RUNNING
Checkpoint Lag       00:00:00 (updated 00:00:04 ago)
Process ID           27776
Log Read Checkpoint  File /software/ogg/191/dirdat/01/out/ei000002506
                     2020-11-17 14:25:07.000000  RBA 8905

on server 38
Now the file on the remote server is being created: ei000000000
oracle@server_38:/software/ogg/191/dirdat/01/in>% ls -ltr
total 45884
-rw-r----- 1 oracle dba    58431 Nov 16 14:25 disc01s.txt
-rw-r----- 1 oracle dba    13947 Nov 16 14:25 disc01p.txt
-rw-r----- 1 oracle dba    20850 Nov 17 11:55 disc01i.txt
-rw-r----- 1 oracle dba 22915034 Nov 17 14:28 es000000003
-rw-r----- 1 oracle dba    13214 Nov 17 14:28 ei000000000
-rw-r----- 1 oracle dba 23709458 Nov 17 14:28 ep000000003

================================
Next step -Troubleshoot REPLICAT on server 38
================================
GGSCI (server_38) 2> SEND REPLICAT REP_I_01 STATS
Sending STATS request to REPLICAT REP_I_01 ...
No active replication maps.
GGSCI (server_38) 8> INFO REP_I_01
REPLICAT   REP_I_01  Last Started 2020-11-17 14:29   Status RUNNING
Checkpoint Lag       00:00:00 (updated 00:00:03 ago)
Process ID           18863
Log Read Checkpoint  File /software/ogg/191/dirdat/01/in/ei000000001
                     First Record  RBA 0
But...
The actual file is : /software/ogg/191/dirdat/01/in/ei000000000
Need manually to set REPLICAT to look into the correct file
GGSCI (server_38) 10> STOP REPLICAT REP_I_01
Sending STOP request to REPLICAT REP_I_01 ...
Request processed.
GGSCI (server_38) 15> ALTER REPLICAT REP_I_01, EXTSEQNO 0
2020-11-17 14:35:09  INFO    OGG-06594  Replicat REP_I_01 has been altered. Even the start up position might be updated, duplicate suppression remains active in next startup. To override duplicate suppression, start REP_I_01 with NOFILTERDUPTRANSACTIONS option.
REPLICAT altered.
GGSCI (server_38) 16> START REPLICAT REP_I_01
Sending START request to MANAGER ...
REPLICAT REP_I_01 starting
GGSCI (server_38) 17> INFO REPLICAT REP_I_01
REPLICAT   REP_I_01  Last Started 2020-11-17 14:35   Status RUNNING
Checkpoint Lag       00:00:15 (updated 00:00:01 ago)
Process ID           32390
Log Read Checkpoint  File /software/ogg/191/dirdat/01/in/ei000000000
                     2020-11-17 14:35:07.001959  RBA 20771
Now, REPICAT is reading records.  
GGSCI (server_38) 45> SEND REPLICAT REP_I_01 STATS
Sending STATS request to REPLICAT REP_I_01 ...
Start of Statistics at 2020-11-17 14:36:08.
Replicating from OGG.GG_HEARTBEAT_SEED to OGG.GG_HEARTBEAT:
*** Total statistics since 2020-11-17 14:35:21 ***
        Total inserts                                      0.00
        Total updates                                     15.00
        Total deletes                                      0.00
        Total upserts                                      0.00
        Total discards                                     0.00
        Total operations                                  15.00
*** Daily statistics since 2020-11-17 14:35:21 ***
        Total inserts                                      0.00
        Total updates                                     15.00
        Total deletes                                      0.00
        Total upserts                                      0.00
        Total discards                                     0.00
        Total operations                                  15.00
*** Hourly statistics since 2020-11-17 14:35:21 ***
        Total inserts                                      0.00
        Total updates                                     15.00
        Total deletes                                      0.00
        Total upserts                                      0.00
        Total discards                                     0.00
        Total operations                                  15.00
*** Latest statistics since 2020-11-17 14:35:21 ***
        Total inserts                                      0.00
        Total updates                                     15.00
        Total deletes                                      0.00
        Total upserts                                      0.00
        Total discards                                     0.00
        Total operations                                  15.00
Replicating from OGG.GG_HEARTBEAT_SEED to OGG.GG_HEARTBEAT_HISTORY:
*** Total statistics since 2020-11-17 14:35:21 ***
        Total inserts                                      0.00
        Total updates                                     15.00
        Total deletes                                      0.00
        Total upserts                                      0.00
        Total discards                                     0.00
        Total operations                                  15.00
*** Daily statistics since 2020-11-17 14:35:21 ***
        Total inserts                                      0.00
        Total updates                                     15.00
        Total deletes                                      0.00
        Total upserts                                      0.00
        Total discards                                     0.00
        Total operations                                  15.00
*** Hourly statistics since 2020-11-17 14:35:21 ***
        Total inserts                                      0.00
        Total updates                                     15.00
        Total deletes                                      0.00
        Total upserts                                      0.00
        Total discards                                     0.00
        Total operations                                  15.00
*** Latest statistics since 2020-11-17 14:35:21 ***
        Total inserts                                      0.00
        Total updates                                     15.00
        Total deletes                                      0.00
        Total upserts                                      0.00
        Total discards                                     0.00
        Total operations                                  15.00
Replicating from MY_USER.IPN_INVALIDATE_TRIGGER to MY_USER.IPN_INVALIDATE_TRIGGER:
        Total inserts                                      1.00
        Total updates                                      0.00
        Total deletes                                      0.00
        Total upserts                                      0.00
        Total discards                                     0.00
        Total ignores                                      2.00
        Total operations                                   1.00
*** Daily statistics since 2020-11-17 14:35:21 ***
        Total inserts                                      1.00
        Total updates                                      0.00
        Total deletes                                      0.00
        Total upserts                                      0.00
        Total discards                                     0.00
        Total ignores                                      2.00
        Total operations                                   1.00
*** Hourly statistics since 2020-11-17 14:35:21 ***
        Total inserts                                      1.00
        Total updates                                      0.00
        Total deletes                                      0.00
        Total upserts                                      0.00
        Total discards                                     0.00
        Total ignores                                      2.00
        Total operations                                   1.00
*** Latest statistics since 2020-11-17 14:35:21 ***
        Total inserts                                      1.00
        Total updates                                      0.00
        Total deletes                                      0.00
        Total upserts                                      0.00
        Total discards                                     0.00
        Total ignores                                      2.00
        Total operations                                   1.00
End of Statistics.

Now Replicat is finally getting input from the other server.
Issue Fixed!

All processes are working OK

GGSCI (server_38) 2> INFO ALL

*** Total statistics since 2020-11-17 14:35:21 ***

ERROR OGG-02191 Incompatible record

=====================
General
=====================

REPLICAT abends with error
ERROR   OGG-02171  Error reading LCR from data source. 

Need to skip the bad record in file, and restart REPLICAT

=====================
Evidences
=====================

 2020-10-28 15:47:29  INFO    OGG-02243  Opened trail file /software/ogg/191/dirdat/01/in/es000000000 at 2020-10-28 15:47:29.963515.

2020-10-28 15:47:50  ERROR   OGG-02171  Error reading LCR from data source. Status 509, data source type TrailDataSource.

2020-10-28 15:47:50  ERROR   OGG-02191  Incompatible record 101 in /software/ogg/191/dirdat/01/in/es000000000, rba 696,172 when getting trail header.

2020-10-28 15:47:50  INFO    OGG-02333  Reading /software/ogg/191/dirdat/01/in/es000000000, current RBA 696,172, 0 records, m_file_seqno = 0, m_file_rba = 696,172.

=====================
Solution
=====================

Use logdump utility to get next record position.
In general, need to skip the bad record in file, and restart REPLICAT.
First, position the position on the failed record.

The tool will display the next record.

Optionally, get the next+1 record.

Then, set the REPLICAT to start at the next record.

ALTER REP_S_01, EXTSEQNO 0, EXTRBA <THE SAVED RBA>

cd $GG_HOME

oracle@qanfv-1-dbs-1b:/software/ogg/191>% ./logdump

Oracle GoldenGate Log File Dump Utility for Oracle Version 19.1.0.0.1 OGGCORE_19.1.0.0.0_PLATFORMS_190524.2201

Copyright (C) 1995, 2019, Oracle and/or its affiliates. All rights reserved.

Logdump 1 >open

Error: Expected a filename

Logdump 2 >open /software/ogg/191/dirdat/01/in/es000000000

Current LogTrail is /software/ogg/191/dirdat/01/in/es000000000

Logdump 3 >ghdr on

Logdump 4 >detail on

Logdump 5 >pos 696172

Reading forward from RBA 696172

Logdump 6 >scanforheader

___________________________________________________________________

Hdr-Ind    :     E  (x45)     Partition  :     L  (x4c)
UndoFlag   :     .  (x00)     BeforeAfter:     A  (x41)
RecLength  :   962  (x03c2)   IO Time    : 2020/10/26
0:19:07.005.706
IOType     :   135  (x87)     OrigNode   :   255  (xff)
TransInd   :     .  (x03)     FormatType :     R  (x52)
SyskeyLen  :     0  (x00)     Incomplete :     .  (x00)
AuditRBA   :     200132       AuditPos   : 122219168
Continued  :     N  (x00)     RecCount   :     1  (x01)

2020/10/26 10:19:07.005.706 GGSUnifiedPKUpdate   Len   962 RBA 696345
Name: OGG.GG_HEARTBEAT_SEED  (TDR Index: 2)
After  Image:                                             Partition x4c   G  s

 df01 0000 0000 0700 0000 0300 4947 5401 001f 0000 | ............IGT.....
 0032 3032 302d 3130 2d32 363a 3130 3a31 383a 3037 | .2020-10-26:10:18:07
 2e32 3530 3233 3330 3030 0200 0400 ffff 0000 0300 | .250233000..........
 0400 ffff 0000 0400 0400 ffff 0000 0500 0400 ffff | ....................
 0000 0600 1f00 ffff 0000 0000 0000 0000 0000 0000 | ....................
 0000 0000 0000 0000 0000 0000 0000 0000 0007 001f | ....................
 00ff ff00 0000 0000 0000 0000 0000 0000 0000 0000 | ....................
Before Image          Len   483 (x000001e3)
BeforeColumnLen    479 (x000001df)
Column     0 (x0000), Len     7 (x0007)
Column     1 (x0001), Len    31 (x001f)
Column     2 (x0002), Len     4 (x0004)
Column     3 (x0003), Len     4 (x0004)
Column     4 (x0004), Len     4 (x0004)
Column     5 (x0005), Len     4 (x0004)
Column     6 (x0006), Len    31 (x001f)
Column     7 (x0007), Len    31 (x001f)
Column     8 (x0008), Len    31 (x001f)
Column     9 (x0009), Len    31 (x001f)
Column    10 (x000a), Len    12 (x000c)
Column    11 (x000b), Len    14 (x000e)
Column    12 (x000c), Len     4 (x0004)
Column    13 (x000d), Len    31 (x001f)
Column    14 (x000e), Len    31 (x001f)
Column    15 (x000f), Len    31 (x001f)
Column    16 (x0010), Len    31 (x001f)
Column    17 (x0011), Len     4 (x0004)
Column    18 (x0012), Len    14 (x000e)
Column    19 (x0013), Len    14 (x000e)
Column    20 (x0014), Len    31 (x001f)

After Image           Len   479 (x000001df)
Column     0 (x0000), Len     7 (x0007)
Column     1 (x0001), Len    31 (x001f)
Column     2 (x0002), Len     4 (x0004)
Column     3 (x0003), Len     4 (x0004)
Column     4 (x0004), Len     4 (x0004)
Column     5 (x0005), Len     4 (x0004)
Column     6 (x0006), Len    31 (x001f)
Column     7 (x0007), Len    31 (x001f)
Column     8 (x0008), Len    31 (x001f)
Column     9 (x0009), Len    31 (x001f)
Column    10 (x000a), Len    12 (x000c)
Column    11 (x000b), Len    14 (x000e)
Column    12 (x000c), Len     4 (x0004)
Column    13 (x000d), Len    31 (x001f)
Column    14 (x000e), Len    31 (x001f)
Column    15 (x000f), Len    31 (x001f)
Column    16 (x0010), Len    31 (x001f)
Column    17 (x0011), Len     4 (x0004)
Column    18 (x0012), Len    14 (x000e)
Column    19 (x0013), Len    14 (x000e)
Column    20 (x0014), Len    31 (x001f)

Logdump 7 >n
___________________________________________________________________
Hdr-Ind    :     E  (x45)     Partition  :     .  (x0c)
UndoFlag   :     .  (x00)     BeforeAfter:     A  (x41)
RecLength  :   271  (x010f)   IO Time    : 2020/10/26
0:19:15.005.532
IOType     :     5  (x05)     OrigNode   :   255  (xff)
TransInd   :     .  (x00)     FormatType :     R  (x52)
SyskeyLen  :     0  (x00)     Incomplete :     .  (x00)
AuditRBA   :     200132       AuditPos   : 131688464
Continued  :     N  (x00)     RecCount   :     1  (x01)

2020/10/26 10:19:15.005.532 Insert               Len   271 RBA 697424
Name: LAB_QANFV_ALLQQ.GA_W_COUNTERS_HISTORY  (TDR Index: 1)
After  Image:                                             Partition
0c   G  b

 0000 1300 0000 0f00 3130 3330 3130 3438 3130 3030 |
.......103010481000
 3030 3001 0009 0000 0005 0033 3635 3839 0200 0600 |
00........36589....
 0000 0200 3136 0300 0500 0000 0100 3004 0005 0000 | ....16........0.....
 0001 0030 0500 0500 0000 0100 3006 0005 0000 0001 | ...0........0.......
 0030 0700 0500 0000 0100 3008 000a 0000 0000 0000 | .0........0.........
 0000 0000 0009 000a 0000 0000 0000 0000 0000 000a | ....................
 0015 0000 0032 3032 302d 3130 2d32 363a 3130 3a31 | .....2020-10-26:10:1
Column     0 (x0000), Len    19 (x0013)
Column     1 (x0001), Len     9 (x0009)
Column     2 (x0002), Len     6 (x0006)
Column     3 (x0003), Len     5 (x0005)
Column     4 (x0004), Len     5 (x0005)
Column     5 (x0005), Len     5 (x0005)
Column     6 (x0006), Len     5 (x0005)
Column     7 (x0007), Len     5 (x0005)
Column     8 (x0008), Len    10 (x000a)
Column     9 (x0009), Len    10 (x000a)
Column    10 (x000a), Len    21 (x0015)
Column    11 (x000b), Len    10 (x000a)
Column    12 (x000c), Len     7 (x0007)
Column    13 (x000d), Len    10 (x000a)
Column    14 (x000e), Len    21 (x0015)
Column    15 (x000f), Len     4 (x0004)
Column    16 (x0010), Len     4 (x0004)
Column    17 (x0011), Len     4 (x0004)
Column    18 (x0012), Len     4 (x0004)
Column    19 (x0013), Len     4 (x0004)
Column    20 (x0014), Len    10 (x000a)
Column    21 (x0015), Len     5 (x0005)
Logdump 8 >
Logdump 8 >exit

./ggsci

GGSCI (qanfv-1-dbs-1b) 2> ALTER REPLICAT REP_S_01, EXTSEQNO 0 ,EXTRBA 696345

2020-10-28 15:58:14  INFO    OGG-06594  Replicat REP_S_01 has been altered. Even the start up position might be updated, duplicate suppression remains active in next startup. To override duplicate suppression, start REP_S_01 with NOFILTERDUPTRANSACTIONS option.

REPLICAT altered.

GGSCI (qanfv-1-dbs-1b) 3> START REP_S_01

Sending START request to MANAGER ...

REPLICAT REP_S_01 starting

GGSCI (qanfv-1-dbs-1b) 111>  INFO ALL

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING
EXTRACT     RUNNING     DPM_I_01    00:00:00      00:00:04
EXTRACT     RUNNING     DPM_P_01    00:00:00      00:00:00
EXTRACT     RUNNING     DPM_S_01    00:00:05      00:00:08
EXTRACT     RUNNING     EXT_I_01    00:00:03      00:00:06
EXTRACT     RUNNING     EXT_P_01    00:00:03      00:00:03
EXTRACT     RUNNING     EXT_S_01    00:00:02      00:00:08
REPLICAT    RUNNING     REP_I_01    00:00:00      00:00:01
REPLICAT    RUNNING     REP_P_01    00:00:00      00:00:01
REPLICAT    RUNNING     REP_S_01    00:00:00      00:00:01

Golden Gate REPLICAT Error: ERROR OGG-01091 Unable to open file "XXX" (error 2, No such file or directory).

=========================
General
=========================
Oracle Golden Gate Replicat Process abends with following error:
ERROR   OGG-01091  Unable to open file "/software/ogg/191/dirdat/01/in/es000004665" (error 2, No such file or directory).

Checking the file system, indeed there is no such file:
There is indeed no such file.
First file being es000000000

ls -ltr   /software/ogg/191/dirdat/01/in/es*/
-rw-r----- 1 oracle dba 499997096 Feb 13 08:26 es000000000
-rw-r----- 1 oracle dba 499996646 Feb 13 08:26 es000000001
-rw-r----- 1 oracle dba 499999334 Feb 13 08:27 es000000002
-rw-r----- 1 oracle dba 499998299 Feb 13 08:27 es000000003
-rw-r----- 1 oracle dba 499998846 Feb 13 08:27 es000000004

=========================
Solution
=========================

Reset Replicat to start at first available file:

EXTSEQNO 0 - point to file es000000000

EXTRBA 0 - point to start position of the file

GGSCI> DBLOGIN USERID USER PASSWORD PASS

GSCI (qanfv-1-dbs-1b) 17> INFO ALL

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING

EXTRACT     RUNNING     DPM_I_01    00:00:00      00:00:09

EXTRACT     RUNNING     DPM_P_01    00:00:00      00:00:05

EXTRACT     RUNNING     DPM_S_01    00:00:03      00:00:04

EXTRACT     RUNNING     EXT_I_01    00:00:03      00:00:02

EXTRACT     RUNNING     EXT_P_01    00:00:02      00:00:09

EXTRACT     RUNNING     EXT_S_01    00:00:02      00:00:05

REPLICAT    RUNNING     REP_I_01    00:00:00      00:00:04

REPLICAT    RUNNING     REP_P_01    00:00:00      00:00:04

REPLICAT    ABENDED     REP_S_01    00:00:00      07:10:31

GGSCI> ALTER REPLICAT REP_S_01, EXTSEQNO 0 EXTRBA 0

REPLICAT altered.

GGSCI> START REPLICAT REP_S_01

REPLICAT REP_P_01 starting

GGSCI> INFO ALL

MANAGER     RUNNING

EXTRACT     RUNNING     DPM_I_01    00:00:00      00:00:02

EXTRACT     RUNNING     DPM_P_01    00:00:00      00:00:01

EXTRACT     RUNNING     DPM_S_01    00:00:00      00:00:04

EXTRACT     RUNNING     EXT_I_01    00:00:03      00:00:04

EXTRACT     RUNNING     EXT_P_01    00:00:00      00:00:03

EXTRACT     RUNNING     EXT_S_01    00:00:02      00:00:04

REPLICAT    RUNNING     REP_I_01    00:00:00      00:00:01

REPLICAT    RUNNING     REP_P_01    00:00:00      00:00:00

REPLICAT    RUNNING     REP_S_01    43:50:16      00:00:00

Since many files were pending replication, there is a backlog, which should be cleared alter some time

REPLICAT    RUNNING     REP_S_01    43:49:57      00:00:00

REPLICAT    RUNNING     REP_S_01    42:12:40      00:00:00

Note: This is not good!!
Because when starting without specific RBA, it will default to RBA from the old file, instead of starting with RBA=0. For example, it will try to start with EXTSEQNO 0 EXTRBA 8132, instead of EXTSEQNO 0 EXTRBA 0

GGSCI> ALTER REPLICAT REP_S_01 EXTSEQNO 0

REPLICAT altered.

GGSCI> INFO ALL

GGSCI> REPLICAT    ABENDED     REP_S_01    00:00:00      07:10:31

=========================
Additional info
=========================

Over weekend GG Replication was broken, and files filled up 100% of /software/ogg/191/dirdat/01/

Since this was a test environment, as a quick solution, existing files were deleted and GG started from scratch.

Golden Gate, Unregister and Register Extract by Example

===========================

Unregister and Register Extract

===========================
DBLOGIN USERID OGG PASSWORD XXXXXX

STOP EXTRACT EXT_I_01

    

DELETE EXTRACT EXT_I_01

REGISTER EXTRACT EXT_I_01 DATABASE

ADD EXTRACT EXT_I_01 INTEGRATED TRANLOG, BEGIN NOW

ADD EXTTRAIL /software/ogg/191/dirdat/01/out/ei EXTRACT EXT_I_01

 

===========================
Unregister and Register Datapump

===========================

DELETE EXTRACT DPM_I_01

ADD EXTRACT DPM_I_01 EXTTRAILSOURCE /software/ogg/191/dirdat/01/out/ei

ADD RMTTRAIL /software/ogg/191/dirdat/01/in/ei EXTRACT DPM_I_01

START EXTRACT DPM_I_01

INFO DPM_I_01

===========================

Unregister and Register Replicat

===========================

STOP REPLICAT REP_I_01

DELETE REPLICAT REP_I_01

ADD REPLICAT REP_I_01,  EXTTRAIL /software/ogg/191/dirdat/01/in/ei

START REPLICAT REP_I_01

===========================

Unregister and Register Group

===========================

DBLOGIN USERID OGG, PASSWORD XXXXXX

STOP EXTRACT EXT_N_01

UNREGISTER EXTRACT EXT_N_01 DATABASE 

DELETE EXTRACT EXT_N_01

REGISTER EXTRACT EXT_N_01 DATABASE

ADD EXTRACT EXT_N_01 INTEGRATED TRANLOG, BEGIN NOW

ADD EXTTRAIL /software/ogg/191/dirdat/01/out/en EXTRACT EXT_N_01

START EXTRACT EXT_N_01

STOP EXTRACT DPM_N_01

DELETE EXTRACT DPM_N_01

ADD EXTRACT DPM_N_01 EXTTRAILSOURCE /software/ogg/191/dirdat/01/out/en

ADD RMTTRAIL /software/ogg/191/dirdat/01/in/en EXTRACT DPM_N_01

START EXTRACT DPM_N_01

STOP REPLICAT REP_N_01

DELETE REPLICAT REP_N_01

ADD REPLICAT REP_N_01, INTEGRATED EXTTRAIL /software/ogg/191/dirdat/01/in/en 

START REPLICAT REP_N_01

Whenever starting a GG processes, it is logged

alert log:

GoldenGate Apply Reader AS01 for OGG$REP_N_01 with pid=158 OS id=17350 stopped

2021-06-28T06:32:25.564814+00:00

GoldenGate APPLY AP05 for OGG$REP_N_01 with pid=157, OS id=17346 stopped

2021-06-28T06:32:33.662778+00:00

GoldenGate Apply: OGG$REP_N_01 APPLY Dropped

2021-06-28T06:32:33.663002+00:00

APPLY OGG$REP_N_01: Apply User: OGG

2021-06-28T06:32:33.663083+00:00

APPLY OGG$REP_N_01: Apply Tag: 00

2021-06-28T06:32:47.704457+00:00

GoldenGate Apply: OGG$REP_N_01 APPLY Created

2021-06-28T06:32:47.752593+00:00

APPLY OGG$REP_N_01: Apply User: OGG

2021-06-28T06:32:47.752693+00:00

APPLY OGG$REP_N_01: Apply Tag: 00

2021-06-28T06:32:48.323311+00:00

GoldenGate APPLY AP05 for OGG$REP_N_01 started with pid=108, OS id=22864

2021-06-28T06:32:48.329577+00:00

APPLY OGG$REP_N_01: Apply User: OGG

2021-06-28T06:32:48.329670+00:00

APPLY OGG$REP_N_01: Apply Tag: 00

2021-06-28T06:32:48.330235+00:00

APPLY OGG$REP_N_01: Parameter Set by User: ALLOW_DUPLICATE_ROWS Value: Y

2021-06-28T06:32:48.330341+00:00

APPLY OGG$REP_N_01: Parameter Set by User: PRESERVE_ENCRYPTION Value: N

2021-06-28T06:32:48.330442+00:00

APPLY OGG$REP_N_01: Parameter Set by User: EAGER_SIZE Value: 15100

2021-06-28T06:32:48.330524+00:00

APPLY OGG$REP_N_01: Parameter Set by User: CDGRANULARITY Value: ROW

2021-06-28T06:32:48.330606+00:00

APPLY OGG$REP_N_01: Parameter Set by User: SUPPRESSTRIGGERS Value: Y

2021-06-28T06:32:48.330687+00:00

APPLY OGG$REP_N_01: Parameter Set by User: _GG_ERROR_HANDLING_MODE Value: ASYNC

2021-06-28T06:32:48.330907+00:00

APPLY OGG$REP_N_01: Parameter Set by User: PARALLELISM Value: 8

2021-06-28T06:32:48.893939+00:00

In ggserr.log

2021-06-28T06:32:26.092+0000  INFO    OGG-00994  Oracle GoldenGate Delivery for Oracle, rep_n_01.prm:  REPLICAT REP_N_01 stopped normal

ly.

2021-06-28T06:32:26.141+0000  INFO    OGG-00987  Oracle GoldenGate Command Interpreter for Oracle:  GGSCI command (oracle): STOP REPLIC

AT REP_N_01.

2021-06-28T06:32:26.601+0000  ERROR   OGG-15163  Oracle GoldenGate Command Interpreter for Oracle:  There was a problem sending a messa

ge to REPLICAT REP_N_01 (Connection reset by peer).

2021-06-28T06:32:29.114+0000  INFO    OGG-00987  Oracle GoldenGate Command Interpreter for Oracle:  GGSCI command (oracle): STOP REPLIC

AT REP_N_01.

2021-06-28T06:32:33.465+0000  INFO    OGG-00987  Oracle GoldenGate Command Interpreter for Oracle:  GGSCI command (oracle): DELETE REPL

ICAT REP_N_01.

2021-06-28T06:32:34.340+0000  INFO    OGG-02529  Oracle GoldenGate Command Interpreter for Oracle:  Successfully unregistered REPLICAT

REP_N_01 inbound server OGG$REP_N_01 from database.

2021-06-28T06:32:34.343+0000  INFO    OGG-14034  Oracle GoldenGate Command Interpreter for Oracle:  Heartbeat entries with [REP_N_01] d

eleted.

2021-06-28T06:32:39.946+0000  INFO    OGG-00987  Oracle GoldenGate Command Interpreter for Oracle:  GGSCI command (oracle): ADD REPLICA

T REP_N_01  INTEGRATED EXTTRAIL /software/ogg/191/dirdat/01/in/en.

2021-06-28T06:32:43.378+0000  INFO    OGG-00987  Oracle GoldenGate Command Interpreter for Oracle:  GGSCI command (oracle): START REPLI

CAT REP_N_01.

2021-06-28T06:32:43.380+0000  INFO    OGG-00963  Oracle GoldenGate Manager for Oracle, mgr.prm:  Command received from GGSCI on host 10

.10.51.39:50218 (START REPLICAT REP_N_01 ).

2021-06-28T06:32:43.385+0000  INFO    OGG-00975  Oracle GoldenGate Manager for Oracle, mgr.prm:  REPLICAT REP_N_01 starting.

2021-06-28T06:32:43.804+0000  INFO    OGG-00995  Oracle GoldenGate Delivery for Oracle, rep_n_01.prm:  REPLICAT REP_N_01 starting.

2021-06-28T06:32:43.804+0000  INFO    OGG-03059  Oracle GoldenGate Delivery for Oracle, rep_n_01.prm:  Operating system character set i

dentified as UTF-8.

2021-06-28T06:32:43.804+0000  INFO    OGG-02695  Oracle GoldenGate Delivery for Oracle, rep_n_01.prm:  ANSI SQL parameter syntax is use

d for parameter parsing.

2021-06-28T06:32:43.804+0000  INFO    OGG-02095  Oracle GoldenGate Delivery for Oracle, rep_n_01.prm:  Successfully set environment variable ORACLE_SID=igt.

2021-06-28T06:32:43.804+0000  INFO    OGG-02095  Oracle GoldenGate Delivery for Oracle, rep_n_01.prm:  Successfully set environment variable ORACLE_HOME=/software/oracle/122.

2021-06-28T06:32:43.804+0000  INFO    OGG-02095  Oracle GoldenGate Delivery for Oracle, rep_n_01.prm:  Successfully set environment variable NLS_LANG=AMERICAN_AMERICA.AL32UTF8.

2021-06-28T06:32:43.944+0000  INFO    OGG-01360  Oracle GoldenGate Delivery for Oracle, rep_n_01.prm:  REPLICAT is running in Integrated mode.

2021-06-28T06:32:44.513+0000  INFO    OGG-06451  Oracle GoldenGate Delivery for Oracle, rep_n_01.prm:  Triggers will be suppressed by default.

2021-06-28T06:32:44.524+0000  INFO    OGG-01815  Oracle GoldenGate Delivery for Oracle, rep_n_01.prm:  Virtual Memory Facilities for: COM

                                  anon alloc: mmap(MAP_ANON)  anon free: munmap

                                  file alloc: mmap(MAP_SHARED)  file free: munmap

                                  target directories:

                                  /software/ogg/191/dirtmp.

2021-06-28T06:32:44.535+0000  INFO    OGG-06604  Oracle GoldenGate Delivery for Oracle, rep_n_01.prm:  Database IGT CPU info: CPU Count 16, CPU Core Count 16, CPU Socket Count 2.

2021-06-28T06:32:44.618+0000  INFO    OGG-02545  Oracle GoldenGate Delivery for Oracle, rep_n_01.prm:  Parameter GROUPTRANSOPS is ignored by Integrated Replicat when parallelism is greater than 1.

2021-06-28T06:32:44.618+0000  INFO    OGG-02527  Oracle GoldenGate Delivery for Oracle, rep_n_01.prm:  Integrated Replicat does not populate a trace table.

2021-06-28T06:32:47.777+0000  INFO    OGG-02528  Oracle GoldenGate Delivery for Oracle, rep_n_01.prm:  REPLICAT REP_N_01 successfully registered with database as inbound server OGG$REP_N_01.

2021-06-28T06:32:48.271+0000  INFO    OGG-00996  Oracle GoldenGate Delivery for Oracle, rep_n_01.prm:  REPLICAT REP_N_01 started.

2021-06-28T06:32:49.439+0000  INFO    OGG-02530  Oracle GoldenGate Delivery for Oracle, rep_n_01.prm:  Integrated replicat successfully attached to inbound server OGG$REP_N_01.

===================

Starhome Stuff

===================

DBLOGIN USERID OGG PASSWORD rad21mif

I_01

STOP EXTRACT EXT_I_01

SEND EXTRACT EXT_I_01, FORCESTOP

UNREGISTER EXTRACT EXT_I_01 DATABASE 

DELETE EXTRACT EXT_I_01

REGISTER EXTRACT EXT_I_01 DATABASE

ADD EXTRACT EXT_I_01 INTEGRATED TRANLOG, BEGIN NOW

ADD EXTTRAIL /software/ogg/191/dirdat/01/out/ei EXTRACT EXT_I_01

START EXTRACT EXT_I_01

STOP EXTRACT DPM_I_01

SEND EXTRACT DPM_I_01, FORCESTOP

DELETE EXTRACT DPM_I_01

ADD EXTRACT DPM_I_01 EXTTRAILSOURCE /software/ogg/191/dirdat/01/out/ei

ADD RMTTRAIL /software/ogg/191/dirdat/01/in/ei EXTRACT DPM_I_01

START EXTRACT DPM_I_01

INFO DPM_I_01

STOP REPLICAT REP_I_01

SEND REPLICAT REP_I_01, FORCESTOP

DELETE REPLICAT REP_I_01

ADD REPLICAT REP_I_01, EXTTRAIL /software/ogg/191/dirdat/01/in/ei

START REPLICAT REP_I_01

S_01

STOP EXTRACT EXT_S_01
SEND EXTRACT EXT_S_01, FORCESTOP

UNREGISTER EXTRACT EXT_S_01 DATABASE 

DELETE EXTRACT EXT_S_01

REGISTER EXTRACT EXT_S_01 DATABASE

ADD EXTRACT EXT_S_01 INTEGRATED TRANLOG, BEGIN NOW

ADD EXTTRAIL /software/ogg/191/dirdat/01/out/es EXTRACT EXT_S_01

START EXTRACT EXT_S_01

STOP EXTRACT DPM_S_01

SEND EXTRACT DPM_S_01, FORCESTOP

DELETE EXTRACT DPM_S_01

ADD EXTRACT DPM_S_01 EXTTRAILSOURCE /software/ogg/191/dirdat/01/out/es

ADD RMTTRAIL /software/ogg/191/dirdat/01/in/es EXTRACT DPM_S_01

START EXTRACT DPM_S_01

STOP REPLICAT REP_S_01

SEND REPLICAT REP_S_01, FORCESTOP

DELETE REPLICAT REP_S_01

ADD REPLICAT REP_S_01, INTEGRATED EXTTRAIL /software/ogg/191/dirdat/01/in/es

START REPLICAT REP_S_01

P_01

STOP EXTRACT EXT_P_01

SEND EXTRACT EXT_P_01, FORCESTOP

UNREGISTER EXTRACT EXT_P_01 DATABASE 

DELETE EXTRACT EXT_P_01

REGISTER EXTRACT EXT_P_01 DATABASE

ADD EXTRACT EXT_P_01 INTEGRATED TRANLOG, BEGIN NOW

ADD EXTTRAIL /software/ogg/191/dirdat/01/out/ep EXTRACT EXT_P_01

START EXTRACT EXT_P_01

STOP EXTRACT DPM_P_01

SEND EXTRACT DPM_P_01, FORCESTOP

DELETE EXTRACT DPM_P_01

ADD EXTRACT DPM_P_01 EXTTRAILSOURCE /software/ogg/191/dirdat/01/out/ep

ADD RMTTRAIL /software/ogg/191/dirdat/01/in/ep EXTRACT DPM_P_01

START EXTRACT DPM_P_01

STOP REPLICAT REP_P_01

SEND REPLICAT REP_P_01, FORCESTOP

DELETE REPLICAT REP_P_01

ADD REPLICAT REP_P_01, INTEGRATED EXTTRAIL /software/ogg/191/dirdat/01/in/ep

START REPLICAT REP_P_01

===================

Check Status

===================