REPLICAT is stuck at same point

 ===========================
General
===========================
Replicat process is stuck.

It has big lag, and it is not coming down
What to check?

===========================
Steps - check REPLICAT
===========================

When running the SEND REPLICAT REP_I_01 STATUS

It seems that the replicat is stuck on the same file.

When checking for dirdat/in files, this is the last file received from the other node.

Need to check on DATAPUMP EXTRACT on the other node

===========================
Steps - check DATAPUMP
===========================

Checking on Datapump Extract

The extract status is RUNNING but it is stuck.

When running SEND EXTRACT DPM_I_01 STATUS

It is stuck on the same trunsaction

===========================
Steps - Fix DATAPUMP issue
===========================

GGSCI (qanfv-1-dbs-1b) 68> STOP EXTRACT DPM_S_01

Sending STOP request to EXTRACT DPM_S_01 ...

STOP request pending end-of-transaction (1,034 records so far).

GGSCI (qanfv-1-dbs-1b) 69> kill EXTRACT DPM_S_01

Sending KILL request to MANAGER ...

Killed process (31058) for EXTRACT DPM_S_01

GGSCI (qanfv-1-dbs-1b) 90> INFO DPM_S_01

EXTRACT    DPM_S_01  Last Started 2021-06-27 12:22   Status ABENDED

Checkpoint Lag       22:57:23 (updated 00:00:01 ago)

Log Read Checkpoint  File /software/ogg/191/dirdat/01/out/es000021373

                     2021-06-26 13:25:39.000000  RBA 450922877

2021-06-27 12:23:02  ERROR   OGG-01091  Unable to open file "/software/ogg/191/dirdat/01/out/es000021373" (error 2, No such file or directory).

****************************************************************

DATAPUMP Extract is expecting for next file, 21373, but this file does not exists.

Extract should be skipped to next available file 21603

GGSCI (qanfv-1-dbs-1b) 93> ALTER EXTRACT DPM_S_01 extseqno 21603

EXTRACT altered.

GGSCI (qanfv-1-dbs-1b) 94> START  EXTRACT DPM_S_01

Sending START request to MANAGER ...

EXTRACT DPM_S_01 starting

Now, Datapump is sending records to the other node, and the REPLICAT is processing data, and the Lag is reducing.