General
=======================================
Large Oracle trace file (~600 Mb) are generated.
What could be the cause?
=======================================
Evidences
=======================================
In alert.log there is only this non informative error message:
Stopping background process CJQ0
Checking the trace files timestamp, they all were generated at apprximately same time each day, at ~23:51.
In all trace files, there is same error message: "dump suspect buffer"
But the trace file has also session detalis.
-rw-r----- 1 oracle dba 650,767,781 Feb 3 23:52
/software/oracle/diag/rdbms/igt/igt/trace/igt_j000_16517.trc
Trace file /software/oracle/diag/rdbms/igt/igt/trace/igt_j000_16517.trc
Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
With the Partitioning, Oracle Label Security, OLAP, Data Mining,
Oracle Database Vault and Real Application Testing options
ORACLE_HOME = /software/oracle/111
System name: Linux
Node name: arg-cti-2-aps-1
Release: 2.6.18-274.el5
Version: #1 SMP Fri Jul 8 17:36:59 EDT 2011
Machine: x86_64
Instance name: igt
Redo thread mounted by this instance: 1
Oracle process number: 175
Unix process pid: 16517, image: oracle@arg-cti-2-aps-1 (J000)
*** 2018-02-03 23:50:24.551
*** SESSION ID:(164.47150) 2018-02-03 23:50:24.551
*** CLIENT ID:() 2018-02-03 23:50:24.551
*** SERVICE NAME:(SYS$USERS) 2018-02-03 23:50:24.551
*** MODULE NAME:() 2018-02-03 23:50:24.551
*** ACTION NAME:() 2018-02-03 23:50:24.551
kcbzib: dump suspect buffer
buffer tsn: 4 rdba: 0x01007af2 (4/31474)
scn: 0x0dac.ef6f7cc4 seq: 0x01 flg: 0x04 tail: 0x7cc42301
frmt: 0x02 chkval: 0xe4b3 type: 0x23=PAGETABLE SEGMENT HEADER
Hex dump of block: st=0, typ_found=1
Dump of memory from 0x0000000082AA8000 to 0x0000000082AAA000
=======================================
Investigation
=======================================
Checking DBA_HIST tables by session_id.
FROM DBA_HIST_ACTIVE_SESS_HISTORY
WHERE session_id = '164' AND session_serial# = 47150;
user_id = 49
sql_id cnsp07td49muk
SELECT * FROM SBA_HIST_SQLTEXT
WHERE sql_id = 'cnsp07td49muk';
SELECT * FROM DBA_USERS WHERE user_id = 49;
--MY_USER
SELECT * FROM DBA_JOBS WHERE schema_user = 'MY_USER'
Indeed there is a job, which is scheduled to run at 23:50 Daily.
Checking that job code, there is the SQL which sql_id is referenced by 'cnsp07td49muk';
Error during procedure: ORA-08103: object no longer exists
=======================================
Resolution
=======================================
Following the error message ORA-08103: object no longer exists, additional jobs which where running at same time were investigated.
Apparently, there was another job, performing a purge of old data from same table, by ALTER TABLE XXX TRUNCATE PARTITION YYY;
This was the cause for error ORA-08103: object no longer exists error.
Issue was resolved, by rescheduling the failing job, so it would not run at he same time with the purge data job.
No comments:
Post a Comment