Some time back I was asked the question as to how do we use GoldenGate in a situation where on the target database we only want to capture records inserted in the source database and ignore any updates being made to existing rows in the source database.
For this we can use the IGNOREUPDATES parameter which is valid for both the Extract as well as Replicat parameter files to inform GoldenGate to selectively ignore any update operations. This parameter is table specific and will apply to all tables mentioned in the subsequent TABLE or MAP statements until the GETUPDATES parameter is used. Note that GETUPDATES is the default.
In this example we will also see how delete operations on source database are ignored using the IGNOREDELETES parameter.
Let us create a simple table on both source as well as target database with the following structure:
SQL> create table mytab
2 (id number, comments varchar2(20));
Table created.
SQL> alter table mytab add constraint pk_mytab primary key (id);
Table altered.
We then create the extract process Testext on source and replicat process Testrep on target.
This is our Extract parameter file:
extract testext
userid ggs_owner, password ggs_owner
rmthost 10.32.20.62, mgrport 7809
rmttrail /u01/app/goldengate/dirdat/gg
table sh.mytab;
This is our Replicat parameter file:
REPLICAT testrep
ASSUMETARGETDEFS
USERID ggs_owner,PASSWORD ggs_owner
IGNOREDELETES
IGNOREUPDATES
MAP SH.MYTAB, TARGET SH.MYTAB;
Let us now test the same by inserting a row into the source table
SQL> insert into mytab
2 values
3 (1,’INSERTED row’);
1 row created.
SQL> commit;
Then check the target table for the inserted row.
SQL> select * from mytab;
ID COMMENTS
———- ——————–
1 INSERTED row
We now go and update the existing row on the target.
SQL> update mytab
2 set comments=’UPDATED row’
3 where id=1;
1 row updated.
SQL> commit;
SQL> select * from mytab;
ID COMMENTS
———- ——————–
1 UPDATED row
On the target, we see that the update to the row has not happened on the target database.
SQL> select * from mytab;
ID COMMENTS
———- ——————–
1 INSERTED row
Let us now delete the existing record on the source database.
SQL> delete mytab;
1 row deleted.
SQL> commit;
Commit complete.
Check the target. We see that the row has not been deleted from the target database.
SQL> select * from mytab;
ID COMMENTS
———- ——————–
1 INSERTED row
On the source GoldenGate environment let us examine the statistics for the Extract process. We see that 3 operations have happened. This is made up of one insert, update and delete operation.
GGSCI (pdemora061rhv.asgdemo.asggroup.com.au) 66> stats extract testext
Sending STATS request to EXTRACT TESTEXT …
Start of Statistics at 2012-07-21 04:51:36.
Output to /u01/app/goldengate/dirdat/gg:
Extracting from SH.MYTAB to SH.MYTAB:
*** Total statistics since 2012-07-21 04:48:33 ***
Total inserts 1.00
Total updates 1.00
Total deletes 1.00
Total discards 0.00
Total operations 3.00
On the target however we see that only one single Insert operation has taken place.
GGSCI (pdemora062rhv.asgdemo.asggroup.com.au) 27> stats replicat testrep
Sending STATS request to REPLICAT TESTREP …
Start of Statistics at 2012-07-21 04:52:35.
Replicating from SH.MYTAB to SH.MYTAB:
*** Total statistics since 2012-07-21 04:48:29 ***
Total inserts 1.00
Total updates 0.00
Total deletes 0.00
Total discards 0.00
Total operations 1.00
Ok. Now the source table has no rows while the target table has one row.
What happens when we insert two rows into the source table?
SQL> insert into mytab
2 values
3 (1,’INSERTED row’);
1 row created.
SQL> insert into mytab
2 values
3 (2,’INSERTED row’);
1 row created.
SQL> commit;
Commit complete.
Since the row with ID=1 already existed in the target database (because it was not deleted when the delete happened on the source), the subsequent insert fails and we see this error in the replicat log file.
2012-07-21 04:54:17 WARNING OGG-00869 OCI Error ORA-00001: unique constraint (SH.PK_MYTAB) violated (status = 1). INSERT INTO “SH”.”MYTAB” (“ID”,”COMMENTS”) VALUES (:a0,:a1).
2012-07-21 04:54:17 WARNING OGG-01004 Aborted grouped transaction on ‘SH.MYTAB’, Database error 1 (OCI Error ORA-00001: unique constraint (SH.PK_MYTAB) violated (status = 1).
We need to tell the Replicat process that it needs to ignore the insert for the row which already exists and for this purpose we use the GoldenGate utility Logdump to examine the contents of the trail files.
We then find the RBA (Relative Byte Address) for the second insert (ID=2) and will use that RBA to tell the Replicat process to start processing not from the beginning of the trail but from a point in the trail file indicated by the RBA value which we will provide to the ALTER REPLICAT command.
We navigate through the trail file using the ‘n’ command until we find the record where the ID=2.
We can see the first INSERT, then the UPDATE and then the DELETE operation. We then see the second INSERT which we are interested in.
[oracle@pdemora062rhv goldengate]$ logdump
Oracle GoldenGate Log File Dump Utility for Oracle
Version 11.2.1.0.1 OGGCORE_11.2.1.0.1_PLATFORMS_120423.0230
Copyright (C) 1995, 2012, Oracle and/or its affiliates. All rights reserved.
Logdump 41 >open /u01/app/goldengate/dirdat/gg000000
Current LogTrail is /u01/app/goldengate/dirdat/gg000000
Logdump 42 >ghdr on
Logdump 43 >detail on
Logdump 44 >n
2012/07/21 04:45:13.522.696 FileHeader Len 1087 RBA 0
Name: *FileHeader*
3000 01cd 3000 0008 4747 0d0a 544c 0a0d 3100 0002 | 0…0…GG..TL..1…
0003 3200 0004 2000 0000 3300 0008 02f1 eb7c 6dc9 | ..2… …3……|m.
9208 3400 003f 003d 7572 693a 7064 656d 6f72 6130 | ..4..?.=uri:pdemora0
3631 7268 763a 6173 6764 656d 6f3a 6173 6767 726f | 61rhv:asgdemo:asggro
7570 3a63 6f6d 3a61 753a 3a75 3031 3a61 7070 3a67 | up:com:au::u01:app:g
6f6c 6465 6e67 6174 6536 0000 2500 232f 7530 312f | oldengate6..%.#/u01/
6170 702f 676f 6c64 656e 6761 7465 2f64 6972 6461 | app/goldengate/dirda
Logdump 45 >n
___________________________________________________________________
Hdr-Ind : E (x45) Partition : . (x04)
UndoFlag : . (x00) BeforeAfter: A (x41)
RecLength : 29 (x001d) IO Time : 2012/07/21 04:48:15.066.791
IOType : 5 (x05) OrigNode : 255 (xff)
TransInd : . (x03) FormatType : R (x52)
SyskeyLen : 0 (x00) Incomplete : . (x00)
AuditRBA : 401 AuditPos : 32627632
Continued : N (x00) RecCount : 1 (x01)
2012/07/21 04:48:15.066.791 Insert Len 29 RBA 1095
Name: SH.MYTAB
After Image: Partition 4 G s
0000 0005 0000 0001 3100 0100 1000 0000 0c49 4e53 | ……..1……..INS
4552 5445 4420 726f 77 | ERTED row
Column 0 (x0000), Len 5 (x0005)
Column 1 (x0001), Len 16 (x0010)
Logdump 46 >n
___________________________________________________________________
Hdr-Ind : E (x45) Partition : . (x04)
UndoFlag : . (x00) BeforeAfter: A (x41)
RecLength : 28 (x001c) IO Time : 2012/07/21 04:50:37.094.598
IOType : 15 (x0f) OrigNode : 255 (xff)
TransInd : . (x03) FormatType : R (x52)
SyskeyLen : 0 (x00) Incomplete : . (x00)
AuditRBA : 401 AuditPos : 33013776
Continued : N (x00) RecCount : 1 (x01)
2012/07/21 04:50:37.094.598 FieldComp Len 28 RBA 1235
Name: SH.MYTAB
After Image: Partition 4 G s
0000 0005 0000 0001 3100 0100 0f00 0000 0b55 5044 | ……..1……..UPD
4154 4544 2072 6f77 | ATED row
Column 0 (x0000), Len 5 (x0005)
Column 1 (x0001), Len 15 (x000f)
Logdump 47 >n
___________________________________________________________________
Hdr-Ind : E (x45) Partition : . (x04)
UndoFlag : . (x00) BeforeAfter: B (x42)
RecLength : 9 (x0009) IO Time : 2012/07/21 04:51:00.119.007
IOType : 3 (x03) OrigNode : 255 (xff)
TransInd : . (x03) FormatType : R (x52)
SyskeyLen : 0 (x00) Incomplete : . (x00)
AuditRBA : 401 AuditPos : 33041936
Continued : N (x00) RecCount : 1 (x01)
2012/07/21 04:51:00.119.007 Delete Len 9 RBA 1374
Name: SH.MYTAB
Before Image: Partition 4 G s
0000 0005 0000 0001 31 | ……..1
Column 0 (x0000), Len 5 (x0005)
Logdump 48 >n
___________________________________________________________________
Hdr-Ind : E (x45) Partition : . (x04)
UndoFlag : . (x00) BeforeAfter: A (x41)
RecLength : 29 (x001d) IO Time : 2012/07/21 04:54:12.124.542
IOType : 5 (x05) OrigNode : 255 (xff)
TransInd : . (x03) FormatType : R (x52)
SyskeyLen : 0 (x00) Incomplete : . (x00)
AuditRBA : 401 AuditPos : 33273872
Continued : N (x00) RecCount : 1 (x01)
2012/07/21 04:54:12.124.542 Insert Len 29 RBA 1494
Name: SH.MYTAB
After Image: Partition 4 G s
0000 0005 0000 0001 3100 0100 1000 0000 0c49 4e53 | ……..1……..INS
4552 5445 4420 726f 77 | ERTED row
Column 0 (x0000), Len 5 (x0005)
Column 1 (x0001), Len 16 (x0010)
Logdump 49 >n
___________________________________________________________________
Hdr-Ind : E (x45) Partition : . (x04)
UndoFlag : . (x00) BeforeAfter: A (x41)
RecLength : 29 (x001d) IO Time : 2012/07/21 05:02:04.056.719
IOType : 5 (x05) OrigNode : 255 (xff)
TransInd : . (x03) FormatType : R (x52)
SyskeyLen : 0 (x00) Incomplete : . (x00)
AuditRBA : 401 AuditPos : 36183568
Continued : N (x00) RecCount : 1 (x01)
2012/07/21 05:02:04.056.719 Insert Len 29 RBA 1634
Name: SH.MYTAB
After Image: Partition 4 G s
0000 0005 0000 0001 3200 0100 1000 0000 0c49 4e53 | ……..2……..INS
4552 5445 4420 726f 77 | ERTED row
Column 0 (x0000), Len 5 (x0005)
Column 1 (x0001), Len 16 (x0010)
The Replicat process which has abended, is now altered to start at a specific RBA and then restarted.
We use the ALTER REPLICAT testrep EXTRBA 1634 command to reposition the replicat process to start reading records from a specific position in the trail file.
We now see that the replicat has started running and has processed the second insert statement.
SQL> select * from mytab;
ID COMMENTS
———- ——————–
1 INSERTED row
2 INSERTED row
Statistics now show 2 insert operations – note no deletes and updates processed ….
GGSCI (pdemora062rhv.asgdemo.asggroup.com.au) 2> stats replicat testrep
Sending STATS request to REPLICAT TESTREP …
Start of Statistics at 2012-07-21 06:52:43.
Replicating from SH.MYTAB to SH.MYTAB:
*** Total statistics since 2012-07-21 05:19:12 ***
Total inserts 2.00
Total updates 0.00
Total deletes 0.00
Total discards 0.00
Total operations 2.00