We will look at an example of GoldenGate replication of a table having a BLOB column and how an INSERT and UPDATE statement on the table with BLOB data is handled by GoldenGate.

We create an APEX 4.2 application to illustrate this example where we create a form and report based on the DOCUMENTS table and upload and download documents. We will observe how changes to the BLOB column data are replicated in real-time to the target database via GoldenGate change data capture.

Thanks to ACE Director Eddie Awad’s article which made me understand how APEX handles file upload and downloads.
Read the article by Eddie.

On the source database we create the DOCUMENTS table and a sequence and trigger to populate the primary key column ID.

CREATE TABLE documents
(
   ID              NUMBER PRIMARY KEY
  ,DOC_CONTENT    BLOB
  ,MIME_TYPE       VARCHAR2 (255)
  ,FILENAME        VARCHAR2 (255)
  ,LAST_UPDATED    DATE
  ,CHARACTER_SET   VARCHAR2 (128)
);

CREATE SEQUENCE documents_seq;

CREATE OR REPLACE TRIGGER documents_trg_bi
   BEFORE INSERT
   ON documents
   FOR EACH ROW
BEGIN
   :new.id := documents_seq.NEXTVAL;
END;
/

Create the Extract and Replicat processes

GGSCI (vindi-a) 3> add extract ext9 tranlog begin now
EXTRACT added.

GGSCI (vindi-a) 4> add rmttrail /u01/app/oracle/product/st_goldengate/dirdat/xx extract ext9
RMTTRAIL added.

GGSCI (vindi-a) 5> edit params ext9
extract ext9
CACHEMGR CACHESIZE 8G
userid gg_owner@testdb password gg_owner
DDL include ALL
ddloptions  addtrandata, report
rmthost poc-strelis-vindi, mgrport 7810
rmttrail  /u01/app/oracle/product/st_goldengate/dirdat/xx
dynamicresolution
SEQUENCE GG_OWNER.*;
TABLE GG_OWNER.DOCUMENTS;

GGSCI (vindi-a) 1> start extract ext9

Sending START request to MANAGER ...
EXTRACT EXT9 starting

GGSCI (vindi-a) 2> info extract ext9

EXTRACT    EXT9      Last Started 2014-05-22 08:43   Status RUNNING
Checkpoint Lag       00:00:00 (updated 00:04:54 ago)
Process ID           17984
Log Read Checkpoint  Oracle Redo Logs
                     2014-05-22 08:38:44  Seqno 122, RBA 9039376
                     SCN 0.0 (0)

On Target GoldenGate 

GGSCI (vindi-a) 3> add replicat rep9 exttrail /u01/app/oracle/product/st_goldengate/dirdat/xx
REPLICAT added.

GGSCI (vindi-a) 4> edit params rep9
replicat rep9
assumetargetdefs
ddlerror default ignore
userid gg_owner@strelis password gg_owner
MAP GG_OWNER.DOCUMENTS ,TARGET GG_OWNER.DOCUMENTS;

GGSCI (vindi-a) 5> start replicat rep9

Sending START request to MANAGER ...
REPLICAT REP9 starting

GGSCI (vindi-a) 6> info replicat rep9

REPLICAT   REP9      Last Started 2014-05-22 08:43   Status RUNNING
Checkpoint Lag       00:00:00 (updated 00:00:04 ago)
Process ID           17919
Log Read Checkpoint  File /u01/app/oracle/product/st_goldengate/dirdat/xx000000
                     First Record  RBA 0

Since we have configured DDL replication as well we see that the DOCUMENTS table has been created on the target database as well.

oracle@vind-a:/export/home/oracle $ sqlplus gg_owner/gg_owner@targetdb

SQL*Plus: Release 11.2.0.3.0 Production on Thu May 22 08:35:55 2014

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> desc documents
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 ID                                        NOT NULL NUMBER
 DOC_CONTENT                                        BLOB
 MIME_TYPE                                          VARCHAR2(255)
 FILENAME                                           VARCHAR2(255)
 LAST_UPDATED                                       DATE
 CHARACTER_SET                                      VARCHAR2(128)

We now launch APEX to create our demo application.

In Application Builder click on Create

 
Select Database
 

 

 
Click Add Page
 

 
Click Next
 

 
Accept default value
 

 
Accept default values
 

 

 
Click Create Application
 

 
Click Page 1 link
 

 
From Regions menu click Create
 

 
Select Form and click Next
 

 
Select Form on a Table or View
 

 
Select DOCUMENTS table from LOV and click Next
 

 
Enter the Page name  and click Next
 

 
Select the primary key column of the DOCUMENTS table and click Next
 

 
Primary key of the table is populated via the sequence called by the trigger
 
Select Existing trigger and click Next
 

 
Select columns to display on the form and click Next
 

 
Change the label of the Create button to Upload and hide other buttons – click Next
 

 
Select the current page as the page to branch to and click Next
 

 
Click Create
 

 
Click Edit Page
 

 
Select the P1_DOC_CONTENT item and click Edit from the menu
 

 
In the Settings section of the page add the table column names against the columns as shown above
 
Click on Apply Changes
 

 

 
Click on Run
 

 
Enter workspace or application login credentials
 

 
Click the Browse button and select the file to upload
 

 
Click Upload
 

 
In GoldenGate we check the extract and replication stats and we can see the capture and apply of the change we just made
 

GGSCI (vind-a) 3> stats extract ext9 latest

Sending STATS request to EXTRACT EXT9 ...

Start of Statistics at 2014-05-22 09:01:44.

DDL replication statistics (for all trails):

*** Total statistics since extract started     ***
        Operations                                         0.00
        Mapped operations                                  0.00
        Unmapped operations                                0.00
        Other operations                                   0.00
        Excluded operations                                0.00

Output to /u01/app/oracle/product/st_goldengate/dirdat/xx:

Extracting from GG_OWNER.DOCUMENTS_SEQ to GG_OWNER.DOCUMENTS_SEQ:

*** Latest statistics since 2014-05-22 08:59:16 ***
        Total updates                                      1.00
        Total discards                                     0.00
        Total operations                                   1.00

Extracting from GG_OWNER.DOCUMENTS to GG_OWNER.DOCUMENTS:

*** Latest statistics since 2014-05-22 08:59:16 ***
        Total inserts                                      1.00
        Total updates                                      1.00
        Total deletes                                      0.00
        Total discards                                     0.00
        Total operations                                   2.00

End of Statistics.

 
Connect to the target database and see if the record has been inserted.
 
Note the colums like MIME_TYPE, LAST_UPDATED, FILENAME etc are automatically populated.
 

SQL> col filename format a60
SQL> col mime_type format a30
SQL> set linesize 120
SQL> select id,filename,mime_type from documents;

        ID FILENAME                                                     MIME_TYPE
---------- ------------------------------------------------------------ ------------------------------
         1 Consultant Profile - Gavin Soorma.doc                        application/msword

Note the size of the document

SQL> select id,filename,dbms_lob.getlength(doc_content)  from documents;

        ID FILENAME                                                     DBMS_LOB.GETLENGTH(DOC_CONTENT)
---------- ------------------------------------------------------------ -------------------------------
         1 Consultant Profile - Gavin Soorma.doc                                                 532992

We will now add a new page to the application. Click Create Page

Select Form and click Next

Select Form on a Table with Report and click Next

Change the Region Title to Edit Documents and click Next

Select the table and click Next

Give a name for the tab for the new page we are creating  and click Next

Select the columns to include in the report and click Next

Accept default and click Next

Change the Region Title and click Next

Select Primary key column and click Next

Select the columns to include in the form and click Next

Click Create

Click Run Page

Click the Edit icon

Click the Edit Page link at the bottom of the page

In the Settings section of the page add the table column names as shown

Click Apply Changes and then Run

We will now download the document from the table, edit the document and upload it back into the database again

Click on the Download link and save the document

Open the document and we will make some changes

We will delete the “Technical Skills” table from the document, save it and then upload it back again

Click on Browse and upload the document which we just downloaded and edited

 Click Apply Changes

Oracle GoldenGate has applied this change and we can see that the size of the document has reduced in the target database from 532992 bytes to 528896 bytes as we had deleted some lines from the document.

Connect to the target database and issue the query

Previous:

SQL> select id,filename,dbms_lob.getlength(doc_content)  from documents;

        ID FILENAME                                                     DBMS_LOB.GETLENGTH(DOC_CONTENT)
---------- ------------------------------------------------------------ -------------------------------
         1 Consultant Profile - Gavin Soorma.doc                                                 532992


Current:
SQL> select id,filename,dbms_lob.getlength(doc_content)  from documents;

        ID FILENAME                                                     DBMS_LOB.GETLENGTH(DOC_CONTENT)
---------- ------------------------------------------------------------ -------------------------------
         1 Consultant Profile - Gavin Soorma.doc                                                 528896

We can see that the Replicat process which had earlier applied the INSERT statement when the document was uploaded to the database the first time has now applied some UPDATE statements as well

GGSCI (vind-a) 1> stats replicat rep9 latest

Sending STATS request to REPLICAT REP9 ...

Start of Statistics at 2014-05-22 10:08:47.

Replicating from GG_OWNER.DOCUMENTS to GG_OWNER.DOCUMENTS:

*** Latest statistics since 2014-05-22 08:59:20 ***
        Total inserts                                      1.00
        Total updates                                      3.00
        Total deletes                                      0.00
        Total discards                                     0.00
        Total operations                                   4.00

In GoldenGate 12.2 we can now validate parameter files before deployment.

There is a new utility called chkprm which can be used for this purpose.’

To run the chkprm utility we provide the name of the parameter file and can optionally indicate what process this parameter file belongs to using the COMPONENT keyword.

Let us look at an example.

Oracle Cloud Marketplace (https://cloudmarketplace.oracle.com) is an online store selling a wide variety of business applications and services which can complement an existing Oracle Cloud implementation. All these applications and services available on the Cloud marketplace are offered by Oracle approved and registered partners and developers.

Oracle GoldenGate is one of the many applications available on Oracle Cloud Marketplace and described very simply, it is the cloud-based version of Oracle GoldenGate that also provides us with all the core functionality of GoldenGate as a product including the scalability, reliability and manageability.

After we provision GoldenGate on Cloud Marketplace, we have access to a compute node which includes an image of the latest 19c Oracle GoldenGate software (19.1.0.0.4) as well as Oracle Database Client software (11g, 12g, 18c, and 19c) pre-installed.

GoldenGate on Cloud Marketplace is available with the choice of deploying using Classic Architecture as well as Microservices Architecture.

This provides us a replication solution from not only on-premise to the cloud as well as cloud-to-cloud platform.

We need an Oracle Cloud Account and access to compute node resources within Oracle Cloud Tenant. Supported shapes are VM.Standard2.4, VM.Standard2.8, VM.Standard2.16 and VM.Standard2.24.

In addition, we also need to provide an SSH Public Key which will enable us to connect to the Oracle Cloud Infrastructure hosted on the GoldenGate compute node as the opc user.

In this example we will be configuring real-time replication from an Oracle 12c Release 1 database on OCI-Classic (Gen 1) to an Oracle 18c database hosted on OCI ATP (Gen 2) using Oracle GoldenGate running on Oracle Cloud Marketplace.

In Part 1, we will look at the various screen shots from the process of  deploying Oracle GoldenGate Microservices on Oracle Cloud Marketplace.
 
 

Connect to the GoldenGate Cloud Marketplace Node

Identify the Public IP Address of the GoldenGate Cloud Marketplace compute node and connect via PuTTY using the saved PPK  private key. (note we had to provide the SSH Public Key details when we deployed Oracle GoldenGate Cloud Marketplace)  

Note the directory structure of the GoldenGate Cloud Marketplace Node

The trail files are stored in the directory /u02/trails and we have the two deployments (‘Source’ and ‘Target’) related directories under /u02/deployments

The GoldenGate software is available under /u01/app/ogg. Note the different versions of the software available.

Change the oggadmin password

Open the ogg-credentials.json file located in the /home/opc directory. Use the credentials included in that file to connect via Service Manager URL as oggadmin user.

-bash-4.2$ pwd
/home/opc

-bash-4.2$ ls -l
total 0
lrwxrwxrwx. 1 opc opc 37 Mar 26 05:22 ogg-credentials.json -> /u02/deployments/ogg-credentials.json

-bash-4.2$ cat ogg-credentials.json
{“username”: “oggadmin”, “credential”: “k%KaBo.2XDmIFRCa”}

Copy wallet zip file which has been downloaded from ATP database

This is required so we can connect to the ATP target database from the GoldenGate Cloud Marketplace compute node.

-bash-4.2$ mkdir wallet

-bash-4.2$ ls

ogg-credentials.json wallet Wallet_GSATP.zip

-bash-4.2$ cp Wallet_GSATP.zip ./wallet/

-bash-4.2$ cd wallet

-bash-4.2$ unzip Wallet_GSATP.zip
Archive: Wallet_GSATP.zip
inflating: cwallet.sso
inflating: tnsnames.ora
inflating: readme.md
inflating: truststore.jks
inflating: ojdbc.properties
inflating: sqlnet.ora
inflating: ewallet.p12
inflating: keystore.jks

Configure Network Connectivity to source OCI Classic database

Edit the network configuration files for deployment “Source”.

Note: replace the host name with the public IP address .

-bash-4.2$ pwd
/u02/deployments/Source/etc

-bash-4.2$ vi tnsnames.ora

ORCL =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 150.136.58.202)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = ORCL.svcsubnetad1.svcvcn.oraclevcn.com)
)
)

PDB1 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 150.136.58.202)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = pdb1.svcsubnetad1.svcvcn.oraclevcn.com)
)
)

Configure source OCI Classic Database for GoldenGate Replication

SQL> SELECT supplemental_log_data_min, force_logging FROM v$database;

SUPPLEME FORCE_LOGGING
——– —————————————
YES YES

SQL> show parameter goldengate

NAME TYPE VALUE
———————————— ———– ——————————
enable_goldengate_replication boolean TRUE

SQL> alter system set streams_pool_size=1024m scope=both;

System altered.

SQL> select username from dba_users where username like ‘%GG%’;

USERNAME
——————————————————————————–
C##GGADMIN

SQL> conn C##GGADMIN/Dreamliner787##
Connected.

Update Credential Store with source database credentials

Enter the credentials for the common user c##oggadmin which connects to the source container database ORCL.

Add SCHEMATRANDATA

Configure connectivity to ATP target database from GoldenGate Cloud Marketplace node

-bash-4.2$ pwd

/u02/deployments/Target/etc

-bash-4.2$ vi sqlnet.ora

WALLET_LOCATION = (SOURCE = (METHOD = file) (METHOD_DATA = (DIRECTORY=”/home/opc/wallet”)))

SSL_SERVER_DN_MATCH=yes

-bash-4.2$ cat tnsnames.ora

gsatp_high = (description= (retry_count=20)(retry_delay=3)(address=(protocol=tcps)(port=1522)(host=adb.us-ashburn-1.oraclecloud.com))(connect_data=(service_name=zjiver6hk5luiwo_gsatp_high.atp.oraclecloud.com))(security=(ssl_server_cert_dn=”CN=adwc.uscom-east-1.oraclecloud.com,OU=Oracle BMCS US,O=Oracle Corporation,L=Redwood City,ST=California,C=US”)))

gsatp_low = (description= (retry_count=20)(retry_delay=3)(address=(protocol=tcps)(port=1522)(host=adb.us-ashburn-1.oraclecloud.com))(connect_data=(service_name=zjiver6hk5luiwo_gsatp_low.atp.oraclecloud.com))(security=(ssl_server_cert_dn=”CN=adwc.uscom-east-1.oraclecloud.com,OU=Oracle BMCS US,O=Oracle Corporation,L=Redwood City,ST=California,C=US”)))

gsatp_medium = (description= (retry_count=20)(retry_delay=3)(address=(protocol=tcps)(port=1522)(host=adb.us-ashburn-1.oraclecloud.com))(connect_data=(service_name=zjiver6hk5luiwo_gsatp_medium.atp.oraclecloud.com))(security=(ssl_server_cert_dn=”CN=adwc.uscom-east-1.oraclecloud.com,OU=Oracle BMCS US,O=Oracle Corporation,L=Redwood City,ST=California,C=US”)))

gsatp_tp = (description= (retry_count=20)(retry_delay=3)(address=(protocol=tcps)(port=1522)(host=adb.us-ashburn-1.oraclecloud.com))(connect_data=(service_name=zjiver6hk5luiwo_gsatp_tp.atp.oraclecloud.com))(security=(ssl_server_cert_dn=”CN=adwc.uscom-east-1.oraclecloud.com,OU=Oracle BMCS US,O=Oracle Corporation,L=Redwood City,ST=California,C=US”)))

gsatp_tpurgent = (description= (retry_count=20)(retry_delay=3)(address=(protocol=tcps)(port=1522)(host=adb.us-ashburn-1.oraclecloud.com))(connect_data=(service_name=zjiver6hk5luiwo_gsatp_tpurgent.atp.oraclecloud.com))(security=(ssl_server_cert_dn=”CN=adwc.uscom-east-1.oraclecloud.com,OU=Oracle BMCS US,O=Oracle Corporation,L=Redwood City,ST=California,C=US”)))

-bash-4.2$ pwd

/u01/app/client/oracle19/network/admin

-bash-4.2$ cp /u02/deployments/Target/etc/sqlnet.ora .

-bash-4.2$ cp /u02/deployments/Target/etc/tnsnames.ora .

Unlock the ggadmin user in the ATP database and grant privileges on DATA tablespace

-bash-4.2$ export TNS_ADMIN=/u01/app/client/oracle19/network/admin

-bash-4.2$ cd ../..

-bash-4.2$ cd bin

-bash-4.2$ export ORACLE_HOME=/u01/app/client/oracle19

-bash-4.2$ export PATH=$ORACLE_HOME/bin:$PATH

-bash-4.2$ sqlplus admin/Dreamliner787##@gsatp_high

SQL*Plus: Release 18.0.0.0.0 – Production on Wed Apr 1 01:45:29 2020

Version 18.3.0.0.0

Copyright (c) 1982, 2018, Oracle.  All rights reserved.

Connected to:

Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 – Production

Version 19.5.0.0.0

SQL> alter user ggadmin identified by Dreamliner787## ;

User altered.

SQL> alter user ggadmin account unlock;

User altered.

SQL> alter user ggadmin quota unlimited on data;

User altered

Create the directory to store the trail files for each deployment

bash-4.2$ cd /u02/

-bash-4.2$ ls

cacheManager  deployments  trails

-bash-4.2$ cd trails/

–bash-4.2$ mkdir Source

-bash-4.2$ mkdir Target