At the recent Oracle Open World 2015 conference I was fortunate to attend a series of very informative presentations on Oracle GoldenGate from senior members of the Product Development team.
Among them was the presentation titled GoldenGate 12.2 New Features Deep Dive which is now available for download via the official OOW15 website.
While no official release date was announced for Goldengate 12.2, the message was being communicated that the release was going to happen ‘very soon’.
So while we eagerly wait for the official product release, here are some of the new 12.2 features which we can look forward to.
No more usage of SOURCEDEFS and ASSUMETARGETDEFS parameter –Metadata included as part of Trail File
In earlier versions if the structure of the table between the source and target database was different in terms of column names, data types and even column positions (among other things), we had to create a flat file which contained the table definitions and column mapping via the DEFGEN utility. Then we had to transfer this file to the target system.
If we used the parameter ASSUMETARGETDEFS, the assumption was that the internal structure of the target tables was the same as the source – which was not always the case – and we encountered issues
Now in 12.2, GoldenGate Trail Files are Self-Describing. Metadata information is included in the Trail Files called Table Definition Record (TDR) before the first occurrence of DML on that particular table and this TDR contains the table and column definition like the column number, data type, column length etc .
For new installations which will use the GoldenGate 12.2 software, metadata gets automatically populated in trail files by default. For existing installations we can use the parameter FORMAT RELEASE 12.2 and then any SOURCEDEFS or ASSUMETARGETDEFS parameters are no longer required or are ignored.
Automatic Heartbeat Table
In earlier versions, one of the recommendations to monitor lag was to create a heartbeat table.
Now in 12.2, there is a built-in mechanism to monitor replication lag. There is a new GGSCI command called ADD HEARTBEATTABLE .
This ADD HEARTBEATTABLE will automatically create the heartbeat tables and views as well as database jobs which updates heartbeat tables every 60 seconds.
One of the views created is called GG_LAG and it contains columns like INCOMING_LAG which will show the period of time between a remote database generating heartbeat and a local database receiving heartbeat.
Similarly to support an Active-Active Bi-Directional GoldenGate configuration, there is also a column called OUTGOING_LAG which is the period of time between local database generating heartbeat and remote database receiving heartbeat.
The GG_HEARTBEAT table is one of the main tables on which other heartbeat views are built and it will contain lag information for each component – Extract, Pump as well as Replicat. So we can quite easily identify where the bottleneck is when faced with diagnosing a GoldenGate performance issue.
Historical heartbeat and lag information is also maintained in tables like GG_LAG_HISTORY and GG_HEARTBEAT_HISTORY tables.
Parameter Files – checkprm , INFO PARAM, GETPARAMINFO
A new utility is available in 12.2 called checkprm which can be used to validate parameter files before they are deployed.
The INFO PARAM command will give us a lot of information about a particular parameter – like what is the default value and what are valid range of values. It is like accessing the online documentation from the GGSCI command line.
When a process like replicat or extract is running, we can use the SEND [process] GETPARAMINFO command to identify the runtime parameters – not only parameters included in the process parameter file, but also any other parameters the process has accessed which are say not included in the parameter file. Sometimes we are not aware of the many default parameters a process will use and this command will show this information real-time while the extract or replicat or manager is up and running.
Transparent Integration with Oracle Clusterware
In earlier releases, when we used the Grid Infrastructure Agent (XAG) to provide high availability capability for Oracle GoldenGate, we had to use the AGCTL to manage the GoldenGate instance like stop and start. If we used the GGSCI commands to start or stop the manager it could cause issues and the recommendation was to only use AGCTL and not GGSCI in that case.
Now in 12.2, once the GoldenGate instance has been registered with Oracle Clusterware using AGCTL, we can then continue to use GGSCI to start and stop GoldenGate without concern of any issues arising because AGCTL was not used. A new parameter for the GLOBALS file is now available called XAG_ENABLE.
Integration of GoldenGate with Datapump
In earlier releases when we added new tables to an existing GoldenGate configuration, we had to obtain the CURRENT_SCN from v$DATABASE view, pass that SCN value to the FLASHBACK_SCN parameter of expdp and then when we started the Replicat we had to use the AFTERCSN parameter with the same value.
Now in 12.2, the ADD TRANDATA or ADD SCHEMATRANDATA will prepare the tables automatically. Oracle Datapump export (expdp) will automatically generate import actions to set the instantiation CSN when that table is imported. We just have to include the new parameter for the Replicat called DBOPTIONS_ENABLE_INSTANTIATION_FILTERING which will then filter out any DML or DDL records based on the instantiation CSN of that table.
Improved Trail File Recovery
In earlier releases if a trail file was missing or corrupt, the Replicat used to abend.
Now in 12.2, if we have a corrupted or missing trail file, we can delete the corrupted trail file and the trail file is rebuilt by restarting the Extract Pump – the same is the case for a missing trail file which can be automatically rebuilt by bouncing the Extract Pump process. Replicat will automatically filter duplicate transactions by default to transactions already applied in the regenerated trail files.
Support for INVISIBLE Columns
The new MAPINVISIBLECOLUMNS parameter in 12.2 now enables replication support for tables (Oracle database only ) which contained any such INVISIBLE columns.
Extended Metrics and Fine-grained Performance Monitoring
Release 12.2 now provides real-time process and thread level Metrics for Extract, Pump and Replicat which can be accessed through RESTful Web Services. Real time database statistics for Extract and Replicat, Queues, as well as network statistics for the Extract Pump can be accessed using a URL like:
http://<hostname>:<manager port>/mpointsx
ENABLEMONITORING parameter needs to be included in the GLOBALS file.
The Java application is also available for free download (and can also be modified and customised) via the URL:
https://java.net/projects/oracledi/downloads/download/GoldenGate/OGGPTRK.jar
GoldenGate Studio
New in Release 12.2 is GoldenGate Studio – a GUI tool which will enable us to quickly design and deploy GoldenGate solutions. It separates the logical from the physical design and enables us to create a one-click and drag and drop logical design based on business needs without knowing all the details.
It has a concept of Projects and Solutions where one Project could contain a number of solutions and Solution contains one logical design and possibly many physical deployments. Rapid design is enabled with a number of out of the box Solution templates like Cascading, Bi-Directional, Unidirectional, Consolidation etc.
GoldenGate Studio enables us to design once and deploy it to many environments like Dev,Test, QA and Production with one click deployment.
GoldenGate Cloud Service
GoldenGate Cloud Service is the public cloud-based offering on a Subscription or Hourly basis.
The GoldenGate Cloud Service provides the delivery mechanisms to move Oracle as well as non-Oracle databases from On Premise to DBaaS – Oracle Database Cloud Service as well as Exadata Cloud Service delivery via GoldenGate. GoldenGate Cloud Service also provides Big Data Cloud Service delivery to Hadoop and NoSQL.
Nine Digit Trail File Sequence Length
In 12.2, the default is to create trail files with 9 digit sequence numbers instead of the earlier 6 digit sequence. This now will allow 1000 times more files per trail – basically 1 billion files per trail!.
We can upgrade existing trail files from 6 to 9 digit sequence numbers using a utility called convchk and there is also backward compatibility support for existing 6 digit sequences using a GLOBAL parameter called TRAIL_SEQLEN_6D.