Increase GoldenGate 26ai Log Retention

GoldenGate logs are a powerful source of information when debugging or analyzing your deployments. However, some of these logs have a rather low retention period in active deployments. They might then not even be useful for debugging if you send them to your dbi consultants or the Oracle support for analysis. So how can you increase GoldenGate log retention ?

In a previous blog, I presented all the log files available in GoldenGate. Each of them has its own format and characteristics, but they have one common aspect: they can be customized.

Standard GoldenGate logging configuration files are located in the $OGG_HOME/lib/utl/logging directory.

> ll $OGG_HOME/lib/utl/logging
-rw-r-----. 1 oracle oinstall  1066 Nov 17  2018 app-adminsrvr-debug.xml
-rw-r-----. 1 oracle oinstall  1076 Jan 10  2019 app-adminsrvr-events.xml
-rw-r-----. 1 oracle oinstall  1066 Nov 17  2018 app-distsrvr-debug.xml
-rw-r-----. 1 oracle oinstall  1040 Apr 17  2017 app-extract-events.xml
-rw-r-----. 1 oracle oinstall  1066 Nov 17  2018 app-pmsrvr-debug.xml
-rw-r-----. 1 oracle oinstall  1397 Apr  1  2018 app-pmsrvr-default.xml
-rw-r-----. 1 oracle oinstall  1066 Nov 17  2018 app-recvsrvr-debug.xml
-rw-r-----. 1 oracle oinstall  1048 Jan 22  2020 app-replicat-debug509.xml
-rw-r-----. 1 oracle oinstall  1040 Apr 17  2017 app-replicat-events.xml
-rw-r-----. 1 oracle oinstall  1066 Nov 17  2018 app-ServiceManager-debug.xml
-rw-r-----. 1 oracle oinstall  2459 Jan 17  2024 app-ServiceManager-services.xml
-rw-r-----. 1 oracle oinstall  1282 Dec 19 18:44 ogg-AIService.xml
-rw-r-----. 1 oracle oinstall  4946 May 14  2020 ogg-audit.xml
-rw-r-----. 1 oracle oinstall  1582 Jun 28  2023 ogg-ConfigService.xml
-rw-r-----. 1 oracle oinstall  4487 Jan 10  2019 ogg-ggserr.xml
-rw-r-----. 1 oracle oinstall 18162 Jun 26  2020 ogg-loggers.json
-rw-r-----. 1 oracle oinstall  1095 Sep 25  2019 ogg-loggers.xml
-rw-r-----. 1 oracle oinstall  2180 Sep 11  2024 sca-default.xml
-rw-r-----. 1 oracle oinstall  1211 Jan 10  2019 sca-restapi.xml
-rw-r-----. 1 oracle oinstall  1210 Jun  6  2022 sca-stdout.xml

The process to modify logging properties of any GoldenGate log file is to copy one of these files in your deployment and update it. It means that you can have different logging properties between your deployments.

Oracle GoldenGate Microservices uses a hierarchical logger framework with Log4j-style appenders, layouts, logger inheritance, and category namespaces. I will not dwell on all the configuration files in this blog, but let’s try to describe the most useful ones.

sca-restapi.xml

<?xml version="1.0"?>
<configuration>

  <!--
   /- ============================================================= -\
   !-   s c a - r e s t a p i . x m l                               -|
   !-                                                               -|
   !-   Logging control file for recording all REST API calls       -|
   !-   to an OGG deployment.                                       -|
   \- ============================================================= -/
  ! -->

  <appender  name="sca-restapi.log"  class="RollingFileAppender">
    <level  value="info"/>
    <param   name="File"             value="restapi.log"/>
    <param   name="MaxFileSize"      value="10MB"/>
    <param   name="MaxBackupIndex"   value="9"/>
    <param   name="BufferedIO"       value="false"/>
    <param   name="Append"           value="true"/>
    <layout class="PatternLayout">
      <param name="Pattern"          value="%d{%Y-%m-%d %H:%M:%S%z} %-5p|%-36.36c| %m%n"/>
    </layout>
  </appender>

  <!--
   !-   M i c r o s e r v i c e s   A r c h i t e c t u r e
  ! -->
  <logger          name="RestAPI">
    <appender-ref  name="sca-restapi.log"/>
    <level        value="info"/>
  </logger>

</configuration>

The sca-restapi.xml file is the logging configuration file for the restapi.log file.

REST API logs are the most verbose across all GoldenGate log files. In production environments where the REST API is often called, you could easily go over the 10 log files in a day or even less. If you have enough space, I would strongly recommend increasing the retention and/or the maximum size of a single log file. This way, you ensure that you keep enough logs for debugging and analysis.

To modify the retention, change MaxFileSize to set the maximum log file size and MaxBackupIndex to choose the number of files you want to keep (on top of the active log file).

Let’s copy the file in the deployment home (it could be any deployment, including the Service Manager home).

cd $OGG_DEPLOYMENT_HOME/etc/conf/logging
cp -p $OGG_HOME/lib/utl/logging/sca-restapi.xml .
vim sca-restapi.xml

For instance, to have 5 files of 50 MB each, edit the following lines:

    <param   name="MaxFileSize"      value="50MB"/>
    <param   name="MaxBackupIndex"   value="4"/>

Once this is done, just restart the administration service.

What about the other log files ?

If you want to increase the retention or the log file size of any other log file in GoldenGate, just use the following mapping and repeat the same process. If you want to edit the Service Manager logging properties, you should also restart it.

Warning: If you want to use the same configuration across all your deployments, you could modify the standard files in $OGG_HOME/lib/utl/logging and create links from the deployments to this file. However, make sure you do not lose these changes when patching out-of-place !

Can I increase the retention to more than 10 files ?

Yes, there is no problem having more than 10 log files. Just increase the MaxBackupIndex (9, by default) to the number of log files you want, minus 1. For 20 log files, set MaxBackupIndex to 19.