NGINX Secured Distribution Path with GoldenGate REST API

In a previous blog, I presented how to set up a distribution path between two GoldenGate deployments both secured with NGINX. The method I used there was purely through the Web UI. But GoldenGate also exposes a full REST API, and everything you can do in the UI can be done through the API as well, which is useful for automation, scripting, or when the UI is not reachable.

This blog covers the exact same setup, using the REST API instead. I will show two ways of doing it :

• Using oggrestapi.py, the GoldenGate REST client I released in another blog.
• Using the requests library to call the REST API directly.

Prerequisites

The prerequisites are the same as in the previous blog :

• Two GoldenGate Microservices deployments, ogg_test_01 (source) on oggvm1 and ogg_test_02 (target) on oggvm2. I will use the latest 26ai version.
• Both OGG setups secured with NGINX acting as a reverse proxy, so everything goes through port 443.
• A running extract on the source, writing to a trail (aa in my case).

Just like in the Web UI, there are three steps to get a working distribution path :

• Create a path connection on the source, to authenticate against the target.
• Register the target’s CA certificate on the source Service Manager.
• Create and start the distribution path.

A quick note on URLs before we start. Behind an NGINX reverse proxy, each service has its own path prefix :

• Administration Service : /services/<deployment>/adminsrvr/v2/...
• Distribution Service : /services/<deployment>/distsrvr/v2/...
• Service Manager : /services/ServiceManager/v2/...

The oggrestapi.py client builds these for you as soon as you pass reverse_proxy=True and the deployment name, so let’s connect once and reuse the client. If you don’t provide the password argument, you will be prompted for it.

from oggrestapi import OGGRestAPI

ogg_source = OGGRestAPI(
    url="https://oggvm1",
    username="ogg",
    deployment="ogg_test_01",
    reverse_proxy=True,
)

Create the path connection

As explained in Creating Path Connections with GoldenGate REST API, a path connection is simply an alias in the Network domain. It stores the credentials of a user that exists on the target deployment, and its alias is only known on the source side.

With the client, just call the create_alias method :

ogg_source.create_alias(
    alias="ogg_target",
    domain="Network",
    data={
        "userid": "ogg_user_on_target",
        "password": "***",
    },
)

As mentioned in the introduction, here is the same call with requests, calling the Administration Service of oggvm1 through NGINX :

import requests

auth = ("ogg", "ogg_password")

response = requests.post(
    "https://oggvm1/services/ogg_test_01/adminsrvr/v2/credentials/Network/ogg_target",
    auth=auth,
    json={
        "userid": "ogg_user_on_target",
        "password": "***",
    },
)

After refreshing the source Web UI, the new path connection is visible under the Path Connections tab :

But of course, you can also view the new path connection by calling the REST API:

# Since path connections are aliases of the Network domain, we use the get_alias method to retrieve them
>>> ogg_source.get_alias('Network', 'ogg_target')
{'$schema': 'ogg:credentials', 'userid': 'ogg_user_on_target', 'type': 'PASSWORD'}

Register the target’s CA certificate

Because the deployments are secured with NGINX, the source has to trust the certificate authority that signed the target’s certificate. This is done on the source Service Manager, by registering the target’s root CA certificate.

With the client, use create_deployment_certificate against the source deployment. The certificate type to use is truststore, and the certificate content goes under trustpointBundle.trustpointPem:

target_ca = open("rootCA_ogg_test_02.pem").read()

ogg_source.create_deployment_certificate(
    deployment="ogg_test_01",
    type="truststore",
    certificate="rootCA_ogg_test_02",
    data={
        "trustpointBundle": {
            "trustpointPem": target_ca,
        }
    },
)

The same call with requests, this time on the Service Manager prefix :

target_ca = open("rootCA_ogg_test_02.pem").read()

response = requests.post(
    "https://oggvm1/services/ServiceManager/v2/deployments/ogg_test_01/certificates/truststore/rootCA_ogg_test_02",
    auth=auth,
    json={
        "trustpointBundle": {
            "trustpointPem": target_ca,
        }
    },
)

Registering under the specific deployment (ogg_test_01) is the equivalent of the Local option in the Web UI. To get the Shared behavior instead, register the same certificate under the ServiceManager deployment name, so it becomes available to every deployment on that node.

If the certificate file contains a chain of certificates, you must register each certificate individually, since GoldenGate does not accept them in one go. I described that issue in detail in a blog about the OGG-30007 error.

Create and start the distribution path

We can now create the distribution path itself. It has a source endpoint (the local trail) and a target endpoint (the target’s Receiver Service, reached over wss through NGINX). Because the target is NGINX-secured, the target URI :

• uses the wss protocol on port 443,
• points at the Receiver Service path prefix, recvsrvr, not distsrvr (that prefix is only for the Distribution Service on the source side),
• does not carry the path connection alias itself. The alias goes in a separate authenticationMethod key.

With the client :

ogg_source.create_distribution_path(
    distpath="path12",
    name="path12",
    source={
        "uri": "trail://localhost/services/v2/sources?trail=PDB1/aa",
    },
    target={
        "uri": "wss://oggvm2/services/ogg_test_02/recvsrvr/v2/targets?trail=PDB1/bb",
        "authenticationMethod": {
            "domain": "Network",
            "alias": "ogg_target",
        },
    },
    begin="now",
    status="running",
)

And the equivalent requests call, on the Distribution Service prefix (/services/ogg_test_01/distsrvr/):

response = requests.post(
    "https://oggvm1/services/ogg_test_01/distsrvr/v2/sources/path12",
    auth=auth,
    json={
        "name": "path12",
        "source": {
            "uri": "trail://localhost/services/v2/sources?trail=PDB1/aa",
        },
        "target": {
            "uri": "wss://oggvm2/services/ogg_test_02/recvsrvr/v2/targets?trail=PDB1/bb",
            "authenticationMethod": {
                "domain": "Network",
                "alias": "ogg_target",
            },
        },
        "begin": "now",
        "status": "running",
    },
)

The trail value in both URIs also has to match the path the extract actually registers, EXTTRAIL PDB1/aa on the source becomes trail=PDB1/aa in the source URI, and the same logic applies to the target’s bb trail. A bare trail=aa without the PDB path segment matches neither what the extract writes nor what the target’s own directory layout expects.

Once the path is created with status: "running", the trail files start flowing. You can confirm it on the target :

oracle@oggvm2:~/ ll $OGG_DEPLOYMENT_HOME/var/lib/data/PDB1
total 0
-rw-r-----. 1 oracle oinstall 0 Mar 22 07:34 bb000000000

The remote peer submitted a certificate that failed validation

If your distribution path doesn’t start and generates a “certificate that failed validation” error, it means that you incorrectly registered your certificates. Make sure that the target deployment’s CA certificate is registered on the source Service Manager, and not the other way around.

And that’s it. With three REST calls, through oggrestapi.py or using the requests module, you get the exact same NGINX-secured distribution path as the Web UI method, but in a form you can script and repeat.