Bulk Update of GoldenGate Extract Profiles

After writing blogs about Default profile updates and a bug affecting profiles in GoldenGate 26ai, I realized it would be a good idea to present a quick way to update all processes at once with the REST API. Because even if you set a new profile as default with the {"isDefault": true} flag, previously created extracts and replicats will not be updated !

To explain this further, let’s look at an extract setting using the retrieve_extract endpoint. I’m using the Python client I presented in another blog. Everything I’m talking about here is not specific to extracts, and can be extended to replicats. However, it doesn’t apply to distribution paths.

Extract with no profile assigned

If an extract was created in the most basic configuration, it inherits the Default profile implicitly, but this information is not registered in the extract definition. Below is the configuration of such an extract. To save you time, just remember that there is no mention of any profile setting.

>>> ogg_client.retrieve_extract('EXTSRC')
{'$schema': 'ogg:extract', 'credentials': {'alias': 'cdb01', 'domain': 'OracleGoldenGate'}, 'begin': 'now', 'encryptionProfile': 'LocalWallet', 'targets': [{'name': 'aa', 'path': 'PDB1', 'sizeMB': 2000, 'sequenceLength': 9, 'sequenceLengthFlip': False, 'sequence': 1, 'offset': 1399, 'remote': False}], 'config': ['--- Auto generated Parameter File, do not edit ---', 'EXTRACT EXTSRC', 'USERIDALIAS cdb01 DOMAIN OracleGoldenGate', 'EXTTRAIL PDB1/aa', '--- End of auto generated Parameter File ---', 'SOURCECATALOG PDB1', 'TABLE APP_PDB1.*;'], 'source': 'tranlogs', 'type': 'Integrated', 'registration': {'share': False, 'containers': ['PDB1'], 'csn': 2809889}, 'status': 'running'}

Extract with a profile set

If a profile is assigned to the extract, the managedProcessSettings key is added to the extract configuration. This is true whether you assign a custom profile or re-assign the Default profile after having changed the extract configuration. Below, we can see the difference with the following key/value pair : 'managedProcessSettings': 'ogg:managedProcessSettings:dbiProfile'

{'$schema': 'ogg:extract', 'credentials': {'alias': 'cdb01', 'domain': 'OracleGoldenGate'}, 'begin': 'now', 'encryptionProfile': 'LocalWallet', 'managedProcessSettings': 'ogg:managedProcessSettings:dbiProfile', 'targets': [{'name': 'aa', 'path': 'PDB1', 'sizeMB': 2000, 'sequenceLength': 9, 'sequenceLengthFlip': False, 'sequence': 1, 'offset': 1399, 'remote': False}], 'config': ['--- Auto generated Parameter File, do not edit ---', 'EXTRACT EXTSRC', 'USERIDALIAS cdb01 DOMAIN OracleGoldenGate', 'EXTTRAIL PDB1/aa', '--- End of auto generated Parameter File ---', 'SOURCECATALOG PDB1', 'TABLE APP_PDB1.*;'], 'source': 'tranlogs', 'type': 'Integrated', 'registration': {'share': False, 'containers': ['PDB1'], 'csn': 2809889}, 'status': 'running'}

Extract with a custom configuration

If an extract has a custom unique configuration that is not a profile, the full configuration is added as a JSON to the extract definition under the same managedProcessSettings key. I give below an example:

{'$schema': 'ogg:extract', 'credentials': {'alias': 'cdb01', 'domain': 'OracleGoldenGate'}, 'begin': 'now', 'encryptionProfile': 'LocalWallet', 'managedProcessSettings': {'autoStart': {'enabled': True, 'delay': 60}, 'autoRestart': {'enabled': False, 'onSuccess': False, 'delay': 0, 'retries': 9, 'window': 0, 'disableOnFailure': True, 'failures': 0}}, 'targets': [{'name': 'aa', 'path': 'PDB1', 'sizeMB': 2000, 'sequenceLength': 9, 'sequenceLengthFlip': False, 'sequence': 1, 'offset': 1399, 'remote': False}], 'config': ['--- Auto generated Parameter File, do not edit ---', 'EXTRACT EXTSRC', 'USERIDALIAS cdb01 DOMAIN OracleGoldenGate', 'EXTTRAIL PDB1/aa', '--- End of auto generated Parameter File ---', 'SOURCECATALOG PDB1', 'TABLE APP_PDB1.*;'], 'source': 'tranlogs', 'type': 'Integrated', 'registration': {'share': False, 'containers': ['PDB1'], 'csn': 2809889}, 'status': 'running'}

How to modify a single extract’s profile with the REST API ?

Modifying the profile of an extract or replicat with the REST API means updating the managedProcessSettings key of the process. For an extract, you would use the update_extract endpoint, with a data payload that only contains the managedProcessSettings key, and its value. It can be the profile name or the full profile definition, as shown above. Here is an example of a single update of an extract:

>>> ogg_client.update_extract(
extract='EXTSRC',
    data={
        'managedProcessSettings': 'ogg:managedProcessSettings:dbiProfile'
    }
)

And do not worry about entering a wrong name for the profile. If you do this, the API returns the following error:

>>> ogg_client.update_extract(
    extract='EXTSRC',
    data={
        'managedProcessSettings': 'ogg:managedProcessSettings:NonExistingProfile'
    }
)
Exception: ERROR - https://vmogg/services/ogg_test_01/adminsrvr/v2/extracts/EXTSRC: The managed process settings profile 'ogg:managedProcessSettings:NonExistingProfile' does not exist. ; INFO - https://vmogg/services/ogg_test_01/adminsrvr/v2/extracts/EXTSRC: The item type  with name 'ogg:managedProcessSettings:NonExistingProfile' does not exist.

How to modify all extracts and replicats ?

Based on the above, modifying all extracts and replicats is easy. You should just iterate over your extracts and replicats and apply the profile that you want. Below is an example script where you can modify the profile_name variable and the connection details to the Administration Service of your deployment. You could even decide to have different profiles assigned to extracts and replicats, if needed.

from oggrestapi import OGGRestAPI

profile_name = 'dbiProfile'

ogg_client = OGGRestAPI(
    url='https://vmogg',
    user='ogg',
    password='ogg',
    reverse_proxy=True,
    deployment='ogg_test_01'
)

for extract in ogg_client.list_extracts():
    extract_name = extract['name']
    print(f"Updating extract {extract_name}")
    ogg_client.update_extract(
        extract=extract_name,
        data={
            'managedProcessSettings': f'ogg:managedProcessSettings:{profile_name}'
        }
    )

for replicat in ogg_client.list_replicats():
    replicat_name = replicat['name']
    print(f"Updating replicat {replicat_name}")
    ogg_client.update_replicat(
        replicat=replicat_name,
        data={
            'managedProcessSettings': f'ogg:managedProcessSettings:{profile_name}'
        }
    )

After running this against the Administration Service, all your extracts and replicats will now follow the same new profile.