The goal was to evaluate OCI-managed MCP servers and determine whether extending their available toolsets with additional database operations could reduce reliance on agent skills. By exposing reusable operations as MCP tools, we aimed to reduce token consumption and associated costs while improving execution efficiency, without compromising the security of databases hosted in private VCN subnets.

Provision OCI-resources: When RTFM Isn’t Enough

Provisioning an MCP server requires an underlying database connection that combines database credentials with network connectivity to the target database, typically through a private endpoint. This connection allows the MCP server to access the database securely while keeping it within the private VCN.

The detailed setup procedure is beyond the scope of this article, as Oracle provides official documentation covering the prerequisites, configuration, and provisioning for Database Tools Private Endpoints

In short, the documentation mentions using Zero Trust Packet Routing (ZPR) alongside, or instead of, Network Security Groups (NSGs) to control network access.

So far, so good. We already use NSGs exclusively to manage network traffic.

However, the documentation does not clearly spell out which specific NSG rules are required or whether ingress rules alone are sufficient. After digging deeper into the documentation, we found the missing piece of the puzzle:

Creating a Database Management Private Endpoint

The key takeaway is that the Network Security Groups (NSGs) associated with the private endpoint must have appropriate ingress and egress rules to permit the required network traffic.

This is where things got more interesting. Our environment already hosted several databases interconnected through Oracle database links, with ingress and egress rules in place for the relevant subnets, including the newly provisioned private endpoint. Yet, despite these existing network configurations, the MCP server still could not connect to the target database.

ORA-12170: Cannot connect. TCP connect timeout

Clearly, something was missing. We therefore dug deeper into the private endpoint’s specific network security requirements to identify the root cause of the connectivity issue.

Still Getting ORA-12170, But Why?

Hold on a minute! We quickly went through the OCI Console to double-check our network configuration:

• Verified the private endpoint’s NSG ingress and egress rules.
• Confirmed that the databases and the private endpoint were deployed in the same subnet.

Everything looked correct at first glance, yet the dreaded ORA-12170: TNS:Connect timeout occurred error persisted.

What were we missing? Time to bring in Oracle Support!

While working with Oracle Support, we took a closer look at the NSG configuration and the private endpoint’s VNIC. Something caught our attention: an entry was missing from the list, but we hadn’t initially considered it relevant.

This prompted us to inspect the private endpoint configuration in greater detail using the OCI Command Line Interface (OCI CLI).

jerome_witt@cloudshell:~ (eu-zurich-1)$ oci dbtools private-endpoint get \
  --private-endpoint-id ${EDP_OCID}  \
  --query 'data.{"nsg-ids": "nsg-ids", "private-endpoint-vnic-id": "private-endpoint-vnic-id", "security-attributes": "security-attributes"}' \
  --output json
{
  "nsg-ids": null,
  "private-endpoint-vnic-id": "ocid1.vnic.oc1.eu-zurich-1.xxxxxxxxxxxxxxxxx",
  "security-attributes": {}
}

Oh! The nsg-ids field is null! That was the missing piece of the puzzle.

After updating the private endpoint with the appropriate NSG, we ran the following OCI CLI command:

oci dbtools private-endpoint update \ 
  --private-endpoint-id ${EDP_OCID} \
  --nsg-ids '["${NSG_ATTACHED_TO_DB_OCID}"]'

And then, the magic happened! The database connection succeeded, and the dreaded ORA-12170 error was gone.

After explicitly associating the appropriate NSG with the Database Tools private endpoint, connectivity was restored. The OCI CLI output had revealed that the nsg-ids field was null, a detail we had overlooked when initially checking the configuration through the OCI Console.

This behavior may be related to recent changes in OCI as Oracle continues to rapidly develop and roll out its core Deep Data Security capabilities.

What had initially looked like a correctly configured network turned out to be missing a crucial piece: the NSG association explicitly configured on the Database Tools private endpoint itself.

Lessons Learned

• Never blindly trust the console! Always verify the underlying configuration using the OCI CLI.
• Have fun troubleshooting! Sometimes, the smallest configuration detail makes all the difference.

By the way, it seems we’re not alone. Others have apparently encountered similar database connectivity issues when configuring OCI Database Tools but none were using NSGs exclusively.

For further reference, these Oracle Support knowledge base articles may be useful:

• KB64948 — How to Connect to the Autonomous Database (Private Endpoint Access)
• KB42277 — OCI — Enable Database Management Fails

L’article OCI-managed MCP Server: Troubleshooting ORA-12170 errors with Private Endpoint connectivity est apparu en premier sur dbi Blog.

A customer recently asked me to disable FileTable/FileStream on their instance because the databases no longer use it, and to ensure that this is indeed the case.
The aim is to migrate to a new architecture without FileTable/FileStream in a second time.

Link to FileTable and FileStream if you need to know a bit more about these technologies

The environment is in a high-availability (HA) configuration using AlwaysOn.

As this isn’t something we do every day, here is a 9-step procedure to achieve this:

•    Check that there are no more FileTables

•    Disable non-transactional access

•    Check that non-transactional access is disabled

•    Check the columns in the tables

•    Identify FILESTREAM filegroups & check FILESTREAM files

•    Delete FILESTREAM containers & the FILESTREAM filegroup

•    Disable FILESTREAM on the SQL Server instance

•    Disable FILESTREAM on the server

•    Disable FILESTREAM on the other servers

Let’s get started!

Step 1: Checking the FileTables

Let’s start with the first, fairly simple step, which is to check that there are no more FileTables

Query used:

SELECT name,object_id,is_filetable FROM <database>.sys.tables WHERE is_filetable = 1;

Example:

In this example, it is clear that there is no longer a FileTable in the tables of the two databases checked here.

Step 2: Disabling non-transactional access

To ensure that no further data is sent to FileTable/FileStream, we disable non-transactional access:

Query used:

ALTER DATABASE <database> SET FILESTREAM ( NON_TRANSACTED_ACCESS = OFF );

Example:

The next step is, of course, to check carefully that it is disabled.

Step 3: Check that non-transactional access is disabled

Query used:

SELECT db_name(database_id),* FROM sys.database_filestream_options;

Example:

As can be seen in the example, all databases are set to OFF

Step 4: Checking the columns in the tables

This is a very important step, as it is the one I missed in my example by the customer…

In fact, in step 6, if there is still a column in a table that has a FileStream defined, it is not possible to delete the associated files and filegroups.

Query used:

SELECT * FROM <database>.sys.columns WHERE is_filestream = 1;
select * from sys.objects where object_id= xxxx

The first query returns the columns that contain FileStream data, and the second provides information about the table.

You will then find a structure of the following type:
CREATE TABLE [schema].[table](
 [ID] [uniqueidentifier] ROWGUIDCOL  NOT NULL,
…..
[FS] [varbinary](max) FILESTREAM  NULL
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY] FILESTREAM_ON [FileStreamFileGroup]

We can therefore see that there is indeed a FileStream that needs to be removed at column level, but also at table level with the FILESTREAM_ON [FileStreamFileGroup].

Initially, creating a new column with [FS2] [varbinary](max) NULL, copying the data and renaming it, seemed like a good idea but was not sufficient, as the table still had FILESTREAM_ON [FileStreamFileGroup].
To resolve this, you need to recreate a table, copy the data and add the keys, indexes and constraints.

Then check everything again before proceeding to the next steps.

Example:

Step 5:  Identify FILESTREAM filegroups & Check FILESTREAM files

In this step, the aim is to identify the FILESTREAM filegroups defined in the databases so that they can be deleted.

Query used:

SELECT name, type_desc FROM <database>.sys.filegroups WHERE type = 'FD';
SELECT name,physical_name,type_desc FROM <database>.sys.database_files WHERE type_desc = 'FILESTREAM';

Example:

As there are two databases in my example that have a FileStream, there are four lines.

Step 6: Delete the FILESTREAM containers and the FILESTREAM FILEGROUP

Once identified, delete them using the following command.
Query used:

ALTER DATABASE [Database]  REMOVE FILE [FileStreamFile]

Example:

Step 7: Disable FILESTREAM on the SQL Server instance

At this point, we disable FILESTREAM at instance level.

Query used:

EXEC sp_configure 'filestream access level';
GO
EXEC sp_configure 'show advanced options', 1;
RECONFIGURE;
EXEC sp_configure 'filestream access level',0;
GO
RECONFIGURE;
GO
EXEC sp_configure 'show advanced options', 0;
RECONFIGURE;
GO

Example:

As can be seen from the example, the `config_value` is indeed 0, but the `run_value` has changed to 1.

In such cases, as always, we’ll check what the documentation says:

We are in T-SQL access only, it’s not really desactivated… Need a restart!

At this stage, we could also carry this out on the secondary node(s) of our AlwaysOn high availability cluster, but it is preferable to do it first on a single node and verify that it works.

Now it is time to disable Filestream on the SQL Server Engine service.

Step 8: Disabling FileStream on the server

For this step, we have already performed a failover of the AAGs to a synchronous secondary node.

We launch SQL Server Configuration Manager to disable FileStream in the Engine properties by unticking the box, as shown in the example above:

Then restart SQL Server and check that FileStream is disabled on the instance:

Step 9: Disabling FileStream on the other servers

At this point, you will just need to repeat steps 7 and 8 on the other nodes in your AlwaysOn high-availability cluster.

Conclusion

The steps are not complex in themselves, but where you can lose time is in the step 4 if you have to recreate a table without the FileStream and copy all data from this table. I was happy to have only one with not so much rows…

All the steps up to the restart of the SQL Server Engine service in step 8 are carried out online without any service interruption or failover.

I hope these steps will help anyone facing a similar issue!

L’article SQL Server: Disabling FileTable/FileStream on an AlwaysOn high-availability instance est apparu en premier sur dbi Blog.

This part is about putting it back: translating the views, rebuilding the keys and indexes under a Standard Edition licence that makes every build offline, and compressing 3.2 TB of heaps that turned out to be mostly empty space.

The logbook resumes at day 5.

Day 5. Four thousand views to convert

4,658 of the 4,784 views live in a single schema, and nobody needed all of them. We went through the list with the client first and kept the ones that actually matter to the business, which brought the scope down to about 710. SSMA converted around 600 of those on its own.

SSMA fails mostly on CONNECT BY, START WITH and SYS_CONNECT_BY_PATH. For some easy cases, DECODE to CASE and NVL handles it well, for the record. For more complex use case, we needed to find an automatic solution process. So first, we discussed with the client to know which view were critical and needed to be migrated. Then, with this list, we leveraged the power of AI to help us converting the remaining views based on the following workflow:

The workflow

SSMA converts what it can. The failures get extracted from dba_views into files. Then, and this is the step that changes the economics, they get classified by pattern in the Oracle source rather than one at a time by error message.

Translation itself was done in batches of one family at a time, outside production, on extracted files, with an AI assistant and a context file describing the type mapping decisions, the column types of the referenced tables and the target version. Then human review, tests against Oracle and finally, deployment.

The catalogue of families is the real deliverable. Eight of them closed the great majority of the cases:

And a word on the AI, since that is the part people ask about. No data ever left the environment: only the view definitions were processed, and every test against the live database was run by hand so we kept control of the process. We could have added MCP connectors and let a local model run the tests itself, but with the number of days allocated to the project and the small number of views left to fix, doing it manually was simpler. Around 85 percent of the views wanted by the client had already come through SSMA.

Day 6. Putting the indexes back and making the database smaller

Recovering what I had thrown away

The primary keys I dropped to load into heaps had left no trace on the SQL Server side. Nothing in sys.indexes, no disabled definition, nothing.

What Standard Edition costs

Index builds are offline and single-threaded. The table is locked for the duration, metadata reads queue behind it, but we managed to run 4 builds in parallel. SORT_IN_TEMPDB = OFF for the clustered builds. The sort covers the entire table, and 36 GB of tempdb was never going to hold a 900 GB sort.

Meanwhile percent_complete sits at 0 for the whole build, with the serene confidence of something that has never been asked to account for itself. It works for BACKUP DATABASE. It does not work here. Real progress lives in sys.dm_exec_query_profiles, operator by operator :

SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
SELECT physical_operator_name, row_count, estimate_row_count,
       CAST(row_count*100.0/NULLIF(estimate_row_count,0) AS decimal(5,1)) AS pct
FROM sys.dm_exec_query_profiles WITH (NOLOCK)
WHERE session_id = <spid of the build> 
ORDER BY node_id;

Those are not three parallel counters. They are three sequential phases of one pipeline, and only one of them moves at a time. The scan reads the source table, and while it runs its percentage is the only honest progress figure available. The sort is a blocking operator: it cannot emit a single row until it has consumed all 31.9 million, so it stays at zero through the whole scan and then through the whole sort. That is the blind spot. Once the scan reaches 100 percent you get a long stretch where all three counters report nothing while the server works flat out, and if the sort spills to disk that stretch can outlast the other two phases combined. Index Insert starts moving only when the sort begins handing rows over, and it is the first number that means the end is actually in sight.

Why compress, and what it returned

On a read-only database, PAGE compression has no recurring write cost. You keep the read benefit and pay nothing back. The argument is rarely this clean.

Compression trades CPU for I/O, and on an archive that trade is one-sided. These pages are read rarely, and when they are, nothing heavy is competing for the processor, so the decompression cost lands on a machine with nothing better to do. What you get back is density: more rows per page, which means fewer reads to find the one row somebody asked for and more of the table sitting in memory at once.

The data was unusually well suited to it, for a reason worth spelling out. These tables carry roughly 200 numeric columns, mostly null, plus five nvarchar(200) aggregation labels that repeat endlessly. Here is the part that surprises people: a fixed-length numeric column that is NULL still occupies its full width in SQL Server. “Mostly null” saves nothing before compression. It is precisely why the heap was that large, and precisely what the page dictionary destroys.

The allocated space per row tells the story:

Three tables hold 2.7 TB of the 3.2 TB total. At around 4 KB of allocated space per row against an 8 KB page, the top two fit around two rows per page and waste most of what is left.

Then the first build, on 505,819,544 rows with a five-column primary key:

ALTER TABLE <TABLE6>
ADD CONSTRAINT PK_TABLE6 PRIMARY KEY CLUSTERED
    (<COLUMNS_NAME>)
WITH (DATA_COMPRESSION = PAGE, SORT_IN_TEMPDB = OFF);

117 GB to 30.7 GB. A ratio of 3.81.

Finally, as you can see above, more than 2.5 T have been reclaimed (datafiles were full with the heaps). Compression has made huge benefits and that means less data on disks, which means fewer pages to take in the buffer (less IO).

The script that puts everything back

Across every schema that is two cursors, one rebuilding the disabled indexes with compression and one re-enabling the foreign keys. It is driven by the same #Schemas table as the disable script in part 1:

SET NOCOUNT ON;
DECLARE @Execute bit = 0;   -- 0 = preview ; 1 = execute
  
DECLARE @cmd nvarchar(max);
DECLARE @n   int = 0;
  
DECLARE reb_cur CURSOR LOCAL FAST_FORWARD FOR
    SELECT N'ALTER INDEX ' + QUOTENAME(i.name)
         + N' ON ' + QUOTENAME(s.name) + N'.' + QUOTENAME(t.name)
         + N' REBUILD WITH (DATA_COMPRESSION = PAGE);'
    FROM   sys.indexes  i
    JOIN   sys.tables   t ON i.object_id = t.object_id
    JOIN   sys.schemas  s ON t.schema_id = s.schema_id
    WHERE  s.name IN (SELECT schema_name FROM #Schemas)
      AND  i.is_disabled = 1          
    ORDER BY s.name, t.name, i.name;
  
OPEN reb_cur;
FETCH NEXT FROM reb_cur INTO @cmd;
WHILE @@FETCH_STATUS = 0
BEGIN
    SET @n += 1;
    PRINT @cmd;
    IF @Execute = 1
    BEGIN
        BEGIN TRY EXEC sys.sp_executesql @cmd; END TRY
        BEGIN CATCH PRINT N'   -- FAILED: ' + ERROR_MESSAGE(); END CATCH
    END
    FETCH NEXT FROM reb_cur INTO @cmd;
END
CLOSE reb_cur;
DEALLOCATE reb_cur;
  
PRINT N'---';
PRINT CONVERT(nvarchar(10), @n) + N' index '
    + CASE WHEN @Execute = 1 THEN N'rebuilt.' ELSE N'to be rebuilt (preview).' END;
GO
  
  
SET NOCOUNT ON;
DECLARE @Execute bit = 1;   -- 0 = preview ; 1 = executer
  
DECLARE @cmd nvarchar(max);
DECLARE @n   int = 0;
  
DECLARE fkc_cur CURSOR LOCAL FAST_FORWARD FOR
    SELECT N'ALTER TABLE ' + QUOTENAME(s.name) + N'.' + QUOTENAME(t.name)
         + N' WITH NOCHECK CHECK CONSTRAINT ' + QUOTENAME(fk.name) + N';'
    FROM   sys.foreign_keys fk
    JOIN   sys.tables  t ON fk.parent_object_id = t.object_id
    JOIN   sys.schemas s ON t.schema_id = s.schema_id
    WHERE  s.name IN (SELECT schema_name FROM #Schemas);
  
OPEN fkc_cur;
FETCH NEXT FROM fkc_cur INTO @cmd;
WHILE @@FETCH_STATUS = 0
BEGIN
    SET @n += 1;
    PRINT @cmd;
    IF @Execute = 1
    BEGIN
        BEGIN TRY EXEC sys.sp_executesql @cmd; END TRY
        BEGIN CATCH PRINT N'   -- FAILED : ' + ERROR_MESSAGE(); END CATCH
    END
    FETCH NEXT FROM fkc_cur INTO @cmd;
END
CLOSE fkc_cur;
DEALLOCATE fkc_cur;
  
PRINT N'---';
PRINT CONVERT(nvarchar(10), @n) + N' FK '
    + CASE WHEN @Execute = 1 THEN N'activated and validated.' ELSE N'to be activated (preview).' END;
GO

The foreign keys, which turned out to be free

All 6,072 foreign keys were already back, active and marked not trusted, because the loading script re-enabled them with NOCHECK. Revalidating them with WITH CHECK would mean full joins against the largest tables in the database.

On a read-only archive whose source integrity was enforced by Oracle for fifteen years, that buys one thing: the optimizer regains the right to eliminate redundant joins. Worth having on a schema of deeply nested views, not worth days of scanning here. They stayed not trusted.

Day 7. Testing 600 views

The principle is simple: run every view, record what happened, and never lose the record. Each one gets a SELECT TOP (100) * inside a TRY/CATCH, so a view that fails is logged rather than fatal, and the outcome lands in a table with its duration in milliseconds. The cursor only picks up views that are not already recorded, which means an interrupted run resumes where it stopped instead of starting over, and the duration column is what you sort on once it is done:

SET NOCOUNT ON;
 
IF OBJECT_ID('dbo.ViewTestResults') IS NULL
    CREATE TABLE dbo.ViewTestResults (
        schema_name  sysname,
        view_name    sysname,
        statut       varchar(20),
        duree_ms     int,
        message      nvarchar(2048),
        tested_at    datetime2 DEFAULT SYSDATETIME(),
        CONSTRAINT PK_ViewTestResults PRIMARY KEY (schema_name, view_name)
    );
 
DECLARE @sch sysname, @vue sysname, @sql nvarchar(max), @t0 datetime2, @ms int, @i int = 0, @n int;
 
SELECT @n = COUNT(*)
FROM   sys.views v JOIN sys.schemas s ON v.schema_id = s.schema_id
WHERE  s.name = 'SCDAT'
  AND  NOT EXISTS (SELECT 1 FROM dbo.ViewTestResults r
                   WHERE r.schema_name = s.name AND r.view_name = v.name);
 
RAISERROR('=== %d view remaining ===', 0, 1, @n) WITH NOWAIT;
 
DECLARE c CURSOR FAST_FORWARD FOR
  SELECT s.name, v.name
  FROM   sys.views v JOIN sys.schemas s ON v.schema_id = s.schema_id
  WHERE  s.name = 'SCDAT'
    AND  NOT EXISTS (SELECT 1 FROM dbo.ViewTestResults r
                     WHERE r.schema_name = s.name AND r.view_name = v.name)
  ORDER  BY v.name;
 
OPEN c;
FETCH NEXT FROM c INTO @sch, @vue;
WHILE @@FETCH_STATUS = 0
BEGIN
    SET @i += 1;
    RAISERROR('[%d/%d] %s.%s', 0, 1, @i, @n, @sch, @vue) WITH NOWAIT;
 
    SET @t0 = SYSDATETIME();
    BEGIN TRY
        SET @sql = N'SELECT TOP (100) * FROM ' + QUOTENAME(@sch) + N'.' + QUOTENAME(@vue) + N';';
        EXEC sp_executesql @sql;
        SET @ms = DATEDIFF(MILLISECOND, @t0, SYSDATETIME());
        INSERT dbo.ViewTestResults (schema_name, view_name, statut, duree_ms, message)
        VALUES (@sch, @vue, 'OK', @ms, NULL);
    END TRY
    BEGIN CATCH
        SET @ms = DATEDIFF(MILLISECOND, @t0, SYSDATETIME());
        INSERT dbo.ViewTestResults (schema_name, view_name, statut, duree_ms, message)
        VALUES (@sch, @vue, 'FAILED', @ms, ERROR_MESSAGE());
        RAISERROR('    -> FAILED: %s', 0, 1, @vue) WITH NOWAIT;
    END CATCH
 
    FETCH NEXT FROM c INTO @sch, @vue;
END
CLOSE c; DEALLOCATE c;
 
RAISERROR('=== Done ===', 0, 1) WITH NOWAIT;
 
-- summary
SELECT statut, COUNT(*) AS nb, SUM(duree_ms) AS duree_totale_ms
FROM dbo.ViewTestResults GROUP BY statut;
 
-- failed
SELECT schema_name, view_name, duree_ms, message
FROM dbo.ViewTestResults WHERE statut='FAILED' ORDER BY view_name;
 
-- all
SELECT schema_name, view_name, statut, duree_ms
FROM dbo.ViewTestResults ORDER BY duree_ms DESC;

The result looked excellent: 604 views OK, zero errors, against real data. Six skipped because they are not implemented to be run without an explicit filter added. SELECT TOP 100 * without a filter is not a representative test but still confirms that definition and joins are working fine. So: test with the filter the users will actually write. A view that only misbehaves when queried in a way nobody queries it is not a broken view, but you cannot know that from a green tick.

Day 8. Closing the hatch

Database set to read-only, final backup taken and user’s specific queries tested with success, the story now comes to a happy end.

What I would tell the next person

An archive migration is not a cheap application migration. It is a different decision set, and read-only is the input that drives most of it. 25,000 triggers are pointless, PAGE compression free, and untrusted foreign keys perfectly acceptable. It even tells you how to index: broadly and in advance, because nobody can tell you which row an auditor will want in three years. Before anything else, ask what the target will be used for. The answer rewrites the plan before the first byte moves.

Measure, and distrust anything that presents itself as progress. The SSMA counter refreshes per batch and sits still for minutes at a time. percent_complete stays at zero for an entire index build. A green tick on SELECT TOP 100 confirms that the joins resolve and nothing beyond that. In every one of those cases the real number was sitting in a DMV.

On views, the conversion rate is the least useful number in the report. What made four thousand views a finite piece of work was not the translation but the two steps around it: agreeing the real scope with the client first, then classifying what remained by pattern in the Oracle source rather than one error message at a time. An AI assistant translates a family of twenty views very well. It validates nothing, and the test against the source stays yours.

Compression on an archive is the rare decision with nothing to weigh against it. 117 GB down to 30.7 GB on the first table, more than 2.5 TB reclaimed across the set, and not a single write to pay it back with. If your heap is mostly null numeric columns, it is bigger than you think, and it will compress better than you expect.

L’article Archiving 5 TB from Oracle into SQL Server: views, indexes and compression (part 2) est apparu en premier sur dbi Blog.

Eighty-one hours. For one table. There were five more behind it.

SSMA is well documented but migrations at this scale are not done often enough to have plenty of materials. Most of what you find is either a step-by-step guide against a sample database, or a vendor case study with no numbers in it. This post has the numbers, in the order I actually got them, including the ones I got wrong first.

To keep my sanity through a long and intricate migration, I kept a logbook. This first part is about getting the data across: the brief, the type negotiation, and the settings that took the load from 788 rows per second to 42,170. Part 2 picks up exactly where this one stops, with 3.2 TB sitting in heaps and not a single index on it.

Day 1. The brief, and the one sentence that mattered

An Oracle 19c production database running an asset management package, to be archived into SQL Server 2025 Standard Edition. Roughly 5,600 tables, 4,800 views, a little under 5 TB depending on how you count (more on that later).

Three facts shaped everything that followed.

The target is read-only. Nothing will ever write to it once the load is finished, because the whole point of the exercise is to keep fifteen years of records available to people who need to look something up rather than to keep an application running. Auditors come by, rarely and unpredictably, to find one specific row.

That is the entire performance requirement, and it decided more of this project than any measurement did. It is why the indexes are broad rather than targeted at known queries, why PAGE compression costs nothing here, why 25,000 triggers went straight in the bin, and why two views that take thirty seconds are perfectly acceptable. Every one of those decisions comes back later in this series.

The freeze has no deadline. This is unusual enough to state plainly. Most migrations are built around a cutover window measured in hours, and every technical decision bends to it. Here the source was frozen since the beginning and simply stayed frozen.

The data travels in two hops, which becomes the story of the whole load:

Getting in without asking for the keys

The source is critical production, so the migration account got the minimum that actually works: CREATE SESSION, dictionary read, and SELECT on the tables in scope.

SSMA also asks for CREATE ANY PROCEDURE, CREATE ANY TYPE and CREATE ANY TRIGGER. Those three serve server-side extraction and SSMA’s own testing feature and at the end of the project a single DROP USER <user_name> CASCADE removes the whole thing.

There is a decision hiding here that is worth making deliberately. SELECT ANY TABLE is convenient and gives your account read access to the entire database, including schemas you are not migrating. Granting SELECT table by table on the schemas in scope is more work and produces an auditable perimeter.

Day 2. Negotiating with numbers

We agreed the type mapping with the client before converting anything, which I recommend for the political reasons as much as the technical ones:

38 digits against 51

Then the conversions started failing on decimal overflow, and the reason is more interesting than “the numbers were too big“.

Oracle NUMBER without precision is a floating decimal: 38 significant digits, with an exponent range running from 1e-130 to just under 1e126. SQL Server DECIMAL is fixed point: the precision bounds the magnitude. DECIMAL(38,0) stops at 1e38 and there is no exponent to escape with.

So the values that broke were not unusually precise, they were unusually large. A value carrying 51 digits and two significant figures is perfectly ordinary in Oracle, sails through every constraint it meets on the way out, and then has no DECIMAL representation waiting for it at the other end.

The answer is FLOAT, which is also a floating type and reaches ±1.79e308. Eleven tables were reloaded after switching the mapping.

Be careful what you promise the client here, because FLOAT keeps 15 significant digits, not 38. For the values that actually overflowed, large magnitude and few significant figures, the loss is zero. For a value in those columns carrying more than 15 significant digits, the loss is real. That is countable rather than arguable, so count it before you write the reassuring email.

25,000 triggers, none converted

The source carried roughly 25,000 triggers, almost all in one schema. I converted none of them and unticked them all before Convert Schema.

The reasoning fits in one line: on a read-only database, a trigger never fires. Converting and testing 25,000 objects that will never execute would have consumed weeks and produced nothing.

Tables went first, with their types and sequences, to unblock the data migration. Packages, views and procedures came later. Indexes are not a separate step: SSMA converts them alongside their table, turning bitmap indexes into nonclustered ones, handling function-based indexes case by case, and skipping XMLTYPE indexes entirely.

Then 103 tables failed extraction with ORA-00904, all because of hidden SYS_C…$ columns left behind by function-based indexes. I had already set “Ignore hidden system columns” to Yes, which is why this took longer than it should have: in our project that option governed schema conversion and did not stop the extraction from asking for the column. The fix is a custom select that aliases NULL into the missing name (see my previous blog about this problem).

Day 3. Turning everything off

The loading method is three switches and a recovery model.

Disable the nonclustered indexes. A disabled index stores nothing and is not maintained row by row during the load.

Disable the foreign keys with NOCHECK CONSTRAINT.

Drop the clustered primary keys on the largest tables and load into heaps.

Then set the recovery model to SIMPLE. With heaps and bulk insert, logging is already minimal.

The script that does this across every schema needs to be restartable and needs to log what it did. A load that runs for days will be interrupted, and you want to know exactly which of 22’000 indexes were disabled when it was.

Script to disable all nonclustered indexes:

USE <DB_NAME>;  
GO
  
  
IF OBJECT_ID(N'tempdb..#Schemas') IS NOT NULL DROP TABLE #Schemas;
CREATE TABLE #Schemas (schema_name sysname PRIMARY KEY);
  
INSERT INTO #Schemas (schema_name) VALUES (N'<SCHEMA_NAME>'); --Insert your schema name here
    ;
  
SELECT schema_name AS schemas_a_traiter FROM #Schemas;
GO
  
  
-- Disable all NC Indexes
 
SET NOCOUNT ON;
DECLARE @Execute bit = 0;   -- 0 = preview ; 1 = execute
  
DECLARE @cmd nvarchar(max);
DECLARE @n   int = 0;
  
DECLARE idx_cur CURSOR LOCAL FAST_FORWARD FOR
    SELECT N'ALTER INDEX ' + QUOTENAME(i.name)
         + N' ON ' + QUOTENAME(s.name) + N'.' + QUOTENAME(t.name)
         + N' DISABLE;'
    FROM   sys.indexes  i
    JOIN   sys.tables   t ON i.object_id = t.object_id
    JOIN   sys.schemas  s ON t.schema_id = s.schema_id
    WHERE  s.name IN (SELECT schema_name FROM #Schemas)
      AND  i.type_desc            in (N'NONCLUSTERED') 
      AND  i.is_disabled          = 0                
    ORDER BY s.name, t.name, i.name;
  
OPEN idx_cur;
FETCH NEXT FROM idx_cur INTO @cmd;
WHILE @@FETCH_STATUS = 0
BEGIN
    SET @n += 1;
    PRINT @cmd;
    IF @Execute = 1
    BEGIN
        BEGIN TRY
            EXEC sys.sp_executesql @cmd;
        END TRY
        BEGIN CATCH
            PRINT N'   -- ECHEC : ' + ERROR_MESSAGE();
        END CATCH
    END
    FETCH NEXT FROM idx_cur INTO @cmd;
END
CLOSE idx_cur;
DEALLOCATE idx_cur;
  
PRINT N'---';
PRINT CONVERT(nvarchar(10), @n) + N' index nonclustered '
    + CASE WHEN @Execute = 1 THEN N'disabled.' ELSE N'te be disabled (preview).' END;
GO
  
  
----------------------------------------------
 
 
SET NOCOUNT ON;
DECLARE @Execute bit = 0;   -- 0 = preview ; 1 = executer
  
DECLARE @cmd nvarchar(max);
DECLARE @n   int = 0;
  
DECLARE fk_cur CURSOR LOCAL FAST_FORWARD FOR
    SELECT N'ALTER TABLE ' + QUOTENAME(s.name) + N'.' + QUOTENAME(t.name)
         + N' NOCHECK CONSTRAINT ' + QUOTENAME(fk.name) + N';'
    FROM   sys.foreign_keys fk
    JOIN   sys.tables  t ON fk.parent_object_id = t.object_id
    JOIN   sys.schemas s ON t.schema_id = s.schema_id
    WHERE  s.name IN (SELECT schema_name FROM #Schemas)
      AND  fk.is_disabled = 0;
  
OPEN fk_cur;
FETCH NEXT FROM fk_cur INTO @cmd;
WHILE @@FETCH_STATUS = 0
BEGIN
    SET @n += 1;
    PRINT @cmd;
    IF @Execute = 1
    BEGIN
        BEGIN TRY EXEC sys.sp_executesql @cmd; END TRY
        BEGIN CATCH PRINT N'   -- FAILED: ' + ERROR_MESSAGE(); END CATCH
    END
    FETCH NEXT FROM fk_cur INTO @cmd;
END
CLOSE fk_cur;
DEALLOCATE fk_cur;
  
PRINT N'---';
PRINT CONVERT(nvarchar(10), @n) + N' FK '
    + CASE WHEN @Execute = 1 THEN N'disabled.' ELSE N'to be disabled (preview).' END;
GO 

There is a second script that puts all of this back, with PAGE compression, once the data is in. It belongs with the rebuild rather than with the load, so it is in part 2.

Day 4. ×53, on the same architecture

The first load ran client-side: the hop server reads from Oracle and writes to SQL Server.

On the SQL Server side the dominant wait was ASYNC_NETWORK_IO, which is the engine telling you it has finished its work and is sitting there waiting for the next batch to arrive. Meanwhile CPU and memory on the hop VM stayed low throughout. Nothing anywhere in the chain was short of power; the pipeline was simply underfed.

The answer for this was not a different architecture (with for example a linked server directly from Oracle to SQL Server), it was three settings on the one I already had.

Load into heaps with the nonclustered indexes disabled. Parallelise across tables, not within one, which is the part most people have backwards about SSMA: Thread Count 14 with Table lock = No, because TABLOCK plus multithreading deadlocks. Batch size at 200’000.

Measured with the two-timestamp method: 7,000,000 rows in 166 seconds, about 42,170 rows per second. Against 788, that is ×53.

It is worth being explicit about why write-side changes fixed a network wait, because the two do not obviously connect. Of the three levers only one touches writing. Parallelising across tables fills a pipe that was carrying a single stream and fast writes remove the back-pressure that was leaving the sender idle.

Two volumetry traps

Ranking tables by dba_segments with segment_type = 'TABLE' ignores LOBs, which live in their own segment. One table that looked unremarkable carried 123 GB of LOB. Redo the ranking with LOB segments included before you plan anything. And row count does not rank the same way as volume. One table held 2.8 billion rows in 67 GB. Another held a tenth of that in fourteen times the space.

What I would tell the next person, about the load

The bottleneck was the feed rate rather than the hardware. 788 rows per second, with ASYNC_NETWORK_IO at the top of the wait list, on a chain where nothing was short of CPU or memory. Three settings took it to 42,170: heaps, disabled indexes and parallelism across tables. Reach for the settings before you reach for a new architecture, because the new architecture takes a week to build and may not be the thing that was slow.

On types, the failure was one of magnitude and not of precision, which is worth knowing before you try to diagnose it. Oracle NUMBER is a floating decimal whose exponent runs far past anything DECIMAL can address, so a value with 51 digits and two significant figures breaks a conversion that a value with 38 significant digits survives without trouble. And when you move those columns to FLOAT, count what you are actually giving up before you write the email saying nothing was lost.

At this point the data is across and the database is useless. Every nonclustered index is disabled, the largest tables have no primary key, the foreign keys are off, and none of the four thousand views compile. Part 2 is about putting all of that back.

L’article Archiving 5 TB from Oracle into SQL Server: a migration logbook (part 1) est apparu en premier sur dbi Blog.

You are using 2 or more LDAP servers for Oracle name resolution to be highly available?
Do you kown what happens if an ldap server is down or not reachable? You probably expect that it tries the next one?

Short answer: Yes, but probably not with a behavior you expect…

Let’s see my test configuration There are 2 ldap servers ol9a and ol9b. For the setup, see my former blog post: Setup an LDAP server for Oracle naming on Enterprise Linux 9

# ldap.ora
DIRECTORY_SERVERS=(ol9a:389:636,ol9b:389:636)
DEFAULT_ADMIN_CONTEXT = "dc=dbi-services,dc=com"
DIRECTORY_SERVER_TYPE = OID

# sqlnet.ora
NAMES.DEFAULT_DOMAIN=dbi-services.com
NAMES.DIRECTORY_PATH=(LDAP)

With NAMES.DIRECTORY_PATH=(LDAP) we enforce to only use LDAP. Often, you will allow other methods like (LDAP,TNSNAMES,EZCONNECT). Oracle tries the name resolution in this order.

Tests with an Oracle client installation (tnsping)

Now let’s see what happens if ldap is not reachable. Note the difference: “server” refers to the ldap server process, “host” refers to the machine on which the ldap server runs (If the server is up, then logically the host is also up). “down” of a host is from the Oracle-client’s point of view and means that it is unreachable (will not respond), regardless of whether it is a host problem or just a network interruption. “1st” means the 1st host/server in the DIRECTORY_SERVERS parameter (in the example ora9a).

time tnsping DB1970

We remember: the order is DIRECTORY_SERVERS=(ol9a:389:636,ol9b:389:636)

Both up

If both ldap servers are up, we get an answer in a fraction of a second (as expected).

2nd down

If the 2nd server/host (ora9b) is down, we get an answer in a fraction of a second. I repeated this test several times, and it always responds immediately. That means, there is no round-robin selection of the servers in place.

1st down

if the 1st ldap server on ora9a is down (the process, not the host), we get the answer within a fraction of a second from ora9b. the Oracle client immediately detects that there is no ldap server running (error “Connection refused” on ldap port).

But if the host itself does not respond (stopped, network interruption), then the Oracle client has to wait for a timeout and that is usually more than 10 seconds. Did you expect that? Most customers have never noticed this problem as they have never had an LDAP host failure.

Used LDAP adapter to resolve the alias
Attempting to contact (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=ltp14s-i)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=DB1970A.erfinder)))
OK (30 msec)

real    0m15.090s

Is that acceptable for your application to wait for >10 seconds? For an interactive login once a day, it is annoying, but probably OK. But with applications that make thousands of short-term logins? In the worst case, the application will get a timeout.

Configuration optimization

Can we get around this long timeout? Yes, we can. Add the following parameter to sqlnet.ora:

NAMES.LDAP_CONN_TIMEOUT=1  #value in seconds

time tnsping db1970

TNS Ping Utility for Linux: Version 19.0.0.0.0 - Production on 18-SEP-2026 16:46:37

Copyright (c) 1997, 2019, Oracle.  All rights reserved.

Used parameter files:
/u00/app/oracle/network/admin/ldap/sqlnet.ora

Used LDAP adapter to resolve the alias
Attempting to contact (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=ltp14s-i)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=DB1970A)))
OK (10 msec)

real    0m1.040s

Note: the timeouts may have some variations (after a few calls, the timeout was reduced to about 3 seconds for me).
The value OK (10 msec) refers only to the time for the listener connect, not for the name resolution. The output of time includes the total time of the tnsping execution.

Conclusion:

• We can configure the ldap failover timeout in seconds. Minimum is 1 second, default seems to be 10-15 sec.
• Timeouts only happen when the host is unreachable, not when the LDAP server (software) is stopped.
• LDAP servers are used in the order of the DIRECTORY_SERVERS parameter. There is no round-robin for load-balancing.

Tests with Java/JDBC

To do the tests with Java jdbc, I will use Oracle’s SQL Developer .

I create a new connection using “Custom JDBC” and add the following string:

jdbc:oracle:thin:@ldap://ol9a:389/DB1970,cn=OracleContext,dc=dbi-services,dc=com ldap://ol9b:389/DB1970,cn=OracleContext,dc=dbi-services,dc=com

The behavior is like before. But wait… it isn’t!
I have now noticed that the ldap servers are used round-robin. Sometimes the connect is instantaneous, sometimes it takes the time of the timeout to failover to the other server.

The jdbc:thin driver does not use our sqlnet.ora settings. So how can we configure this in Java? Fortunately there is a connection property
com.sun.jndi.ldap.connect.timeout
In sqldeveloper you will configure it in the “Advanced” tab:

With JDBC, the values are in milliseconds. Thus, in contrast to the normal client, we can also specify values of less than 1 second (in the example 500 = 0.5 sec.).

In sqldeveloper it works. Does it also work with a “normal” java program (thanks to duck.ai to generate me the code). The class is named T (like Test).

// T.java
import java.sql.*;
import java.util.Properties;

class T {
  public static void main(String[] args) throws Exception {
    String url =
        "jdbc:oracle:thin:@ldap://ol9a:389/DB1970,cn=OracleContext,dc=dbi-services,dc=com ldap://ol9b:389/DB1970,cn=OracleContext,dc=dbi-services,dc=com";

    Properties env = new Properties();
    env.put("user", "system");
    env.put("password", "SystemPassword");
    env.put("com.sun.jndi.ldap.connect.timeout", "2000");

    try (Connection c = DriverManager.getConnection(url, env);
         Statement s = c.createStatement();
         ResultSet r = s.executeQuery("select host_name from v$instance")) {
      if (r.next())
        System.out.println(r.getString(1));
    }
  }
}

Then run the program

 $ORACLE_HOME/jdk/bin/javac -cp $ORACLE_HOME/jdbc/lib/ojdbc8.jar T.java
 time $ORACLE_HOME/jdk/bin/java -cp $ORACLE_HOME/jdbc/lib/ojdbc8.jar:. T

We will see the same round-robin behavior as in sqldeveloper. Sometimes it takes 0.04-0.6 seconds (tries ol9b first), sometimes it takes 2.05-2.6 seconds (tries ol9a, wait configured timeout of 2sec. , try ol9b and connect)

Conclusion:

• We can configure the ldap failover timeout in milliseconds (in contrast to seconds with OCI client like sqlplus)
• Timeouts only happen when the host is unreachable, not when the LDAP server (software) is stopped.
• LDAP servers defined in the DIRECTORY_SERVERS parameter are used in a rond-robin order which is ideal for load-balancing.

General remarks and conclusions

• a timeout does not occur when the ldap service is stopped. The client immediately receives an error message from the host and can ask the other ldap
• If the first ldap server does not know the requested name, it will not try other ldap servers, but will try (if configured in names.directory_path) other naming methods (e.g. tnsnames.ora)
• The configured timeout is the maximum timeout. If it finds out that the host is not reachable, it tries the next one (e.g. the hostname is unknown).

For normal sql*net clients using the *.ora files (TNS_ADMIN / …/network/admin; see my blog post), you can configure the timeout in sqlnet.ora

NAMES.LDAP_CONN_TIMEOUT=1  #value in seconds

For Java JDBC clients, set the property

com.sun.jndi.ldap.connect.timeout=1000  #value in milliseconds

L’article Oracle LDAP Timeouts est apparu en premier sur dbi Blog.

To centralise Oracle naming resolution, you can use an LDAP server. You can use the commercial products from Oracle like Oracle Internet Directory (OID) or Oracle Universal Directory (OUD). Alternatively, you can also use OpenLDAP from your Linux distribution. I have used this successfully with customers for many years.

I now wanted to build a new test environment for myself on an Enterprise Linux 9 system (RedHat, Oracle Linux). But then I noticed that the opendap server is no longer available. After some internet searching, I found the following blog post that describes how to install OpenLDAP on EL9: https://seanstuber.com/2019/12/29/setting-up-openldap-on-oracle-linux-8-for-tns-name-resolution/

But is there no way to do this with on-board tools? A further search then revealed that RedHat has switched to 389-ds-base (https://www.port389.org/). So I want to try it with this native tool. And as I fear, almost everything is different here again… To spare you this try-and-error, I’ll give you the commands here on how to get an executable environment.

Installation and configuration

First of all, install the software:

[root@ol9b ~]# dnf install -y 389-ds-base openldap-clients

Caution! I stongly recommend to do a dnf upgrade before. I had the problem that you couldn’t log in with ssh afterwards and the ssh daemon couldn’t be restarted.

OpenSSL version mismatch. Built against 30000070, you have 30500080

At least a dnf upgrade "openssh*" was required and restart of sshd.

Afterwards, you can configure and start the LDAP server. We will use “orcladmin” as administrative user and “dbi” as instance name.

[root@ol9b ~]# dscreate interactive
Install Directory Server (interactive mode)
===========================================

Enter system's hostname [ol9b]:

Enter the instance name [ol9b]: dbi

Enter port number [389]:

Create self-signed certificate database [yes]:

Enter secure port number [636]:

Enter Directory Manager DN [cn=Directory Manager]: cn=orcladmin

Enter the Directory Manager password:
Confirm the Directory Manager Password:

Choose whether mdb or bdb is used. [bdb]:

Enter the database suffix (or enter "none" to skip) [dc=ol9b]: none

Do you want to start the instance after the installation? [yes]:

Are you ready to install? [no]: yes
Starting installation ...
Validate installation settings ...
Create file system structures ...
Create self-signed certificate database ...
Perform SELinux labeling ...
Perform post-installation tasks ...
Completed installation for instance: slapd-dbi

[root@ol9b ~]#  systemctl status dirsrv@dbi
● dirsrv@dbi.service - 389 Directory Server dbi.
     Loaded: loaded (/usr/lib/systemd/system/dirsrv@.service; enabled; preset: disabled)
    Drop-In: /usr/lib/systemd/system/dirsrv@.service.d
             └─custom.conf
...

After configuration, the daemon is already started and the service is also set to enabled. If you use a firewall on your server, open the ldap ports:

[root@ol9b ~]# firewall-cmd --permanent --add-service=ldap
[root@ol9b ~]# firewall-cmd --permanent --add-service=ldaps
[root@ol9b ~]# firewall-cmd --reload

Now we have to extend the schema with the extensions for Oracle naming resolution. Since Oracle 26ai (23.26), a schema file is available in $ORACLE_HOME/network/ldap/schema/openldap/orclnetnaming.schema; This file has to be converted to an ldif file for the ldapmodify tool. From:

attribute ( 2.16.840.1.113894.7.1.1
  NAME 'orclVersion'
  EQUALITY octetStringMatch
  SYNTAX '1.3.6.1.4.1.1466.115.121.1.40' )

attribute ( 2.16.840.1.113894.7.1.2
  NAME 'orclOracleHome'
  EQUALITY octetStringMatch
  SYNTAX '1.3.6.1.4.1.1466.115.121.1.40' )
...

to:

dn: cn=schema
changetype: modify
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.7.1.1
  NAME 'orclVersion'
  EQUALITY octetStringMatch
  SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.7.1.2
  NAME 'orclOracleHome'
  EQUALITY octetStringMatch
  SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
...

(the full version you will find at the end of this post)

And now you can load this schema extensions

[root@ol9b ~]# ldapmodify -x -H ldap://ol9b:389 -D "cn=orcladmin" -W -f orclnetnaming.ldif
Enter LDAP Password:
modifying entry "cn=schema"

We will create the DN dbi-services,dn=com

[root@ol9b ~]# dsconf dbi backend create --suffix="dc=com" --be-name="com" --create-suffix
The database was sucessfully created

[root@ol9b ~]# ldapadd -x -H ldap://ol9b:389 -D "cn=orcladmin"  -W <<EOD
dn: dc=dbi-services,dc=com
objectClass: top
objectClass: dcObject
objectClass: organization
o: Oracle Internet Directory
dc: dbi-services

dn: cn=OracleContext,dc=dbi-services,dc=com
objectclass: orclContext
cn: OracleContext
EOD
Enter LDAP Password:
adding new entry "dc=dbi-services,dc=com"

adding new entry "cn=OracleContext,dc=dbi-services,dc=com"

Oracle tools need the so-called “anonymous bind” so that they can query the data without logging in. We still have to activate this.

[root@ol9b ~]# ldapmodify -x -H ldap://ol9b:389 -D "cn=orcladmin"  -W <<EOD
 dn: dc=dbi-services,dc=com
changetype: modify
add: aci
aci: (targetattr = "*")(version 3.0; acl "Anonymous read access"; allow (read, search, compare) userdn = "ldap:///anyone";)
EOD

The configuration is now complete, and we can insert the first sqlnet alias

[root@ol9b ~]# ldapadd -x -H ldap://ol9b:389 -D "cn=orcladmin"  -W <<EOD
dn: cn=DB1970,cn=OracleContext,dc=dbi-services,dc=com
objectclass: top
objectclass: orclNetService
cn: DB1970
orclNetDescString: (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=ltp14s-i)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=DB1970A.erfinder)))
EOD
Enter LDAP Password:
adding new entry "cn=DB1970,cn=OracleContext,dc=dbi-services,dc=com"

Hint: this is the syntax of OpenLdap. As oracle user, use /usr/bin/ldapadd, or adapt the syntax for Oracle’s ldapadd.

Testing

Now it gets exciting: does the Oracle name resolution work?

Probably we have to adapt our ldap.ora and sqlnet.ora files. For the test, I will create a temporary TNS_ADMIN directory.

[oracle@ltp14 ~]# mkdir /tmp/ldap
[oracle@ltp14 ~]# export TNS_ADMIN=/tmp/ldap

[oracle@ltp14 ~]# echo 'DIRECTORY_SERVERS= (ol9b:389:636)
DEFAULT_ADMIN_CONTEXT = "dc=dbi-services,dc=com"
DIRECTORY_SERVER_TYPE = OID' >$TNS_ADMIN/ldap.ora

[oracle@ltp14 ~]# echo 'NAMES.DEFAULT_DOMAIN=dbi-services.com
NAMES.DIRECTORY_PATH=(LDAP)' >sqlnet.ora

[oracle@ltp14 ~]# tnsping DB1970

TNS Ping Utility for Linux: Version 19.0.0.0.0 - Production on 18-SEP-2026 13:48:26

Copyright (c) 1997, 2019, Oracle.  All rights reserved.

Used parameter files:
/u00/app/oracle/network/admin/ldapdbi/sqlnet.ora

Used LDAP adapter to resolve the alias
Attempting to contact (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=ltp14s-i)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=DB1970A.erfinder)))
OK (0 msec)

And YES! it works 🙂

Disclaimer: I’m not the expert of LDAP. I wanted to be able to quickly set up an LDAP server for Oracle name resolution. The setup certainly still has some optimization potential. Feel free to adapt it for your needs.

Annex

orclnetnaming.ldif (converted from 26ai $ORACLE_HOME/network/ldap/schema/openldap/orclnetnaming.schema)

dn: cn=schema
changetype: modify
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.7.1.1 NAME 'orclVersion' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.7.1.2 NAME 'orclOracleHome' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.7.1.3 NAME 'orclSystemName' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.7.1.4 NAME 'orclServiceType' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.7.1.5 NAME 'orclSid' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.7.1.6 NAME 'orclProductVersion' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.1 NAME 'orclNetSourceRoute' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.2 NAME 'orclNetLoadBalance' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.3 NAME 'orclNetFailover' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.4 NAME 'orclNetSdu' EQUALITY numericStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.36 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.5 NAME 'orclNetServer' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.6 NAME 'orclNetServiceName' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.7 NAME 'orclNetInstanceName' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.8 NAME 'orclNetHandlerName' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.9 NAME 'orclNetParamList' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.10 NAME 'orclNetAuthenticationType' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.11 NAME 'orclNetAuthParams' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.12 NAME 'orclNetDescName' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.13 NAME 'orclNetDescString' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.14 NAME 'orclNetAddressString' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.15 NAME 'orclNetProtocol' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.16 NAME 'orclNetShared' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.17 NAME 'orclNetAddrList' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.18 NAME 'orclNetProtocolStack' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.19 NAME 'orclNetDescList' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.20 NAME 'orclNetConnParamList' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: attributeTypes
attributeTypes: ( 2.16.840.1.113894.3.1.21 NAME 'orclNetAuthenticationService' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: objectClasses
objectClasses: ( 2.16.840.1.113894.7.2.3 NAME 'orclContext' SUP top MAY ( cn $   orclNetDescName ) )
-
add: objectClasses
objectClasses: ( 2.16.840.1.113894.7.2.6 NAME 'orclSchemaVersion' SUP top MUST ( cn $   orclProductVersion ) )
-
add: objectClasses
objectClasses: ( 2.16.840.1.113894.3.2.1 NAME 'orclNetAddress' SUP top MUST (cn) MAY ( orclNetAddressString $ orclNetProtocol $ orclNetShared $ orclVersion $description) )
-
add: objectClasses
objectClasses: ( 2.16.840.1.113894.3.2.2 NAME 'orclNetAddressList' SUP top MUST (cn)
     MAY ( orclNetAddrList $ orclNetSourceRoute $ orclNetLoadBalance $ orclNetFailover $ orclNetShared $ orclVersion $ description ) )
-
add: objectClasses
objectClasses: ( 2.16.840.1.113894.3.2.3
     NAME 'orclNetDescription' SUP top
     MUST (cn)
     MAY ( orclVersion  $ orclNetSourceRoute  $  orclNetSdu $ orclNetServer $ orclNetServiceName $ orclNetInstanceName $ orclNetHandlerName $ orclNetParamList $ orclNetAuthenticationType $ orclNetAuthParams $ orclNetShared $ orclNetAddrList $  orclNetProtocolStack $ orclNetConnParamList $ orclNetAuthenticationService $ orclNetLoadBalance $   orclNetDescList $ orclNetFailover $ orclNetShared $ orclNetAddrList ) )
-
add: objectClasses
objectClasses: ( 2.16.840.1.113894.3.2.30
     NAME 'orclNetDescriptionAux1' SUP top
     AUXILIARY MUST (cn)
     MAY ( orclVersion  $ orclNetSourceRoute  $  orclNetSdu $ orclNetServer $ orclNetServiceName $ orclNetInstanceName $ orclNetHandlerName $ orclNetParamList $ orclNetAuthenticationType $ orclNetAuthParams $ orclNetShared $ orclNetAddrList $  orclNetProtocolStack $ orclNetConnParamList $ orclNetAuthenticationService $ orclNetLoadBalance $   orclNetDescList $ orclNetFailover $ orclNetShared $ orclNetAddrList ) )
-
add: objectClasses
objectClasses: ( 2.16.840.1.113894.3.2.4
     NAME 'orclNetDescriptionList' SUP top
     MUST (cn)
     MAY ( orclNetLoadBalance $   orclNetDescList $ orclNetFailover $ orclNetShared $ orclNetAddrList ) )
-
add: objectClasses
objectClasses: ( 2.16.840.1.113894.3.2.5
     NAME 'orclNetService' SUP top
     MUST (cn)
     MAY ( orclNetServiceName $ orclNetDescString $ orclNetDescName $ orclNetSourceRoute ) )

L’article Setup an LDAP server for Oracle naming on Enterprise Linux 9 est apparu en premier sur dbi Blog.

The problem

Migrating ~5,600 Oracle tables to SQL Server with SSMA. Most load fine; ~100 tables fail Migrate Data with the same error:

ERROR [42S22] [Oracle][ODBC][Ora]ORA-00904:
"SYS_C00004_21081414:28:22$": invalid identifier

The named column exists in no DDL anyone wrote. The [Ora] prefix says Oracle itself is rejecting the query: SSMA built an extraction SELECT naming a column Oracle refuses to resolve. Tellingly, SELECT * and COUNT(*) run fine against the same table, whatever this column is, Oracle is happy to ignore it, but not to be asked for it by name.

Where the column actually comes from

The name is the giveaway: SYS_C00004_21081414:28:22$ is what Oracle calls a column after ALTER TABLE … SET UNUSED COLUMN.

Oracle offers two ways to get rid of a column: a logical delete and a physical one. The physical delete (ALTER TABLE … DROP COLUMN) is the honest one, but on a large table it is very time- and resource-consuming. That’s why people reach for the logical delete instead:

ALTER TABLE table_name SET UNUSED (column_name);

That statement is metadata-only and instant. The column immediately stops being visible to users, and the physical removal is deferred to whenever there is time for it (see Oracle Documentation):

ALTER TABLE table_name DROP UNUSED COLUMNS;
-- on large tables, cap undo growth by checkpointing every N rows:
ALTER TABLE table_name DROP UNUSED COLUMNS CHECKPOINT 250;

To free the original name for reuse, Oracle renames the column to SYS_C<internal column number>_<YYMMDDHH24:MI:SS>$, sets USER_GENERATED to NO, HIDDEN_COLUMN to YES and releases its COLUMN_ID. So the timestamp is not when the column was added, it is the second someone ran SET UNUSED. Ours says 14 August 2021, 14:28:22.

That also explains the error pattern. The operation is one-way and the column is unreadable by design, so naming it gets you ORA-00904 «invalid identifier». SELECT * and COUNT(*) keep working because the column no longer has a COLUMN_ID and is simply excluded from the star. SSMA, however, lists it and builds an explicit column list Oracle then refuses.

Not to be confused with SYS_NC…$. Those are a different animal: virtual columns backing a function-based index or extended statistics.

Bottom line: returning NULL costs you nothing. This is a column its owner already decided to delete, holding data Oracle itself will no longer hand out. There is no information left to lose.

What doesn’t work for the migration

• SSMA setting Ignore hidden system columns = Yes was not making any effect on this use case
• Dropping the column on SQL Server resolves nothing because the error is on the source SELECT, unaffected.
• Dropping it on Oracle could not be done in our scenario because the source is frozen; DDL not allowed.
• Custom select, column removed or bare NULL: SSMA still expects the name in its mapping and fails with “key not present” or “does not match up” before the query ever reaches Oracle.

Find them all first

Before editing anything, get the full list. Discovering the affected tables one failed migration at a time is a waste of an afternoon because the data dictionary already knows.

The reason the columns are findable at all is an asymmetry between two views: an unused column is gone from ALL_TAB_COLUMNS, but still listed in ALL_TAB_COLS with HIDDEN_COLUMN = 'YES'. That second view is what you query:

SELECT owner,
table_name,
column_name,
data_type,
internal_column_id,
TO_DATE(REGEXP_SUBSTR(column_name, '\d{8}:\d{2}:\d{2}'),
'YYMMDDHH24:MI:SS') AS set_unused_at
FROM dba_tab_cols
WHERE hidden_column = 'YES'
AND user_generated = 'NO'
AND REGEXP_LIKE(column_name, '^SYS_C\d+_\d{8}:\d{2}:\d{2}\$$')
-- AND owner = '<SCHEMA_NAME>'
ORDER BY owner, table_name, internal_column_id;

The regex is deliberately strict: it matches only the SET UNUSED naming pattern, so virtual columns and other system-generated names stay out of the result.

One more view is worth a look, as a cross-check:

SELECT owner, table_name, count AS unused_columns
FROM   dba_unused_col_tabs
--WHERE owner = '<SCHEMA_NAME>'
ORDER  BY count DESC, table_name;

DBA_UNUSED_COL_TABS gives the number of unused columns per table. Sorting by that number puts the dangerous tables first: those with two or three hidden columns are the ones where you’ll forget a line in the custom select and be back at square one.

The fix

Keep the hidden column’s name as an alias, but return a literal NULL instead of reading it. SSMA’s mapping finds the name (no “key not present”); Oracle never resolves the real column (no ORA-00904).

• Tools → Project Settings → General → Migration → enable Extended data migration options.

• Data Migration Settings tab → tick Use custom select → replace each hidden-column line with:

 SELECT ...   
    TO_CHAR("<COLUMN_NAME>", 'TM', 'NLS_NUMERIC_CHARACTERS = ''.,''') as "<COLUMN_NAME>",
    NULL as "SYS_C00004_21081414:28:22$"
from <OWNER>.<TABLE_NAME> t

• Migrate Data → 100%. Drop the NULL-filled column(s) on SQL Server in post-migration cleanup.

Takeaway

SYS_C…$ is not an exotic Oracle feature, it’s an ordinary column someone deleted years ago, logically. Oracle keeps the name on file; SSMA finds it, insists on naming it, and Oracle refuses to hand it over. Aliasing a NULL satisfies both, then you drop the column on the target. No source DDL, no external tooling, everything inside SSMA, behind a project setting that’s hidden by default.

L’article Fixing ORA-00904 on Oracle hidden columns during SSMA data migration est apparu en premier sur dbi Blog.

This is something we often see among our customers. If we assume that we carry out a migration project roughly every five years, this means we’re keeping databases that are taking up space for no reason.

We, however, are Green SQL Server DBA; we’ll look to manage it correctly.

The AUDIT

First of all, what do we need to analyse?

This does not apply to all databases, but only to those that are obsolete.

They can be categorised into two types:

• Databases Never Accessed
  • No application connections detected
  • No recent transactions or queries after a period (3 months for exemple)
• Abandoned Databases
  • Linked to cancelled projects
  • No activity and nobody tell it

To see databases never accessed and abandoned databases I use the DMV sys.dm_db_index_usage_stats to see if I have an access to the database (read or write):

SELECT d.name AS DatabaseName, MAX(ius.last_user_seek) AS LastSeek,MAX(ius.last_user_scan) AS LastScan,
    MAX(ius.last_user_lookup) AS LastLookup,

    (
        SELECT MAX(v.ActivityDate)
        FROM (VALUES
                (MAX(ius.last_user_seek)),
                (MAX(ius.last_user_scan)),
                (MAX(ius.last_user_lookup))
             ) v(ActivityDate)
    ) AS LastReadActivity,

    MAX(ius.last_user_update) AS LastWriteActivity,

    DATEDIFF(DAY,
        (
            SELECT MAX(v.ActivityDate)
            FROM (VALUES
                    (MAX(ius.last_user_seek)),
                    (MAX(ius.last_user_scan)),
                    (MAX(ius.last_user_lookup))
                 ) v(ActivityDate)
        ),
        GETDATE()
    ) AS DaysSinceLastRead,

    DATEDIFF(DAY,
        MAX(ius.last_user_update),
        GETDATE()
    ) AS DaysSinceLastWrite

FROM sys.databases d
LEFT JOIN sys.dm_db_index_usage_stats ius
    ON d.database_id = ius.database_id
WHERE d.database_id > 4  -- Exclude system databases
GROUP BY d.name
ORDER BY LastWriteActivity ASC;

We can see on the first databases in the list, that we have null everywhere… This means that the databases were not solicited since the last restart… Good candidate for unused databases!

This brings us to the next point.

The ANALISYS

If we can see that there is little activity on the database, we can, of course, carry out an initial analysis using criteria such as these:

< 30 days –> the database is still active

30–180 days –> the database is on hold pending to be dropped

> 180 days –> the database is a strong candidate to be dropped

Of course, it also depends on your environment – whether it’s dev, test or prod – and the criteria will vary accordingly.

The IMPACT

Like in my precedent post, we will have same impacts but not for queries of courses!

The first impact is the storage and the backup

If the database represents ~100 GB on the disk , it’s also 100 GB backed up unnecessarily.

If you have Dev, Test, PreProd & Prod and all in HA… I let you do the calculation but it’s more 1 TB!

The third impact is on the maintenance plan

The following operations must also handle the unused database:

– CheckDB
– Backups

As the database is not used, the index rebuild/reorg & update Stats will be fast!

Each unused database extends the maintenance windows for the checkdb and backups…

The ADVISE

The first advice would be to plan well in advance.

Indeed, when creating or restoring a database, it is important to know its lifecycle.

Don’t hesitate to challenge the application owners on a deadline and make sure to add it to your calendar +1day.

The second advice is to set up a monthly monitoring process prior to Windows or SQL Server patch cycles so that you can identify these unnecessary databases.

And before dropping a database, I advice you to put the database offline and see if after some days somebody complaints…

In the majority of cases, you will not have reactions, believe me!

The Green SQL Server DBA Score

Like for my precedent post, I setup a ‘SQL Server DBA Score’ to use for my tips on the subject:

Conclusion

A traditional SQL Server DBA or system administrator will let the unused databases until the next migration.

A Green SQL Server DBA will check it periodically and manage it.

An unused database is like a car that never leaves the garage.
 It still costs money, requires maintenance, and consumes resources.
The Green SQL Server DBA notices it before it becomes waste…

Think about it and begin now to see to monitor your unused databases!

See you soon for the next one! 

L’article Green SQL Server DBA Tips #3 – The hidden cost of forgotten databases est apparu en premier sur dbi Blog.

Can this also be applied to PostgreSQL?

In short, yes, it is possible. Let’s see how.

Here is the list of the previous blog posts:

• https://www.dbi-services.com/blog/sql-server-snapshot-backup-and-restore-with-proxmox-zfs/
• https://www.dbi-services.com/blog/sql-server-snapshot-backup-and-restore-with-proxmox-zfs-2-3/
• https://www.dbi-services.com/blog/sql-server-snapshot-backup-and-restore-with-proxmox-zfs-rest-api-with-sql-server-2025-3-3/

Partitioning and filesystem

We will reuse the sqlpool ZFS pool created in the first part of this series.

We identify the 300 GB disk attached to the VM. In our case, it is /dev/sdb, backed by the sqlpool/pve/vm-307-disk-0 zvol on the Proxmox side:

lsblk

We create a single partition of type Linux filesystem:

sudo sgdisk -n 1:0:0 -t 1:8300 /dev/sdb

We format the partition with XFS, which is the most commonly recommended filesystem for PostgreSQL data directories:

sudo mkfs.xfs -L pgdata /dev/sdb1 -f

We verify the result:

ahi@pgl:~$ sudo blkid /dev/sdb1
/dev/sdb1: LABEL="pgdata" UUID="028afa2f-7bb3-4a40-92aa-91c1a33f18ae9" BLOCK_SIZE="512" TYPE="xfs" PARTUUID="bd54f285-092e-4b1b-ba5e-6877f054fa7"

Mountpoint:

We create the mount point:

sudo mkdir -p /pgdata

Persistent mount via fstab:

We add the mount entry to /etc/fstab using the filesystem label rather than the device name. The device name (/dev/sdb) may change if disks are added or removed while the label remains stable:

echo 'LABEL=pgdata /pgdata xfs noatime,nodiratime 0 2' | sudo tee -a /etc/fstab
sudo systemctl daemon-reload
sudo mount /pgdata

We verify that the volume is mounted:

df -h /pgdata

PostgreSQL installation

We install PostgreSQL 18 from the official PGDG repository, which provides the latest PostgreSQL versions for Ubuntu:

sudo apt install -y postgresql-common
sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh -y
sudo apt install -y postgresql-18

Cluster creation on /pgdata:

The Ubuntu packages create a default cluster under /var/lib/postgresql. This is not what we want. The data files and the WAL must both reside on the ZFS-backed volume, so that a single ZFS snapshot captures a consistent state of the database. If they were on different volumes, the snapshot would not be atomic.

We drop the default cluster and recreate it on /pgdata:

ahi@pgl:~$ sudo pg_dropcluster --stop 18 main
sudo install -d -o postgres -g postgres -m 700 /pgdata/18
sudo pg_createcluster -d /pgdata/18/main 18 main
sudo systemctl enable --now postgresql@18-main
Creating new PostgreSQL cluster 18/main ...
/usr/lib/postgresql/18/bin/initdb -D /pgdata/18/main --auth-local peer --auth-host scram-sha-256 --no-instructions
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are enabled.

fixing permissions on existing directory /pgdata/18/main ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default "max_connections" ... 100
selecting default "shared_buffers" ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
             Ver Cluster Port Status Owner     Data directory  Log file
             18  main    5432 down   postgres  /pgdata/18/main /var/log/postgresql/postgresql-18-main.log
Created symlink /etc/systemd/system/multi-user.target.wants/postgresql@18-main.service → /usr/lib/systemd/system/postgresql@.service.

We verify that the cluster is online and located on the right volume:

ahi@pgl:~$ pg_lsclusters
Ver Cluster Port Status Owner     Data directory  Log file
18  main    5432 online <unknown> /pgdata/18/main /var/log/postgresql/postgresql-18-main.log
ahi@pgl:~$ sudo -u postgres psql -c "SHOW data_directory;"
 data_directory
-----------------
 /pgdata/18/main
(1 row)

ahi@pgl:~$ sudo -u postgres psql -c "SELECT version();"
                                                             version
-------------------------------------------------------------------------------------------------------------------------------
 PostgreSQL 18.4 (Ubuntu 18.4-1.pgdg24.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0, 64-bit
(1 row)

We can also confirm that the WAL directory lives inside the data directory, and therefore on the zvol:

ls -ld /pgdata/18/main/pg_wal

Creating a large database

We need a database large enough to make traditional backup and restore operations time-consuming. In the SQL Server part of this series, we used the StackOverflow database (about 207 GB). For PostgreSQL, we use pgbench, the benchmarking tool shipped with PostgreSQL.

We create the database and initialize it with a scale factor of 10000. This produces a database of approximately 146 GB, with 1 billion rows in the pgbench_accounts table:

sudo -u postgres createdb bench
sudo -u postgres pgbench -i -s 10000 --partitions=8 bench

A few minutes later:

We can monitor the data growth during the initialization:

watch -n 30 'df -h /pgdata'

A few minutes later:

On the Proxmox side:

After some time, the process completes:

vacuuming...

creating primary keys...
done in 1559.55 s (drop tables 0.00 s, create tables 0.02 s, client-side generate 598.12 s, vacuum 675.53 s, primary keys 285.88 s).
ahi@pgl:~$

We check the database size:

We run a checkpoint before taking the snapshot. The recovery process starts replaying the WAL from the last checkpoint. By running it right before the snapshot, almost nothing needs to be replayed when the database starts after a restore:

sudo -u postgres psql -c "CHECKPOINT;"

Comparison with SQL Server:

On the SQL Server side, we had to run SUSPEND_FOR_SNAPSHOT_BACKUP and BACKUP WITH METADATA_ONLY. On the PostgreSQL side, none of that is needed.

The data files and the WAL are on the same zvol. An atomic ZFS snapshot therefore captures a state equivalent to a power loss, and PostgreSQL is designed to recover cleanly from that state through crash recovery: the WAL is replayed from the last checkpoint. This is documented and officially supported.

The snapshot is the backup. There is no .bkm file, no metadata backup.

Snapshot process flow

On the Proxmox side, we create the snapshot and protect it with a hold:

SNAP="sqlpool/pve/vm-307-disk-0@pg_bench_$(date +%Y%m%dT%H%M%S)" 
zfs snapshot "$SNAP" 
zfs hold sqlsnap "$SNAP" 
echo "$SNAP"

The hold protects the snapshot from an accidental destruction, as we did in part 2 of this series. We note the exact snapshot name, it will be needed for the restore.

The database stays online during the whole operation. No I/O freeze is required.

We list the snapshots:

zfs list -t snapshot -r sqlpool/pve/vm-307-disk-0

We drop the database then we restore the snapshot:

We run the snapshot restore procedure. On the VM, we stop the cluster and unmount the volume:

sudo systemctl stop postgresql@18-main
sudo umount /pgdata

On the Proxmox side, we want to restore our snapshot. We can list the available snapshots:

zfs list -t snapshot -r sqlpool/pve/vm-307-disk-0

We roll back the snapshot:

zfs rollback -r sqlpool/pve/vm-307-disk-0@pg_bench_20260803T165409

On the VM, we mount the volume and start the service:

sudo mount /pgdata
sudo systemctl start postgresql@18-main

We check a few elements in the logs:

sudo tail -30 /var/log/postgresql/postgresql-18-main.log

The service shutdown, then the restart:

UTC [232108] LOG:  database system is shut down
UTC [233968] LOG:  starting PostgreSQL 18.4 (Ubuntu 18.4-1.pgdg24.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0, 64-bit
UTC [233968] LOG:  listening on IPv4 address "0.0.0.0", port 5432
UTC [233968] LOG:  listening on IPv6 address "::", port 5432
UTC [233968] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"

PostgreSQL detects that the database was not shut down properly and replays the WAL. This is the same crash recovery mechanism as in SQL Server. Finally, the database starts.

UTC [233974] LOG:  database system was not properly shut down; automatic recovery in progress
UTC [233974] LOG:  redo starts at 20/2CB76278
UTC [233974] LOG:  invalid record length at 20/2CB76380: expected at least 24, got 0
UTC [233974] LOG:  redo done at 20/2CB76348 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
UTC [233974] LOG:  checkpoint starting: end-of-recovery immediate wait
UTC [233972] LOG:  checkpoint complete: wrote 0 buffers (0.0%), wrote 3 SLRU buffers; 0 WAL file(s) added, 0 removed, 0 recycled; write=0.002 s, sync=0.009 s, total=0.030 s; sync files=0, longest=0.000 s, average=0.005 s; distance=0 kB, estimate=0 kB; lsn=20/2CB76380, redo lsn=20/2CB76380
UTC [233968] LOG:  database system is ready to accept connections

We then verify that the database is available again:

sudo -u postgres psql -d bench -c "SELECT pg_size_pretty(pg_database_size('bench'));"

Consistency proof under load

The previous test was done on a quiesced database: we ran a CHECKPOINT right before the snapshot, and nothing was writing. The real question is: what happens if the snapshot is taken while the database is being written to?

This is where PostgreSQL differs the most from SQL Server. There is no SUSPEND_FOR_SNAPSHOT_BACKUP. We take the snapshot in the middle of the write activity and we let the WAL replay do the work.

We start the load. The built-in pgbench script runs a TPC-B-like transaction: three UPDATE statements on the accounts, tellers and branches tables and one INSERT into the history table:

sudo -u postgres pgbench -c 8 -j 4 -T 300 bench &

While the load is running, we take a snapshot on the Proxmox side:

zfs snapshot sqlpool/pve/vm-307-disk-0@pg_bench_$(date +%Y%m%dT%H%M%S)

zfs hold sqlsnap sqlpool/pve/vm-307-disk-0@pg_bench_20260803T224551

No CHECKPOINT this time, no freeze. The database is actively writing while the snapshot is taken.

The state after some time under load:

We stop the service:

sudo systemctl stop postgresql@18-main 
sudo umount /pgdata

We restore the snapshot:

zfs rollback -r sqlpool/pve/vm-307-disk-0@pg_bench_20260803T224551 

We mount the volume, start the service and check the logs:

sudo mount /pgdata 
sudo systemctl start postgresql@18-main

This time the log shows a real recovery:

2026-08-03 20:49:03.244 UTC [234405] LOG:  database system is shut down
2026-08-03 20:50:34.333 UTC [234620] LOG:  starting PostgreSQL 18.4 (Ubuntu 18.4-1.pgdg24.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0, 64-bit
2026-08-03 20:50:34.333 UTC [234620] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2026-08-03 20:50:34.333 UTC [234620] LOG:  listening on IPv6 address "::", port 5432
2026-08-03 20:50:34.335 UTC [234620] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2026-08-03 20:50:34.343 UTC [234626] LOG:  database system was interrupted; last known up at 2026-08-03 20:44:36 UTC
2026-08-03 20:50:34.381 UTC [234626] LOG:  database system was not properly shut down; automatic recovery in progress
2026-08-03 20:50:34.384 UTC [234626] LOG:  redo starts at 20/7A3618B0
2026-08-03 20:50:38.024 UTC [234626] LOG:  invalid record length at 20/8BC0A4F8: expected at least 24, got 0
2026-08-03 20:50:38.024 UTC [234626] LOG:  redo done at 20/8BC0A4D0 system usage: CPU: user: 0.71 s, system: 0.63 s, elapsed: 3.63 s
2026-08-03 20:50:38.028 UTC [234624] LOG:  checkpoint starting: end-of-recovery immediate wait
2026-08-03 20:50:53.936 UTC [234624] LOG:  checkpoint complete: wrote 105355 buffers (53.6%), wrote 5 SLRU buffers; 0 WAL file(s) added, 17 removed, 0 recycled; write=3.648 s, sync=12.232 s, total=15.911 s; sync files=189, longest=12.223 s, average=0.065 s; distance=287395 kB, estimate=287395 kB; lsn=20/8BC0A4F8, redo lsn=20/8BC0A4F8
2026-08-03 20:50:53.948 UTC [234620] LOG:  database system is ready to accept connections

Three differences compared to the first test:

• The “last known up at” timestamp (20:44:36) does not match a checkpoint we ran manually. It matches the last automatic checkpoint triggered during the load.

• The redo is not instantaneous anymore: 3.63 seconds, replaying about 280 MB of WAL (from LSN 20/7A3618B0 to 20/8BC0A4D0). All the write activity between the last checkpoint and the snapshot had to be replayed. The transactions committed before the snapshot are recovered, the ones that were in flight are rolled back.

• The end-of-recovery checkpoint then writes everything the redo rebuilt in memory: 105355 buffers, 53.6% of the buffer pool. The database is ready to accept connections about 19 seconds after the service start.

The crash recovery completed correctly and the database has been restored. We verify the TPC-B invariant. Each pgbench transaction applies the same delta to the accounts, tellers and branches tables in a single transaction. On a consistent database, the three sums must be equal:

Major drawbacks

• The snapshot covers the whole zvol. All the databases of the cluster are captured and restored together. There is no per-database restore, unlike the METADATA_ONLY approach on SQL Server which targets a single database.
• There is no backup history. SQL Server records the metadata backup in msdb. Here, the only trace is the snapshot itself on the ZFS side.
• Point-in-time recovery is not covered. The snapshot alone brings the database back to the moment it was taken. For PITR, WAL archiving would still be required on top of it.

Conclusion

• The snapshot backup and restore model of the SQL Server series applies to PostgreSQL (no I/O freeze, no metadata backup).
• One important rule: data files and WAL must reside on the same zvol so the snapshot is atomic.
• A 146 GB database was restored in a few seconds and in less than 20 seconds under active load, WAL replay included.

Thank you. Amine Haloui

L’article PostgreSQL Snapshot Backup and Restore with Proxmox ZFS (4/4) est apparu en premier sur dbi Blog.

After creating the Ticket, I connect to the instance and see that the msdb system database is in a Suspect State:

After taking a deep breath because I don’t see that every day, I begin my investigation by asking the state with the system view sys.databases first:

msdb is really in a suspect mode…

Now, to begin the analysis, I go to read the error log.:

The Error Log give me the error that I comments in SSMS query to read it better:

Error: 3314, Severity: 21, State: 1.
During undoing of a logged operation in database ‘msdb’, an error occurred at log record ID (821931:128397:5).
Typically, the specific failure is logged previously as an error in the Windows Event Log service. Restore the database or file from a backup, or repair the database.

If you have this error message, the best is to restore the msdb;
As the information of the backups is in the msdb, I cannot do a right-click to restore the database…  

I looked for the latest backup in the error log and find it:

But I also sse that the checkdb before each backup give 12 errors…

My question is at this point: Wasn’t there an alert saying that the database was corrupted?

As the users are blocked, I need to react quickly.
This is why, I do my restore and a CHECKDB with  REPAIR_ALLOW_DATA_LOSS:

RESTORE DATABASE msdb FROM DISK = 'xxx\msdb\msdb_fullbackup_xxxx20260801xxxx.BAK' WITH REPLACE;
GO
 ALTER DATABASE msdb SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
GO
DBCC CHECKDB ('msdb', REPAIR_ALLOW_DATA_LOSS) WITH NO_INFOMSGS, ALL_ERRORMSGS;
GO
ALTER DATABASE msdb SET MULTI_USER;

The commands completed successfully! YES!

The database msdb is online and working fine. SUPER!

To be sure, I go in the errorlog to see the restore and the checkdb with REPAIR_ALLOW_DATA_LOSS:

The restore was successful and in the checkDB with can see 12 errors and repaired 12 errors.

If I will investigate deeper, I will go the dump file and analyse it.

I finally managed to sort out and resolve the critical alert on monday morning.
Users can connect and read their tables again…
The customer knows why he comes to us for an SLA…

A good victory for a monday morning in our support SLA dbi-services!
See you soon for sharing again customer cases!

L’article SQL Server:  msdb is in Suspect State est apparu en premier sur dbi Blog.