SQLDay 2026 offers a full-day workshop program on 11 May 2026, before the main conference scheduled for 12–13 May 2026 in Wrocław, with onsite and online participation options depending on the session. The workshops cover several areas of the modern data platform: advanced BI, AI and MLOps, SQL performance tuning, PostgreSQL adoption, and Microsoft Fabric automation.

DAX – Beyond the Basics

This workshop is designed for Power BI users who already know the basics of DAX but now need to solve more complex business problems. The focus is on moving from simple reports to reusable, efficient and business-oriented DAX patterns.

Participants will work on practical scenarios such as advanced slicer logic, hierarchical calculations, year-to-date reporting, visual calculations, cumulative totals, ranking, and relative-period analysis. The main objective is to extend the participant’s DAX toolbox and help them write expressions that are both more powerful and better performing

AI in Databricks: Training, Deployment and Monitoring

This Polish onsite workshop covers the complete lifecycle of machine learning models in Databricks. The goal is to show how to move from data preparation to training, automation, deployment and monitoring in a production-oriented environment.

The workshop focuses on the practical implementation of MLOps using Databricks and MLflow. Topics include AI/ML architecture, data pipelines, feature engineering, model training, deep learning, CI/CD, orchestration, model versioning and monitoring. It is mainly targeted at engineers, data scientists and architects who are already working with machine learning or planning to start.

Building an Intelligent Agent in One Day with Copilot Studio

This Polish onsite workshop focuses on building conversational and autonomous agents with Microsoft Copilot Studio. The format is highly practical, with most of the time dedicated to hands-on exercises.

Participants will build agents that automate business processes, use multimodal data, generate data-driven answers and connect to enterprise data sources. The workshop also covers Dataverse grounding, flows, plugins, actions, autonomous triggers, Responsible AI, moderation and access control. It is a good fit for participants who want to understand how Copilot Studio can be used beyond simple chatbot scenarios.

Advanced T-SQL Triage: The Art of Fixing Terrible Code

This workshop is focused on real-world SQL Server troubleshooting and refactoring. The starting point is familiar to many DBAs and developers: complex stored procedures, poor query patterns, blocking data modifications, bad use of CTEs, problematic window functions, indexed views, dynamic SQL, user-defined functions and execution plans that are difficult to understand.

The objective is not only to identify what is slow, but also to understand why it is slow and how to rewrite it properly. This session is especially relevant for people who regularly inherit problematic T-SQL code and need a structured way to fix it without guessing.

Adding PostgreSQL to your SQL Server Skill Set

This workshop targets SQL Server professionals who need to add PostgreSQL to their technical scope. The context is clear: many organizations are adding PostgreSQL without immediately replacing SQL Server, which creates a need for people who understand both platforms.

The workshop compares the two database engines, explains the areas of overlap, and highlights the differences that can make PostgreSQL challenging for SQL Server users. It also covers tooling, documentation, cloud options and practical resources to support the learning path.

Automating Your Microsoft Fabric Data Platform: From Blueprint to Reality

This onsite hands-on lab focuses on automation in Microsoft Fabric. The goal is to help participants automate the full lifecycle of a Fabric data platform, from design and setup to deployment and documentation.

The workshop covers platform setup using code and configuration scripts, metadata-driven ingestion, semantic model foundations, CI/CD with GitHub and Azure DevOps, Fabric CLI, REST APIs and the fabric-cicd Python library. The expected outcome is a more robust, scalable and repeatable approach to building Fabric solutions, with less manual work and lower operational risk.

Conclusion

The SQLDay 2026 workshop program is clearly oriented toward practical implementation. Each session addresses a common challenge faced by data teams: improving analytical models, industrializing AI, fixing complex SQL code, extending SQL Server skills to PostgreSQL, or automating a modern Microsoft Fabric platform.

The common thread is operational efficiency. These workshops are not only about learning features; they are about applying them in real environments, with constraints such as performance, maintainability, automation, governance and production readiness.

Thank you. Amine Haloui.

References :

L’article SQLDay 2026 Workshops Overview est apparu en premier sur dbi Blog.

The business requirements to be able to get hands on clones of production databases are nowadays countless:

• developer agility (needing separate and isolated databases),
• production-like shared environment providing same model and code but with a smaller footprint,
• test databases for software update testing,
• data sharing,
• point-in-time reporting, what-if scenario modeling or data audit.

For environments with big databases (tens of terabytes and more), Exascale is very well suited to do the job, with the following main features and advantages:

• source database flexibility: you can now clone read-write and read-only 23/26ai databases, at the PDB and CDB level
• no upstream dependencies: unlike ASM, Exascale no longer requires a test-master read-only copy
• storage reclamation is automatically performed by Exascale for extents no longer referenced by a file
• redirect-on-write techniques enable quick clone creation and maintenance with unprecedented space efficiency
• and last but not least, clone/snapshot is independent of source, which can be dropped without impacting clone

Before going through the details on how to make clones and snapshots, let’s remind what clones and snapshots are.

Snapshot

A snapshot is a read-only point-in-time copy of a file. Possible sources for snapshot creation are regular files, clones or another snapshot.

Clone

A clone is a read-write point-in-time copy of a file, possibly created from a regular file, another clone or a snapshot.

There are actually two types of clones: thin (space efficient) or full (byte-for-byte copy of source).

Cloning implicitly creates snapshots.

How do we create clones or snapshots of PDBs ?

This is where database kernel and Exascale integration really shines: physical storage operations are encapsulated in single SQL commands, thus there is no need to learn new tricky commands and clauses to leverage Exascale cloning and snapshoting capabilities. You use SQL commands you are already familiar with:

CREATE|ALTER PLUGGABLE DATABASE

Creating PDB snapshots

To create a read-only point-in-time reference to a PDB, use:

ALTER PLUGGABLE DATABASE pdb_name SNAPSHOT pdb_snapshot_name;

Remember that snapshots are inconsistent by default (which precisely makes quick creation possible); so cloning a snapshot triggers recovery based on redo data before the new clone can be opened read-write. Thus, required redo data has to be retained or restored from backup.

The DBA_PDB_SNAPSHOTS dictionary view is available to display snapshots information.

Creating PDB thin clones

A PDB thin clone is a space-efficient clone of an existing PDB. You need to use the SNAPSHOT COPY clause like in the following syntax:

CREATE PLUGGABLE DATABASE pdb_name FROM source_pdb_name SNAPSHOT COPY;

Thin cloning a PDB

CREATE PLUGGABLE DATABASE thin_clone_pdb_name FROM source_pdb_name SNAPSHOT COPY;
ALTER PLUGGABLE DATABASE thin_clone_pdb_name OPEN INSTANCES=ALL;

Thin cloning a PDB snapshot

CREATE PLUGGABLE DATABASE thin_clone_pdb_name FROM source_pdb_name USING SNAPSHOT pdb_snapshot_name SNAPSHOT COPY;
ALTER PLUGGABLE DATABASE thin_clone_pdb_name OPEN INSTANCES=ALL;

Creating PDB full clones

As already mentioned, a full clone is a byte-for-byte copy of an existing PDB and therefore takes the same storage space than source PDB. The command to be used is a simple:

CREATE PLUGGABLE DATABASE … FROM …

The missing SNAPSHOT COPY clause directs the database to create a complete block-for-block copy of the PDB.

CREATE PLUGGABLE DATABASE full_clone_pdb_name FROM source_pdb_name;
ALTER PLUGGABLE DATABASE full_clone_pdb_name OPEN INSTANCES=ALL;

Existing snapshots or thin clones can also be referenced as a source in the preceding commands.

What about cloning an entire container database ?

Space-efficient clones of an entire container database are possible thanks to the powerful gDBClone utility written by Ruggero Citton, to whom we owe other great utilities ODABR (consistent and incremental backup and restore for Oracle Database Appliance nodes), KVMBR (backup and restore of KVM domains) or more recently DSCM & DSCMREST (deployment of Oracle database containers with efficient storage snapshots).

The gDBClone utility enables:

• End-to-end thin and full cloning of a CDB with all its PDB
• File clone operations
• RMAN calls to make cloned files consistent
• Instance related files (such as password, spfile or TDE wallet files) creation
• GI registration

Here are some simple examples to respectively thin clone and full clone an entire CDB:

gDBClone.bin snap –sdbname source_db_name –tdbname target_db_name

gDBClone.bin clone –sdbname source_db_name –tdbname target_db_name

More on this powerful tool in Exascale documentation and Oracle Support KB145187 article.

Performance

Thin cloning a 2.4 TB PDB takes less than 12 seconds while opening the new thin clone on a two instances cluster database takes another 11 seconds, for a total elapsed time of the entire operation under 30 seconds !!!

Additional details can be found in this Alex Blyth blog post.

Conclusion

By leveraging state-of-the-art redirect-on-write technologies, Exascale is a leap forward and game changing technology when it comes to clone and snapshot features and capabilities. It is CI/CD friendly and the ideal tool to address the business requirements for test and qualification environments by letting DBAs provide very quickly space efficient copies of production or standby databases or PDBs.

Give it a try to be definitely convinced.

L’article Exascale new clone and snapshot capabilities est apparu en premier sur dbi Blog.

Setting up oci-cli

If you are already connected to your OCI tenancy with the command line, you can skip this part. Otherwise, let’s begin.

The first step is to install and configure the oci-cli utility. The full list of installation steps is available in the documentation and depends on the platform from which you want to administer OCI.

On macOS, just run this command:

brew install oci-cli

Some Linux distributions (like OEL 7 or 8) come with the utility already installed. Otherwise, you can run the following command.

bash -c "$(curl -L https://raw.githubusercontent.com/oracle/oci-cli/master/scripts/install/install.sh)"

Once this is installed, you need the connection information from your OCI account. You can finish the setup either by running the oci setup config command, which will ask for all the information needed, or by doing the following if you are not sure about the information required.

• Log in the OCI interface, and go in your Profile > User settings

• In the User settings tab, click on Tokens and keys, and Add API key

• Here, you have the option to generate a new key pair or to register an existing public key. Make sure to save the private key if you generate a new pair.

• After clicking on Add key, OCI will provide a configuration file that you can use.

• Edit this configuration file with the location of your private key and place it at ~/.oci/config

> cat ~/.oci/config
[DEFAULT]
user=<user_ocid>
fingerprint=<fingerprint>
tenancy=<tenancy_ocid>
region=<your_region>
key_file=/path/to/private_key.pem

And that’s it ! To test the connection, run the oci iam region list command. It should return a JSON document containing all the available regions.

> oci iam region list
{
  "data": [
    {
      "key": "AMS",
      "name": "eu-amsterdam-1"
    },
    ...
    {
      "key": "ZRH",
      "name": "eu-zurich-1"
    }
  ]
}

Creating your first VM with the oci CLI

An OCI virtual machine is always created within a specific set of cloud components, for which you will have to retrieve the OCIDs.

• Compartment: it can be the root compartment or any other compartment, but the user defined in the .oci/config file should have access to it
• Availability domain: depending on your OCI setup, you will have access to specific availability domains. We’ll see how to retrieve the correct one.
• Subnet: public subnet in which the VM will be available.
• Shape: shape of the VM.
• Image: the OS image used to create the VM.

Retrieve your compartment OCID

For the compartment, you can retrieve the list of all compartments with the following command:

> oci iam compartment list --all
{
  "data": [
    {
      "compartment-id": "compartment_ocid",
      "defined-tags": {
        "Oracle-Tags": {
          "CreatedBy": "...",
          "CreatedOn": "..."
        }
      },
      "id": "compartment_ocid",
      "inactive-status": null,
      "is-accessible": null,
      "lifecycle-state": "ACTIVE",
      "name": "compartment_name",
      "time-created": "..."
    },
    ...
}

If you know the name of the compartment you want to retrieve, you can get the OCID with this command. Just replace the compartment_name.

> oci iam compartment list --compartment-id-in-subtree true -all --query "data[?name=='compartment_name'].id" --raw-output
[
  "ocid.compartment..."
]

Choose an availability domain

If you don’t know it already, you can easily retrieve the availability domains available with the compartment OCID.

> oci iam availability-domain list --compartment-id ocid1.compartment.oc1... --query 'data[].name' --raw-output
[
  "AlWD:EU-ZURICH-1-AD-1"
]

Retrieve the subnet OCID

In a basic configuration, you should use the OCID of the only public subnet available. Let’s display the subnet names and OCID to help you choose the correct one.

> oci network subnet list --compartment-id ocid1.compartment.oc1... --query "data[].{name:\"display-name\", ocid:id}" --output table
+--------------------+---------------------+
| name               | ocid                |
+--------------------+---------------------+
| private subnet-vcn | ocid1.subnet.oc1... |
| public subnet-vcn  | ocid1.subnet.oc1... |
+--------------------+---------------------+

Select the desired shape

You can list the available shapes with the oci compute shape list command. Because it contains a lot of useless information for us, we will filter it. Below is a display of all available shapes and whether they are resizable. Indeed, some shapes cannot be resized and come with specific CPU/memory settings.

> oci compute shape list --compartment-id ocid1.compartment.oc1... --query  "data[].{name:shape,resizable:\"is-flexible\"}" --output table
+------------------------+-----------+
| name                   | resizable |
+------------------------+-----------+
| BM.Standard.E5.192     | False     |
| BM.Standard.A1.160     | False     |
| BM.Standard3.64        | False     |
| BM.Standard2.52        | False     |
| BM.Optimized3.36       | False     |
| BM.Standard.E4.128     | False     |
| BM.Standard.E3.128     | False     |
| BM.Standard.E2.64      | False     |
| BM.DenseIO2.52         | False     |
| VM.Optimized3.Flex     | True      |
| VM.Standard.E4.Flex    | True      |
| VM.Standard.E3.Flex    | True      |
| VM.Standard.A1.Flex    | True      |
| VM.Standard2.1         | False     |
| VM.Standard2.2         | False     |
| VM.Standard2.4         | False     |
| VM.Standard2.8         | False     |
| VM.Standard2.16        | False     |
| VM.Standard2.24        | False     |
| VM.Standard.E2.1       | False     |
| VM.Standard.E2.2       | False     |
| VM.Standard.E2.4       | False     |
| VM.Standard.E2.8       | False     |
| VM.Standard.E2.1.Micro | False     |
| VM.Standard3.Flex      | True      |
| VM.DenseIO2.8          | False     |
| VM.DenseIO2.16         | False     |
| VM.DenseIO2.24         | False     |
| VM.Standard.E5.Flex    | True      |
+------------------------+-----------+

Choose a specific image

Use the oci compute image list command to retrieve the available images inside your compartment. You might already have a name shape in mind, so just find the one you are interested in.

> oci compute image list --compartment-id ocid1.compartment.oc1... --query --query "data[].{name:\"display-name\",ocid:id}" --all --output table
+----------------------------------+--------------------+
| name                             | ocid               |
+----------------------------------+--------------------+
| Windows-Server-2025-...          | ocid1.image.oc1... |
| Oracle-Linux-9.7-2026.01.29-0    | ocid1.image.oc1... |
| Oracle-Linux-8.10-2026.01.29-0   | ocid1.image.oc1... |
| CentOS-8-Stream-2024.04.25-0     | ocid1.image.oc1... |
| custom_image                     | ocid1.image.oc1... |
+----------------------------------+--------------------+

Build your first virtual machine

You now have all the necessary information to create your own VM from the CLI ! Combine it together with the following flags, and you’re good to go:

• --display-name: Name of the virtual machine
• --assign-public-ip: Whether to assign a public IP.
• --metadata: Useful to add an ssh key to the default user.
• --shape-config: If the shape you selected can be customized, you can select its attributes here, like ocpus or memoryInGBs.

At the end, we have the following command:

oci compute instance launch \
    --compartment-id ocid1.compartment.oc1... \
    --availability-domain "EiFV:EU-ZURICH-1-AD-1" \
    --shape "VM.Standard.E5.Flex" \
    --shape-config '{"ocpus":1,"memoryInGBs":4}' \
    --subnet-id ocid1.subnet.oc1... \
    --image-id ocid1.image.oc1... \
    --display-name "vm_dbi" \
    --assign-public-ip true \
    --metadata '{"ssh_authorized_keys": "ssh-rsa ssh_key_content== user@host"}'

Of course, there are other options that you could use:

• --wait-for-state: useful if you want to wait for the VM to be RUNNING before ending the command. It is very useful in combination with the --query 'data.id' option, which will give you the OCID of the newly created VM, that you can then use to retrieve its IP, among other things.
• --user-data-file: this is especially useful if you want to add a custom cloud-init.yaml file for the VM setup.
• --boot-volume-size-in-gbs: size of the boot volume, in GB.

With this, you have everything you need for a basic VM creation in OCI.

L’article Creating an Oracle Cloud Infrastructure VM Using the CLI est apparu en premier sur dbi Blog.

While I was working for a customer, I was tasked to create an Azure infrastructure, using Terraform and Azure DevOps (ADO). I thought about doing it like I usually do with GitLab but it wasn’t possible with ADO as it doesn’t store the state file itself. Instead I have to use an Azure Storage Account. I configured it, blocked public network, and realized that my pipeline couldn’t push the state in the Storage Account

In fact, ADO isn’t supported as a “Trusted Microsoft Service” and so it can’t bypass firewall rules using that option in Storage Accounts. For this to work, I had to create a self-hosted agent that run on Azure VM Scale Set and that will be the topic of this blog.

Azure resources creation

Agent Creation

First thing, we create a Azure VM Scale Set. I kept most parameters to their default values but it can be customized. I chose Linux as operating system as it was what I needed. One important thing is to set the “Orchestration mode” to “Uniform”, else ADO pipelines won’t work.

Storage account

For the storage account that will store our state, any storage account should work. Just note that you also need to create a container inside of it to fill your terraform provider. Also, for network preferences we will go with “Public access” and “Enable from selected networks”. This is will allow public access only from restricted networks. I do this to avoid creating a private endpoint to connect to a fully private storage account.

Entra ID identity for the pipeline

We also need to create an Entra ID Enterprise Application that we will provide to the pipeline. This identity must have Contributor (or any look alike) role over the scope you target. Also, it must have at least Storage Blob Data Contributor on the Storage Account to be able to write in it.

Azure DevOps setup

Terraform code

You can use any Terraform code you want, for my example I only use one which creates a Resource Group and a Virtual Network. Just note that your provider should look like this

Pipeline code

I’m used to split my pipeline in two files, the plan.yml will be given to the ADO pipeline and it will call the template to run its code. The things done in the pipeline are pretty simple. It installs Terraform on the VM Scale Set instance, then run the Terraform commands. The block of code can be reused for the “apply”.

Few things to note, in my plan.yml I set a Variable Group “Terraform_SPN” that I will show you just after. That’s where we will find the information about our previously created Entra Id Enterprise Application

In the template.yml, what is important to note is the pool definition. Here I point just a name, which correspond to ADO Agent Pool that I created. I’ll also show this step a bit further.

For the pipeline creation itself, we will go to Pipeline -> Create a new pipeline -> Azure Repos Git

Then Existing Azure Pipelines YAML file, and pick our file from our repo.

We will also create a Variable Group, the name doesn’t matter, just remember to put the same in your YAML code. Here you create 4 variables which are information coming from your tenant and your enterprise application. That’s gonna be used during the pipeline run to deploy your resources.

ADO Agent Pool

In the Project Settings, look for Agent Pools. Then create a new one and fill it as follow:

The Authorize button will appear after you select the subscription you want, and to accept this your user must have the Owner role, as it adds rights. This will allow ADO to communicate with Azure by creating a Service Principal. Then you can fill the rest as follow:

ADO pipeline run

When you first run your pipeline you must authorize it to use the Variable Group and the Agent pool.

One this is done, everything should go smoothly and end like this.

I hope that this blog was useful and could help you troubleshoot that king of problem between Azure and Azure DevOps.

L’article Deploying Azure Terraform code with Azure DevOps and a storage account as remote backend est apparu en premier sur dbi Blog.

In this blog post, we will explore the Exascale storage architecture and processes pertaining to Exascale. But before diving in, a quick note to avoid any confusion about Exascale: even though in the previous article, we focused on Exascale Infrastructure and its various benefits in terms of small footprint and hyper-elasticity based on modern cloud characteristics, it is important to keep in mind that Exascale is an Exadata technology and not a cloud-only technology. You can benefit from it in non-cloud deployments as well such as on Exadata Database Machine deployed in your data centers.

Exascale storage components

Here is the overall picture:

Exascale cluster

An Exascale Cluster is composed of Exadata storage servers to provide storage to Grid Infrastructure clusters and the databases. An Exadata storage server can belong to only one Exascale cluster.

Software services (included in the Exadata System Software stack) run on each Exadata cluster for managing the cluster resources made available to GI and databases, namely pool disks, storage pools, vaults, block volumes and many other.

With Exadata Database Service on Exascale Infrastructure, the number of Exadata storage servers included in the Exascale cluster can be quite huge, hundreds maybe even thousands storage servers, to enable cloud-scale storage resource pooling.

Storage pools

A storage pool is a collection of pool disks (see below for details about pool disks).

Each Exascale cluster requires at least one storage pool.

A storage pool can be dynamically reconfigured by changing pool disks size, allocating more pool disks or adding Exadata storage servers.

The pool disks found inside a storage pool must be of the same media type.

Pool disks

A pool disk is physical storage space allocated from an Exascale-specific cell disk to be integrated in a storage pool.

Each storage server physical disk has a LUN in the storage server OS and a cell disk is created as a container for all Exadata-related partitions within the LUN. Each partition in a cell disk is then designated as a pool disk for Exascale (or grid disk in case of ASM is used instead of Exascale).

A media type is associated to each pool disk based on the underlying storage device and can be one of the following :

• HC : points to high capacity storage using hard disk drives
• EF : based on extreme flash storage devices
• XT : corresponds to extended storage using hard disk drives found in Exadata Storage Server X11M Extended (XT) hardware

Vaults

Vaults are logical storage containers used for storing files and are allocated from storage pools. By default, without specific provisioning attributes, vaults can use all resources from all storage pools of an Exascale cluster.

Here are the two main services provided by vaults:

1. Security isolation: a security perimeter is associated with vaults based on user access controls which guarantee a strict isolation of data between users and clusters.
2. Resource control: storage pool resources usage is configured at the vault level for attributes like storage space, IOPS, XRMEM and flash cache sizing.

For those familiar with ASM, think of vaults as the equivalent to ASM disk groups. For example, instance parameters like ‘db_create_file_dest’ and ‘db_recovery_file_dest’ reference vaults instead of ASM disk groups by using ‘@vault_name’ syntax.

Since attributes like redundancy, type of file content, type of media are positioned at the file level instead at the logical storage container, there is no need to organize vaults in the same manner as we did for disk groups. For instance, we don’t need to create a first vault for data and a second vault for recovery files as we are used to with ASM.

Beside database files, vaults can also store other types of files even though it is recommended to store non database files on block volumes. That’s because Exascale is optimized for storing large files such as database files whereas regular files are typically much smaller and fit more on block volumes.

Files

The main files found on Exascale storage are Database and Grid Infrastructure files. Beyond that, all objects in Exascale are represented as files of a certain type. Each file type has storage attributes defined in a template. The file storage attributes are :

• mediaType : HC, EF or XT
• redundancy : currently high
• contentType : DATA and RECO

This makes a huge difference with ASM where different redundancy needs required to create different disk groups. With Exascale, it is now possible to store files with different redundancy requirements in the same storage structure (vaults). This also enables to optimize usage of the storage capacity.

Files are composed of extents of 8MB in size which are mirrored and stripped across all vault’s storage resources.

The tight integration of the database kernel with Exascale makes it possible for Exascale to automatically understand the type of file the database asks to create and thus applies the appropriate attributes defined in the file template. This prevents Exascale to store data and recovery files extents (more on extents in the next section) on the same disks and also guarantees that mirrored extents are located on different storage servers than the primary extent.

File extents

Remember that in Exascale, storage management moved from the compute servers to the storage servers. Specifically, this means the building blocks of files, namely extents, are managed by the storage servers.

The new data structure used for extent management is a mapping table which tracks for each file extent the location of the primary and mirror copy extents in the storage servers. This mapping table is cached by each database server and instance to retrieve its file extents location. Once the database has the extent location, it can directly make an I/O call to the appropriate storage server. In case the mapping table is no more up-to-date because of database physical structure changes or storage servers addition or removal, an I/O call can be rejected, triggering a mapping table refresh allowing the I/O call to be retried.

Exascale Block Store with RDMA-enabled block volumes

Block volumes can be allocated from storage pools to store regular files on file systems like ACFS or XFS. They also enable centralization of VC VM images, thus cutting the dependency of VM images to internal compute node storage and streamlining migrations between physical database servers. Clone, snapshot and backup and restore features for block volumes can leverage all resources of the available storage servers.

Exascale storage services

From a software perspective, Exascale is composed of a number of software services available in the Exadata System Software (since release 24.1). These software services run mainly on the Exadata Storage Servers but also on the Exadata Database Servers.

Exascale storage server services

Exascale database server services

Below diagram depicts how the various Exascale services are dispatched on the storage and compute nodes:

Wrap-up

By rearchitecting Exadata storage management with focus on space efficiency, flexibility and elasticity, Exascale can now overcome the main limitations of ASM:

• diskgroup resizing complexity and time-consuming rebalance operation
• space distribution among DATA and RECO diskgroups requiring rigorous estimation of storage needs for each
• sparse diskgroup requirement for cloning with a read-only test master or use of ACFS without smart scan
• redundancy configuration at the diskgroup level

The below links will provide you further details on the matter:

Oracle Exadata Exascale blog

Oracle Exadata Exascale advantages blog

Oracle Exadata Exascale storage fundamentals blog

Oracle Exascale documentation

Oracle Exadata architecture

Oracle And Me blog – Why ASM Needed an Heir Worthy of the 21st Century

Oracle And Me blog – New Exascale architecture

More on the exciting Exascale technology in coming posts …

L’article Exascale storage architecture est apparu en premier sur dbi Blog.

Introduction

Exadata on Exascale Infrastructure is a new deployment option for Exadata Database Service. It comes in addition to the well known Exadata Cloud@Customer or Exadata on Dedicated Infrastructure options already available. It is based on new storage management technology decoupling database and Grid Infrastructure clusters from the Exadata underlying storage servers by integrating the database kernel directly with the Exascale storage structures.

What is Exascale Infrastructure ?

Simply put, Exascale is the next-generation of Oracle Exadata Database Services. It combines a cloud storage approach for flexibilty and hyper-elasticity with the performance of Exadata Infrastructure. It introduces a loosely-coupled shared and multitenant architecture where Database and Grid Infrastructure clusters are decoupled from the underlying Exadata storage servers which become a pool of shared storage resources available for multiple Grid Infrastructure clusters and databases.

Strict data isolation provides secure storage sharing while storage pooling enables flexible and dynamic provisioning combined with better storage space and processing utilization.

Advanced snapshot and cloning features, leveraging Redirect-On-Write technology instead of Copy-On-Write, enable space-efficient thin clones from any read/write database or pluggable database. Read-only test master are therefore a thing of the past. These features alone make Exascale a game-changer for database refreshes, CI/CD workflows and pipelines, development environments provisioning, all done with single SQL commands and much faster than before.

Block storage services allow the creation of arbitrary-sized block volumes for use by numerous applications. Exascale block volumes are also used to store Exadata database server virtual machines images enabling :

• creation of more virtual machines
• removal of local storage dependency inside the Exadata compute nodes
• seamless migration of virtual machines between different compute nodes

Finally, the following hardware and software considerations complete this brief presentation of Exascale:

• runs on 2-socket Oracle Exadata system hardware with RoCE Network Fabric (X8M-2 or later)
• Database 23ai release 23.5.0 or later is required for full-featured native Database file storage in Exascale
• Exascale block volumes support databases using older Database software releases back to Oracle Database 19c
• Exascale is built into Exadata System Software (since release 24.1)

Architecture

The main point with Exascale architecture is cloud-scale, multi-tenant resource pooling, both for storage and compute.

Storage pooling

Exascale is based on pooling storage servers which provide services such as Storage Pools, Vaults and Volume Management. Vaults, which can be considered an equivalent of ASM diskgroups, are directly accessed by the database kernel.

File and extent management is done by Exadata System Software, thus freeing the compute layer from ASM processes and memory structures for database files access and extents management (with Database 23ai or 26ai; for Database 19c, ASM is still required). With storage management moving to storage servers, resource management becomes more flexible and more memory and CPU resources are available on compute nodes to process database tasks.

Exascale also provides redundancy, caching, file metadata management, snapshots and clones as well as security and data integrity features.

Of course, since Exascale is built on top of Exadata, you benefit from features like RDMA, RoCE, Smart Flash Cache, XRMEM, Smart Scan, Storage Indexes, Columnar Caching.

Compute pooling

On the compute side, we have database-optimized servers which run Database 23ai or newer and Grid Infrastructure Cluster management software. The physical database servers host the VM Clusters and are managed by Oracle. Unlike Exadata on Dedicated Infrastructure, there is no need to provision an infrastructure before going ahead with VM Cluster creation and configuration : in Exascale, you only deal with VM Clusters.

VM file systems are centrally-hosted by Oracle on RDMA-enabled block volumes in a system-vault. VM images used by the VM Clusters are no more hosted on local storage on the database servers. This enables the number of VMs running on the database servers to raise from 12 to 50.

Each VM Cluster accesses a Database Vault storing the database files with strict isolation from other VM Clusters database files.

Virtual Cloud Network

Client and backup connectivity is provided by Virtual Cloud Network (VCN) services.

This loosely-coupled, shared and multitenant architecture enables far greater flexibilty than what was possible with ASM or even Exadata on Dedicated Infrastructure. Exascale’s hyper-elasticity enables to start with very small VM Clusters and then scale as the workloads increase, with Oracle managing the infrastructure automatically. You can start as small as 1 VM per cluster (up to 10), 8 eCPUs per VM (up to 200), 22GB of memory per VM, 220GB file system storage per VM and 300GB Vault storage per VM Cluster (up to 100TB). Memory is tightly coupled to eCPUs configuration with 2.75GB per eCPU and thus does not scale independently from the eCPUs number.

For those new to eCPU, it is a standard billing metric based on the number of cores per hour elastically allocated from a pool of compute and storage servers. eCPUs are not tied to the make, model or clock speed of the underlying processor. By contrast, an OCPU is the equivalent of one physical core with hyper-threading enabled.

To summarize

To best understand what Exascale Infrastructure is and introduces, here is the wording which best describes this new flavor of Exadata Database Service :

• loosely-coupling of compute and storage cloud
• hyper-elasticity
• shared and multi-tenant service model
• ASM-less (for 23ai or newer)
• Exadata performance, reliability, availability and security at any scale
• CI/CD friendly
• pay-per-use model

Stay tuned for more on Exascale …

L’article Exascale Infrastructure : new flavor of Exadata Database Service est apparu en premier sur dbi Blog.

So, I thought I’d write a little catch-up blog post, to help you to find all LinkedIn Posts!

First, a bit of humour with this image of SQL Server scrolling through LinkedIn and wondering if anyone still loves it… Meanwhile, Power BI and Azure Data Fabric are stealing the show.

Let’s talk about SQL Server, and in particular all tests done on SQL Server 2025:  

SSRS Deprecation – It’s time to rethink your reporting architecture explain you how with our Team, we can help you to go forward!

dbi services’ blog on fire for SQL Server 2025 with this Tips to find all our SQL Server blogs : just Google: site:dbi-services.com/blog SQL Server 2025

Exploring SQL Server 2025 blogs with tests on SQL Server 2025 new features

Time to review your SQLServer licenses with 10% price increase for SQL Server licences, we can help you to optimize your SQL Server Architecture!

SQL Server 2025: AI, Performance… and real-world Lessons with a lot of deep tests and new thinking about  Tuning, Performance & AI

Your SQL Server might be exposed right now is a very important topic because Microsoft put in Augaust and Septembre critical patches in High importance

SQL Server 2025 – What’s new & What’s next? withgood feedback and ideas from SQLBits 2025  in London

It’s time to rethink your data strategy with SQL Server 2025 with the end of DQS, MDS and Synapse Link, it’s  the opportunity to see our expertise in Microsoft Fabic

And also, publications about how we can help beside SQL Server and share these experiences & expertise:

Struggling with rising Azure Data Fabric & Power BI costs? with our experience to help to optimize the cost of your Azure Platform using the BI stack

When M-Files meets SQL Server with our SQL Server & M-Files expertise, we are on the top to help you to configure, customize, optimize, secure  your M-File infrastructure

Continue to follow us in our blogs and linkedIn posts!

L’article SQL Server’s LinkedIn Posts catch-up! est apparu en premier sur dbi Blog.

In response to this, Oracle (and others) use a not-so-native adaptation of SQL to query JSON documents. And if you’ve ever had to deal with these SQL-based JSON queries, you may have found yourself thinking (in hindsight) that querying JSON documents with MQL (MongoDB Query Language) wasn’t that painful after all. And rightly so : JSON documents are very different from relational tables, and there is no easy way to continue using SQL without learning some new syntax.

Oracle likely came to the same conclusion, and offers an API dedicated to MongoDB natives. The idea is simple : to ease the work of developers and database administrators when deciding to migrate from MongoDB to Oracle (and the other way around ?…)

On Oracle Cloud Infrastructure (OCI), this means being able to connect to Autonomous Databases (ADB). As a reminder, you have two of them included in the Always Free Tier of OCI, so you can play around with this feature for free before deciding to migrate to the cloud.

Setting up the Oracle MongoDB API on OCI

When creating an Autonomous Database, you can decide on multiple workloads types, including a JSON workload. However, the MongoDB API does not strictly require the JSON workload type to work.

However, the network access setting of your Autonomous Database must be non-default with one of the following options enabled :

• Secure access from allowed IPs and VCNs only
• Private endpoint access only

For instance, when using the Secure access from allowed IPs and VCNs only option, you can add any IP address to the Access Control List (ACL), including your own.

After creating the ADB (Autonomous Database), you can check in the Tool configuration tab the MongoDB API is enabled (it should be, by default).

Then you can go in Database actions > Database Users to either create a user or modify an existing one, with the following privileges : CREATE SESSION, CREATE RESOURCE (default), and the SODA_APP role. After creation, you should enable REST as shown below. This will allow the API to work on that particular schema.

Back on the Tool configuration tab of your ADB instance, you’ll find the Public access URL. Copy it, this will be your connection string when using the API.

Connection String Example

For this to work, you have to replace the user and password in the connection string. For instance, if your user is called USER_DB, and your password is userpwd, then you would use this connection string (without the brackets). Make sure to escape any special character in the password when doing so (see documentation).

mongodb://user_db:userpwd@A49E7CHE9B320E6-PCCVX8PAE2NZPVBQ.adb.eu-zurich-1.oraclecloudapps.com:27017/user_db?authMechanism=PLAIN&authSource=$external&ssl=true&retryWrites=false&loadBalanced=true'

Use this connection string with mongosh or any tool provided by MongoDB. With mongosh, you would connect with this command :

mongosh 'mongodb://user_db:userpwd@A49E7CHE9B320E6-PCCVX8PAE2NZPVBQ.adb.eu-zurich-1.oraclecloudapps.com:27017/adb_admin?authMechanism=PLAIN&authSource=$external&ssl=true&retryWrites=false&loadBalanced=true'

You are now connected to the Oracle ADB instance through the mongosh utility.

Viewing Oracle tables with the API

If you connect to an existing schema with tables, you will notice that running show collections (or show tables) doesn’t return anything. In Oracle 19c, only collections/tables created through the API appear in this list. Standard tables create with a CREATE TABLE SQL statement won’t be displayed.

Included and excluded features in the API

Once connected, you can query the different databases (schemas, in Oracle terms) of your instance. However, unlike in MongoDB, you can’t administer the instance through mongosh.

You can look at the documentation to check the features included in the API, but here is an extract of the most notable changes :

• All the most basic aggregated pipelines are available in the API, but more complex ones like $currentOp or $merge are not yet supported.
• Other aggregation operators, like $bucket, $documents or $sample are only available starting with Oracle 23ai.
• db.dropDatabase() doesn’t drop the Oracle schema. In fact, it doesn’t even deletes everything in the schema. It only deletes all real collections inside of it. So if you have a traditional table that is not a considered as a collection (not shown in show collections), it will not be deleted by this command.

Troubleshooting MongoServerError: Database connection unavailable

You might encounter the following error :

MongoServerError: Database connection unavailable. Ensure that the user exists and the schema is enabled for use with Oracle REST Data Services. A schema can be enabled by calling the PL/SQL procedure ORDS.ENABLE_SCHEMA

If you see this message, you can either follow the procedure or remember to enable REST directly on the OCI Database Actions panel, as shown above.

In short, Oracle’s MongoDB API provides an interesting bridge between the relational and document worlds. Even though some MongoDB features are supported yet, the API offers a straightforward way to connect MongoDB and Oracle Autonomous Database. It makes it ideal for testing, for migration or even for hybrid setups. For developers used to MQL, it can significantly ease the transition to Oracle’s ecosystem.

L’article Using Oracle API for MongoDB on OCI est apparu en premier sur dbi Blog.

Quand j’ai commencé avec les ETL, j’ai appris “à l’ancienne” : toutes les règles de gestion codées directement dans les packages. Une approche rigide, lourde à maintenir, et qui surtout créait une dépendance forte de mes clients envers moi.

Assez vite, j’ai développé une autre méthode, que j’ai fini par appliquer dès que je le pouvais :

• utiliser l’ETL comme orchestrateur,
• confier le cœur du traitement au SQL,
• et donner à mes clients plus d’autonomie, tout en simplifiant la maintenance.

Dans ce blog, je vais partager cette approche et comparer deux contextes :

• La structure de base que j’utilise pour SSIS depuis de longues années
• Ce même modèle ré-adapté à MS Fabric et aux nouvelles options que Pipeline offre

Le Master Package SSIS : ma première brique de factorisation

Quand j’ai commencé à travaillé avec SSIS, j’ai vite cherché à éviter de répéter les mêmes flux. Dupliquer dix fois un package juste parce qu’on change le nom d’une table, pour moi, ça n’a jamais eu de sens.

La solution que j’ai mise en place, c’était le Master Package, ce n’est pas une idée nouvelle, mais je l’ai adapté à ma façon de travailler. L’idée était simple :

• construire des packages enfants très unitaires (un fichier, une table, une logique claire),
• centraliser l’exécution dans un Master qui lit une table de pilotage,
• et exécuter les sous-packages selon un scénario défini en base.

Avantages obtenus :

• factorisation maximale,
• possibilité pour mes clients d’activer/désactiver un flux par simple mise à jour en table,
• orchestration gérée sans redéploiement du projet,
• centralisation des logs de process en un seul point.

Un mécanisme de logging centralisé dans le Master Package me permettait de :

• tracer l’exécution de chaque sous-package,
• uniformiser les retours d’erreurs,
• simplifier la maintenance.

Plus besoin d’ouvrir 15 packages pour comprendre où ça avait cassé : tout remontait dans la même table de suivi.

Exemple de modèle utilisé :

CREATE TABLE dbo.Packages
(
    PackageID    INT            IDENTITY(1,1) PRIMARY KEY,
    PackageName  VARCHAR( 100 ) NOT NULL, -- Nom du .dtsx
    Description  VARCHAR( 400 ) NULL,
    IsEnabled    BIT            NOT NULL DEFAULT (1),
    CreatedAt    DATETIME2(0)   NOT NULL DEFAULT (SYSUTCDATETIME()),
    CONSTRAINT UQ_Packages__PackageName UNIQUE (PackageName)
);
GO

/* Paramètres par défaut au niveau du package (surchargés par scénario si besoin) */
CREATE TABLE dbo.PackageParameters
(
    PackageID    INT           NOT NULL FOREIGN KEY REFERENCES dbo.Packages(PackageID),
    ParamName    VARCHAR(128)  NOT NULL,
    DefaultValue VARCHAR(4000) NOT NULL,
    CONSTRAINT PK_PackageParameters PRIMARY KEY (PackageID, ParamName)
);
GO


/* =========================================================================
   SCÉNARIOS (contextes d’exécution) & ORCHESTRATION
   ========================================================================= */
CREATE TABLE dbo.Scenarios
(
    ScenarioID   INT          IDENTITY(1,1) PRIMARY KEY,
    ScenarioName VARCHAR(100) NOT NULL, -- DAILY / BACKFILL / CLIENT_X ...
    Description  VARCHAR(400) NULL,
    IsActive     BIT          NOT NULL DEFAULT (1),
    CreatedAt    DATETIME2(0) NOT NULL DEFAULT (SYSUTCDATETIME()),
    CONSTRAINT UQ_Scenarios__ScenarioName UNIQUE (ScenarioName)
);
GO

CREATE TABLE dbo.ScenarioSteps
(
    ScenarioStepID  INT IDENTITY(1,1) PRIMARY KEY,
    ScenarioID      INT NOT NULL FOREIGN KEY REFERENCES dbo.Scenarios(ScenarioID),
    PackageID       INT NOT NULL FOREIGN KEY REFERENCES dbo.Packages(PackageID),
    ExecutionOrder  INT NOT NULL,
    IsActive        BIT NOT NULL DEFAULT (1),
    RetryCount      INT NOT NULL DEFAULT (0),
    ContinueOnError BIT NOT NULL DEFAULT (0),
    TimeoutSec      INT NULL,                -- optionnel : timeouts par step
    CONSTRAINT UQ_Steps__Scenario_Order UNIQUE (ScenarioID, ExecutionOrder)
);
GO

/* Surcharges de paramètres au niveau scénario→package */
CREATE TABLE dbo.ScenarioParameters
(
    ScenarioID INT            NOT NULL FOREIGN KEY REFERENCES dbo.Scenarios(ScenarioID),
    PackageID  INT            NOT NULL FOREIGN KEY REFERENCES dbo.Packages(PackageID),
    ParamName  VARCHAR(128)   NOT NULL,
    ParamValue VARCHAR(4000)  NOT NULL,
    CONSTRAINT PK_ScenarioParameters PRIMARY KEY (ScenarioID, PackageID, ParamName)
);
GO

/* Vue des paramètres “effectifs” (Scenario override > défaut package) */
CREATE VIEW dbo.vw_EffectiveParameters
AS
SELECT s.ScenarioID,
       p.PackageID,
       pp.ParamName,
       COALESCE(sp.ParamValue, pp.DefaultValue) AS EffectiveValue
  FROM dbo.Scenarios          AS s
       INNER JOIN dbo.ScenarioSteps      AS ss ON ss.ScenarioID = s.ScenarioID AND ss.IsActive = 1
       INNER JOIN dbo.Packages           AS p  ON p.PackageID   = ss.PackageID AND p.IsEnabled = 1
        LEFT JOIN dbo.PackageParameters  AS pp ON pp.PackageID  = p.PackageID
        LEFT JOIN dbo.ScenarioParameters AS sp ON sp.ScenarioID = s.ScenarioID
                                              AND sp.PackageID  = p.PackageID
                                              AND sp.ParamName  = pp.ParamName;
GO


/* =========================================================================
   LOGS CENTRALISÉS (header run + détails par step)
   ========================================================================= */
CREATE TABLE dbo.ProcessRun
(
    RunID         BIGINT       IDENTITY(1,1) PRIMARY KEY,
    ScenarioName  VARCHAR(100) NOT NULL,
    Status        VARCHAR(20)  NOT NULL, -- Started / Succeeded / Failed / PartiallyFailed
    StartTime     DATETIME2(0) NOT NULL DEFAULT (SYSUTCDATETIME()),
    EndTime       DATETIME2(0) NULL,
    DurationSec   INT          NULL,
    TriggeredBy   VARCHAR(128) NULL, -- Agent, utilisateur, API, etc.
    CorrelationID UNIQUEIDENTIFIER NOT NULL DEFAULT (NEWID())
);
GO

CREATE TABLE dbo.ProcessRunStep
(
    RunStepID      BIGINT        IDENTITY(1,1) PRIMARY KEY,
    RunID          BIGINT        NOT NULL FOREIGN KEY REFERENCES dbo.ProcessRun(RunID),
    ScenarioStepID INT           NULL,          -- facultatif : rattacher au modèle
    ScenarioName   VARCHAR(100)  NOT NULL,
    PackageName    SYSNAME       NOT NULL,
    Status         VARCHAR(20)   NOT NULL,      -- Started / Succeeded / Failed / Skipped / Retried
    StartTime      DATETIME2(0)  NOT NULL DEFAULT (SYSUTCDATETIME()),
    EndTime        DATETIME2(0)  NULL,
    DurationSec    INT           NULL,
    Retries        INT           NOT NULL DEFAULT (0),
    ErrorMessage   VARCHAR(2000) NULL
);
GO

/* Index utiles */
CREATE INDEX IX_ProcessRun__StartTime   ON dbo.ProcessRun     (StartTime DESC);
CREATE INDEX IX_ProcessRunStep__RunID   ON dbo.ProcessRunStep (RunID);
CREATE INDEX IX_Steps__Scenario_Order   ON dbo.ScenarioSteps  (ScenarioID, ExecutionOrder);
GO

Requêtes “maître” utilisées par le Master Package

/* 1) Plan d’exécution pour un scénario donné */
DECLARE @ScenarioName VARCHAR(100) = @ScenarioNameInput;

SELECT s.ScenarioName,
       p.PackageName,
       ss.ExecutionOrder,
       ss.RetryCount,
       ss.ContinueOnError,
       ss.TimeoutSec
  FROM dbo.ScenarioSteps AS ss
       INNER JOIN dbo.Scenarios AS s ON s.ScenarioID = ss.ScenarioID
	                                AND s.IsActive  = 1
       INNER JOIN dbo.Packages  AS p ON p.PackageID = ss.PackageID
	                                AND p.IsEnabled = 1
 WHERE s.ScenarioName = @ScenarioName
   AND ss.IsActive    = 1
 ORDER BY
       ss.ExecutionOrder ASC;

/* 2) Paramètres effectifs pour (Scénario, Package) */
DECLARE @ScenarioName VARCHAR(100) = @ScenarioNameInput;
DECLARE @PackageName  SYSNAME       = @PackageNameInput;

SELECT ep.ParamName,
       ep.EffectiveValue
  FROM dbo.vw_EffectiveParameters AS ep
       INNER JOIN dbo.Scenarios AS s  ON s.ScenarioID = ep.ScenarioID
       INNER JOIN dbo.Packages  AS p  ON p.PackageID  = ep.PackageID
 WHERE s.ScenarioName = @ScenarioName
   AND p.PackageName  = @PackageName;

Explication du modèle SSIS

Ce modèle repose sur un principe simple : séparer la logique technique de l’orchestration.

• Packages : liste des packages disponibles (chargement d’une table, traitement d’un fichier…).
  • 1 package = 1 traitement.
  • Exemple : j’importe une table, puis j’exécute la procédure associée.
• PackageParameters : paramètres par défaut au niveau package (source, cible, dates…).
  • Exemple : dans un traitement différentiel, je stock la dernière date de traitement et la réinjecte automatiquement lors de l’exécution.
• Scenarios : contextes d’exécution (DAILY, BACKFILL, CLIENT_X…).
  • Exemple : certains clients gèrent plusieurs bases identiques. Le scénario permet de passer dynamiquement d’un client à l’autre (connexion, serveur, etc.).
• ScenarioSteps : plan d’exécution qui relie un scénario à une suite de packages (ordre, retries, tolérance aux erreurs).
  • Gestion dynamique des étapes, réduction de l’effort de maintenance.
  • Au final : un simple job SQL Agent lié à cette table, simplifiant l’exploitation.
• ScenarioParameters : surcharges de paramètres pour un scénario donné.
  • Exemple : un client peut demander un filtrage particulier (plage de dates, règles locales).
• ProcessRun : entête de chaque exécution (statut global, durée, déclencheur).
  • Exemple : branchement Power BI pour suivi en temps réel des traitements.
• ProcessRunStep : détail de chaque étape reliée au RunID (statut, durée, erreurs).
  • Exemple : visualisation simple des erreurs dans Power BI, envoi d’emails en cas de problème, rapport quotidien automatisé.

Intérêt du modèle SSIS :

1. Pilotage dynamique : tout changement se fait en table, pas dans les packages.
2. Logs centralisés : un seul point de vérité pour le suivi complet.
3. Reporting intégré : mise en place rapide de dashboards ou rapports automatisés.

Le metadata-driven dans Fabric : la continuité naturelle

Avec Microsoft Fabric, je retrouve la même philosophie que celle de SSIS : séparer la logique technique de l’orchestration.

La différence : Fabric va beaucoup plus loin en termes de :

• souplesse (paramétrage),
• scalabilité (Spark, clusters dynamiques),
• observabilité (logs centralisés exploitables).

Modèle de tables Fabric — complet & factorisé :

/* =========================================================================
   UNITÉS DE TRAVAIL (WorkUnits) = l’équivalent des packages SSIS
   ========================================================================= */
CREATE TABLE dbo.WorkUnits
(
    WorkUnitID   INT          IDENTITY(1,1) PRIMARY KEY,
    WorkUnitName VARCHAR(128) NOT NULL,   -- ex : Load_Customers, Load_Sales
    WorkType     VARCHAR(50)  NOT NULL,   -- COPY / SPARK_JOB / STORED_PROC
    Target       VARCHAR(128) NULL,       -- Lakehouse, DWH, fichier…
    IsEnabled    BIT          NOT NULL DEFAULT (1),
    CreatedAt    DATETIME2(0) NOT NULL DEFAULT (SYSUTCDATETIME()),
    CONSTRAINT UQ_WorkUnits__Name UNIQUE (WorkUnitName)
);
GO

/* Paramètres par défaut au niveau WorkUnit */
CREATE TABLE dbo.WorkUnitParameters
(
    WorkUnitID   INT           NOT NULL FOREIGN KEY REFERENCES dbo.WorkUnits(WorkUnitID),
    ParamName    VARCHAR(128)  NOT NULL,
    DefaultValue VARCHAR(4000) NOT NULL,
    CONSTRAINT PK_WorkUnitParameters PRIMARY KEY (WorkUnitID, ParamName)
);
GO


/* =========================================================================
   SCÉNARIOS (contextes d’exécution) & ORCHESTRATION
   ========================================================================= */
CREATE TABLE dbo.FabricScenarios
(
    ScenarioID       INT          IDENTITY(1,1) PRIMARY KEY,
    ScenarioName     VARCHAR(100) NOT NULL,
    Description      VARCHAR(400) NULL,
    IsActive         BIT          NOT NULL DEFAULT (1),
    ConcurrencyLevel INT          NULL,                  -- tuning possible par scénario
    CreatedAt        DATETIME2(0) NOT NULL DEFAULT (SYSUTCDATETIME()),
    CONSTRAINT UQ_FabricScenarios__Name UNIQUE (ScenarioName)
);
GO

CREATE TABLE dbo.FabricScenarioSteps
(
    ScenarioStepID  INT          IDENTITY(1,1) PRIMARY KEY,
    ScenarioID      INT          NOT NULL FOREIGN KEY REFERENCES dbo.FabricScenarios(ScenarioID),
    WorkUnitID      INT          NOT NULL FOREIGN KEY REFERENCES dbo.WorkUnits(WorkUnitID),
    ExecutionOrder  INT          NOT NULL,
    IsActive        BIT          NOT NULL DEFAULT (1),
    RetryCount      INT          NOT NULL DEFAULT (0),
    ContinueOnError BIT          NOT NULL DEFAULT (0),
    ClusterSize     VARCHAR(20)  NULL,                  -- Small / Medium / Large
    SkipHeaderRows  INT          NULL,                  -- pratique pour les CSV
    Comments        VARCHAR(400) NULL,
    CONSTRAINT UQ_FabricSteps__Scenario_Order UNIQUE (ScenarioID, ExecutionOrder)
);
GO

/* Surcharges de paramètres par scénario → WorkUnit */
CREATE TABLE dbo.FabricScenarioParameters
(
    ScenarioID INT           NOT NULL FOREIGN KEY REFERENCES dbo.FabricScenarios(ScenarioID),
    WorkUnitID INT           NOT NULL FOREIGN KEY REFERENCES dbo.WorkUnits(WorkUnitID),
    ParamName  VARCHAR(128)  NOT NULL,
    ParamValue VARCHAR(4000) NOT NULL,
    CONSTRAINT PK_FabricScenarioParameters PRIMARY KEY (ScenarioID, WorkUnitID, ParamName)
);
GO

/* Vue des paramètres effectifs */
CREATE VIEW dbo.vw_FabricEffectiveParameters
AS
SELECT s.ScenarioID,
       w.WorkUnitID,
       wp.ParamName,
       COALESCE(sp.ParamValue, wp.DefaultValue) AS EffectiveValue
  FROM dbo.FabricScenarios          AS s
       INNER JOIN dbo.FabricScenarioSteps      AS ss ON ss.ScenarioID = s.ScenarioID
	                                                AND ss.IsActive = 1
       INNER JOIN dbo.WorkUnits                AS  w ON w.WorkUnitID = ss.WorkUnitID
	                                                AND w.IsEnabled = 1
        LEFT JOIN dbo.WorkUnitParameters       AS wp ON wp.WorkUnitID = w.WorkUnitID
        LEFT JOIN dbo.FabricScenarioParameters AS sp ON sp.ScenarioID = s.ScenarioID
                                                    AND sp.WorkUnitID = w.WorkUnitID
                                                    AND sp.ParamName  = wp.ParamName;
GO


/* =========================================================================
   LOGS CENTRALISÉS (entête + détails steps)
   ========================================================================= */
CREATE TABLE dbo.FabricRun
(
    RunID        BIGINT       IDENTITY(1,1) PRIMARY KEY,
    ScenarioName VARCHAR(100) NOT NULL,
    Status       VARCHAR(20)  NOT NULL, -- Started / Succeeded / Failed / Partial
    StartTime    DATETIME2(0) NOT NULL DEFAULT (SYSUTCDATETIME()),
    EndTime      DATETIME2(0) NULL,
    DurationSec  INT          NULL,
    TriggeredBy  VARCHAR(128) NULL, -- utilisateur, service, API
    ProcessID    UNIQUEIDENTIFIER NOT NULL DEFAULT (NEWID())
);
GO

CREATE TABLE dbo.FabricRunStep
(
    RunStepID      BIGINT        IDENTITY(1,1) PRIMARY KEY,
    RunID          BIGINT        NOT NULL FOREIGN KEY REFERENCES dbo.FabricRun(RunID),
    ScenarioStepID INT           NULL, -- rattachement au modèle
    ScenarioName   VARCHAR(100)  NOT NULL,
    WorkUnitName   VARCHAR(128)  NOT NULL,
    Status         VARCHAR(20)   NOT NULL, -- Started / Succeeded / Failed / Skipped
    StartTime      DATETIME2(0)  NOT NULL DEFAULT (SYSUTCDATETIME()),
    EndTime        DATETIME2(0)  NULL,
    DurationSec    INT           NULL,
    Retries        INT           NOT NULL DEFAULT (0),
    ErrorMessage   VARCHAR(2000) NULL
);
GO

CREATE INDEX IX_FabricRun__StartTime    ON dbo.FabricRun     (StartTime DESC);
CREATE INDEX IX_FabricRunStep__RunID    ON dbo.FabricRunStep (RunID);
CREATE INDEX IX_FabricSteps__Scenario_Order ON dbo.FabricScenarioSteps (ScenarioID, ExecutionOrder);
GO

Sélection des steps pour un scénario

SELECT s.ScenarioName,
       w.WorkUnitName,
       w.WorkType,
       w.Target,
       fs.ExecutionOrder,
       fs.SkipHeaderRows,
       fs.ClusterSize
  FROM dbo.FabricScenarioSteps AS fs
       INNER JOIN dbo.FabricScenarios AS s ON s.ScenarioID = fs.ScenarioID
	                                      AND s.IsActive = 1
       INNER JOIN dbo.WorkUnits       AS w ON w.WorkUnitID = fs.WorkUnitID
	                                      AND w.IsEnabled = 1
WHERE s.ScenarioName = @ScenarioName
  AND fs.IsActive     = 1
ORDER BY
      fs.ExecutionOrder ASC;

Explication du modèle Fabric

• WorkUnits : équivalent des packages SSIS. Une unité = 1 tâche (COPY, Spark, procédure).
• WorkUnitParameters : paramètres par défaut (source, cible, filtres).
• FabricScenarios : contextes d’exécution (DAILY, BACKFILL, CLIENT_X, TEST…), avec parallélisme possible.
• FabricScenarioSteps : orchestration des étapes (ordre, retries, tolérance aux erreurs, taille de cluster, options).
• FabricScenarioParameters : surcharges de paramètres (filtres spécifiques à un client).
• FabricRun : entête de chaque exécution (statut global, durée, déclencheur, ProcessID).
• FabricRunStep : détail de chaque étape (statut, durée, erreurs, reliée au RunID).

Intérêt (similaire à SSIS, mais en mieux) :

• pilotage dynamique (tout en table, zéro code dur),
• logs centralisés avec ProcessID unique,
• scalabilité (taille de cluster, parallélisme modulables),
• observabilité (liaison native avec Power BI).

Comparaison SSIS vs Fabric

Conclusion

Que ce soit avec SSIS ou avec Fabric, mon approche reste la même : séparer la logique technique de l’orchestration et piloter les traitements par des tables.

• Avec SSIS, le Master Package m’a permis de factoriser mes flux, centraliser mes logs et donner de l’autonomie à mes clients.
• Avec Fabric, je pousse cette logique plus loin : WorkUnits remplacent les packages, les scénarios pilotent toujours, mais les logs deviennent encore plus exploitables, notamment via Power BI.

Ce que je défends, ce n’est pas un outil en particulier, mais une philosophie :

• écrire une fois,
• piloter par métadonnées,
• donner aux équipes visibilité et autonomie.

C’est cette approche qui, selon moi, fait la différence entre un projet qui devient vite un cauchemar de maintenance, et un projet qui s’adapte naturellement aux évolutions.

L’article Metadata-driven pipelines : la continuité du Master Package SSIS est apparu en premier sur dbi Blog.

Amid the infinite sessions about AI, I could also find some on other topics. I started my journey with a session on security best practices.

Then came the keynote, where I could see a demo of an automated delivery robot shaped like a dog that can climb stairs. During the keynote, it was announced that Oracle Exadata would be available in the Zurich region starting 2026.

Among the other sessions I attended, I can spotlight one on the IAM service, where the speakers deep-dived into how to manage the service more accurately.

I also attended a session addressing the Sovereignty topic where we were explained the impact of the CLOUD act and the data protection subject.

I also went to a network focused presentation, the speaker showed us the new features of AWS Cloud WAN and how it works as a backbone network for hybrid solutions. The biggest news was that now we can connect AWS Direct Connections directly to AWS Cloud WAN without the need of a transit gateway.

They also introduced to us VPC Lattice, a fully managed application networking service that you use to connect, secure, and monitor the services and resources for your application.

Finally, I followed a session on cloud cost optimization. We were presented some dashboards that we can use to optimize our cloud costs.

That’s it for this year. Yet another summit where I’m able to learn new stuff. I hope I can go again next year and still learn new skills.

L’article AWS Summit Zurich 2025 est apparu en premier sur dbi Blog.