Joan Frey, auteur/autrice sur dbi Blog

Highly Available, Load-Balanced PostgreSQL with Patroni, HAProxy, and Keepalived

Joan Frey — Fri, 26 Jun 2026 19:11:41 +0000

Patroni runs your PostgreSQL cluster and handles failover, promoting a replica the moment the primary dies and recording the change in its distributed store (etcd, Consul, or ZooKeeper). That part works on its own.

Your applications still need one stable address to connect to, and they need writes to reach the primary while reads spread across replicas. HAProxy handles that routing, with a floating IP from Keepalived in front of it.

How the tools work together

Three components stands between your application and the database.

Patroni manages replication and failover, and it runs an agent on every PostgreSQL node. Each agent exposes a small REST API (port 8008 by default) that reports that node’s role.

HAProxy accepts client connections and forwards them to the right node. It asks Patroni’s REST API which node is the primary and which are replicas, then sends each connection to a matching node.

Keepalived publishes a virtual IP that floats between your HAProxy hosts using VRRP. Your application connects to the VIP, so one HAProxy host going down doesn’t take the whole entry point with it.

Your application talks to the VIP. Keepalived points the VIP at a live HAProxy. HAProxy forwards the connection to whichever PostgreSQL node Patroni reports as healthy for that role.

The health-check method

HAProxy checks one port and routes to another.

Patroni’s REST API returns an HTTP status that depends on the node’s role:

GET / returns 200 only on the leader (the primary). A non-leader node returns 503.
GET /primary is the explicit name for the same leader check.
GET /replica returns 200 only on a running replica.
GET /read-only returns 200 on the primary or a replica, any node that can serve a read.

In our case, HAProxy runs its health check against the API port (8008) and reads that status code, then forwards the SQL connection to the database port (5432). A node receives traffic only when its API answers 200 for the role that listener cares about. Point a listener’s check at / and it follows the primary. Point it at /replica and it follows the replicas. Patroni promotes a new leader, the status codes change, and HAProxy moves traffic to match within a couple of health-check cycles.

A first and simple working configuration

A two-node setup with 10.5.5.147 and 10.5.5.148 looks like this. One listener handles writes, the other handles reads.

listen PG1
    bind *:5000
    option httpchk
    http-check expect status 200
    default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions
    server postgresql_10.5.5.147_5432 10.5.5.147:5432 maxconn 100 check port 8008
    server postgresql_10.5.5.148_5432 10.5.5.148:5432 maxconn 100 check port 8008

listen PG1_ro
    bind *:5001
    option httpchk GET /replica
    http-check expect status 200
    default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions
    server postgresql_10.5.5.147_5432 10.5.5.147:5432 maxconn 100 check port 8008
    server postgresql_10.5.5.148_5432 10.5.5.148:5432 maxconn 100 check port 8008

This example runs PostgreSQL on port 5432 and the Patroni API on 8008, so swap in whatever ports your deployment uses (the defaults are 5432 and 8008).

Line by line:

bind *:5000 and bind *:5001 are the two addresses your applications connect to. Send writes to 5000 and reads to 5001.
option httpchk (with no path) on the first listener checks Patroni’s root endpoint. Only the leader answers 200, so HAProxy sends port 5000 traffic to the current primary.
option httpchk GET /replica on the second listener checks the replica endpoint, so HAProxy sends port 5001 traffic to a replica.
http-check expect status 200 tells HAProxy that 200 means healthy and anything else means down.
inter 3s fall 3 rise 2 checks every 3 seconds, marks a server down after 3 failures, and brings it back after 2 successes.
on-marked-down shutdown-sessions kills existing connections to a server the instant HAProxy marks it down, so clients reconnect and get rerouted instead of hanging on a dead node.
check port 8008 is the trick in action: health checks hit the Patroni API on 8008 while HAProxy forwards traffic to PostgreSQL on 5432.
maxconn 100 limit connections per server so you don’t exhaust PostgreSQL’s connection slots.

For a primary plus one or more replicas, this routes writes and reads to the right node and survives a failover.

The failure mode hiding in the read path

Imagine a two-node cluster: one primary, one replica. The replica goes down. Maybe it crashed, maybe Patroni is mid-switchover and no standby exists for a few seconds.

Your read traffic hits port 5001. That listener marks a server up only when GET /replica returns 200, and right now no node is a replica. HAProxy has zero usable servers in the pool, so it refuses the connection. Read queries start failing.

The primary is up the entire time, and it can serve those reads. Your config won’t send them there, because you told the read listener to look for replicas and nothing else. You’ve turned a degraded cluster that could still serve reads into a read outage. You feel this most on small clusters, and each failover passes through a window where the old primary becomes a replica and no standby is available yet. In the worst case, your replica is down, and one of your application is connecting to port 5001, resulting in errors.

The fix: fall back to the primary

Send reads to the primary when the read listener runs out of replicas, instead of dropping them.

listen PG1
    bind *:5000
    option httpchk
    http-check expect status 200
    default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions
    server postgresql_10.5.5.147_5432 10.5.5.147:5432 maxconn 100 check port 8008
    server postgresql_10.5.5.148_5432 10.5.5.148:5432 maxconn 100 check port 8008

listen PG1_ro
    bind *:5001
    option httpchk GET /replica
    http-check expect status 200
    use_backend PG1_ro_leader if { nbsrv(PG1_ro) eq 0 }
    default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions
    server postgresql_10.5.5.147_5432 10.5.5.147:5432 maxconn 100 check port 8008
    server postgresql_10.5.5.148_5432 10.5.5.148:5432 maxconn 100 check port 8008

backend PG1_ro_leader
    option httpchk GET /primary
    http-check expect status 200
    default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions
    server postgresql_10.5.5.147_5432 10.5.5.147:5432 maxconn 100 check port 8008
    server postgresql_10.5.5.148_5432 10.5.5.148:5432 maxconn 100 check port 8008

This new line carries the whole fix:

use_backend PG1_ro_leader if { nbsrv(PG1_ro) eq 0 }

nbsrv(PG1_ro) counts the usable servers in the PG1_ro pool, which here means the number of available replicas, since those servers pass the check only when GET /replica returns 200. While at least one replica is up, the count stays above zero, the condition is false, and reads stay on the replicas. The moment the last replica drops, nbsrv(PG1_ro) hits zero, the condition fires, and HAProxy diverts reads to the PG1_ro_leader backend.

That backend health-checks with GET /primary, so the only server it counts as up is the current primary. HAProxy sends reads to the primary until a replica returns, then shifts them back to the replica pool once a replica passes its check again.

Three names have to agree for this to work. The backend you define (backend PG1_ro_leader), the backend you route to (use_backend PG1_ro_leader), and the pool you count (nbsrv(PG1_ro)) all reference the real section names. Drop in a stale name from an earlier version and HAProxy either refuses to start or counts the wrong pool.

The /read-only shortcut and what it costs

Patroni offers GET /read-only, which returns 200 on the primary and the replicas alike. Point the read listener there and both the primary and the replicas serve reads, no fallback backend needed.

The cost is read load on the primary even when your replicas are healthy and idle. The fallback approach keeps reads off the primary until the replicas are gone, then leans on it as a safety net. You protect the primary’s write capacity during normal operation and still keep reads alive during a replica outage.

To keep lagging replicas out of the read pool, Patroni accepts a threshold on the replica check, for example GET /replica?lag=10MB, which fails any replica more than 10 MB behind. Pair that with the fallback and HAProxy drops the lagging replicas from rotation while reads still have somewhere to go.

Keepalived: removing HAProxy as a single point of failure

One HAProxy host fronting the cluster moves the single point of failure up a layer. Run HAProxy on two hosts and let Keepalived float a virtual IP between them with VRRP. Your application connects to the VIP, and whichever HAProxy holds it answers.

A minimal keepalived.conf on the primary HAProxy host:

vrrp_script chk_haproxy {
    script "killall -0 haproxy"   # succeeds while the haproxy process is alive
    interval 2
    weight 2
}

vrrp_instance VI_1 {
    interface eth0
    state MASTER
    virtual_router_id 51
    priority 101
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass changeme
    }
    virtual_ipaddress {
        10.0.0.1
    }
    track_script {
        chk_haproxy
    }
}

The second HAProxy host runs the same file with state BACKUP and a lower priority (100). Both advertise over VRRP, and the higher priority holds the VIP. chk_haproxy runs every two seconds. If HAProxy dies on the active host, its priority drops and the backup takes the VIP, so an HAProxy crash on one host no longer takes the entry point down with it.

Point your applications at 10.0.0.1:5000 for writes and 10.5.5.100:5001 for reads. Your applications never see which physical HAProxy does the work.

Summary

Patroni keeps the cluster healthy and picks the leader. HAProxy turns Patroni’s REST API into routing, sending writes to the primary and reads to the replicas by health-checking the API port while forwarding to the database port. The naive read-only listener drops reads when the last replica goes down, even though the primary could serve them. Adding use_backend ... if { nbsrv(...) eq 0 } with a primary-checking backend closes that gap, and a lag threshold on the replica check keeps stale standbys out of rotation. Keepalived puts a floating VIP in front of two HAProxy instances so the proxy layer survives a host failure too.

Writes reach the primary and reads spread across the replicas. Reads stay up as long as one node in the cluster is alive.

Let me know if you find any improvements to this configuration

L’article Highly Available, Load-Balanced PostgreSQL with Patroni, HAProxy, and Keepalived est apparu en premier sur dbi Blog.

Upgrade RHEL from 9.6 to 10.1 (when running PostgreSQL/Patroni)

Joan Frey — Fri, 26 Jun 2026 10:39:40 +0000

Upgrading from RHEL 9.6 to 10.1 is not just a routine update, it’s a major platform shift. When your server runs PostgreSQL compiled from source and a Patroni-managed cluster, the complexity increases significantly. System libraries change, Python environments break, ICU versions evolve, and your database binaries may no longer start after reboot.

In this guide, I walk through a real-world in-place upgrade using Leapp, covering preparation, resolving high-severity warnings, executing the upgrade, recompiling PostgreSQL, fixing collation mismatches, and restoring Patroni.

I. Preparation

Before the upgrade, you must ensure the current OS is healthy and fully patched.

1. Pause High Availability

Prevent Patroni from triggering a failover during the reboot cycles. If you are using a single PostgreSQL cluster, stop it by stopping the service or by using pg_ctl stop.

patronictl -c /etc/patroni/patroni.yml pause
systemctl stop patroni

2. Fix Subscription & Perform Full Update

I’m using an old VM for this blog, and If just like me, you see 403 Forbidden errors on repositories like codeready-builder, refresh your registration:

[root@patroni2 ~]# sudo dnf update -y
Updating Subscription Management repositories.

This system is registered with an entitlement server, but is not receiving updates. You can use subscription-manager to assign subscriptions.

Red Hat CodeReady Linux Builder for RHEL 9 x86_64 (RPMs)                                                                                                     761  B/s | 480  B     00:00
Errors during downloading metadata for repository 'codeready-builder-for-rhel-9-x86_64-rpms':
  - Status code: 403 for https://cdn.redhat.com/content/dist/rhel9/9/x86_64/codeready-builder/os/repodata/repomd.xml (IP: 23.206.57.92)
Error: Failed to download metadata for repo 'codeready-builder-for-rhel-9-x86_64-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

[root@patroni2 ~]# sudo subscription-manager clean
[root@patroni2 ~]# sudo subscription-manager register --force
[root@patroni2 ~]# sudo subscription-manager attach --auto
[root@patroni2 ~]# sudo subscription-manager refresh

[root@patroni2 ~]# sudo dnf update -y
...
Complete!

[root@patroni2 ~]# reboot

II. The Leapp Upgrade to 10.1

1. Install & Analyze

In this first phase, we are preparing the system for a major in-place upgrade using Leapp, the official upgrade framework for Red Hat–based distributions. When we install the package:

[root@patroni2 ~]# dnf install leapp-upgrade -y
...
Installed:
  leapp-0.20.0-1.el9.noarch                leapp-deps-0.20.0-1.el9.noarch          leapp-upgrade-el9toel10-0.23.0-1.el9.noarch       leapp-upgrade-el9toel10-deps-0.23.0-1.el9.noarch
  libdb-utils-5.3.28-57.el9_6.x86_64       python3-leapp-0.20.0-1.el9.noarch       systemd-container-252-55.el9_7.7.x86_64

Complete!

When running leapp preupgrade –target 10.1, we are not performing the upgrade. Instead, Leapp performs a full system audit to determine if the server is ready for RHEL 10.1. It checks:

Installed packages and their compatibility
Deprecated or removed libraries
Kernel drivers that will not exist in RHEL 10
Bootloader configuration (GRUB2)
GPG key validity
Custom system-level modifications (like dynamic linker changes)
…

Think of this step as a dry-run with intelligence.

[root@patroni2 ~]# sudo leapp preupgrade --target 10.1

...

============================================================
                      REPORT OVERVIEW
============================================================

HIGH and MEDIUM severity reports:
    1. GRUB2 core will be automatically updated during the upgrade
    2. Detected customized configuration for dynamic linker.
    3. Leapp detected loaded kernel drivers which are no longer maintained in RHEL 10.
    4. Failed to read GPG keys from provided key files
    5. Berkeley DB (libdb) has been detected on your system

Reports summary:
    Errors:                      0
    Inhibitors:                  0
    HIGH severity reports:       4
    MEDIUM severity reports:     1
    LOW severity reports:        1
    INFO severity reports:       3

Before continuing, review the full report below for details about discovered problems and possible remediation instructions:
    A report has been generated at /var/log/leapp/leapp-report.txt
    A report has been generated at /var/log/leapp/leapp-report.json

After running the pre-upgrade analysis, the next step is to carefully review:

/var/log/leapp/leapp-report.txt

What we are looking for first is simple:

Errors: 0
Inhibitors: 0

If an Inhibitor is present, the upgrade will be blocked entirely.
In my case, there were no blockers, but I did have several high severity warnings.

High severity does not mean the upgrade will fail.
It means: This could break something, review it carefully.

Let’s look at one concrete example from my system.

2. High Severity Example – Dynamic Linker Customization

Leapp detected that my system had a custom dynamic linker configuration:

Risk Factor: high

Title: Detected customized configuration for dynamic linker.

Summary: Custom configurations to the dynamic linker could potentially impact the upgrade in a negative way. The custom configuration includes modifications to /etc/ld.so.conf, custom or modified drop in config files in the /etc/ld.so.conf.d directory and additional entries in the LD_LIBRARY_PATH or LD_PRELOAD variables. These modifications configure the dynamic linker to use different libraries that might not be provided by Red Hat products or might not be present during the whole upgrade process. The following custom configurations were detected by leapp:

- The following drop in config files were marked as custom:

    - /etc/ld.so.conf.d/postgres.conf

Remediation: [hint] Remove or revert the custom dynamic linker configurations and apply the changes using the ldconfig command. In case of possible active software collections we suggest disabling them persistently.

Key: cc9bd972af70b7a27f66a37b11a00dcfcb73b1bc

----------------------------------------

What does this actually mean?

The dynamic linker (ld.so) is responsible for loading shared libraries at runtime.

By modifying:

/etc/ld.so.conf
files in /etc/ld.so.conf.d/
LD_LIBRARY_PATH
LD_PRELOAD

we are telling the system to load non-standard or custom libraries. In PostgreSQL environments (especially with custom builds or extensions), this is common practice. However, during a major OS upgrade, these custom paths might:

Point to libraries that do not exist in RHEL 10
Override new system libraries
Break dependency resolution mid-upgrade

Leapp flags this because it cannot guarantee consistency during the transition phase. In my case, it shouldn’t be an issue, because inside postgres.conf, I only have a path aiming to the lib directories of my PostgreSQL installation, which will not change, but we will still see how to prevent an error.

Understanding the Remediation

The report clearly suggests:

Remove or revert the custom dynamic linker configurations and apply the changes using the ldconfig command.

In my case, the configuration was related to PostgreSQL, so temporarily removing it is safe for the upgrade preparation phase. Instead of deleting it permanently, I moved it aside:

[root@patroni2 ~]# sudo mv /etc/ld.so.conf.d/postgres.conf /tmp/postgres.conf.bak
[root@patroni2 ~]# sudo ldconfig

ldconfig rebuilds the system library cache now based only on standard paths.

Re-Run the Preupgrade Check

After remediation, always re-run the preupgrade command and check the report again. If the fix was successful:

The severity should disappear from the REPORT OVERVIEW
The issue should no longer appear in leapp-report.txt

This validation loop is important. We are progressively cleaning the system until it is fully compliant for upgrade.

============================================================
                      REPORT OVERVIEW
============================================================

HIGH and MEDIUM severity reports:
    1. Leapp detected loaded kernel drivers which are no longer maintained in RHEL 10.
    2. GRUB2 core will be automatically updated during the upgrade
    3. Failed to read GPG keys from provided key files
    4. Berkeley DB (libdb) has been detected on your system

Reports summary:
    Errors:                      0
    Inhibitors:                  0
    HIGH severity reports:       3
    MEDIUM severity reports:     1
    LOW severity reports:        1
    INFO severity reports:       3

Before continuing, review the full report below for details about discovered problems and possible remediation instructions:
    A report has been generated at /var/log/leapp/leapp-report.txt
    A report has been generated at /var/log/leapp/leapp-report.json

Only once the report is clean, or fully understood, should we proceed to the actual upgrade execution.

2. Execute the upgrade

Once all errors and inhibitors are resolved, and high-severity findings have been reviewed or remediated, we are finally ready to perform the actual in-place upgrade. This is the moment where Leapp transitions from analysis mode to execution mode.

About the Repository Warning

During the preupgrade phase, Leapp informed us that codeready-builder-… repositories are not officially supported during the upgrade process and are excluded by default.

This is expected behavior as Leapp only enables a minimal, controlled set of repositories to ensure:

Package consistency
Dependency resolution stability
Predictable upgrade paths

However, in PostgreSQL environments, some packages (extensions, development headers, libraries) may depend on CodeReady Builder. If a repository is truly required during the upgrade, we must explicitly enable it using:

--enablerepo

Running the Upgrade

Since I need CodeReady Builder for PostgreSQLdependencies, I ran:

leapp upgrade --target 10.1 --enablerepo codeready-builder-for-rhel-10-x86_64-rpms

What happens when we run this command?

At this stage, Leapp:

Resolves and downloads required RHEL 10 packages
Builds a temporary upgrade environment
Prepares a special upgrade initramfs
Modifies the bootloader (GRUB) to boot into the upgrade environment on next reboot

The system is not upgraded immediately. The actual OS transition happens during the next boot.

After the command completes, Leapp generates another report. Just like in the preupgrade phase, verify:

Errors: 0
Inhibitors: 0

If everything looks clean, we can proceed.

Reboot – The Real Upgrade Begins

# reboot

This is where the real upgrade starts. During boot:

The system enters a temporary upgrade environment
Packages are replaced
Obsolete components are removed
Configuration files are migrated
The new RHEL 10 kernel is installed

This phase can take several minutes depending on your VM/server resources. My VM doesn’t have many resources and it took me around 30 minutes. Be patient, interrupting this process can leave the system in an inconsistent state.

Verifying the Upgrade

Once the server is back online, confirm the OS version:

[root@patroni2 ~]# cat /etc/os-release
NAME="Red Hat Enterprise Linux"
VERSION="10.1 (Coughlan)"
ID="rhel"
ID_LIKE="centos fedora"
VERSION_ID="10.1"
PLATFORM_ID="platform:el10"
PRETTY_NAME="Red Hat Enterprise Linux 10.1 (Coughlan)"
ANSI_COLOR="0;31"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:redhat:enterprise_linux:10.1"
HOME_URL="https://www.redhat.com/"
VENDOR_NAME="Red Hat"
VENDOR_URL="https://www.redhat.com/"
DOCUMENTATION_URL="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/10"
BUG_REPORT_URL="https://issues.redhat.com/"

REDHAT_BUGZILLA_PRODUCT=”Red Hat Enterprise Linux 10″
REDHAT_BUGZILLA_PRODUCT_VERSION=10.1
REDHAT_SUPPORT_PRODUCT=”Red Hat Enterprise Linux”
REDHAT_SUPPORT_PRODUCT_VERSION=”10.1″

This confirms that we are now running RHEL 10.1.

III. Post-Upgrade Database Recovery

Once you login to RHEL 10.1, your Postgres binaries in /u01 will fail because libicuuc.so.67 (from RHEL 9) is missing. It can also fail because of other libraries.

14:45:55 postgres@patroni2:/home/postgres/ [test-op-patroni] pgstart
/u01/app/postgres/product/17/db_6/bin/postgres: error while loading shared libraries: libicuuc.so.67: cannot open shared object file: No such file or directory
no data was returned by command ""/u01/app/postgres/product/17/db_6/bin/postgres" -V"
command not found
program "postgres" is needed by pg_ctl but was not found in the same directory as "/u01/app/postgres/product/17/db_6/bin/pg_ctl"

1. Recompile PostgreSQL

Since you installed from source, you must re compile PostgreSQL with the new RHEL 10 system libraries. Here is the command I personally use to build it, with the postgres user:

postgres@patroni2:/home/postgres/ [dummy] MAJOR="17"
postgres@patroni2:/home/postgres/ [dummy] MINOR="6"
postgres@patroni2:/home/postgres/ [dummy] tar axf postgresql-${MAJOR}.${MINOR}.tar.gz
postgres@patroni2:/home/postgres/ [dummy] mkdir build; cd $_
postgres@patroni2:/home/postgres/ [dummy] export PGHOME="/u01/app/postgres/product/${MAJOR}/db_${MINOR}"
postgres@patroni2:/home/postgres/ [dummy] export SEGSIZE=2
postgres@patroni2:/home/postgres/ [dummy] export BLOCKSIZE=8
postgres@patroni2:/home/postgres/ [dummy] meson setup . ../postgresql-${MAJOR}.${MINOR}
postgres@patroni2:/home/postgres/ [dummy] meson configure -Dprefix=${PGHOME}                   -Dbindir=${PGHOME}/bin                   -Ddatadir=${PGHOME}/share                   -Dincludedir=${PGHOME}/include                   -Dlibdir=${PGHOME}/lib                   -Dsysconfdir=${PGHOME}/etc                   -Dpgport=5432                   -Dplperl=enabled                   -Dplpython=enabled                   -Dssl=openssl                   -Dpam=enabled                   -Dldap=enabled                   -Dlibxml=enabled                   -Dlibxslt=enabled                   -Dsegsize=${SEGSIZE}                   -Dblocksize=${BLOCKSIZE}                   -Dllvm=enabled                   -Duuid=ossp                   -Dzstd=enabled                   -Dlz4=enabled                   -Dzstd=enabled                   -Dgssapi=enabled                   -Dsystemd=enabled                   -Dicu=enabled                   -Dsystem_tzdata=/usr/share/zoneinfo                   -Dextra_version=" dbi services build"
postgres@patroni2:/home/postgres/ [dummy] ninja
postgres@patroni2:/home/postgres/ [dummy] ninja install

2. Restore Library Paths

[root@patroni2 ~]# mv /tmp/postgres.conf.bak /etc/ld.so.conf.d/postgres.conf
[root@patroni2 ~]# ldconfig

3. Start & Fix Collation Mismatch

Postgres will now start, but will warn you about Collation Version Mismatches (2.34 vs 2.39).

15:09:55 postgres@patroni2:/home/postgres/build/ [test-op-patroni] pgstart
waiting for server to start.... done
server started
15:10:28 postgres@patroni2:/home/postgres/build/ [test-op-patroni] psql
WARNING:  database "postgres" has a collation version mismatch
DETAIL:  The database was created using collation version 2.34, but the operating system provides version 2.39.
HINT:  Rebuild all objects in this database that use the default collation and run ALTER DATABASE postgres REFRESH COLLATION VERSION, or build PostgreSQL with the right library version.
psql (17.6 dbi services build)
Type "help" for help.

Inside Postgres, run for every database:

ALTER DATABASE postgres REFRESH COLLATION VERSION;
ALTER DATABASE

-- Repeat for other DBs if applicable
REINDEX DATABASE postgres;

Your PostgreSQL is now starting properly and your server has been upgraded.

4. In case of a patroni cluster

Since the system Python version has changed after the OS upgrade, your old .local venv is invalid. You must recreate it. Here is how I install patroni using the postgres user:

$ python3 -m venv .local
$ .local/bin/pip3 install –upgrade pip
$ .local/bin/pip3 install –upgrade setuptools
$ .local/bin/pip3 install wheel
$ .local/bin/pip3 install psycopg[binary]
$ .local/bin/pip3 install python-etcd
$ .local/bin/pip3 install patroni
$ .local/bin/patroni version

Check if the cluster sees the member again. If patronictl list is empty, a restart of the service is usually required to re-register with etcd.

10:45:40 postgres@patroni2:/home/postgres/ [test-op-patroni] patronictl list
+ Cluster: test-op-patroni (7565882985963789761) -+-----+------------+-----+
| Member | Host | Role | State | TL | Receive LSN | Lag | Replay LSN | Lag |
+--------+------+------+-------+----+-------------+-----+------------+-----+
+--------+------+------+-------+----+-------------+-----+------------+-----+
10:45:48 postgres@patroni2:/home/postgres/ [test-op-patroni] sudo systemctl restart patroni
10:45:55 postgres@patroni2:/home/postgres/ [test-op-patroni] patronictl list
+ Cluster: test-op-patroni (7565882985963789761) --+---------+----+-------------+-----+------------+-----+
| Member                 | Host           | Role   | State   | TL | Receive LSN | Lag | Replay LSN | Lag |
+------------------------+----------------+--------+---------+----+-------------+-----+------------+-----+
| patroni-tst-op-geapg02 | 192.168.56.142 | Leader | running | 11 |             |     |            |     |
+------------------------+----------------+--------+---------+----+-------------+-----+------------+-----+

IV. Conclusion

Upgrading from RHEL 9.6 to 10.1 is a big move. It’s not just a simple update; it’s a total shift in the system’s foundation. Between hardware driver changes and library updates, you really have to pay attention to the details to keep your database running.

RHEL 10.1 is a great, modern platform, but you can’t just click “update” and hope for the best. By planning the upgrade, planning to rebuild your binaries, refreshing your database objects, you can make the jump without the drama. Take the pre-upgrade report seriously, it’s there for a reason!

That said, for a production PostgreSQL cluster, especially one managed with Patroni and etcd, I would not recommend this in-place upgrade approach. Even if Leapp makes the process technically possible, you are still:

Modifying the operating system in place
Replacing core libraries underneath a running database stack
Trusting automated dependency resolution during a major version jump

In production, risk reduction should always be the priority.

Instead, I strongly recommend provisioning new VMs or physical servers, installing RHEL 10.1 from scratch, deploying PostgreSQL, Patroni, and etcd cleanly, rebuilding the cluster from best practices, and then migrating the data from the old environment to the new one using replication or another appropriate method.

Sometimes the safest upgrade… is a new cluster.

L’article Upgrade RHEL from 9.6 to 10.1 (when running PostgreSQL/Patroni) est apparu en premier sur dbi Blog.

PostgreSQL Anonymizer: Simple Data Masking for DBAs

Joan Frey — Fri, 27 Feb 2026 10:08:50 +0000

Sensitive data (names, emails, phone numbers, personal identifiers…) should not be freely exposed outside production. When you refresh a production database to a test or staging environment, or when analysts need access to real-looking data, anonymization becomes critical.

The PostgreSQL Anonymizer extension (often called anon) is an open‑source extension that lets you mask, fake, shuffle, or generalize data directly inside PostgreSQL, using simple SQL rules. This article explains what it is, how it works, how to install it on PostgreSQL 18.1 running on Red Hat Enterprise Linux 10.1, and how to use it with clear, beginner‑friendly examples.

The target audience is everyone, but especially beginner DBAs who want a practical, command‑line–focused introduction.

I. What is PostgreSQL Anonymizer?

PostgreSQL Anonymizer is an extension that helps protect sensitive data by replacing it with fake or obfuscated values.

Instead of exporting data and anonymizing it with scripts, the rules live inside the database itself. You declare how a column should be anonymized, and PostgreSQL applies that rule automatically.

Typical use cases:

Refreshing production data into test / staging environments
Giving developers or analysts access to realistic but non‑sensitive data
Producing anonymized database dumps for external sharing
Helping comply with GDPR and other privacy regulations

The extension supports three main approaches:

Static masking – permanently replaces data in tables
Dynamic masking – masks data on the fly for specific users
Anonymous dumps – exports an already anonymized dump

II. How does it work ?

PostgreSQL Anonymizer uses PostgreSQL’s security labels mechanism. You attach a label to a column that says: “When this data is anonymized, use this function.”

Example:

SECURITY LABEL FOR anon ON customer.email IS 'MASKED WITH FUNCTION anon.fake_email()';

Once declared:

Static masking rewrites the table using those rules
Dynamic masking rewrites query results for masked users
Dumps automatically apply the same rules

The rules stay attached to the schema, not to scripts or applications.

III. Installing PostgreSQL Anonymizer on RHEL 10.1 (PostgreSQL 18.1)

In this guide, PostgreSQL 18.1 is already installed following dbi services standard. PostgreSQL binaries are located in:

/u01/app/postgres/product/18/db_1/bin

The PostgreSQL data directory (PGDATA) is:

/u02/pgdata/18/demo-cluster

We will only focus on installing and enabling the PostgreSQL Anonymizer extension.

1. Install the Anonymizer extension

Since we are installing from source, we’ll start by installing the necessary prerequisites. We’ll use cargo to handle the PGRX system requirements. You can find the official documentation here, but keep in mind that I’ve updated the commands to reflect a more recent version.

#Cargo install

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env
rustc --version
cargo --version

# postgresql_anonymizer install

cargo install cargo-pgrx --version 0.16.1 --locked
cargo pgrx init --pg18 /u01/app/postgres/product/18/db_1/bin/pg_config
git clone https://gitlab.com/dalibo/postgresql_anonymizer.git
cd postgresql_anonymizer/
make extension PG_CONFIG=/u01/app/postgres/product/18/db_1/bin/pg_config PGVER=pg18
sudo make install PG_CONFIG=/u01/app/postgres/product/18/db_1/bin/pg_config PGVER=pg18

2. Enable Anonymizer in postgresql.conf

Because PostgreSQL Anonymizer hooks into query execution, it must be loaded at session start. Edit the configuration file:

vi /u02/pgdata/18/demo-cluster/postgresql.conf

Add or update:

session_preload_libraries = 'anon'

Restart PostgreSQL:

pg_ctl -D /u02/pgdata/18/demo-cluster restart

3. Install and enable the Anonymizer extension

Connect as the PostgreSQL superuser:

psql -U postgres

Create a database:

CREATE DATABASE anonymizer_demo;

Create and initialize the extension:

postgres=# \c anonymizer_demo
You are now connected to database "anonymizer_demo" as user "postgres".
anonymizer_demo=# CREATE EXTENSION anon CASCADE;
CREATE EXTENSION

anonymizer_demo=# \dx
                                  List of installed extensions
  Name   | Version | Default version |   Schema   |                 Description
---------+---------+-----------------+------------+---------------------------------------------
 anon    | 3.0.0   | 3.0.0           | public     | Anonymization & Data Masking for PostgreSQL
 plpgsql | 1.0     | 1.0             | pg_catalog | PL/pgSQL procedural language
(2 rows)

anonymizer_demo=# SELECT anon.init();
 init
------
 t
(1 row)

anon.init() loads fake data dictionaries (names, companies, cities, etc.) used by anonymization functions.

IV. Demo: anonymizing a simple table

For this demo, I will keep it simple and create everything inside the postgres database and default schema, but I recommend you to follow the best practices and use a dedicated database, user and schema.

Create sample data:

CREATE TABLE customer (
  id         SERIAL PRIMARY KEY,
  first_name TEXT,
  last_name  TEXT,
  birthdate  DATE,
  email      TEXT,
  company    TEXT
);

INSERT INTO customer (first_name, last_name, birthdate, email, company) VALUES
('Alice', 'Martin', '1987-02-14', 'alice.martin@example.com', 'Acme Corp'),
('Bob',   'Dupont', '1979-11-03', 'bob.dupont@example.com',   'Globex');

anonymizer_demo=# select * from customer;
 id |  full_name   | birthdate  |          email           |  company
----+--------------+------------+--------------------------+-----------
  1 | Alice Martin | 1987-02-14 | alice.martin@example.com | Acme Corp
  2 | Bob Dupont   | 1979-11-03 | bob.dupont@example.com   | Globex
(2 rows)

1. Static masking (permanent anonymization)

Static masking is a “fire and forget” approach where the original sensitive data is physically overwritten on the disk with faked or scrambled values. This process is destructive. Once the data is masked, the original values are gone forever. This should only be performed on non-production environments (like Staging or Dev) or on a backup copy of your database.

Before applying any rules, you must explicitly enable the static masking engine at both the database and role levels. This acts as a safety switch to prevent accidental data loss.

-- Enable the extension for the current database
anonymizer_demo=# ALTER DATABASE anonymizer_demo SET anon.static_masking = TRUE;
ALTER DATABASE

-- Grant the postgres user permission to execute static masking operations
anonymizer_demo=# ALTER ROLE postgres SET anon.static_masking = TRUE;
ALTER ROLE

Next, you define how the data should be transformed. We use SECURITY LABEL to attach masking logic to specific columns. This doesn’t change the data yet; it simply tells the anon extension which functions to use during the anonymization process.

-- Replace names with realistic dummy values
anonymizer_demo=# SECURITY LABEL FOR anon ON COLUMN customer.first_name IS 'MASKED WITH FUNCTION anon.dummy_first_name()';
SECURITY LABEL
anonymizer_demo=# SECURITY LABEL FOR anon ON COLUMN customer.last_name IS 'MASKED WITH FUNCTION anon.dummy_last_name()';
SECURITY LABEL

-- Generate a random date within a specific age range (1950-2000)
anonymizer_demo=# SECURITY LABEL FOR anon ON column customer.birthdate IS 'MASKED WITH FUNCTION anon.random_date_between(''1950-01-01'', ''2000-12-31'')';
SECURITY LABEL

-- Generate syntactically correct but fake emails and company names
anonymizer_demo=# SECURITY LABEL FOR anon ON column customer.email IS 'MASKED WITH FUNCTION anon.fake_email()';
SECURITY LABEL
anonymizer_demo=# SECURITY LABEL FOR anon ON column customer.company IS 'MASKED WITH FUNCTION anon.fake_company()';
SECURITY LABEL

This is the final execution step. Running anonymize_database() triggers the engine to scan your rules and overwrite the table data globally. Depending on the size of your database, this may take some time as it performs UPDATE operations on the disk.

anonymizer_demo=# SELECT anon.anonymize_database();
 anonymize_database
--------------------
 t
(1 row)

The data rules have now been applied and the data have been anonymized. You can check the result with the following query:

anonymizer_demo=# SELECT * FROM customer;
 id | birthdate  |        email         |    company     | first_name | last_name
----+------------+----------------------+----------------+------------+------------
  1 | 1960-11-30 | lpeters@example.net  | Brown and Sons | Chyna      | Mertz
  2 | 1997-09-08 | avaughan@example.com | Rice PLC       | Damien     | Williamson
(2 rows)

You can now turn off static masking to continue with the next demo, dynamic masking:

anonymizer_demo=# ALTER SYSTEM SET anon.static_masking TO off;
ALTER SYSTEM
anonymizer_demo=# ALTER ROLE postgres SET anon.static_masking TO off;
ALTER ROLE

2. Dynamic masking

Dynamic masking allows you to hide sensitive information from specific users (like developers or analysts) while preserving the original data for administrators or the application itself. The masking happens in memory at the moment the query is executed.

First, we tell the database to activate the transparent masking engine. This allows the anon extension to intercept queries from specific roles and apply masking rules before the results are returned.

anonymizer_demo=# ALTER DATABASE anonymizer_demo SET anon.transparent_dynamic_masking = TRUE;
ALTER DATABASE

To see show case masking in action, we will use two types of users: a masked user who sees fake data, and an unmasked user (like the superuser) who sees the actual data stored on disk.

In this step, we create demo_user and “tag” them with a security label that forces the masking engine to engage whenever they log in.

anonymizer_demo=# CREATE ROLE demo_user LOGIN;
CREATE ROLE
anonymizer_demo=# SECURITY LABEL FOR anon ON ROLE demo_user IS 'MASKED';
SECURITY LABEL
anonymizer_demo=# GRANT pg_read_all_data to demo_user;
GRANT ROLE

anonymizer_demo=# SECURITY LABEL FOR anon ON ROLE demo_user IS 'MASKED';
SECURITY LABEL

-- As PostgreSQL user:
-- Ensure the postgres user remains unmasked so we can see the 'real' data

anonymizer_demo=# SECURITY LABEL FOR anon ON ROLE postgres IS NULL;
SECURITY LABEL

We don’t need to redefine our masking rules (for names, emails, etc.) because the SECURITY LABEL definitions we created in the static masking section are still stored in the database schema.

Watch what happens when we query the table as demo_user:

anonymizer_demo=> select * from customer;
 id | birthdate  |          email           |     company      | first_name | last_name
----+------------+--------------------------+------------------+------------+-----------
  1 | 1957-02-07 | walterkristi@example.org | Fernandez-Tucker | Emelie     | Rohan
  2 | 1950-05-15 | hannah76@example.com     | Leonard Group    | Jane       | Durgan
(2 rows)

If we run the exact same command again, the output changes:

anonymizer_demo=> select * from customer;
 id | birthdate  |         email          |    company    | first_name | last_name
----+------------+------------------------+---------------+------------+------------
  1 | 1963-12-26 | blairpeter@example.com | Simpson Group | Tod        | Balistreri
  2 | 1971-09-14 | steven27@example.com   | Davis-Hardin  | Sonny      | Wintheiser
(2 rows)

Because the data is being generated “on-the-fly” by the masking functions, the results are dynamic. Each request produces a fresh set of data.

Now, let’s switch back to the postgres user. Since we removed the MASKED label from this role, the engine steps aside and shows us the actual data residing on the disk (which, in this case, is the data we masked statically in the previous step).

anonymizer_demo=# SELECT * FROM customer;
 id | birthdate  |        email         |    company     | first_name | last_name
----+------------+----------------------+----------------+------------+------------
  1 | 1960-11-30 | lpeters@example.net  | Brown and Sons | Chyna      | Mertz
  2 | 1997-09-08 | avaughan@example.com | Rice PLC       | Damien     | Williamson
(2 rows)

3. Anonymized dump

An Anonymized Dump allows you to export your database into a .sql file where the sensitive data is already replaced by fake values. This is incredibly powerful because you can create a “safe” backup that can be shared with developers or consultants without ever giving them access to your live production server.

Rather than using a superuser, we create a specific role whose sole purpose is to retrieve the masked version of the data during the export process.

-- Create a specialized user for the dump process
anonymizer_demo=# CREATE ROLE demo_ano_dumper LOGIN PASSWORD 'secret';
CREATE ROLE

-- Force the masking engine to be active for this user session
anonymizer_demo=# ALTER ROLE demo_ano_dumper SET anon.transparent_dynamic_masking = TRUE;
ALTER ROLE

-- Apply the MASKED label to the role
anonymizer_demo=# SECURITY LABEL FOR anon ON ROLE demo_ano_dumper IS 'MASKED';
SECURITY LABEL

-- Grant permission to read all tables
anonymizer_demo=# GRANT pg_read_all_data TO demo_ano_dumper;
GRANT

Now, we use the standard pg_dump utility. Because we are logging in as demo_ano_dumper, the anon extension intercepts the data export on-the-fly. We use a few specific flags to ensure the resulting file is clean and doesn’t contain the masking logic itself:

/u01/app/postgres/product/18/db_1/bin/pg_dump anonymizer_demo --username=demo_ano_dumper --password --no-security-labels --exclude-extension="anon" --file=anonymized_dump.sql

--no-security-labels: Prevents the “MASKED” tags from being exported (the new database doesn’t need to know how the data was masked).

--exclude-extension="anon": Ensures the recipient doesn’t need the anon extension installed to restore the file.

If you open the generated anonymized_dump.sql file in a text editor, you will see that the COPY commands contain the fake data, not the original sensitive information.

--
-- Data for Name: customer; Type: TABLE DATA; Schema: public; Owner: postgres
--

COPY public.customer (id, birthdate, email, company, first_name, last_name) FROM stdin;
1       1980-12-17      cannonlauren@example.com        Carr-Doyle      Katharina       Shanahan
2       1998-03-27      sfox@example.net        Pollard and Sons        Ayla    Spencer
\.

V. Conclusion

In modern development, the goal is to work with realistic data without the real-world risk. PostgreSQL Anonymizer bridges this gap by allowing you to transform sensitive production information into safe, functional datasets.

Now that we’ve explained how to use pg_anonymizer and covered all three methods, here is a quick guide on when to use each:

Static Masking: Best for “cleaning” a staging database after a production refresh.
Dynamic Masking: Best for internal users (DBAs, support staff) who need to work on the live database but shouldn’t see production data.
Anonymized Dump: Best for sharing data with external partners or creating local development environments.

In the end, whether you choose Static, Dynamic, or Dump masking, the benefits remain the same:

Utility: Because the data is masked with realistic functions (like fake_email or dummy_first_name), your application logic—like email validation or UI layout—still works perfectly.
Compliance: Meet GDPR, HIPAA, and internal security requirements by default.
Safety: Developers and analysts can work on real-world bugs and features without ever seeing a customer’s actual PII (Personally Identifiable Information).

I hope you enjoyed this guide and found these examples clear and easy to follow! My goal was to show that data privacy doesn’t have to be painful for your development workflow. Don’t forget to follow the extension latest news on the official website: https://postgresql-anonymizer.readthedocs.io/en/latest/

If you have any questions about these commands or how to implement them in your own environment, feel free to reach out

L’article PostgreSQL Anonymizer: Simple Data Masking for DBAs est apparu en premier sur dbi Blog.

Monitoring PostgreSQL on Windows with PgHero and Docker

Joan Frey — Thu, 26 Feb 2026 09:35:33 +0000

I know, I know, PostgreSQL usually feels more at home on Linux. But guess what? PgHero works just fine on Windows too, and I gave it a try myself. If you’re running PostgreSQL in a Docker container, this setup is surprisingly easily, and it gives you a great performance dashboard without touching your base system.

In this post, I’ll show you how I set it up with Docker Compose, including running PgHero itself in a container. The goal: a fully contained stack that’s easy to spin up, monitor, and tear down anytime.

What you need

Docker Desktop installed on Windows (I recommend using WSL2 for smooth volume mounting).
A PostgreSQL database running in Docker. We’ll build the stack together if you don’t have one yet. In my case, I decided to install PgHero when I was following one of the NestJs course, and used my existing stack.
Basic familiarity with Docker Compose and command-line.

Optionally, you can create a folder on your PC to store database files, so your data sticks around even if the container stops. I will use a volume in this blog, to easily edit my PostgreSQL configuration.

Step 1: Your Docker Compose setup

Here’s a minimal Compose file for PostgreSQL, Redis, and PgHero. Save it somewhere on your machine, like C:\docker\pghero-setup\docker-compose.yml:

version: "3"
services:
  db:
    image: postgres
    restart: always
    ports:
      - "5438:5432"
    environment:
      POSTGRES_PASSWORD: pass123
      POSTGRES_USER: postgres
      POSTGRES_DB: postgres
    volumes:
      - "C:/Users/frj/Documents/Training/Nestjs/auth-extension/docker/postgres-data:/var/lib/postgresql/data"

  redis:
    image: redis
    ports:
      - "6379:6379"
    restart: always

  pghero:
    image: ankane/pghero
    depends_on:
      - db
    ports:
      - "8080:8080"
    environment:
      DATABASE_URL: postgres://postgres:pass123@db:5432/postgres
      PGHERO_USERNAME: admin
      PGHERO_PASSWORD: secret
    restart: always

Why this works:

db is your PostgreSQL service. The bind mount points to a folder on your PC, so your data survives container restarts.
The command lines ensure PostgreSQL loads pg_stat_statements, which PgHero uses to track queries.
pghero runs in its own container and exposes port 8080 for the dashboard.
depends_on: db ensures PgHero waits for PostgreSQL to start.

Step 2: Monitoring multiple databases

To be able to track more than one database, we create a pghero.yml file instead of using the DATABASE_URL environment variable. Create the file in the same directory as docker-compose.yml:

databases:
  primary:
    url: postgres://postgres:pass123@db:5432/postgres
  analytics:
    url: postgres://postgres:pass123@db:5432/analytics

Then update docker-compose.yml for PgHero:

  pghero:
    image: ankane/pghero
    depends_on:
      - db
    ports:
      - "8080:8080"
    environment:
      PGHERO_USERNAME: admin
      PGHERO_PASSWORD: secret
    volumes:
      - "./pghero.yml:/app/config/pghero.yml:ro"
    restart: always

PgHero will now show a dropdown to switch between databases in the UI.

Step 3: Start everything

In the folder with your docker-compose.yml:

docker-compose down  # stop any previous run
docker-compose up -d

After a few seconds, open your browser at http://localhost:8080. Log in with the username and password you set (admin / secret). You should see the PgHero dashboard connected to your database.

Step 4: Enable query tracking

After starting PostgreSQL, connect to it and enable the pg_stat_statements extension. Inside your container, run:

docker exec -it <db_container> psql -U postgres -d postgres

Then in psql:

CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

This makes sure PgHero can track queries, see long-running statements, and show useful stats. Note that you have to do it for each database.

Step 5: Windows-specific tips

Use forward slashes (C:/path/to/folder) for volume paths in Compose.
Make sure the folder exists before starting PostgreSQL.
Check your pg_hba.conf if you see connection errors — Docker containers have their own internal IPs, and PostgreSQL needs to allow connections from them.

Step 6: Common troubleshooting

500 Internal Server Error / cannot connect: Usually a problem with pg_hba.conf. Make sure your Docker network is allowed.
No stats showing: Ensure pg_stat_statements is loaded and the extension is created.
Dashboard not reachable: Check Docker mapped ports, firewall, and container logs (docker logs ).

Conclusion

And that’s it! You now have a Windows-based Docker setup with PostgreSQL and PgHero. You can see your query stats, monitor performance, and even track multiple databases from the same dashboard.

L’article Monitoring PostgreSQL on Windows with PgHero and Docker est apparu en premier sur dbi Blog.

Unit Testing PostgreSQL with pgTAP

Joan Frey — Fri, 29 Aug 2025 16:23:47 +0000

Introduction

Unit testing is a fundamental practice in software development, ensuring that individual components function correctly. When working with PostgreSQL, testing database logic—such as functions, triggers, and constraints—is crucial for maintaining data integrity and reliability. One powerful tool for this purpose is pgTAP.
pgTAP is a PostgreSQL extension that provides a set of TAP (Test Anything Protocol) functions for writing unit tests directly in SQL. It allows developers to test database functions, schemas, constraints, and much more in an automated and repeatable way.

Installing pgTAP

Before using pgTAP, you need to install it on your PostgreSQL instance. You can install it from source as follows:

10:18:56 postgres@ws-pgdev:/home/postgres/ [sw] wget https://api.pgxn.org/dist/pgtap/1.3.3/pgtap-1.3.3.zip .
--2025-04-02 10:19:53--  https://api.pgxn.org/dist/pgtap/1.3.3/pgtap-1.3.3.zip
Resolving api.pgxn.org (api.pgxn.org)... 88.198.49.178
Connecting to api.pgxn.org (api.pgxn.org)|88.198.49.178|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 329966 (322K) [application/zip]
Saving to: ‘pgtap-1.3.3.zip’

pgtap-1.3.3.zip                         100%[============================================================================>] 322.23K  --.-KB/s    in 0.1s

2025-04-02 10:19:54 (3.18 MB/s) - ‘pgtap-1.3.3.zip’ saved [329966/329966]

--2025-04-02 10:19:54--  http://./
Resolving . (.)... failed: No address associated with hostname.
wget: unable to resolve host address ‘.’
FINISHED --2025-04-02 10:19:54--
Total wall clock time: 0.4s
Downloaded: 1 files, 322K in 0.1s (3.18 MB/s)

10:19:54 postgres@ws-pgdev:/home/postgres/ [sw] unzip pgtap-1.3.3.zip
Archive:  pgtap-1.3.3.zip
b941782fada240afdb7057065eb3261a21e8512c
   creating: pgtap-1.3.3/
  inflating: pgtap-1.3.3/Changes
...
...

10:20:11 postgres@ws-pgdev:/home/postgres/ [sw] cd pgtap-1.3.3/
11:11:58 postgres@ws-pgdev:/home/postgres/pgtap-1.3.3/ [sw] make

GNUmake running against Postgres version 17.0, with pg_config located at /u01/app/postgres/product/17/db_0/bin

Makefile:186: To use pg_prove, TAP::Parser::SourceHandler::pgTAP Perl module
Makefile:187: must be installed from CPAN. To do so, simply run:
Makefile:188: cpan TAP::Parser::SourceHandler::pgTAP
cp sql/pgtap--0.95.0--0.96.0.sql.in sql/pgtap--0.95.0--0.96.0.sql
cp sql/pgtap--0.96.0--0.97.0.sql.in sql/pgtap--0.96.0--0.97.0.sql
cp sql/pgtap--0.97.0--0.98.0.sql.in sql/pgtap--0.97.0--0.98.0.sql
cp sql/pgtap--0.98.0--0.99.0.sql.in sql/pgtap--0.98.0--0.99.0.sql
cp sql/pgtap--0.99.0--1.0.0.sql.in sql/pgtap--0.99.0--1.0.0.sql
cp sql/pgtap.sql.in sql/pgtap.sql
sed -e 's,MODULE_PATHNAME,$libdir/pgtap,g' -e 's,__OS__,linux,g' -e 's,__VERSION__,1.3,g' sql/pgtap.sql > sql/pgtap.tmp
mv sql/pgtap.tmp sql/pgtap.sql
'/usr/bin/perl' -e 'for (grep { /^CREATE /} reverse <>) { chomp; s/CREATE (OR REPLACE )?/DROP /; s/DROP (FUNCTION|VIEW|TYPE) /DROP $1 IF EXISTS /; s/ (DEFAUL                                T|=)[ ]+[a-zA-Z0-9]+//g; print "$_;\n" }' sql/pgtap.sql > sql/uninstall_pgtap.sql
cp sql/pgtap.sql.in sql/pgtap-static.sql.tmp

*** Patching pgtap-static.sql with compat/install-9.6.patch
patching file sql/pgtap-static.sql.tmp

*** Patching pgtap-static.sql with compat/install-9.4.patch
patching file sql/pgtap-static.sql.tmp

*** Patching pgtap-static.sql with compat/install-9.2.patch
patching file sql/pgtap-static.sql.tmp

*** Patching pgtap-static.sql with compat/install-9.1.patch
patching file sql/pgtap-static.sql.tmp
sed -e 's#MODULE_PATHNAME#$libdir/pgtap#g' -e 's#__OS__#linux#g' -e 's#__VERSION__#1.3#g' sql/pgtap-static.sql.tmp > sql/pgtap-static.sql
'/usr/bin/perl' compat/gencore 0 sql/pgtap-static.sql > sql/pgtap-core.sql
'/usr/bin/perl' compat/gencore 1 sql/pgtap-static.sql > sql/pgtap-schema.sql
cp sql/pgtap.sql sql/pgtap--1.3.3.sql
cp sql/pgtap-core.sql sql/pgtap-core--1.3.3.sql
cp sql/pgtap-schema.sql sql/pgtap-schema--1.3.3.sql

11:12:02 postgres@ws-pgdev:/home/postgres/pgtap-1.3.3/ [sw] make install

GNUmake running against Postgres version 17.0, with pg_config located at /u01/app/postgres/product/17/db_0/bin

Makefile:186: To use pg_prove, TAP::Parser::SourceHandler::pgTAP Perl module
Makefile:187: must be installed from CPAN. To do so, simply run:
Makefile:188: cpan TAP::Parser::SourceHandler::pgTAP
mkdir -p '/u01/app/postgres/product/17/db_0/share/extension'
mkdir -p '/u01/app/postgres/product/17/db_0/share/extension'
mkdir -p '/u01/app/postgres/product/17/db_0/share/doc/extension'
/bin/sh /u01/app/postgres/product/17/db_0/lib/pgxs/src/makefiles/../../config/install-sh -c -m 644 .//pgtap.control '/u01/app/postgres/product/17/db_0/share/extension/'
/bin/sh /u01/app/postgres/product/17/db_0/lib/pgxs/src/makefiles/../../config/install-sh -c -m 644 .//sql/pgtap--0.90.0--0.91.0.sql .//sql/pgtap--0.91.0--0.92.0.sql .//sql/pgtap--0.92.0--0.93.0.sql .//sql/pgtap--0.93.0--0.94.0.sql .//sql/pgtap--0.94.0--0.95.0.sql .//sql/pgtap--0.95.0--0.96.0.sql .//sql/pgtap--0.96.0--0.97.0.sql .//sql/pgtap--0.97.0--0.98.0.sql .//sql/pgtap--0.98.0--0.99.0.sql .//sql/pgtap--0.99.0--1.0.0.sql .//sql/pgtap--1.0.0--1.1.0.sql .//sql/pgtap--1.1.0--1.2.0.sql .//sql/pgtap--1.2.0--1.3.0.sql .//sql/pgtap--1.3.0--1.3.1.sql .//sql/pgtap--1.3.1--1.3.2.sql .//sql/pgtap--1.3.2--1.3.3.sql .//sql/pgtap--1.3.3.sql .//sql/pgtap--unpackaged--0.91.0.sql .//sql/pgtap-core--1.3.3.sql .//sql/pgtap-core.sql .//sql/pgtap-schema--1.3.3.sql .//sql/pgtap-schema.sql .//sql/pgtap.sql .//sql/uninstall_pgtap.sql  '/u01/app/postgres/product/17/db_0/share/extension/'
/bin/sh /u01/app/postgres/product/17/db_0/lib/pgxs/src/makefiles/../../config/install-sh -c -m 644 .//doc/pgtap.mmd '/u01/app/postgres/product/17/db_0/share/doc/extension/'

As mentioned in the output of the previous command, we need to run some commands to be able to use pg_prove, which we are going to use later:

11:14:23 postgres@ws-pgdev:/home/postgres/pgtap-1.3.3/ [sw] cpan TAP::Parser::SourceHandler::pgTAP
Loading internal logger. Log::Log4perl recommended for better logging

CPAN.pm requires configuration, but most of it can be done automatically.
If you answer 'no' below, you will enter an interactive dialog for each
configuration option instead.

Would you like to configure as much as possible automatically? [yes] yes

Warning: You do not have write permission for Perl library directories.

To install modules, you need to configure a local Perl library directory or
escalate your privileges.  CPAN can help you by bootstrapping the local::lib
module or by configuring itself to use 'sudo' (if available).  You may also
resolve this problem manually if you need to customize your setup.

What approach do you want?  (Choose 'local::lib', 'sudo' or 'manual')
 [local::lib] sudo
Fetching with HTTP::Tiny:
https://cpan.org/authors/01mailrc.txt.gz
Reading '/home/postgres/.cpan/sources/authors/01mailrc.txt.gz'
............................................................................DONE
Fetching with HTTP::Tiny:
https://cpan.org/modules/02packages.details.txt.gz
Reading '/home/postgres/.cpan/sources/modules/02packages.details.txt.gz'
  Database was generated on Wed, 02 Apr 2025 08:29:02 GMT
..............
  New CPAN.pm version (v2.38) available.
  [Currently running version is v2.33]
  You might want to try
    install CPAN
    reload cpan
  to both upgrade CPAN.pm and run the new version without leaving
  the current session.


..............................................................DONE
Fetching with HTTP::Tiny:
https://cpan.org/modules/03modlist.data.gz
Reading '/home/postgres/.cpan/sources/modules/03modlist.data.gz'
DONE
Writing /home/postgres/.cpan/Metadata
Running install for module 'TAP::Parser::SourceHandler::pgTAP'
Fetching with HTTP::Tiny:
https://cpan.org/authors/id/D/DW/DWHEELER/TAP-Parser-SourceHandler-pgTAP-3.37.tar.gz
Fetching with HTTP::Tiny:
https://cpan.org/authors/id/D/DW/DWHEELER/CHECKSUMS
Checksum for /home/postgres/.cpan/sources/authors/id/D/DW/DWHEELER/TAP-Parser-SourceHandler-pgTAP-3.37.tar.gz ok
'YAML' not installed, will not store persistent state
Configuring D/DW/DWHEELER/TAP-Parser-SourceHandler-pgTAP-3.37.tar.gz with Build.PL
Created MYMETA.yml and MYMETA.json
Creating new 'Build' script for 'TAP-Parser-SourceHandler-pgTAP' version '3.37'
  DWHEELER/TAP-Parser-SourceHandler-pgTAP-3.37.tar.gz
  /usr/bin/perl Build.PL --installdirs site -- OK
Running Build for D/DW/DWHEELER/TAP-Parser-SourceHandler-pgTAP-3.37.tar.gz
Building TAP-Parser-SourceHandler-pgTAP
  DWHEELER/TAP-Parser-SourceHandler-pgTAP-3.37.tar.gz
  ./Build -- OK
Running Build test for DWHEELER/TAP-Parser-SourceHandler-pgTAP-3.37.tar.gz
t/source_handler.t .. ok
All tests successful.
Files=1, Tests=47,  1 wallclock secs ( 0.03 usr  0.00 sys +  0.08 cusr  0.12 csys =  0.23 CPU)
Result: PASS
  DWHEELER/TAP-Parser-SourceHandler-pgTAP-3.37.tar.gz
  ./Build test -- OK
Running Build install for DWHEELER/TAP-Parser-SourceHandler-pgTAP-3.37.tar.gz
Building TAP-Parser-SourceHandler-pgTAP
Installing /usr/local/man/man1/pg_prove.1p
Installing /usr/local/man/man1/pg_tapgen.1p
Installing /usr/local/share/perl/5.36.0/TAP/Parser/SourceHandler/pgTAP.pm
Installing /usr/local/man/man3/TAP::Parser::SourceHandler::pgTAP.3pm
Installing /usr/local/bin/pg_tapgen
Installing /usr/local/bin/pg_prove
  DWHEELER/TAP-Parser-SourceHandler-pgTAP-3.37.tar.gz
  sudo ./Build install  -- OK
11:15:03 postgres@ws-pgdev:/home/postgres/pgtap-1.3.3/ [sw] mkdir -p '/u01/app/postgres/product/17/db_0/share/extension'
11:15:05 postgres@ws-pgdev:/home/postgres/pgtap-1.3.3/ [sw] mkdir -p '/u01/app/postgres/product/17/db_0/share/extension'
mkdir -p '/u01/app/postgres/product/17/db_0/share/doc/extension'
11:15:10 postgres@ws-pgdev:/home/postgres/pgtap-1.3.3/ [sw] /bin/sh /u01/app/postgres/product/17/db_0/lib/pgxs/src/makefiles/../../config/install-sh -c -m 64                                4 .//pgtap.control '/u01/app/postgres/product/17/db_0/share/extension/'
11:15:18 postgres@ws-pgdev:/home/postgres/pgtap-1.3.3/ [sw] /bin/sh /u01/app/postgres/product/17/db_0/lib/pgxs/src/makefiles/../../config/install-sh -c -m 64                                4 .//sql/pgtap--0.90.0--0.91.0.sql .//sql/pgtap--0.91.0--0.92.0.sql .//sql/pgtap--0.92.0--0.93.0.sql .//sql/pgtap--0.93.0--0.94.0.sql .//sql/pgtap--0.94.0--0                                .95.0.sql .//sql/pgtap--0.95.0--0.96.0.sql .//sql/pgtap--0.96.0--0.97.0.sql .//sql/pgtap--0.97.0--0.98.0.sql .//sql/pgtap--0.98.0--0.99.0.sql .//sql/pgtap--0                                .99.0--1.0.0.sql .//sql/pgtap--1.0.0--1.1.0.sql .//sql/pgtap--1.1.0--1.2.0.sql .//sql/pgtap--1.2.0--1.3.0.sql .//sql/pgtap--1.3.0--1.3.1.sql .//sql/pgtap--1.                                3.1--1.3.2.sql .//sql/pgtap--1.3.2--1.3.3.sql .//sql/pgtap--1.3.3.sql .//sql/pgtap--unpackaged--0.91.0.sql .//sql/pgtap-core--1.3.3.sql .//sql/pgtap-core.sql                                 .//sql/pgtap-schema--1.3.3.sql .//sql/pgtap-schema.sql .//sql/pgtap.sql .//sql/uninstall_pgtap.sql  '/u01/app/postgres/product/17/db_0/share/extension/'
11:15:33 postgres@ws-pgdev:/home/postgres/pgtap-1.3.3/ [sw] /bin/sh /u01/app/postgres/product/17/db_0/lib/pgxs/src/makefiles/../../config/install-sh -c -m 64                                4 .//doc/pgtap.mmd '/u01/app/postgres/product/17/db_0/share/doc/extension/'
11:15:37 postgres@ws-pgdev:/home/postgres/pgtap-1.3.3/ [sw] pg_prove
No tests named and 't' directory not found at /usr/share/perl/5.36/App/Prove.pm line 522.

11:15:42 postgres@ws-pgdev:/home/postgres/pgtap-1.3.3/ [sw] pg_prove --version
pg_prove 3.37

You can check if pgTAP was installed properly using the following command:

10:24:09 postgres@ws-pgdev:/home/postgres/pgtap-1.3.3/ [sw] make installcheck

GNUmake running against Postgres version 17.0, with pg_config located at /u01/app/postgres/product/17/db_0/bin

Makefile:186: To use pg_prove, TAP::Parser::SourceHandler::pgTAP Perl module
Makefile:187: must be installed from CPAN. To do so, simply run:
Makefile:188: cpan TAP::Parser::SourceHandler::pgTAP
Using 89 parallel test connections
Rebuilding test/build/all_tests
Schedule changed to test/build/parallel.sch
cp `cat test/build/which_schedule` test/build/run.sch
echo "# +++ regress install-check in  +++" && /u01/app/postgres/product/17/db_0/lib/pgxs/src/makefiles/../../src/test/regress/pg_regress --inputdir=./ --bindir='/u01/app/postgres/product/17/db_0/bin'    --inputdir=test --max-connections=89 --schedule test/schedule/main.sch   --schedule test/build/run.sch
# +++ regress install-check in  +++
# using postmaster on Unix socket, port 5432
ok 1         - build                                     369 ms
...
ok 4         - hastap                                   1309 ms
# parallel group (35 tests):  matching istap do_tap moretap util performs_ok performs_within todotap check cmpok pg73 runjusttests roletap throwtap trigger usergroup enumtap policy runtests runnotests proctap fktap privs inheritance partitions valueset functap resultset aretap extension ownership ruletap pktap index unique
ok 5         + aretap                                   5911 ms
ok 6         + check                                    1558 ms
...
ok 39        + valueset                                 3784 ms
1..39
# All 39 tests passed.

Once installed, enable it in your database:

10:25:06 postgres@ws-pgdev:/home/postgres/pgtap-1.3.3/ [sw] psql
psql (17.0 dbi services build)
Type "help" for help.

postgres=# \c hybrid
You are now connected to database "hybrid" as user "postgres".
hybrid=# CREATE EXTENSION pgtap;
CREATE EXTENSION
hybrid=# \dx
                                                        List of installed extensions
        Name        | Version |    Schema    |                                          Description
--------------------+---------+--------------+-----------------------------------------------------------------------------------------------
 btree_gist         | 1.7     | training_app | support for indexing common datatypes in GiST
 orafce             | 4.14    | training_app | Functions and operators that emulate a subset of functions and packages from the Oracle RDBMS
 pg_stat_statements | 1.11    | public       | track planning and execution statistics of all SQL statements executed
 pg_trgm            | 1.6     | training_app | text similarity measurement and index searching based on trigrams
 pgcrypto           | 1.3     | training_app | cryptographic functions
 pgtap              | 1.3.3   | training_app | Unit testing for PostgreSQL
 plperl             | 1.0     | pg_catalog   | PL/Perl procedural language
 plpgsql            | 1.0     | pg_catalog   | PL/pgSQL procedural language
(8 rows)

Writing Your First pgTAP Tests

pgTAP provides a wide range of assertions for testing various database objects. Let’s go through some examples.

1. Testing a Function

Assume we have a function that verifies a password based on a specific pattern:

CREATE OR REPLACE FUNCTION training_app.f_password_verify (pv_password TEXT) RETURNS BOOLEAN
AS $$
  SELECT pv_password ~ '^(?=.{10,}$)(?=.*[a-z])(?=.*[A-Z])(?=.*[0-9])(?=.*\W).*$';
$$ LANGUAGE sql;

hybrid=# \df f_password_verify
                                List of functions
    Schema    |       Name        | Result data type | Argument data types | Type
--------------+-------------------+------------------+---------------------+------
 training_app | f_password_verify | boolean          | pv_password text    | func
(1 row)

To test this function with pgTAP:

hybrid=# SELECT plan(4);
 plan
------
 1..4
(1 row)

hybrid=# SELECT ok(training_app.f_password_verify('ValidPass1!'), 'Valid password should return true');
                    ok
------------------------------------------
 ok 1 - Valid password should return true
(1 row)

hybrid=# SELECT ok(NOT training_app.f_password_verify('short1!'), 'Too short password should return false');
                      ok
-----------------------------------------------
 ok 2 - Too short password should return false
(1 row)

hybrid=# SELECT ok(NOT training_app.f_password_verify('NoNumberPass!'), 'Password without a number should return false');
                          ok
------------------------------------------------------
 ok 3 - Password without a number should return false
(1 row)

hybrid=# SELECT ok(NOT training_app.f_password_verify('NoSpecialChar1'), 'Password without special character should return false');
                              ok
---------------------------------------------------------------
 ok 4 - Password without special character should return false
(1 row)

hybrid=# SELECT * FROM finish();
 finish
--------
(0 rows)

2. Testing Table Constraints

Consider the users table with the following schema:

hybrid=# \d users
                         Table "training_app.users"
  Column  |   Type    | Collation | Nullable |           Default
----------+-----------+-----------+----------+------------------------------
 id       | integer   |           | not null | generated always as identity
 username | text      |           | not null |
 password | text      |           | not null |
 created  | date      |           |          | now()
 validity | tstzrange |           |          |
Indexes:
    "users_pkey" PRIMARY KEY, btree (id)
    "i_username_trgm" gin (username gin_trgm_ops)
    "i_users_username" btree (username)
    "i_users_username_btree_partial" btree (created) WHERE created >= '2024-11-07'::date AND created < '2024-11-08'::date
Check constraints:
    "user_check_username" CHECK (username ~* '[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}'::text)
    "user_check_username_length" CHECK (length(username) <= 72)
Referenced by:
    TABLE "user_training" CONSTRAINT "fk_user_training_users" FOREIGN KEY (user_id) REFERENCES users(id)
    TABLE "users_history" CONSTRAINT "fk_users_history_user_id_users_id" FOREIGN KEY (user_id) REFERENCES users(id)
Policies:
    POLICY "policy_current_month" FOR SELECT
      TO role_app_read_only
      USING (((EXTRACT(month FROM created))::integer = (EXTRACT(month FROM now()))::integer))
Triggers:
    t_log_user_history BEFORE UPDATE ON users FOR EACH ROW EXECUTE FUNCTION tf_user_history()

create or replace function training_app.tf_user_history() returns trigger as
$$
begin
   insert into training_app.users_history ( user_id, username, password, created, validity)
                                  values ( old.id, old.username, old.password, old.created, old.validity);
   return new;
end;                            
$$ language plpgsql;

create trigger t_log_user_history
   before update on training_app.users
   for each row
   execute procedure training_app.tf_user_history();

To test the constraints, create a test file test_users_constraints.sql:

10:57:07 postgres@ws-pgdev:/u01/app/postgres/local/dmk/tests/ [sw] touch test_users_constraints.sql
10:57:37 postgres@ws-pgdev:/u01/app/postgres/local/dmk/tests/ [sw] cat test_users_constraints.sql

BEGIN;
SELECT plan(2);

-- Test CHECK constraint on username format
SELECT throws_like(
  $$INSERT INTO training_app.users (username, password) VALUES ('invalid_user', 'Password1!')$$,
  'new row for relation "users" violates check constraint "user_check_username"',
  'Invalid username should fail CHECK constraint'
);

-- Test CHECK constraint on username length
SELECT throws_like(
  $$INSERT INTO training_app.users (username, password) VALUES (repeat('a', 73), 'Password1!')$$,
  'new row for relation "users" violates check constraint "user_check_username"',
  'Username exceeding 72 characters should fail CHECK constraint'
);

SELECT * FROM finish();
ROLLBACK;

Running Tests

You can execute pgTAP tests using pg_prove, a command-line tool for running TAP tests. We are now going to test it with the file we just created, test_users_constraints.sql.

Run it with pg_prove:

11:45:14 postgres@ws-pgdev:/u01/app/postgres/local/dmk/tests/ [sw] pg_prove -d hybrid -U postgres -p 5432 test_users_constraints.sql
test_users_constraints.sql .. ok
All tests successful.
Files=1, Tests=2,  0 wallclock secs ( 0.05 usr  0.01 sys +  0.00 cusr  0.01 csys =  0.07 CPU)
Result: PASS

3. Testing Triggers

To verify that our trigger correctly logs changes to the users table, we check:

That the recorded historical data correctly reflects the old values before the update.
That an update on users triggers an insert into users_history.

13:43:55 postgres@ws-pgdev:/u01/app/postgres/local/dmk/tests/ [sw] cat test_trigger.sql
BEGIN;
SELECT plan(2);

-- Insert a test user with a valid email as username
INSERT INTO training_app.users (username, password) VALUES ('testuser@example.com', 'TestPassword123!');

-- Update the user's username (this should activate the trigger)
UPDATE training_app.users SET username = 'updateduser@example.com' WHERE username = 'testuser@example.com';

-- Check if the corresponding row is added to the users_history table
SELECT ok(
    (SELECT COUNT(*) FROM training_app.users_history WHERE user_id = (SELECT id FROM training_app.users WHERE username = 'updateduser@example.com')) > 0,
    'User history should be logged in users_history after update'
);

-- Check if the values in users_history match the old values (before the update)
SELECT is(
    (SELECT username FROM training_app.users_history WHERE user_id = (SELECT id FROM training_app.users WHERE username = 'updateduser@example.com') ORDER BY created DESC LIMIT 1),
    'testuser@example.com',
    'Username in user history should match the old (pre-update) value'
);

SELECT * FROM finish();
ROLLBACK;

Execute the test using:

13:58:38 postgres@ws-pgdev:/u01/app/postgres/local/dmk/tests/ [sw] pg_prove -d hybrid -U postgres -p 5432 test_trigger.sql
test_trigger.sql .. ok
All tests successful.
Files=1, Tests=2,  0 wallclock secs ( 0.04 usr  0.01 sys +  0.01 cusr  0.01 csys =  0.07 CPU)
Result: PASS

Benefits of Using pgTAP

Automated Testing: Helps maintain database integrity by catching errors early.
SQL-Based: No need for external scripting languages; tests are written in SQL.
Integration with CI/CD: Works with CI/CD pipelines to ensure database quality.
Comprehensive Assertions: Supports functions, constraints, indexes, views, and more.

Conclusion

pgTAP is a powerful tool for unit testing PostgreSQL databases. By incorporating it into your workflow, you can ensure that your database logic remains robust and reliable over time. Whether you’re testing functions, constraints, or triggers, pgTAP provides a structured and repeatable approach to database testing. You can find more information about pgTAP on the official website.
Do you use pgTAP in your projects? Let me know in the comments how it has helped you!

L’article Unit Testing PostgreSQL with pgTAP est apparu en premier sur dbi Blog.

How to add Column Filtering to Vuetify Data Table

Joan Frey — Mon, 25 Aug 2025 19:22:27 +0000

For a long time, my v-data-table-server implementation in the YaK only supported global search through a simple text input. While it was good enough for basic queries, I needed something more powerful—the ability to filter by individual columns using custom operators.

Here’s a walkthrough of how I added column filters with v-select and v-text-field embedded in v-menu components right inside the table headers.

The Problem

My users wanted to search not just by a global term but with more granularity—for example:

Show only items where provider contains “aws”
Filter out items with a specific state
View all entries with a name that contains “master”

The default Vuetify Data Table doesn’t support per-column filtering out of the box, so I needed to get creative, even if you can override the default filtering used with the search prop.

The Approach

Attach filter UI elements to each column header
Bind filters to a filters object keyed by column title
Apply the filters to the data using a computed property
Change the icon and icon color on the header to indicate active filters

I. Injecting the Filter UI into Headers

In the v-slot:[header.key], I added a v-menu that displays a filter operator dropdown and a text input:

Each filter’s state is stored in a reactive object:

const filters = ref>({});

II. Initializing Filters for Each Column

Once I loaded the table headers, I made sure to initialize the corresponding filters:

dataHeaders.value.forEach((header) => {
    if (!filters.value[header.title]) {
        filters.value[header.title] = { value: "", operator: "=" };
     }
});

This ensured every column could be filtered independently and to avoid errors related to empty/null data.

III. Applying the Filters to the Data

I used a computed property to transform the data on the fly:

const filteredData = computed(() => {
  return recordData.value.filter((item) => {
    return Object.entries(filters.value).every(([filterKey, filter]) => {
      if (!filter.value) return true;

      const matchingKey = Object.keys(item).find((key) => {
        const normalizedFilterKey = filterKey.toLowerCase();
        return key.toLowerCase() === normalizedFilterKey ||
               key.toLowerCase().endsWith("name") &&
               key.toLowerCase().startsWith(normalizedFilterKey);
      });

      if (!matchingKey) return true;

      const itemValue = String(item[matchingKey] || "").toLowerCase();
      const filterValue = filter.value.toLowerCase();

      return filter.operator === "="
        ? itemValue.includes(filterValue)
        : !itemValue.includes(filterValue);
    });
  });
});

This filtering happens client-side after the data is fetched.

IV. Adding UX Details

A few extra things I did for better UX:

Colored icons: A regular filter icon shows when a filter is active.
Clear filters button (optional): Could be added for each column.
Auto-focus: Focuses on the text field when the menu opens.

What’s Next?

After adding this new feature, the sort indicator next to each column header disappeared—although sorting still works as expected. I haven’t had time to look into it yet, but I’d like to find a solution. If you happen to figure it out, feel free to share it in the comments!

Where to find the complete code ?

You can check out the complete and latest code in the open-source project on GitLab:
👉 https://gitlab.com/yak4all/yak_frontend/yak_ui
The relevant logic lives in the YakGrid.vue component.

Conclusion

By embedding filter controls directly into column headers and managing state reactively, I was able to turn a basic Vuetify data table into a much more powerful data exploration tool.

Let me know if you try this pattern or come up with improvements—I’m always up for iterating! You can also suggest changes or contribute directly on the GitLab repo of the Yak project.

L’article How to add Column Filtering to Vuetify Data Table est apparu en premier sur dbi Blog.

Exploring FlyonUI with Vue.js: A Fresh and Evolving Library

Joan Frey — Thu, 31 Oct 2024 16:13:43 +0000

As someone who has relied on Vuetify for most of my Vue.js projects, I’m always on the lookout for new libraries that can streamline development and keep designs modern. Recently, a new UI library called FlyonUI was released, and I decided to give it a spin. FlyonUI is a Tailwind CSS Components Library. To my surprise, it turned out to be not only lightweight but also incredibly easy to set up and customize. In this post, I’ll walk you through the basics of integrating FlyonUI with Vue.js.

Why I’m Interested About FlyonUI

Released just a few weeks ago, FlyonUI promises a fresh approach to Vue UI development. FlyonUI is an open-source (I love open-source projects) Tailwind CSS Components Library with semantic classes and powerful JS plugins. It’s designed to be intuitive, flexible and to have universal framework compatibility (Vuejs, React, Angular, …). I like their website and the documentation seems very complete, not even mentioning the fact that there is already more than 78 available components and 800 examples.

Prerequisites

A basic understanding of Vue.js
Node.js and npm installed
Vue.js running application set up with Tailwind CSS (https://tailwindcss.com/docs/guides/vite#vue)

I. Install FlyonUI

I’m going to use the application I created for my previous blog, which you can find here: https://www.dbi-services.com/blog/vue-creating-an-awesome-parallax-effect-with-swiper/. To add FlyonUI, open your terminal in the project’s root folder and run:

npm install flyon-ui

II. Configure FlyonUI in Your Vue App

Now that FlyonUI is installed, let’s import it to make its components available across the app. Open tailwind.config.js and edit the file with the following code:

/** @type {import('tailwindcss').Config} */
export default {
  content: [
    "./index.html",
    "./src/**/*.{vue,js,ts,jsx,tsx}",
    "./node_modules/flyonui/dist/js/*.js",
  ],
  theme: {
    extend: {},
  },
  plugins: [
    require('flyonui'),
    require('flyonui/plugin')
  ],
}

Open main.ts and add the following code:

import "flyonui/flyonui";

III. Add a reinitialization helper

Add code in your route file (./src/router/index.ts) to reinitialize components each time the page is refreshed.

import { createRouter, createWebHistory } from 'vue-router'
import HomeView from '../views/HomeView.vue'

import { type IStaticMethods } from "flyonui/flyonui";
declare global {
  interface Window {
    HSStaticMethods: IStaticMethods;
  }
}

const router = createRouter({
  history: createWebHistory(import.meta.env.BASE_URL),
  routes: [
    {
      path: '/',
      name: 'home',
      component: HomeView
    },
    {
      path: '/about',
      name: 'about',
      // route level code-splitting
      // this generates a separate chunk (About.[hash].js) for this route
      // which is lazy-loaded when the route is visited.
      component: () => import('../views/AboutView.vue')
    }
  ]
})

router.afterEach((to, from, failure) => {
  if (!failure) {
    setTimeout(() => {
      window.HSStaticMethods.autoInit();
    }, 100)
  }
});

export default router

IV. Using FlyonUI Components

FlyonUI components are simple and straightforward, so let’s jump into using a few popular ones.

Example 1: Button

Now you can use FlyonUI components throughout your Vue application. Here’s an example of using a FlyonUI responsive button:

Example 2: Add a card component

We’ll make it simple and place the card structure below the title of our swiper slide. You can add this code to your component:

        
          
            
          
          
            Card title
            
              Nike Air Max is a popular line of athletic shoes that feature
              Nike's signature Air cushioning technology in the sole.

Example 3: Use radio buttons to control CSS theme

FlyonUI offers a range of pre-built themes that make it easy to customize the look and feel of your app. Each theme provides a cohesive color scheme applied across all FlyonUI components. To use a theme, simply add its name to your tailwind.config.js.

/** @type {import('tailwindcss').Config} */
export default {
  content: [
    "./index.html",
    "./src/**/*.{vue,js,ts,jsx,tsx}",
    "./node_modules/flyonui/dist/js/*.js",
  ],
  theme: {
    extend: {},
  },
  flyonui: {
    themes: ["light", "dark", "gourmet", "corporate", "luxury", "soft"]
  },
  plugins: [
    require('flyonui'),
    require('flyonui/plugin')
  ],
}

Now let’s add the radio buttons onto our slide and verify that the theme of our card is changing:

Well, without big surprise, this is working as expected, simple and easy to use. I like how the Gourmet theme is even applying rounded class to our buttons.

V. Advanced Features

FlyonUI also offers advanced components, such as tables, charts, and timeline, to build complex interfaces quickly. Refer to the FlyonUI documentation for detailed customization options, methods, and event handling for each component.

Conclusion

FlyonUI seems like an excellent library for Vue.js developers looking to create modern, responsive, and interactive UIs without a significant performance hit. In this guide, we’ve covered the basics of integrating FlyonUI into your Vue project and using several key components.

Still, FlyonUI is a fresh and evolving library, and it’s clear that the team is still working to expand its capabilities. As you may notice on the official website, some sections are marked as “coming soon.” This means there’s more to look forward to as the library grows and introduces new features. So, stay tuned for updates, enjoy exploring the current components, a big thanks to the developers and have fun testing out FlyonUI!

L’article Exploring FlyonUI with Vue.js: A Fresh and Evolving Library est apparu en premier sur dbi Blog.

Vue – Creating an Awesome Parallax Effect with Swiper

Joan Frey — Thu, 31 Oct 2024 07:06:31 +0000

Adding a parallax effect to your website brings a smooth, professional feel, making it more interactive and visually appealing. With Swiper for Vue, you can easily implement parallax effects for a slideshow. In this guide, we’ll create a parallax effect using Vue 3 and explore additional features to enhance this effect.

I. Create a New Vue 3 Project

Let’s start from scratch to create a Vue 3 app and set up Swiper to create a parallax slider. Here’s a step-by-step guide to set up the environment and build an awesome parallax effect.

blogger@PC MINGW64 ~/Documents/Blogs/parallax-vue-awesome
$ npm init vue@latest my-parallax-app
Need to install the following packages:
  create-vue@3.11.2
Ok to proceed? (y) y

Vue.js - The Progressive JavaScript Framework

√ Add TypeScript? ... No / Yes
√ Add JSX Support? ... No / Yes
√ Add Vue Router for Single Page Application development? ... No / Yes
√ Add Pinia for state management? ... No / Yes
√ Add Vitest for Unit Testing? ... No / Yes
√ Add an End-to-End Testing Solution? » No
√ Add ESLint for code quality? ... No / Yes
√ Add Prettier for code formatting? ... No / Yes
√ Add Vue DevTools 7 extension for debugging? (experimental) ... No / Yes

Scaffolding project in C:\Users\blogger\Documents\Blogs\parallax-vue-awesome\my-parallax-app...

Done. Now run:

  cd my-parallax-app
  npm install
  npm run format
  npm run dev

II. Install Vue Swiper

Next, install Swiper:

npm install swiper

III. Set Up the Project Structure

Inside the project, let’s create a new component for our parallax slider. We’ll place this in the components folder for organization.

Create a file named AwesomeParallax.vue in the src/components folder:

src
└── components
    └── ParallaxSlider.vue

I also added the pictures that I’ll be using for the Parallax into the assets directory (Pictures are mine 📷). In the end, you should have at least the following structure:

src
├── assets
│   ├── parallax1.jpg
├── components
│   └── AwesomeParallax.vue
├── views
│   └── HomeView.vue
└── App.vue

IV. Create the Parallax Slider Component

Add the following code to AwesomeParallax.vue. This component will import the required Swiper modules for navigation, pagination, and the parallax effect. You can find the official documentation from Swiper here: https://swiperjs.com/demos#parallax.

To achieve a full-screen parallax effect, I used display: contents; on the #app element. This allows the background to stretch across the viewport without being constrained by the parent element’s dimensions.

However, setting the .parallax-bg width to 130% can cause height issues, as the image might not scale proportionately, leading to excess height. Still, this is needed to achieve the parallax effect in our case, so it means that you need to choose your background image knowing that it is going to be cropped.

V. Use the Component in HomeView.vue

Now that we’ve created the AwesomeParallax component, we can add it to HomeView.vue to display it on the main page.

Open src/HomeView.vue and replace its content with the following:

Then open src/App.vue and replace its content with the following:

VI. Run the application

With everything in place, you can start the development server to see the parallax effect in action:

npm run dev

Navigate to http://localhost:3000 in your browser to see your parallax slider. You should see a smoothly animated slider with a parallax background and Swiper’s navigation and pagination controls.

VII. Enhancing the Parallax Effect

Now that you have a working parallax slider, you can experiment with additional features, such as:

Slide Transition Duration: Customize the transition duration for smoother animations by editing the :speed prop adjustments on the swiper component.
Center the text: To center the text, title, and subtitle in the middle of your screen, you can use flexbox properties on the .swiper-slide elements. Here’s how you can adjust your CSS:

.swiper-slide {
  font-size: 18px;
  color: #fff;
  box-sizing: border-box;
  padding: 40px 60px;
  
  /* Flexbox properties for centering */
  display: flex;
  flex-direction: column; /* Stack elements vertically */
  justify-content: center; /* Center vertically */
  align-items: center; /* Center horizontally */
}

.swiper-slide .title {
  font-size: 41px;
  font-weight: 300;
  text-align: center; /* Center text */
}

.swiper-slide .subtitle {
  font-size: 21px;
  text-align: center; /* Center text */
}

.swiper-slide .text {
  font-size: 14px;
  max-width: 400px;
  line-height: 1.3;
  text-align: center; /* Center text */
}

Adding Image Gradients: Layering gradients over the background images to create depth. To apply a gradient overlay on top of your background image while keeping the image itself, you can modify the .parallax-bg class to use both the background image and the gradient. Here’s how you can do it:

.parallax-bg {
  position: absolute;
  left: 0;
  top: 0;
  width: 130%;
  height: 100%;
  background-image:
    linear-gradient(to right, rgba(255, 128, 0, 0.3), rgba(0, 0, 0, 0.9)),
    /* Gradient overlay */ url('../assets/parallax1.jpg'); /* Background image */
  background-size: cover;
  background-position: center;
}

Vertical parallax: Add the props :direction="'vertical'" to create a vertical parallax. In this case, don’t forget to edit the class parallax-bg to adjust the height and width property as needed:



.parallax-bg {
  position: absolute;
  left: 0;
  top: 0;
  width: 100%;
  height: 130%;
  background-image:
    linear-gradient(to right, rgba(255, 128, 0, 0.3), rgba(0, 0, 0, 0.9)),
    /* Gradient overlay */ url('../assets/parallax1.jpg'); /* Background image */
  background-size: cover;
  background-position: center;
}

Conclusion

This guide demonstrated how to set up a Vue 3 application with a parallax effect, creating visually engaging user experiences. You can further enhance the parallax by adjusting animation speeds, or even incorporating dynamic slide content. This setup is highly flexible for building beautiful, interactive sliders in Vue apps. Don’t forget to check Swiper API to learn more about what is doable with this component.

L’article Vue – Creating an Awesome Parallax Effect with Swiper est apparu en premier sur dbi Blog.

PostgreSQL 17: Enhancing JSON Support for Web Developers

Joan Frey — Fri, 18 Oct 2024 12:47:21 +0000

PostgreSQL 17 introduces new features for working with JSON data. These features align PostgreSQL more closely with the SQL/JSON standard and improve the developer experience when dealing with semi-structured data. In this blog, we will explore the new JSON capabilities by walking through real-world examples using a table with a jsonb column.

The JSON Data Challenge for Web Developers

When building modern web applications, it’s common to handle JSON data—whether it’s from API responses, user data, or configuration files. PostgreSQL has supported JSON for years, and with the release of PostgreSQL 17, working with JSON becomes even more streamlined.
Let’s start by creating a table to store user data in a jsonb column and use it to demonstrate the new features.

I. Creating a Table with a `jsonb` Column

To demonstrate PostgreSQL 17’s new features, let’s create a simple table called users that stores user information in a jsonb column.

postgres=# CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    name TEXT NOT NULL,
    profile JSONB
);
CREATE TABLE

This table has three columns:

id: A unique identifier for each user.
name: The user’s name.
profile: A jsonb column that stores various information about the user, such as their age, preferences, and settings.

Inserting Data

Now, we’ll insert some user data in JSON format into the profile column.

postgres=# INSERT INTO users (name, profile)
VALUES
('John Doe', '{"age": 30, "preferences": {"newsletter": true, "theme": "dark"}}'),
('Jane Smith', '{"age": 25, "preferences": {"newsletter": false, "theme": "light"}}'),
('Alice Brown', '{"age": 35, "preferences": {"newsletter": true, "theme": "dark"}, "address":                        {"city": "Paris", "country": "France"}}');
INSERT 0 3

We now have three users with different JSON profiles, each containing details such as age, preferences for newsletters and themes, and, in Alice’s case, an address.

postgres=# select * from users;
 id |    name     |                                                       profile
----+-------------+----------------------------------------------------------------------------------------------------------------------
  1 | John Doe    | {"age": 30, "preferences": {"theme": "dark", "newsletter": true}}
  2 | Jane Smith  | {"age": 25, "preferences": {"theme": "light", "newsletter": false}}
  3 | Alice Brown | {"age": 35, "address": {"city": "Paris", "country": "France"}, "preferences": {"theme": "dark", "newsletter": true}}
(3 rows)

II. Using PostgreSQL 17’s New JSON Features

With the table set up and populated, let’s explore the new JSON-related features in PostgreSQL 17, such as JSON_TABLE, SQL/JSON query functions, and enhanced jsonpath expressions.

a. `JSON_TABLE`: Converting JSON into Tabular Format

The JSON_TABLE function allows us to transform our jsonb data into rows and columns. This is particularly useful when we want to extract structured data from JSON documents stored in a relational database.

Let’s extract the age and theme from the profile column and convert it into a tabular format:

postgres=# SELECT *
FROM JSON_TABLE(
    (SELECT profile FROM users WHERE name = 'Alice Brown'),
    '$.preferences' COLUMNS (theme TEXT PATH '$.theme', newsletter BOOLEAN PATH '$.newsletter')
) AS jt;
 theme | newsletter
-------+------------
 dark  | t
(1 row)

Here, we extracted Alice Brown’s theme preference and whether she subscribes to the newsletter from her profile. The $ symbol is essential for this operation.

Here’s a breakdown of how this works:

$.preferences: This part refers to the preferences key at the root of the JSON document. Once this key is selected, it acts as the root for the columns inside the preferences object.
Inside the COLUMNS clause:
- $.theme: Here, $ refers to the root of the preferences object, not the root of the entire JSON document. So, it looks for the theme key inside the preferences object.
- $.newsletter: Similarly, $ here refers to the root of the preferences object, and it looks for the newsletter key within that object.

In this context, the $ in the COLUMNS clause is “relative” to the object you are working with, which in this case is preferences. This is a key concept in using the JSON_TABLE function and jsonpath expressions in PostgreSQL—$ adapts based on the context of the object you’re working with at that stage of the query.

b. jsonb_build_object: Creating JSON Data in Queries

PostgreSQL allows you to create JSON directly from SQL expressions, making it easier to work with JSON data dynamically. This function exists since PostgreSQL 12, but I believe that it makes sense to present it here.

Let’s construct some JSON data based on our users table:

postgres=# SELECT name,
       jsonb_build_object(
           'age', profile->>'age',
           'theme', profile->'preferences'->>'theme'
       ) AS constructed_json
FROM users;
    name     |        constructed_json
-------------+---------------------------------
 John Doe    | {"age": "30", "theme": "dark"}
 Jane Smith  | {"age": "25", "theme": "light"}
 Alice Brown | {"age": "35", "theme": "dark"}
(3 rows)

This query dynamically builds a JSON object from the age and theme fields in the profile column.

c. SQL/JSON Query Functions: Simplifying JSON Queries

PostgreSQL 17 introduces several new SQL/JSON query functions, such as JSON_EXISTS, JSON_QUERY, and JSON_VALUE. These functions allow you to query and extract values from JSON documents more efficiently.

Example: Checking for the Existence of a Key

Let’s check if the address key exists in John Doe’s profile:

postgres=# SELECT JSON_EXISTS(profile, '$.address')
FROM users
WHERE name = 'John Doe';
 json_exists
-------------
 f
(1 row)

Example: Extracting Scalar Values

We can use the JSON_VALUE function to extract specific values from a JSON document. For example, let’s extract the city from Alice’s address:

postgres=# SELECT JSON_VALUE(profile, '$.address.city')
FROM users
WHERE name = 'Alice Brown';
 json_value
------------
 Paris
(1 row)

The JSON_VALUE function simplifies the process of extracting individual scalar values from JSON documents. Use JSON_VALUE() only when you expect the extracted value to be a single SQL/JSON scalar. Attempting to retrieve multiple values will result in an error. If the extracted value might be an object or an array, opt for the JSON_QUERY function instead.

Example: extract specific values from a JSON document

The JSON_QUERY function in PostgreSQL provides a powerful way to extract data from JSONB columns using path expressions. By leveraging its various options, developers can customize the output format, handle errors gracefully, and work efficiently with JSON data stored in PostgreSQL. Now, let’s use JSON_QUERY to extract the preferences for each user. We want to get the preferences object for each user’s profile.

postgres=# SELECT
    name,
    JSON_QUERY(profile, '$.preferences') AS user_preferences
FROM users;
    name     |            user_preferences
-------------+-----------------------------------------
 John Doe    | {"theme": "dark", "newsletter": true}
 Jane Smith  | {"theme": "light", "newsletter": false}
 Alice Brown | {"theme": "dark", "newsletter": true}
(3 rows)

d. Enhanced `jsonpath` Expressions

PostgreSQL improves its support for jsonpath expressions, enabling more advanced queries. You can now cast JSON values into native PostgreSQL types, such as integers or booleans.

Let’s extract Jane Smith’s age and cast it as an integer:

postgres=# SELECT JSON_VALUE(profile, '$.age' RETURNING INT) AS age_int
FROM users
WHERE name = 'Jane Smith';
 age_int
---------
      25
(1 row)

This query demonstrates how you can convert JSON data into a native PostgreSQL type using the RETURNING clause.

Conclusion

PostgreSQL 17 brings powerful new features for working with JSON data, making it easier for web developers to query and manipulate JSON in a database. We explored how JSON_TABLE, new SQL/JSON functions, and enhanced jsonpath expressions help convert JSON data into a more usable format, extract values, and even handle complex queries.

These new tools make working with semi-structured data simpler, allowing you to build more efficient and flexible web applications. If you often work with JSON in PostgreSQL, upgrading to version 17 will streamline your workflow and enhance your capabilities.

Stay tuned and don’t forget to check PostgreSQL 17 Release note https://www.postgresql.org/about/news/postgresql-17-released-2936/

L’article PostgreSQL 17: Enhancing JSON Support for Web Developers est apparu en premier sur dbi Blog.

Creating a Global and Custom Snackbar in Vue 3 Using Vuetify and Pinia

Joan Frey — Thu, 17 Oct 2024 16:25:20 +0000

In this post, we’ll walk through how to implement a Global and Custom Snackbar in Vue 3 with Vuetify and Pinia for state management. We’ll set it up so that you can easily trigger the Snackbar from any component in your app.

I. Set Up Your Vue App with Vuetify CLI

To start, we’ll create our Vue 3 app using the Vuetify CLI. This CLI simplifies the process of integrating Vuetify and sets up your project structure.

npm create vuetify@latest

Follow the prompts to choose your project settings and install dependencies. As we are going to use Pinia, don’t forget to choose this option. I decided to keep all the generated files to keep it simple and straightforward, but you can clean the code that was automatically generated. I just cleaned the component HelloWorld.vue, that now looks like this:

II. Set Up the Pinia Store for Snackbar

Next, create a Pinia store (SnackbarStore.ts) to handle the state of the Snackbar (open/close, message, status, etc.). This will allow you to trigger the Snackbar from anywhere in the app.

// SnackbarStore.ts inside stores directory

import { defineStore } from 'pinia'
import { Ref, ref } from 'vue'

export interface CustomAction {
  actionName: string;
  link: any;
}

export const useSnackbarStore = defineStore('SnackbarStore', () => {
  const isOpen = ref(false)
  const message = ref('')
  const status = ref('error')
  const customActions: Ref = ref([])

  const showSnackbar = (msg: string, st: string, optionalActions?: CustomAction[]) => { // Change to accept an array
    message.value = msg
    status.value = st
    isOpen.value = true
    if (optionalActions) {
      customActions.value.push(...optionalActions) // Spread the array into customActions
    }
    if (status.value.toLowerCase() === 'success') {
      setTimeout(() => {
        closeSnackbar()
      }, 2500)
    }
  }

  function closeSnackbar () {
    isOpen.value = false
    customActions.value.splice(0, customActions.value.length) // Clear custom actions
  }

  return { isOpen, message, status, showSnackbar, closeSnackbar, customActions }
})

III. Create the Snackbar Component

We’ll create a reusable Snackbar component (GlobalSnackbar.vue) that can be used anywhere in your application.

IV. Modify `App.vue` to Include the Global Snackbar

To ensure the Snackbar is available across your entire app, import it into App.vue and pass the required props from the SnackbarStore.

This step is essential to making sure the GlobalSnackbar component is part of the root layout and is always available to display messages.

V. Using the Snackbar in a Component

Now, let’s demonstrate how to trigger the Snackbar from any component. For this example, we’ll use a button that, when clicked, shows a success message with an optional action. We can edit HelloWord.vue with following code.

VI. Testing the Snackbar

Once everything is in place, try clicking the button to trigger the Snackbar. You should see the Snackbar with a success message and an optional action button appear at the bottom of the screen. And Logically, if you click on one of the custom action buttons, it will redirect you properly, but you will obviously see a blank page, as the view doesn’t exist.

Conclusion

In this post, we created a global and custom Snackbar component in Vue 3 using Vuetify and Pinia. We set it up so that it can be easily triggered from anywhere in the app. By managing its state with Pinia, we keep the logic centralized and clean, making the Snackbar reusable across the entire application.

Feel free to adjust the styling and functionality of the Snackbar to suit your needs. Now you have a flexible, easy-to-use notification system for your Vue 3 project!

L’article Creating a Global and Custom Snackbar in Vue 3 Using Vuetify and Pinia est apparu en premier sur dbi Blog.