MongoDB DMK 2.5.0 : Sharding support and new features

In this blog, I will present the new features introduced with the latest release (2.5.0) of the MongoDB DMK.

The DMK is a set of standardized tools that aims at easing the work of DBAs by having dbi’s best practices embedded in common scripts across all the database servers of an organization.

dbi services provides the DMK (Database Management Kit) to its customers for multiple technologies: Oracle, Postgres, MongoDB, etc. This toolkit is provided free of charge to all clients who work with dbi services on a consulting project.

New features of the MongoDB DMK in 2.5.0

Sharding support

The biggest addition in DMK 2.5.0 is full sharding support. DMK can now create, start, stop, restart, and display the status of every component of a sharded MongoDB cluster: config server replica sets, shard replica sets, and mongos routers.

Internally, instance management was refactored into a class hierarchy: a MongoInstance base class handles what is common to all MongoDB processes (URI construction, mongosh command execution, status gathering), and MongodInstance / MongosInstance extend it for their respective types. From a day-to-day usage perspective, this is transparent: you use the same dmk_db_ctl.py and dmk_status.py commands for all instance types.

A few behaviours are specific to mongos: stopping a mongos router uses SIGTERM instead of the --shutdown command (which mongos does not support), and trying to back up a mongos instance now raises an explicit error, since mongos processes hold no data. In a sharded configuration, the underlying shards should be backed up instead.

New configuration templates for each sharding component type are available in $DMK_HOME/templates/dbcreate/, in both forked (no systemd) and non-forked (systemd-managed) variants. A dedicated systemd unit file template (mongos.service.template) was also added for mongos routers.

Three new generic JavaScript scripts in $DMK_HOME/js/ handle the initialization steps:

• csrs_initiate.js — initiate a config server replica set
• rs_initiate.js (updated) — initiate a shard or standalone replica set, now with optional arbiter support
• sh_addshards.js — register shards in the cluster via a mongos router

All three receive their parameters via the mongosh --eval flag, keeping them reusable across any cluster topology:

# Initiate the config server replica set
mongosh 127.0.0.1:27017 \
  --eval "var rsName='csrs'; var rsHosts=['127.0.0.1:27017','127.0.0.1:27018','127.0.0.1:27019'];" \
  $DMK_HOME/js/csrs_initiate.js

# Initiate shard 1 with an arbiter
mongosh 127.0.0.1:27020 \
  --eval "var rsName='rs1'; var rsHosts=['127.0.0.1:27020','127.0.0.1:27021']; var arbiterHost='127.0.0.1:27022';" \
  $DMK_HOME/js/rs_initiate.js

# Add all shards from a mongos router
mongosh 127.0.0.1:27030 \
  --eval "var shards=['rs1/127.0.0.1:27020,127.0.0.1:27021','rs2/127.0.0.1:27023,127.0.0.1:27024','rs3/127.0.0.1:27026,127.0.0.1:27027'];" \
  $DMK_HOME/js/sh_addshards.js

New --check-status option in dmk_db_ctl.py

The dmk_db_ctl.py script now accepts a --check-status flag. When provided, it polls the instance after the start, stop, or restart command to confirm the instance has reached the expected state. This was the default behaviour until now, which led to unwanted errors when setting up DMK.

dmk_db_ctl.py -i mdb01 -a start --check-status
dmk_db_ctl.py -i mdb01 -a stop --check-status
dmk_db_ctl.py -i mdb01 -a restart --check-status

Without this flag, the script returns immediately after issuing the command. It should be noted that when using the restart option, the status will be checked after stopping the instance. Otherwise, if the instance is still running when the start was issued, the restart could silently succeed without actually restarting anything.

Changes made to dmk_status.py

Colored status output. STARTED is now displayed in green, STOPPED in red — making it easier to spot unhealthy instances at a glance. To disable colors (for log files, CI pipelines, or terminals that don’t support ANSI), set COLORED_STATUS=no in your local configuration file (dmkl) by uncommenting the following line:

# var::COLORED_STATUS::=::nowarn::no::  # Disable colored status output in dmk_status.py (yes/no)

CSV output. The table view now supports CSV export via the --csv option, useful for piping output to other tools or exporting to a file:

# Export status as semicolon-delimited CSV
> dmk_status.py -t -a --csv ";"
Instance;Status;Version;Port;Bind IPs
mdb01;STARTED;8.0.26;27417;127.0.0.1
mdb02;STARTED;8.0.26;27418;127.0.0.1
mdbcfg01;STARTED;8.0.26;27017;127.0.0.1
mdbmgs01;STARTED;8.0.26;27030;127.0.0.1

The session view (sess) supports the same option.

Process-only check. The new -p / --process option performs a quick process presence check without connecting to MongoDB. This is faster and does not require credentials, making it useful for lightweight health checks:

dmk_status.py -t -a --process

Improved version display. The -mv option now shows all installed MongoDB versions side by side in a single table instead of separate blocks, making it much easier to compare binary versions across multiple installations.

Session display refactoring. Session data gathering was moved into the instance class, and the display logic (including filtering and formatting) was centralized in dmk_status.py. This also fixes the --include-inactive flag, which was not working correctly in the previous release.

New update_mongo_conf.py script

The new update_mongo_conf.py script modifies MongoDB configuration file parameters from the command line using dot-notation for nested keys, without manually editing YAML:

# Enable TLS on an existing instance
update_mongo_conf.py mdb01 net.tls.mode requireTLS
update_mongo_conf.py mdb01 net.tls.certificateKeyFile /u01/app/mongodb/admin/mdb01/secret/server.pem

New set_uri.py script

The new set_uri.py script generates and persists a MONGO_URI in the local DMK configuration for a given instance. It supports both standalone and replica set connection strings:

# Standalone
set_uri.py mdb01 sa 127.0.0.1:27017

# Replica set
set_uri.py mdbrs01 rs 127.0.0.1:27411,127.0.0.1:27412,127.0.0.1:27413 rs1

The resulting URI is stored in the local configuration file and available as $MONGO_URI in the instance context.

Improvements to dmk_dbcreate.py

dmk_dbcreate.py now reads the [instance_name] section from the local DMK configuration file when creating an instance, so instance-specific variables defined in your local config are picked up automatically. Empty values for MONGO_BACKUP, MONGO_JOURNAL, and MONGO_LOG are also now accepted — directories are only created for paths that are explicitly set.

The storageEngine key was removed from all YAML configuration templates. WiredTiger is the only storage engine available in MongoDB Community Edition, making the field redundant.

A new mongod.service.template_fork systemd unit (Type=simple, delegating start/stop to dmk_db_ctl.py) was also added, alongside the existing mongod.service.template that runs mongod in the foreground. dmk_dbcreate.py generates the mongod unit for mongod instances and the mongos unit for routers.

Installation improvements

Auto-creation of .bash_profile. If the .bash_profile of the mongodb user does not contain the DMK block, DMK now creates it automatically from the template on first sourcing. If the file already exists with a DMK block, only that block is updated in place (with a backup created first). This removes a manual step from first-time installation.

DMK temp files. The dmk_env.sh file used to source instance environments is now written to $DMK_HOME/tmp/ rather than /tmp, keeping temporary DMK files within the DMK directory tree.

Security improvements

The default bindIp in all DMK configuration templates was changed from 0.0.0.0 to 127.0.0.1. This is a safer default for freshly created instances that limits network exposure from the start.

A new MONGO_TLS_CERTIFICATE_KEY_FILE environment variable allows specifying a custom client certificate for DMK connections. This is useful for instances secured with serverAuth-only TLS, where the default certificate path may not be the right client certificate to use.

New TLS templates for replica sets (mongo_conf_rs_tls.yaml, mongo_conf_rs_tls_fork.yaml, mongo_conf_rs_keyfile.yaml) were also added.

Aliases and environment variables added in DMK 2.5.0

New aliases in DMK 2.5.0:

• mglogrotate: rotate the MongoDB log file (db.adminCommand({ logRotate: 1 })).
• certrotate: rotate the TLS certificate of an instance (db.rotateCertificates()).

New environment variables:

• COLORED_STATUS: set to no to disable colored status output in dmk_status.py. Enabled by default.
• MONGO_TLS_CERTIFICATE_KEY_FILE: custom client TLS certificate path for DMK connections.
• MONGO_URI: the MongoDB connection URI, now easily managed with set_uri.py.
• INSTANCE_NAMING_CONVENTION: the regular expression used to validate instance names. This variable is now optional — if not set, no naming convention is enforced.

Bug fixes

A few notable bugs were fixed in this release:

• Quoted values in the MongoDB configuration file (viewed with vic / dmkc) could cause the DMK environment to fail loading due to unescaped double quotes. This is now handled correctly.
• Restart could silently succeed without actually restarting the instance if it was still running. The stop phase now always waits for the instance to be fully stopped first.
• --include-inactive flag in the session view was not working correctly and has been fixed.

Upgrading from an earlier version of DMK

If you are upgrading from DMK 2.4.0, replace the DMK folder with the new one:

cd /u01/app/mongodb/local
mv dmk .dmk_old
unzip /path/to/dmk_mongodb-2.5.0.zip
dmk

If you are upgrading from DMK 2.3.x or earlier, read the release notes of DMK 2.4.0 first.