Morgan Patou, auteur/autrice sur dbi Blog

M-Files BD – Trend widgets: line and area

Morgan Patou — Sat, 13 Jun 2026 16:34:04 +0000

This is the second widget post of the series, focused on the two trend widgets: line and area. The previous one covered scalar widgets (Post 4a). Since these two widgets are closely related, it made sense to bundle them together in the same post. Both surface a single value per category or per time bucket, plotted across a horizontal axis. They are best used with the groupByDateBucket aggregation to show evolution over time, but they also accept groupByProperty.

1. Why these two are in the same post

The wire format and the rendering pipeline for line and area are nearly identical. They produce the same JSON on the server, they use the same client code path, they accept the same aggregation types, the same reducers, the same display options. The only meaningful difference is visual: area fills the region under the line with a translucent color, while line leaves it transparent.

In other words: choose area when you want to emphasize cumulative volume or magnitude, choose line when you want to emphasize the trajectory. Both communicate the same thing, it’s more about what you want to focus on.

2. A first example: monthly revenue trend

Here is the canonical example: total invoice amount per month for the ongoing year, displayed as a line chart.

{
  "id": "",
  "title": "Monthly revenue",
  "type": "line",
  "gridColumnSpan": 12,
  "gridRowSpan": 2,
  "display": { "smooth": "Yes", "decimals": 0 },
  "query": {
    "objectType": "Document",
    "class": "Invoice",
    "filters": [
      { "property": "Invoice date", "operator": "between",
        "value": ["@startOfYear", "@endOfYear"], "valueType": "dateToken" }
    ],
    "aggregation": {
      "type": "groupByDateBucket",
      "propertyName": "Invoice date",
      "bucketSize": "month",
      "reducer": "sum",
      "reducerProperty": "Amount",
      "includeEmptyResults": "Yes"
    }
  }
}

A few things to note in this JSON:

The filter restricts the data to the current calendar year using two date tokens. The full date-token vocabulary is covered in the future Post 5.
The aggregation is groupByDateBucket on the same date property, bucketed by month. Each bucket is reduced with sum of the Amount property. In other words, this means that all invoices with a date within this calendar year will be grouped by month and the value coming from the Amount property will all be summed within each month to arrive to a Total per month.
includeEmptyResults: “Yes” ensures that months with no invoices appear as zero values, so the trend line is continuous from January to December rather than skipping empty months (e.g. you were closed for vacations or weren’t able to sell anything for some other reasons).

Switching the widget type from line to area in this exact JSON gives the area-chart variant, nothing else changes:

3. The aggregation types these widgets accept

line and area are valid with two aggregation types:

groupByDateBucket: the natural fit for time series. It groups a date / timestamp property into a certain bucket: day, week, month, quarter, or year.
groupByProperty: groups by the distinct values of any property. It supports date/time properties, but it’s uncommon, as each day would be a data-point. It also support any other type of properties, to display them as categorical trends (e.g. “Monthly revenue per region” rendered as a line for some reason). Depending on requirements, a bar display might be more adapted.

The visual designer will only allow you to select one of these two aggregations. But if you try to set a summary or list aggregation in the JSON editor directly, then an error will be thrown, as it isn’t possible.

4. display.smooth

The single most common display option for these widgets is smooth:

"display": { "smooth": "Yes" }

When set to “Yes”, the line/area charts draws a Bézier-smoothed curve between data points (c.f. example above). When “No” (or omitted), the connecting line is made of straight segments. Smoothing makes monthly or quarterly trends look more visually pleasant. It can also slightly hide outliers, so use it where the trend story is more important than the exact per-bucket value.

5. display.decimals

For numeric reducers (sum, avg, min, max, median), this controls how many decimal places are shown in tooltips (default 0).

"display": { "smooth": "Yes", "decimals": 2 }

In the monthly-revenue example above, if amounts are stored as floats (which is probably the case), you may want decimals: 2 so the tooltip would show 116’520.43 rather than 116’520.

6. display.thresholds in single-series mode

In single-series mode (no seriesProperty), the thresholds array colors each data point marker based on its own value. The connecting line (and the area below) itself stays brand blue. The configuration is the same as on kpiNumber and gauge. You can either use text values representing colors or the HTML color code:

"display": {
  "smooth": "Yes",
  "thresholds": [
    { "value": 0, "color": "red" },
    { "value": 40000, "color": "orange" },
    { "value": 100000, "color": "yellow" },
    { "value": 120000, "color": "green" }
  ]
}

This is useful to highlight specific buckets that crossed a business threshold without changing the overall trend visualization. As mentioned in Post 4a, the default falls back to brand blue for values below the lowest threshold.

7. seriesProperty: multiple lines on the same chart

This is where these widgets become genuinely powerful. When you set a seriesProperty on the aggregation, the engine produces one line per distinct value of that additional property, with an automatic legend and color palette.

For instance, splitting the monthly revenue line into separate lines per country:

"aggregation": {
  "type": "groupByDateBucket",
  "propertyName": "Invoice date",
  "bucketSize": "month",
  "reducer": "sum",
  "reducerProperty": "Amount",
  "seriesProperty": "Country",
  "includeEmptyResults": "Yes"
}

A few notes that matter in practice:

display.smooth applies to every series. There is no per-series smoothing.
display.thresholds are silently ignored in multi-series mode. If they were applied per-point, they would override the series palette colors and break the relationship between line color and legend label. So the engine drops them in multi-series and keeps the legend honest.
includeEmptyResults in multi-series mode fills gaps across both dimensions (every series gets a zero in any time bucket where it has no data). This avoids broken lines.

7.1. Use low-cardinality series only

The seriesProperty is best used with low-cardinality properties: lookup values, booleans, short text ranges. A monthly revenue chart with 5-6 customers is readable, but the same chart with two hundred customers becomes a colored mess where no individual line is distinguishable from the others. If there are too many series, it loses its usefulness. The validation wouldn’t block it though, since it is technically a valid compatibility. Try it, and see what it looks like!

8. includeEmptyResults: gap filling

I mentioned this earlier already, and as you could see above, there are actually months with “0” values/amounts.

With includeEmptyResults: “Yes” present, the graph will fill the gaps between the earliest and the latest data points found with zero values. In above example, we can see in March that there was 0 revenue in total. And similarly, in October, there was 0 revenue for the USA. There is no discontinuity in the lines, all months are present, even if there is “0” as a result.

With includeEmptyResults: “No”, the chart will only display the buckets that have at least one matching object (across the series). So, if no invoice was issued in March, the X axis jumps from February to April directly. For above case, it would still display October, since there is a revenue in Switzerland, therefore the USA line will display a “0” value still, the line cannot be “cut”. Visually, that compresses the time scale and hides the meaningful “we had no revenue in March” signal.

There are valid use cases for both situations, so just select the one that makes sense depending on what you want to see.

Note: Adding includeEmptyResults: “Yes” can also add a (none) bucket for objects whose property used by the groupByProperty has no value. For example, if you group by a boolean state Yes/No, and some objects do not have a value on that boolean, they would end up in this (none) group. For groupByDateBucket, they usually don’t appear since you usually filter on a date in a certain range (like a month, or a year). Therefore if there is no date, usually it is just outside the range, but it technically also behaves in the same way if you fetch all results.

9. The “no date-valued reducer” rule

Same rule as on chart widgets in general: line and area cannot render a reducer that returns a date string. If you write something like:

"aggregation": {
  "type": "groupByProperty",
  "propertyName": "Customer",
  "reducer": "max",
  "reducerProperty": "Invoice date"
}

Then, with a line widget, the display would contain a placeholder and the query validation would show a warning. The reason was discussed in the previous post: ISO dates cannot be plotted on a numeric axis at the moment, so it wouldn’t make sense as a display.

The workaround is the same: use kpiNumber or a gauge in date mode if you only want to display one value, or a table for one or more values.

10. Drill-through on trend widgets

When drillThroughEnabled is “Yes”, clicking a data point opens the drill-through modal with the objects in that bucket. In multi-series mode, the click respects the series the point belongs to: only objects matching both the time bucket and the series value are shown. This makes it easy to answer questions like “show me the May invoices from Switzerland” with a single click on the right point of the line.

11. Wrap-up

line and area are the natural choice for time-series questions. They are fundamentally one widget with two visual modes, sharing the same aggregation contract and the same display vocabulary. The interesting part is what you can do with seriesProperty: turning a single trend into a multi-actor comparison without writing any extra logic.

Post 4c picks up the remaining three widgets: donut, bar, and table. The grouping rationale will be that they all answer distribution questions (how a population breaks down by category) or tabular questions (give me the raw rows), as opposed to the time-trend questions that line and area handle.

Want to know more about this Business Dashboard? Contact us and we will be happy to showcase it on M-Files.

L’article M-Files BD – Trend widgets: line and area est apparu en premier sur dbi Blog.

M-Files BD – Scalar widgets: kpiNumber and gauge

Morgan Patou — Tue, 09 Jun 2026 18:09:44 +0000

This is the first widget-specific post of the series. As a reminder, the previous post mapped the anatomy of a dashboard definition. From here on, I go widget by widget. I have grouped the seven widget types into three families:

Scalar widgets: kpiNumber and gauge both display a single value (a count, a sum, a date) over the matching object set. This is what the current post covers.
Trend widgets: line and area, which are very similar both in form and in configuration, will be the next post, 4b.
Distribution and tabular widgets: donut, bar, table, which are all remaining supported widgets (as of now), c.f. Post 4c later.

The reason kpiNumber and gauge sit together is that they accept exactly the same aggregation type (summary) and they share the same reducer model. So once you understand one, the other is mostly a different visual.

1. The summary aggregation in one paragraph

A full post will be dedicated to aggregations (Post 6 in the series), but to make this one self-contained: the summary aggregation returns a single value over all the matching objects. By default, it counts all matching objects. Its initial name was actually “count”, but later I wanted to introduce mathematical capabilities (c.f. the next sentence) so I had to change the name. With a reducer of sum, avg, median, min, or max (instead of the default count), it reduces a property’s values across the matching set instead.

// Total count of matching objects
"aggregation": {
  "type": "summary"
}

// Sum of an "amount" property for all matching objects
"aggregation": {
  "type": "summary",
  "reducer": "sum",
  "reducerProperty": "Amount"
}

// Latest contract expiry date among all matching objects
"aggregation": {
  "type": "summary",
  "reducer": "max",
  "reducerProperty": "Effective through"
}

Both kpiNumber and gauge accept this aggregation and only this one. The validator’s compatibility matrix is strict on that point. If you ever try to put a groupByProperty (or something else) on a KPI tile, you will not be able to save your dashboard.

2. kpiNumber – the big number tile

The kpiNumber is the simplest widget in the catalog. It shows a large number, with an optional small unit label below.

Minimal example:

{
  "id": "",
  "title": "All Contracts",
  "type": "kpiNumber",
  "gridColumnSpan": 3,
  "gridRowSpan": 1,
  "query": {
    "objectType": "Document",
    "class": "Contract or Agreement",
    "aggregation": { "type": "summary" }
  },
  "display": { "unit": "contracts" }
}

2.1. display.unit

A small text label shown below the number (c.f. above screenshot), indicating the natural unit of the value: “contracts”, “invoices”, “CHF”, “open items” etc… This is purely optional, so you can omit it if the title is already self-explanatory.

2.2. display.decimals

Number of decimal to use for the widget (default 0). This is useful when the reducer isn’t the default count, since a count of object is mandatorily an integer. For example:

"display": { "unit": "CHF", "decimals": 2 }

2.3. display.thresholds

This one is where the KPI tile becomes really useful in management dashboards. The thresholds array colors the number based on the current value. You can either use texts representing colors or the HTML color code:

"display": {
  "unit": "Overdue Invoices",
  "thresholds": [
    { "value": 0,  "color": "green"   },
    { "value": 1, "color": "orange"  },
    { "value": 10, "color": "red" }
  ]
}

The engine applies the color of the highest threshold where currentValue >= threshold.value. So in the example above:

0 overdue invoices: green
1 to 9: orange
10 and above: red
Below 0 (which would be unusual for a count, but possible for a sum on a money property): falls back to brand blue (#006eef) with the above example. But nothing prevents you to set a threshold on negative values.

Please note that thresholds apply only to numeric values. When the KPI shows a date (e.g. max on a date property), thresholds are ignored and the brand color stays. That’s the current behavior, and it might change if there is a need that calls for it.

2.4. Reducer types on kpiNumber

All six reducers are supported, with the standard property-type rules:

count (default): count the number of objects and you do not need to specify the reducerProperty in this case, since it’s taken by default.
sum, avg, median: numeric properties only.
min, max: numeric properties or date / timestamp properties.

A few real-world combinations:

// Count
{ "type": "summary" }

// Total revenue
{ "type": "summary", "reducer": "sum", "reducerProperty": "Amount" }

// Average duration
{ "type": "summary", "reducer": "avg", "reducerProperty": "Project duration days" }

// Latest contract expiry
{ "type": "summary", "reducer": "max", "reducerProperty": "Effective through" }

// Oldest open invoice
{ "type": "summary", "reducer": "min", "reducerProperty": "Invoice date" }

The last two return ISO date strings internally, which is then displayed on the end-user side as DD/MM/YYYY, currently. So, in M-Files Sample Vault you can build a KPI like “Latest contract expiry: 31/12/2026” with three lines of JSON.

2.5. Drill-through on kpiNumber

When drillThroughEnabled is “Yes” at the dashboard level, the entire KPI tile is clickable. A click opens the drill-through modal with all the objects that contributed to the value. This is useful for all types of reducers, to get more details on how the dashboard reached that value.

3. gauge – the dial

The gauge is also driven by a summary aggregation, but it shows the value on a dial against a defined range. It comes in two flavors depending on the property selected for the reducer.

3.1. Numeric mode

This is the default mode and it can be used with any reducer: count, sum, avg, median, or min / max on a numeric property. The scale is in the natural units of the value, with display.min and display.max bracketing the dial.

{
  "id": "",
  "title": "Overdue Invoices",
  "type": "gauge",
  "gridColumnSpan": 6,
  "gridRowSpan": 2,
  "display": { "min": 0, "max": 50, "unit": "invoices" },
  "query": {
    "objectType": "Document",
    "class": "Purchase Invoice",
    "filters": [
      { "property": "Deadline", "operator": "lessThan",
        "value": "@today", "valueType": "dateToken" }
    ],
    "aggregation": { "type": "summary" }
  }
}

3.2. display.thresholds on a gauge

Same shape as the kpiNumber, but the gauge applies the colors to both the pointer and the arc segments, so the dial visually splits into zones:

"display": {
  "min": 0, "max": 50, "unit": "invoices",
  "thresholds": [
    { "value": 0, "color": "green"  },
    { "value": 5, "color": "orange"  },
    { "value": 20, "color": "red" }
  ]
}

This colors the arc as: 0 to 5 in green, 5 to 20 in orange, 20 and above in red. The pointer also takes the color of the zone where the current value lands. In other words, a glance at the dial tells you whether you are in a comfortable zone, a warning zone, or a critical one (c.f. above screenshot for the example with the colors).

3.3. Date mode

When the reducer is min or max and reducerProperty is a date / timestamp property, the gauge switches to date mode automatically. The dial then represents a day offset relative to today:

display.min and display.max are expressed in day units, where 0 is today, negative values are in the past, positive values are in the future.
Graduation labels show dates like DD MMM if all dates are within a 365 timeframe, or it adds the YY if it goes beyond, so you know which date exactly we are talking about.
The center detail shows the actual date DD/MM/YYYY that comes from the reducer.

{
  "id": "",
  "title": "Closest Expiry",
  "type": "gauge",
  "gridColumnSpan": 6,
  "gridRowSpan": 2,
  "display": { "min": 0, "max": 180, "unit": "next expiry" },
  "query": {
    "objectType": "Document",
    "class": "Contract or Agreement",
    "filters": [
      {
        "property": "Effective through",
        "operator": "greaterOrEqual",
        "value": "@today",
        "valueType": "dateToken"
      }
    ],
    "aggregation": {
      "type": "summary",
      "reducer": "min",
      "reducerProperty": "Effective through"
    }
  },
}

In this example, the dial spans from today to 180 days in the future. If the earliest contract expiry is 22 days from today, the needle lands at the date that represents 22 and the center shows the actual date. If it is already in the past (negative offset), it indicates an overdue situation immediately. (in below screenshot, I’m using dates related to the past, as that’s what is available in the Sample Vault from M-Files, but in a real-world, the above JSON is usable)

3.4. When the reducer cannot produce a numeric value

If you try to use a date reducer on a chart widget (bar, line, area, donut), the widget will show a grey placeholder text, explaining that it is not supported (doesn’t really make sense). Moreover, before that, the administrator that designed the dashboard also got the same warning that this isn’t something that would work.

On a gauge, you have the date-mode escape hatch. On a kpiNumber or a table, the value is just formatted as a date string. So as a rule, date-valued reducers belong on kpiNumber, table, or gauge (date mode). They do not belong on chart widgets.

As said, this rule is not arbitrary, it is what the engine actually enforces. The “Test Widget”, “Test Queries” or “Save” will all tell you the same thing in a friendly way, if you slip or forgot about it.

4. Summary of reducers for scalar widgets

To close this post, here is a quick reference of what works on kpiNumber and gauge:

Reducer	What it computes	reducerProperty types	Empty set returns
*count* (default)	Number of matching objects	n/a	0
*sum*	Total of the property’s values	Numeric	*null* (rendered as –)
*avg*	Arithmetic mean	Numeric	*null* (rendered as –)
*median*	Middle value	Numeric	*null* (rendered as –)
*min*	Smallest value	Numeric or date / timestamp	*null* (rendered as –)
*max*	Largest value	Numeric or date / timestamp	*null* (rendered as –)

As described above, there is a subtle difference of “0” vs “-” for the empty results. For the default count, we display “0”, as there was really no objects returned, so the count of them is correct. For all other reducers, if there is no results, then there is nothing to do maths on. To differentiate between “no results” and “sum is really 0” (e.g. -2+2), I decided to display “-” for cases where there is no actual data to compute.

Of course, this empty results set difference is only true when there are no data returned. So if your filter is accurate and you expect some data, then it will never happen.

5. Closing thoughts

kpiNumber and gauge are the two widgets that typically stands at the top of a dashboard. It can be in a row of four KPIs and a gauge below, or side-by-side, depending on the layout defined. They give the at-a-glance answer that motivated the whole module in the first place.

In the next post (4b), I will talk about the trend widgets, line and area, which work in pairs almost as closely as kpiNumber and gauge do here.

Want to know more about this Business Dashboard? Contact us and we will be happy to showcase it on M-Files.

L’article M-Files BD – Scalar widgets: kpiNumber and gauge est apparu en premier sur dbi Blog.

M-Files BD – Anatomy of a dashboard definition

Morgan Patou — Sun, 07 Jun 2026 14:52:03 +0000

In the previous post, I walked through what an end user sees when they open the Business Dashboard pane in M-Files. From this post on, the audience shifts more towards the administrator’s side, looking at what a dashboard definition looks like.

Every dashboard is a single JSON document. The Admin tab in M-Files Admin offers a Visual Designer that builds this JSON for you, but in the end the JSON is the source of truth, so I think it is worth understanding its shape before getting into widget types and queries in the next posts. Think of this post as the reference map the rest of the series will refer back to.

1. The minimum valid dashboard definition

Before discussing every field, here is the (almost) smallest dashboard JSON that the engine accepts. It is intentionally trivial – a single KPI widget counting all Documents in the vault:

{
  "schemaVersion": 1,
  "id": "",
  "name": "My Dashboard",
  "widgets": [
    {
      "id": "",
      "title": "Total Documents",
      "type": "kpiNumber",
      "query": {
        "objectType": "Document",
        "aggregation": { "type": "summary" }
      }
    }
  ]
}

Note 1: Boolean-like fields are exposed as the strings “Yes” or “No”. This is what M-Files shows as booleans, don’t ask me why it’s not True/False like the rest of the world ;).

Note 2: Display names (object types, classes, properties, value-list items) must match what the vault shows in M-Files Admin. They are case-insensitive but must otherwise be exact.

2. The dashboard-level fields

The top-level object can carry the following fields: required ones are marked, all others are optional with default values that make sense, usually.

2.1. Identification and presentation

These are the main parameters of a dashboard that you will usually set at the beginning:

schemaVersion: Integer (currently 1). The Business Dashboard validation process emits a warning when this field is absent or lower than expected. This version is incremented when breaking changes are introduced, so admins will automatically be flagged that they need to update something.
id (required): typically a UUID v4 value (e.g. “12345678-abcd-1234-5678-abcd12345678”) but it isn’t restricted to that (e.g. “d01-contracts” works as well). The ID should be unique for each dashboard, otherwise you risk overriding an existing dashboard when you save. You can leave the field ID empty (as done above), the Business Dashboard will automatically generate one for you.
name (required): Text shown in the end-user dropdown, you should keep it concise (<250 chars).
description: Optional subtitle shown in italic below the top bar (hidden when absent). As described in the previous post, the first two lines will be shown on the user side, with a hover tooltip for the full text.

2.2. Auto-refresh

These are parameters that controls whether or not the dashboard will refresh itself or allows controls about the refresh more globally:

autoRefreshEnabled: “Yes” or “No” (default “No”). Whether auto-refresh is enabled when the dashboard first loads.
autoRefreshIntervalSeconds: Integer (default 300, minimum 15). Typical values can be 60, 300 or 900 for example. This is the interval of time between two auto-refresh of all widgets done when the auto-refresh is enabled (either via autoRefreshEnabled or because userCanToggleAutoRefresh is “Yes” and a user enabled it manually). The minimum is set to 15s to avoid overloading the vault in case of mis-configuration.
userCanToggleAutoRefresh: “Yes” or “No” (default “No”). Whether the user can disable or enable the auto-refresh feature.

2.3. Exports and drill-through

These are the optional features that you can enable:

exportToPdfEnabled: “Yes” or “No” (default “No”). Shows the ⎙ PDF button in the top bar, to allow users to export the dashboard to PDF.
exportToCsvEnabled: “Yes” or “No” (default “No”). Shows the ⊞ CSV button on all widgets (except kpiNumber & gauge) as well as on the drill-through modal, to allow users to export the widget details to CSV.
drillThroughEnabled: “Yes” or “No” (default “No”). Makes chart elements and rows clickable to open the drill-through modal (c.f. previous post if you don’t know what the drill-through is).

These three flags are intentionally off by default. Turning them on is an explicit choice per dashboard, which I think is the right place to make that decision. A dashboard meant for casual “at-a-glance” use might not need them while a dashboard meant for active investigation might.

2.4. Performance settings

These are security and performance settings:

skipTemplateCheck: “Yes” or “No” (default “No”). When “No” (default), objects marked as template (i.e. they have the “is Template” property and it is checked) are automatically excluded from any results. When “Yes”, this additional verification is skipped and templates can therefore be part of the results (faster execution).
skipObjectPermissionCheck: “Yes” or “No” (default “No”). When “No” (default), every queries will respect the user-level accesses / ACLs, so different users might see different results / numbers / lists of objects, depending on their permissions. When “Yes”, server-level accesses are used, so different users will see the same results, always (faster execution).
serverScanMaxResults: Integer (default 500). The maximum number of M-Files objects fetched per widget query. You can set it to 0 for unlimited.
drillThroughMaxResults: Integer (default 300). Maximum rows shown in drill-through modals and list tables. You can set it to 0 for unlimited. The pagination for the opened modal is set at 15 rows per page. Therefore, there can be up to 20 pages by default.

I will dedicate a full post (Post 8 in the series) to these details and to the trade-offs they imply, so I will not spend too much time on them here. The defaults are reasonable for most use-cases but you can change everything, of course.

2.5. Access control

Finally, the last parameter is about access control to the dashboard:

assignedTo: An object with two optional arrays, users and groups. When the arrays are absent or empty, the dashboard is public. This means that ANY M-Files user will be able to select the dashboard from the dropdown. It does NOT mean that he will see something inside the widgets! Therefore, it gives access to the dashboard itself, but widget content still relies on the user’s ACLs by default (or server-level access if skipObjectPermissionCheck is set to “Yes”, c.f. above section). On the other hand, when at least one of the arrays is present, access is granted to any user who appears in the users array OR is a member of any group in the groups array.

In this example, the dashboard will be accessible by the user whose Login Account is Morgan.Patou OR any users part of the Finance Team OR Management groups:

"assignedTo": {
  "users": ["Morgan.Patou"],
  "groups": ["Finance Team", "Management"]
}

2.6. Widgets

widgets (required): An array of widget objects. The order in this array is the order they appear on the grid. The size of the widget (column span, row span) is part of each widget’s definition (i.e. inside the array).

3. The widget object

Each entry in the widgets array has the following shape:

{
  "id": "",
  "title": "My Widget",
  "description": "An example description for My Widget",
  "type": "kpiNumber",
  "gridColumnSpan": 3,
  "gridRowSpan": 1,
  "query": { ... },
  "display": { ... }
}

Here is a quick description of the different fields:

id (required): Same description as for the dashboard
title (required): Name of the widget shown at the top of the tile
description: Optional one-line italic subtitle just below the title. Annotate what the widget measures, or flag a caveat
type (required): One of kpiNumber, gauge, donut, bar, line, area, table (covered in posts 4a, 4b, 4c)
gridColumnSpan: Integer from 1 to 12 (default 4). Represent the width of the widget on the dashboard
gridRowSpan: Integer (default 1). Represent the height of the widget on the dashboard
query (required): The query the widget will execute. This can be pretty complex, so it will be detailed in Post 5 (filters) and Post 6 (aggregations), later
display: Type-specific display options, like color thresholds, unit, decimals, etc. It will be covered per widget type in posts 4a, 4b, 4c

4. The query object in one paragraph

As mentioned, the detailed walkthrough of queries is the topic of Post 5 / 6, but for completeness, here is the basic shape of a query:

"query": {
  "objectType": "Document",
  "class": "Contract or Agreement",
  "filters": [
    {
      "property": "Effective through",
      "operator": "greaterOrEqual",
      "value": "@today",
      "valueType": "dateToken"
    }
  ],
  "aggregation": {
    "type": "groupByProperty",
    "propertyName": "Agreement type",
    "resolveValueListLabels": "Yes",
    "includeEmptyResults": "No"
  }
}

objectType can be a string or an array of strings (multi-object-type query). class is optional but strongly recommended for performance and to check only objects that actually matter… For example, you don’t need to fetch all documents (irrespective of their class) if you only want to count the number of active contracts. filters is an array of 1-to-N filter combined by AND. Finally, aggregation specifies the shape of the result and the reduction applied. To know more about these, please wait for the next posts.

5. The 12-column grid

The layout is computed on a CSS grid with 12 columns. gridColumnSpan sets the width of a widget, while gridRowSpan sets its height in row units.

Therefore, most common column layouts include:

4 widgets per row: gridColumnSpan: 3 each. Typical for KPI tiles.
3 widgets per row: gridColumnSpan: 4 each. Medium tiles or narrow charts.
2 widgets per row: gridColumnSpan: 6 each. Charts and tables.
1 widget per row: gridColumnSpan: 12. Full-width bar chart or wide table.

These are just examples. Nothing prevents you to set 3 widgets on a row with gridColumnSpan: 5, gridColumnSpan: 4, gridColumnSpan: 3 respectively. The Visual Designer is great for that since you get an exact view of what it will look like directly while creating the dashboard. More on that on the Post 9a.

The widgets simply fill the rows from left to right in the order they appear in the widgets array, wrapping to the next row when a row’s spans sum exceeds 12. So planning the layout is largely a matter of ordering the widgets and giving each one a sensible column span.

For row height, a small subtlety: gridRowSpan: 2 does not give you exactly twice the canvas height of gridRowSpan: 1. It actually gives you roughly three times the canvas height, because each row unit subtracts a fixed overhead (header, padding, borders) from the available space. The practical consequence is that chart widgets (bar, line, area, donut, gauge) should almost always use gridRowSpan: 2 or higher. A single-row chart has very little drawing space and ends up cramped.

KPI number tiles, on the other hand, look fine at gridRowSpan: 1 and that is the right default for them. As a rule of thumb: KPIs at 1, everything else at 2.

6. Where the definition is stored

Dashboard definitions are stored in the vault’s Named Value Storage (NVS). They are NOT stored as a specific object in the vault. I initially thought about creating them as standard vault objects but in the end decided the NVS would probably make it easier for deployments and usage in the long run. The trade-offs of that choice:

Pros: no extra object type to provision when installing the module, no ACL setup, no admin training on a new object type, no per-customer deployment friction. Backups of the vault include the dashboards automatically because NVS is part of the vault.
Cons: there is no per-definition version history of the kind you would get with vault objects. I mitigated this by offering Import and Export buttons in the Admin tab that produce .json files. What I would recommend is to manage those files in a Git repository, which gives a clean and conventional version history outside the vault, with diff support.

The Export / Import buttons are the topic of Post 7. The TL;DR is that I would recommend that you should develop your dashboards on a DEV. Then, export them all into your git. Finally, when ready, import them into the TEST/QA/PROD environments, either manually or via CI/CD pipelines.

7. Quick overview of a full dashboard JSON definition

Putting everything together, here is a slightly more complete (but still small) example. It has some of the dashboard-level fields modified from their default values to show-case actions and buttons, and two widgets:

{
  "schemaVersion": 1,
  "id": "",
  "name": "Contracts",
  "description": "Quick overview of contracts",
  "autoRefreshEnabled": "Yes",
  "autoRefreshIntervalSeconds": 60,
  "userCanToggleAutoRefresh": "Yes",
  "exportToPdfEnabled": "Yes",
  "exportToCsvEnabled": "Yes",
  "skipTemplateCheck": "No",
  "drillThroughEnabled": "Yes",
  "skipObjectPermissionCheck": "No",
  "serverScanMaxResults": 500,
  "drillThroughMaxResults": 300,
  "assignedTo": {
    "users": [],
    "groups": ["Legal Team"]
  },
  "widgets": [
    {
      "id": "",
      "title": "Active Contracts",
      "type": "kpiNumber",
      "gridColumnSpan": 3,
      "gridRowSpan": 2,
      "query": {
        "objectType": "Document",
        "class": "Contract or Agreement",
        "filters": [
          {
            "property": "Effective through",
            "operator": "greaterOrEqual",
            "value": "@today",
            "valueType": "dateToken"
          }
        ],
        "aggregation": { "type": "summary" }
      },
      "display": { "unit": "contracts" }
    },
    {
      "id": "",
      "title": "By Agreement Type",
      "type": "donut",
      "gridColumnSpan": 9,
      "gridRowSpan": 2,
      "query": {
        "objectType": "Document",
        "class": "Contract or Agreement",
        "aggregation": {
          "type": "groupByProperty",
          "propertyName": "Agreement type",
          "resolveValueListLabels": "Yes"
        }
      }
    }
  ]
}

This dashboard renders as a single KPI tile and a donut chart on the same row (3 + 9 = 12 columns), with auto-refresh every minute and access restricted to the Legal Team group. Nothing exotic, this is the kind of dashboard most customers should probably start with.

If you wonder what it would look like, you can check the 1st and 2nd posts, they include these same widgets (though arranged slightly differently).

8. What comes next

In the next three posts (4a, 4b, 4c) I will go through the seven widget types one family at a time:

4a – Scalar widgets: kpiNumber and gauge.
4b – Trend widgets: line and area.
4c – Distribution and tabular widgets: donut, bar, table.

After the widget tour, Post 5 covers the query side (object types, classes, filters, date tokens) and Post 6 covers aggregations and reducers. By the end of those, you will have everything needed to write a non-trivial dashboard from scratch – if that’s what you want to do… Or you can just use the Visual Designer and follow along.

In the meantime, I hope this anatomy post gives you a clean mental map. If something is unclear or if you spotted a field whose behavior I have not explained here, please reach out so I can update the post.

Want to know more about this Business Dashboard? Contact us and we will be happy to showcase it on M-Files.

L’article M-Files BD – Anatomy of a dashboard definition est apparu en premier sur dbi Blog.

M-Files BD – End-user experience

Morgan Patou — Thu, 04 Jun 2026 20:48:21 +0000

In the first post of this series, I introduced the Business Dashboard module I built, over the past few weeks, for M-Files. This second post is for the end user: what does it actually look like, where is it, and what can you do with it once your administrator has configured one or more dashboards for you. No JSON in this post, no admin screens, no code. Just your experience as end-users. Please note that in the rest of this blog, when I say “administrator”, I mean the person that will design your dashboards. It might be a Super User or a System Admin, but he needs access to the M-Files Admin UI.

If you have followed my posts for some time, you know I prefer to walk you through a feature (or an issue) exactly as you would discover it. So, let’s do the same here.

1. Where to find the dashboard pane

The Business Dashboard adds a third tab to the right-hand pane of M-Files, next to the standard Metadata and Preview tabs. It is simply labelled Dashboard. Both M-Files Web (in a browser) and M-Files Desktop (the Windows client) show it – same panel, same behavior, with one platform-specific detail I will flag along the way.

As a reminder, this is what it looks like and how to access it:

If you do not see the Dashboard tab, two things are possible: either the module is not installed in your vault, or it is installed but your vault is configured to use the old UI. In both cases, the next step is to ask your administrator.

A small note on the pane width: the right pane is resizable by dragging its left edge. The dashboard AND the widgets inside respond to that resize. Therefore, on very narrow widths, widgets like donuts will have their legends move from the side to the bottom, to give more space to the charts. Don’t hesitate to play with the resize, it can be a good stress-reliever ;).

2. Selecting and switching between dashboards

At the top of the pane, you have a “Current dashboard:” dropdown. It lists every dashboard your administrator has decided you can see, in the order the administrator chose. There might be more of them, but in this case, you don’t have permissions to see them. Click the dropdown, pick a dashboard, the panel updates immediately.

The last dashboard you selected is remembered for the next time you open the pane, so if you mostly use one dashboard (e.g. the Contracts one), you do not need to re-select it every morning. More on that “last seen” vs “favorite” vs “link” below, in following sections.

Under the dropdown, if your administrator added a description to the dashboard, you will see a thin italic strip that contains it. It is clamped to two lines maximum. If the description is longer, hover the strip to see the full text in a tooltip. In addition to that, when the dashboard dropdown is opened, you will also be able to see the description of the dashboard you hover over, which will have a darker background.

In this screenshot, you can see the selected dashboard is about Contracts and its description is Quick view of contracts: active, …, while the description of the Invoices Overview dashboard is Purchase and Sales Invoice breakdown by year, workflow state, and customer.. Therefore, before loading it, you can get a quick idea on what exactly it will contain:

3. The widget grid

Each dashboard is a grid of widgets. As said, the grid is responsive and it contains 12 columns, so each widget can use from 1/12th and up to 100% of the width. Your administrator decides which widgets appear, in what order, at what size.

As of now, there are seven widget types supported:

Widget	What it shows
KPI Number	A single value, typically a count, a sum (e.g. “Active contracts: 142”, “Invoices: 1’520’430 CHF”) or other mathematical operators like min/max/average/median. But it can only display a date (with min/max).
Gauge	A dial with a value against a defined range. Some gauges are numeric (e.g. overdue invoices against a target of 0-50). Some are in date mode, showing a date value as a day offset from today (“days until earliest contract expiry”).
Donut chart	A distribution across categories or time periods (e.g. contracts by agreement type). Slice values appear inline. The legend is shown next to the donut when the pane is wide enough, otherwise below. Also supports multi-series by displaying multiple sub-donuts.
Bar chart	A distribution across categories or time periods, displayed either vertically or horizontally. Also supports multi-series, with stacked or grouped bars.
Line chart	A distribution across categories or time periods, with data points linked to each other. Also supports multi-series, with multiple lines.
Area chart	Same data as a line, with the area below the line filled, to emphasise volume.
Table	A sortable list of objects or groups of objects, with pagination and additional columns that can be added.

Each widget has a title at the top, sometimes a short italic subtitle (the widget description, when your administrator added one), and three small elements in the top-right corner of the widget header that I will detail in the next sections.

4. Refreshing the data

Data on the dashboard is fetched live from M-Files. There are three ways the dashboard can be refreshed:

Auto-refresh. If your administrator enabled auto-refresh, the panel re-queries all widgets at a configured interval (e.g. every five minutes). If your administrator also enabled the user toggle, you see an Auto-refresh: On / Off switch in the top bar that lets you pause and resume it. If the user toggle is disabled, the switch will be greyed-out.
Global manual refresh. A small ↻ button in the top bar refreshes every widget at once. To avoid accidental hammering, the button enters a 15 second cooldown after a click. The per-widget refresh buttons are also locked during this cooldown.
Per-widget refresh. Each widget has its own ↻ button in its top-right corner. Click it to refresh that widget alone. A 15 second cooldown applies to the clicked widget only.

Each widget header also shows a faint “X ago” or “just now” text indicating when the data was last successfully loaded. Hover it to see the exact timestamp in a tooltip: “Last refreshed: hh:mm:ss“. This is useful to spot, at a glance, if all widgets were successfully refreshed or if there is one that is fresher than another (typically because you refreshed it manually).

5. Drill-through: exploring the objects behind the numbers

This is one of the features that makes the difference between an “interesting dashboard” and an “actually useful solution”. If your administrator enabled drill-through on the dashboard, all widgets automatically become clickable, and clicking them opens a detail modal listing the underlying M-Files objects.

What is clickable:

Element	What it shows
KPI tile	All the objects matching that widget’s query.
Gauge face	All the objects matching the widget’s query before applying the reducer (e.g. the minimum date is displayed, but all objects are shown in the drill-through).
Donut slice	Objects in that specific category or time period.
Bar column	Same as donut.
Line point	Same as donut.
Area point	Same as donut.
Row in a *list* table	No modal; clicking an object name navigates directly to it
Row in other tables	Same as donut.

The modal that opens shows a table with one column for the object name (as a clickable link – c.f. navigation section below), plus any additional columns your administrator configured. Columns are sortable: click a column header to sort ascending, click again for descending. Sorting applies across all fetched rows, not just the visible page. So if you sort an “Amount” column descending, you see the top 15 of all the rows that were retrieved, not just the page you happened to be on.

Note: Sometimes the bar shows a small warning like "(⚠ 500 object(s) fetched)". This tells you the server retrieved up to its scan cap and there may be more objects in the vault that did not make it into the modal. Hover the icon for an explanation. If you regularly hit that cap on a widget that should show everything, your administrator can raise (or disable) the cap on the dashboard configuration.

Why drill-through can potentially show more objects than the widget counts

This deserves its own paragraph because it might be a bit counter-intuitive at first. The widget aggregation runs inside a capped scan (set by your administrator), while drill-through runs a focused search with the slice or row category as an additional filter. If the focused search can fit all matching objects under its own cap, you may see more objects in the modal than the widget counted.

The Partial results badge on the widget header (when present) is the warning that the widget aggregate may be incomplete. There are basically two different limits. A first global limit, that ensures we do not request too much from the server and a second one linked to the drill-through that applies on filtered results.

Example: A dashboard has been configured by your admin with 10’000 for the first limit and 5’000 for the second. Let’s take a Documents by Class donut widget (i.e. you want to see ALL documents – without any filters). If your vault has more than 10’000 objects of type Documents, then the widget would display 10’000 with a Partial results badge. In these first then thousand results, there might have been 150 Contract or Agreement (Class) shown in the donut. If you click on the donut slice for Contract or Agreement, the associated modal will open and it will show at least 150 and up to 5’000 results. In the end, if the initial search is “too wide”, you might hit a limit. But the drill-through will include another filter (here “Class=Contract or Agreement”), which only return these specific objects.

Again, this is for performance reasons and as a kind of server-protection. If you have a real need, these limits can be increased or disabled completely by the admin – it is simply like that by default.

6. Navigating to an object

In drill-through modals and in list-type table widgets, object names are blue underlined links. Clicking a link opens that object on the left-side of M-Files.

A small detail that is sometimes useful: the drill-through modal stays open after you click a link, so you can navigate to several objects one after the other without losing your place in the list. Close the modal manually when you are done (using the × button, the Escape key, or clicking outside the modal).

7. Exporting to PDF

If your administrator enabled PDF export, a ⎙ PDF button appears in the top bar of the dashboard. Clicking it generates a snapshot of the current dashboard state (charts as images, KPI numbers, tables) and opens a print dialog automatically (both on Web and Desktop). From there:

Select “Save as PDF” (or “Microsoft Print to PDF”) as the printer and click Save.
Or pick a physical printer to print a hard copy.

The top bar, the refresh buttons and any open modals are excluded from the snapshot, so the PDF stays clean.

Note 1: if your client blocks the automatic print dialog (a rare but possible restriction), an HTML file is downloaded instead. You can open that file in any browser and press Ctrl+P to print it in the exact same way.
Note 2: the PDF export will respect exactly what M-Files is currently showing. This allows you to export the dashboard with a specific configuration. For example a table sorted in a different way that the default one, or some legends unselected from a donut, so that it ignores these details and re-render the donut with the remaining categories or time periods, etc. Whatever M-Files is showing will be what is present in the PDF export.

A PDF export of a dashboard will look like that:

Contracts Snapshot (01-Nov-2011)Download

8. Exporting to CSV

Similarly to the PDF export, your administrator can also enable CSV export. When it is on, a ⊞ CSV button appears in the header of every table, bar, donut, line, and area widget, as well as in the header of the drill-through modal for all widgets.

Clicking ⊞ CSV downloads a .csv file immediately. There is no extra server call; the export uses the data already in memory, so it is fast.

The file starts with a small metadata block describing what the data represents / how it was fetched:

Widget Name,Active Contracts
Object Type,Document
Object Class,Contract or Agreement
Filter #1,Effective through greaterOrEqual @today
Aggregation Details,summary | count
Export Date,2026-06-04 19:30:08

There might be two additional lines if the export is done on a specific group (a part of the initial results (e.g. after opening a donut’s slice modal details)) and if the widget is configured with multi-series. One line will describe which group is exported (category or time period) and another will describe the second dimension (the series).

The above metadata section is followed by a blank separator row (empty line), then the actual data rows (column headers and associated values). All rows currently held in the browser are exported, not just the page you are looking at. For example, if the table shows page 2 only (objects 16-30) but 120 results were fetched, then all 120 rows will be in the CSV.

The CSV file exported can then be opened in your preferred editor (Microsoft Excel, Euro-Office, LibreOffice, etc…). In the below image, I simply added the headers in Bold, but otherwise it’s the default outcome.

Note: similarly to the PDF export, the CSV will also respect table sorting, so that the rows exported will be in the exact same order as the one that you see on your screen.

9. Favorite dashboard

If you use one dashboard far more often than the others, you can pin it as your favorite. There is a small star button (☆) in the top bar:

Select the dashboard you want to pin.
Click the ☆ button. It turns gold (★).
From now on, when you open the Dashboard pane, that dashboard is loaded automatically (instead of the last one you used).

To unpin, click the ★ again. The favorite is stored under your account, so it follows you across clients and devices. Do it once and you can then enjoy forever. Only one dashboard can be pinned at a time. Picking a different favorite simply replaces the previous one.

Right next to the favorite button, a small link button () can be used to share a link to a specific dashboard with someone else.

As far as I know, this is the only feature – as of today – that behaves slightly differently between M-Files Web and M-Files Desktop:

In M-Files Web, the clipboard will be updated to contain the current URL with the specific dashboard ID. If you give this link to someone else, they can open it in their Web and the same dashboard will be opened too. Access control still applies: if the recipient is not entitled to that dashboard, the link is silently ignored and their own default dashboard is shown instead.
In M-Files Desktop, the URL does exist, but it’s a random one specific to your Desktop. Therefore, it wouldn’t be usable by anybody else. Instead of sharing this URL, in Desktop, the link button will only copy the dashboard name. You can still paste the name in a chat or email and somebody else should be able to find the same dashboard, but manually…

It’s not super elegant, I know, but it’s how it works at the moment. I might re-work that later if I find a better way to do it or if there is a real need for it.

11. Indicators worth knowing

A few small visual indicators are worth recognising:

Partial results (orange badge in the widget header). The server scan cap was reached; the widget may not reflect the full data. Section 5 above has the full explanation.
Grey placeholder text inside a widget. The dashboard definition has a small mis-configuration. The administrator was normally warned about it when he designed the dashboard but he probably ignored it.
Red error overlay that replaces a widget. The server returned an unexpected error. The dashboard definition has a configuration problem. The dashboard designer part (admin) doesn’t allow to Save/Publish dashboards with such errors, but someone might have bypassed the administrative tool and updated the database directly.
Spinning overlay. The widget is loading. If it stays in that state for a long time, the vault may be under load. Use the per-widget refresh button to retry once the spinner disappears. I haven’t seen the spinner stays more than 2 seconds myself, but I guess it could happen.
Numbers higher than what you see in your views. Dashboard widgets can use either user-level permissions (default) or server-level permissions, depending on the configuration done by the administrator. Some use-cases might require to give access to all M-Files users to some KPIs for example. In this case, a dashboard could use server-level permissions, so that the KPIs are accurate and the same for everybody, even people with no access to the underlying objects. The navigation (links) to the objects will always respect your actual user-level permissions, obviously. So the only thing that the dashboard can give you access to, if configured in that way by the administrator, is KPIs or some charts (possibly even without the drill-through – so you don’t have any details about the objects themselves).

12. What comes next

What I believe is genuinely useful about this module, is that it removes the small friction of “I just want to know quickly”. Opening a Power BI report takes a context switch; opening a saved view, scrolling and counting takes time. The Business Dashboard sits one click away in the M-Files window the user is already in, and it shows the answer he is looking for with nice visuals.

In the next post of this series, I will move to the other side of the panel: how an administrator configures a dashboard. I will start with the anatomy of a dashboard definition – the JSON structure that drives everything you just saw.

Want to know more about this Business Dashboard? Contact us and we will be happy to showcase it on M-Files.

L’article M-Files BD – End-user experience est apparu en premier sur dbi Blog.

M-Files – Introducing the Business Dashboard module

Morgan Patou — Tue, 02 Jun 2026 19:50:32 +0000

Some time ago, a customer asked whether it was possible, in M-Files, to have dashboards with KPIs and a global overview of what was happening in their vault. It was a topic I had in my radar for a few years already, but I never had – or took – the time to really work on it. I jumped on the occasion to start building a solution in parallel of my daily-job at dbi services. Several evenings (who said nights?), weekends and some holidays later, here we are.

The starting point was a real observation: the views of M-Files are powerful, the metadata model is flexible, the workflows are solid, but getting an at-a-glance picture of the business is still a clicking exercise. You open a view, you count, you open another view, you count again, you switch to the search, you start building an Excel with what you found… However, you might still not have the chart you wanted to present in the management meeting that starts in ten minutes.

This blog post introduces what I called the Business Dashboard module, a custom M-Files extension designed and built to fill that gap. This is the first post of a series that will walk through the module. From the end-user experience, to the authoring side, and finally to a complete worked example. I will use the M-Files Sample Vault for this series since it contains test data. You can quickly and easily deploy that vault via the M-Files installer.

1. The problem to solve

As you might know, M-Files ships with a few visualization features (views, search facets, the right-hand Metadata and Preview panes). With these, you can answer one of the following questions, but never all at once and never at a single glance:

How many contracts are currently active in the vault?
How many of them expire in the next 30 days?
What is the distribution of our contracts by agreement type?
What is the total amount of invoices waiting to be approved, broken down by customer?
How many documents was each employee responsible for last quarter?

These are not exotic questions. They are the kind of things you might face every week. And the honest answer, today, is “I will get back to you later”. Sometimes there is a Power BI dashboard plugged into the vault SQL backend that answers some of them, but that is a second tool, a second login, often a second team to involve, and usually a permission/access story that does not match the per-user M-Files context.

I wanted something simpler. Something that lives inside M-Files, that integrates with the M-Files access model, that an administrator can configure without writing any code, and that the end user can open with one click from within M-Files.

2. What the Business Dashboard module is

The Business Dashboard is an additional panel that appears on the right-hand side of M-Files (alongside Metadata and Preview). It works in both M-Files Web and Desktop (latest UI). End users pick a dashboard to render, which contains widgets (tiles) that display live business data from the vault using KPIs, donut, bar, line, area, tables or gauges. Data is fetched live from M-Files and it can refresh itself. It provides multiple other features such as export to PDF or CSV, favorites, links, even object navigation, etc… From a security point of view, the access control will be in Post 8.

For administrators, dashboards are defined as JSON documents edited inside a dedicated tab in M-Files Admin. The editor offers both a JSON editor and a Visual Designer that lets you build dashboards without any knowledge of what JSON is. Definitions are stored in the vault, so a backup includes them automatically and there is no extra database to provision or manage.

Two design choices are worth pulling out here, because they shape everything else in the series:

The engine is fully generic. There is no hardcoded business concept anywhere in the code. The same module works for contracts, invoices, customers, employees, projects, or whatever objects is contained within your vault. The JSON tells the engine how to query (which object type, which class, which property to group by, which filter to apply), not what values to expect.
The wire format is frozen. Once the dashboard JSON is saved, the calls between the UI and the server speak a fixed shape. This means that when/if I ship a new widget type or a new aggregation, existing dashboards will keep working with no migration.

This is what the Business Dashboard looks like, from end-user point of view (click to enlarge the image):

3. Concrete use cases

Rather than list features, let me describe four concrete scenarios where the Business Dashboard could shine:

Contracts overview. A KPI tile showing the number of active contracts, another showing those expiring in the next 30 days, a donut chart by agreement type, a bar chart showing expiries per month for the next year (e.g. until end of year or next 365 days), and a list of contracts to review this week. One panel, no clicking, refreshed automatically.
Purchase invoices. Total amount waiting for approval, oldest invoice still pending, distribution by customer/countries/regions, a gauge of overdue invoices against a target of zero.
Employees and responsibilities. A list of employees by department, with the count of documents worked on by each one in the past week. Possibly the list of people in holidays with their substitute. Useful for capacity planning and for the “who is on vacation, who covers what” conversation.
Customer activity. Documents by customer in the last quarter, broken down by document class. Quickly spot the customers who went quiet and the ones who suddenly became very active.

None of these are hardcoded in the module. They are examples of what an administrator can configure, once, using the Visual Designer or the JSON editor. The same engine produces all of them and any number of variations.

Note: Whatever the widget used, with at most one more click, you can get the details of which objects correspond to what you are looking for. For example, if you have a KPI for the number of active contracts, if it shows 25, then simply clicking on that 25 will open a drill-through with the list of all 25 objects in that specific situation, with optional additional details. See the follow-up posts for more details on that (and all other features).

4. Where it runs

The module contains two parts:

BusinessDashboard – the server-side, which contains the query engine, the validator, the persistence layer, etc.
BusinessDashboard.UIX – the client-side, which contains all the display part for the end-users.

Both are installed, independently, in the M-Files Admin, via the standard application installation process. In regards to compatibility, it should technically supports all recent versions of M-Files that uses the new UI. I personally tested it extensively on M-Files 26.3 / 26.4 / 26.5.

No external service is contacted; everything runs inside the vault. This makes the module a good fit for both on-premises and cloud deployments (though for cloud, the module would need to first be validated by M-Files), or for any deployment where adding a second tool (Power BI, Grafana) is not desirable (too complex to setup, too much for the actual need, or simply for security reasons).

5. What this module is not

I think it is fair, before closing this introduction, to flag what the Business Dashboard module is not trying to do.

It is not a BI tool. No SQL access, no data warehouse, no cross-vault joins. The engine only queries the M-Files vault it is deployed on and that’s it. If you need to combine data from a vault and an ERP, a dedicated BI stack might still be the right answer. UNLESS you are already merging the ERP data in M-Files (external repository or database connector). In that case, the Business Dashboard would be able to query it as well.
It is not (really) a reporting engine. No scheduled report, no email delivery. However, you can export a dashboard as PDF and you can export a widget (or its drill-through) to CSV (Excel-compatible).
It is not (really) designed for very large aggregations. The engine fetches up to a configurable amount of objects per widget (500 by default – can be increased, decreased or disabled completely…). Then aggregates it in memory, which is fine for any reasonable business question. Though it is not fine for things such as “list all 50 million objects that were created in the last 10 years”… If the limit is hit, the user knows about it with the help of a special badge.
It is not a replacement for M-Files views (or is it…?). Depending on use-cases, views can still be a better starting point. The Business Dashboard sits next to them, not in front of them.

If your need fits inside those boundaries, the module might make your life easier, irrespective of whether you are a decision maker, a power user, a simple user or an administrator. If it does not, please get in touch and I will be happy to discuss what would!

6. What comes next in the series

This series is going to cover, in order:

Post 2 – The end-user experience: opening the panel, switching dashboards, drill-through, exports, favorites, sharing.
Post 3 – The anatomy of a dashboard definition: the JSON structure that drives everything.
Posts 4a, 4b, 4c – The widget catalog, split by type family: scalar widgets (kpiNumber, gauge), trend widgets (line, area), and distribution / tabular widgets (donut, bar, table).
Post 5 – Writing queries: objectType, class, filters, date tokens.
Post 6 – Aggregations and reducers.
Post 7 – The Admin tab: Visual Designer, JSON editor, validation, import / export.
Post 8 – Access control, performance settings, interactive features.
Posts 9a and 9b – The same Contracts dashboard built two ways: through the Visual Designer (9a) and through the JSON editor (9b), on the M-Files Sample Vault so anyone can follow along.

If you have a specific question or a use case you would like to see covered in the series, don’t hesitate to reach out, and I might be able to add it to my to-do list.

In the meantime, the next post will be a tour of what an end user sees the first time they click on the new Dashboard tab. See you there.

L’article M-Files – Introducing the Business Dashboard module est apparu en premier sur dbi Blog.

OTDS – Installation of Replicas fail if the OT Admin password is too long?

Morgan Patou — Mon, 30 Mar 2026 11:36:04 +0000

For simplicity, in this blog, I will refer to the first OTDS instance as the “Primary” (the synchronization master host, installed with ISREPLICA_TOPOLOGY=0=FALSE) and any additional instances as “Replicas” (installed with ISREPLICA_TOPOLOGY=1=TRUE). Over the past few years, I have installed and worked on around 20–30 different OTDS environments, some with a single instance and others with multiple instances (HA). Overall, it is not a bad piece of software, even though it could use improvements in certain areas (e.g.: c.f. this blog). However, it was only after I started installing a few Replicas on recent OTDS versions (using a database backend instead of OpenDJ) that I encountered a rather unusual issue.

1. OTDS Replica installation failure

Single-instance installations using the silent properties file were always successful, and most multi-instance installations worked as well. However, I encountered a very specific issue twice: the Primary instance would install successfully, but the Replica installation would fail with an error stating “parameter JDBC_CONNECTION_STRING not defined“. Since everything runs in automated environments (Kubernetes or Ansible), I knew it was not a human error. When comparing the silent properties files, everything looked correct. The file used on the Primary was exactly the same as the one used on the Replica, except for “ISREPLICA_TOPOLOGY=0” and “ENCRYPTION_KEY=” on the Primary versus “ISREPLICA_TOPOLOGY=1” and “ENCRYPTION_KEY=XXXXXXX” on the Replica.

This is the expected configuration. A Replica needs to take the value of “directory.bootstrap.CryptSecret” from the “otds.properties” file of the Primary and use that value for “ENCRYPTION_KEY“. Therefore, when you install the Primary instance, the value remains empty because nothing is installed yet. During the Replica installation, the automation retrieves this value and populates the parameter accordingly. But then why would the Primary installation succeed while the Replica fails when using the exact same silent properties file? Quite strange, right? First of all, I tried running the installer manually (outside of Kubernetes or Ansible) to see whether additional details would appear in the console:

[tomcat@otds-1 workspace_otds]$ /app/scripts/workspace_otds/otds/setup -qbi -rf /app/scripts/workspace_otds/otds/silent.properties

OpenText Directory Services 24.4.0

Error, parameter JDBC_CONNECTION_STRING not defined.
[tomcat@otds-1 workspace_otds]$

2. Going further into the installation logs

The generated log file was not really helpful either:

[tomcat@otds-1 workspace_otds]$ cat otds.log
...
2025-08-08  6:38:40 chmod ran successfully on /etc/opentext/unixsetup
2025-08-08  6:38:40 Setting environment variable "ACTION" to "-1" : Success
2025-08-08  6:38:40 Setting environment variable "UPGRADE" to "0" : Success
2025-08-08  6:38:40 Setting environment variable "PATCH" to "0" : Success
2025-08-08  6:38:40 Setting environment variable "INSTALLED" to "0" : Success
2025-08-08  6:38:40 Setting environment variable "INSTALLEDVERSION" to "0.0.0" : Success
2025-08-08  6:38:40 Setting environment variable "PRODUCTINSTANCE" to "1" : Success
2025-08-08  6:38:40 Setting environment variable "PRODUCTVERSION" to "24.4.0.4503" : Success
2025-08-08  6:38:40 Setting environment variable "PRODUCTNAME" to "OpenText Directory Services" : Success
2025-08-08  6:38:40 Setting environment variable "PRODUCTID" to "OTDS" : Success
2025-08-08  6:38:40 Setting environment variable "PATCHVERSION" to "0" : Success
2025-08-08  6:38:40 Setting environment variable "ROOTUSER" to "0" : Success
2025-08-08  6:38:40 Setting environment variable "Main_INSTALLED" to "-1" : Success
2025-08-08  6:38:40 Setting environment variable "INST_GROUP" to "tomcat" : Success
2025-08-08  6:38:40 Setting environment variable "INST_USER" to "tomcat" : Success
2025-08-08  6:38:40 Setting environment variable "INSTALL_DIR" to "/app/tomcat/app_data/otds" : Success
2025-08-08  6:38:40 Setting environment variable "TOMCAT_DIR" to "/app/tomcat" : Success
2025-08-08  6:38:40 Setting environment variable "PRIMARY_FQDN" to "otds-1.otds.otdsdev.svc.cluster.local" : Success
2025-08-08  6:38:40 Setting environment variable "ISREPLICA_TOPOLOGY" to "1" : Success
2025-08-08  6:38:40 Setting environment variable "IMPORT_DATA" to "0" : Success
2025-08-08  6:38:40 Setting environment variable "OTDS_PASS" to "*****" : Success
2025-08-08  6:38:40 Setting environment variable "ENCRYPTION_KEY" to "mqLgucZ8UIUnNcLwjwmhNw==" : Success
2025-08-08  6:38:40 Setting environment variable "MIGRATION_OPENDJ_URL" to "" : Success
2025-08-08  6:38:40 Setting environment variable "MIGRATION_OPENDJ_PASSWORD" to "*****" : Success
2025-08-08  6:38:40 Setting environment variable "JDBC_CONNECTION_STRING" to "" : Success
2025-08-08  6:38:40 Setting environment variable "JDBC_USERNAME" to "" : Success
2025-08-08  6:38:40 Setting environment variable "JDBC_PASSWORD" to "*****" : Success
2025-08-08  6:38:40 Setting environment variable "ACTION" to "3" : Success
2025-08-08  6:38:40 Setting environment variable "Main_ACTION" to "3" : Success
2025-08-08  6:38:40 Adding Pre-req "TOMCAT7_HIGHER"
...
2025-08-08  6:38:40 Action #1 ended: OK
2025-08-08  6:38:40 Setting environment variable "PRIMARY_FQDN" to "otds-1.otds.otdsdev.svc.cluster.local" : Success
2025-08-08  6:38:40 Setting environment variable "ISREPLICA_TOPOLOGY" to "1" : Success
2025-08-08  6:38:40 Skipping IMPORT_DATA parameter (condition is false)
2025-08-08  6:38:40 Skipping OTDS_PASS parameter (condition is false)
2025-08-08  6:38:40 Setting environment variable "ENCRYPTION_KEY" to "mqLgucZ8UIUnNcLwjwmhNw==" : Success
2025-08-08  6:38:40 Skipping MIGRATION_OPENDJ_URL parameter (condition is false)
2025-08-08  6:38:40 Skipping MIGRATION_OPENDJ_PASSWORD parameter (condition is false)
2025-08-08  6:38:40 Error, parameter JDBC_CONNECTION_STRING not defined.
2025-08-08  6:38:40 Setup Ended: 1
2025-08-08  6:38:40 ============= Verbose logging Ended =============
[tomcat@otds-1 workspace_otds]$

For reference, here is the content of the “silent.properties” file that this Replica installation uses:

[tomcat@otds-1 workspace_otds]$ cat otds/silent.properties
[Setup]
Id=OTDS
Version=24.4.0.4503
Patch=0
Basedir=/app/scripts/workspace_otds/otds
Configfile=/app/scripts/workspace_otds/otds/setup.xml
Action=Install
Log=/app/scripts/workspace_otds/otds/otds.log
Instance=1
Feature=All

[Property]
INST_GROUP=tomcat
INST_USER=tomcat
INSTALL_DIR=/app/tomcat/app_data/otds
TOMCAT_DIR=/app/tomcat
PRIMARY_FQDN=otds-1.otds.otdsdev.svc.cluster.local
ISREPLICA_TOPOLOGY=1
IMPORT_DATA=0
OTDS_PASS=m1z6GX+HEX81DRpC
ENCRYPTION_KEY=mqLgucZ8UIUnNcLwjwmhNw==
MIGRATION_OPENDJ_URL=
MIGRATION_OPENDJ_PASSWORD=
JDBC_CONNECTION_STRING=jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=db_host.domain.com)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=db_svc.domain.com)))
JDBC_USERNAME=OTDS
JDBC_PASSWORD=Shu#Asd#Tgb;6799
[tomcat@otds-1 workspace_otds]$

(These are the real passwords from that environment. I have changed them since then, obviously, but I included them so you can understand the details below. The encryption key is altered, though – the system originally took the real one from the “Primary” instance.)

3. Status of the installer created/managed files

After the failure, I checked the parameter file that the OTDS installer populates during installation, but it was mostly empty and not yet filled:

[tomcat@otds-1 workspace_otds]$ cat /etc/opentext/unixsetup/OTDS_parameters_1.txt

#GROUP name that should be used to change file group ownership (group of USER)
INST_GROUP=tomcat

#USER name that should be used to change file ownership (user running processes)
INST_USER=tomcat

#Specify the installation directory for OpenText Directory Services
INSTALL_DIR=/usr/local/OTDS

#Specify the directory, where (64-bit) Apache Tomcat 10 or higher is installed
TOMCAT_DIR=/app/tomcat

#This hostname is used by other instances to connect to the synchronization master host.
PRIMARY_FQDN=

#Is this server a supplementary instance to an existing environment?
ISREPLICA_TOPOLOGY=0

#Specify OpenDJ connection for import.
IMPORT_DATA=0

#Specify the data encryption key from an existing instance.
ENCRYPTION_KEY=

#OpenDJ LDAP URL (example: ldap://localhost:1389)
MIGRATION_OPENDJ_URL=

#Specify JDBC connection String (example: jdbc:postgresql://localhost:5432/postgres). NOTE: Enter these values carefully since they cannot be validated here. Refer to the OTDS installation and administration guide for JDBC URL samples for supported databases.
JDBC_CONNECTION_STRING=

#Specify Database User Name
JDBC_USERNAME=
[tomcat@otds-1 workspace_otds]$

Finally, the “otds.properties” file (normally generated during the installation) was also not present yet:

[tomcat@otds-1 workspace_otds]$ ls -l $APP_DATA/otds/config/otds.properties
ls: cannot access '/app/tomcat/app_data/otds/config/otds.properties': No such file or directory
[tomcat@otds-1 workspace_otds]$

4. Attempts to debug the issue

I tried launching the installer multiple times, on that OTDS Replica, while making small changes to the silent properties file to see if something specific would cause it to fail. Starting by modifying the “JDBC_CONNECTION_STRING” parameter, since that is what the installer complained about, but without success. I then suspected the password parameter. Because passwords are masked in the logs (“*“), it is impossible to see whether the value is parsed correctly or not.

Therefore, I replaced the OTDS Admin password in the silent properties file with “dummyPassword“, and the installer suddenly proceeded further… I cancelled the installation because this was not the real password of the “otadmin” account on the Primary instance, but in this case the “JDBC_CONNECTION_STRING” parameter was no longer empty and the installer continued normally.

Note: the OTDS documentation specifies that passwords must contain at least eight characters, including one lowercase letter, one uppercase letter, one number, and one special character. However, it appears that this rule may not be strictly validated during Replica installations (and possibly not for the Primary either?).

At that point it became clear that the password itself was involved in the issue, somehow. Looking at the script “tools/setup.sh“, you can see that the installer extracts the value of “OTDS_PASS“, applies a function called “AsctoHex“, and then encrypts it. My original “otadmin” password was 16 characters long and satisfied all complexity requirements. However, I noticed that the password contained the string “HEX“. Since the installer converts the password to hexadecimal before encryption, I wondered whether the presence of the string “HEX” might interfere with this process. That would be quite unbelievable, right?

5. A problem with the password length or content?

To test this idea, I removed the “E” in the middle, transforming “HEX” into “HX” and effectively reducing the password length by one character:

[tomcat@otds-1 workspace_otds]$ grep OTDS_PASS otds/silent.properties | awk -F= '{print $2}' | wc -c
17
[tomcat@otds-1 workspace_otds]$ # 17 means 16 char since wc -c count the new line in this command
[tomcat@otds-1 workspace_otds]$
[tomcat@otds-1 workspace_otds]$ sed -i 's,HEX,HX,' otds/silent.properties
[tomcat@otds-1 workspace_otds]$
[tomcat@otds-1 workspace_otds]$ grep OTDS_PASS otds/silent.properties | awk -F= '{print $2}' | wc -c
16
[tomcat@otds-1 workspace_otds]$ # 16 means 15 char now

To re-execute the installer after a failure, you must remove the content of the “/etc/opentext” directory (which kind of caches the content from the “silent.properties” file) and also delete the “otds.properties” file if it exists (not in my case):

[tomcat@otds-1 workspace_otds]$ rm -rf /etc/opentext/*
[tomcat@otds-1 workspace_otds]$

In addition to modifying the “silent.properties” file, I also changed the “otadmin” password through the OTDS otds-admin UI (see the OTDS Install & Admin Guide, section 7.2.5 “Resetting a user password”). Then I started a new Replica installation to see whether changing “HEX” to “HX” from the password would resolve the issue:

[tomcat@otds-1 workspace_otds]$ /app/scripts/workspace_otds/otds/setup -qbi -rf /app/scripts/workspace_otds/otds/silent.properties

OpenText Directory Services 24.4.0

------------------------------------------------------------------------------
  OpenText Directory Services
------------------------------------------------------------------------------
Installing OpenText Directory Services Component
Please wait .
Installation of OpenText Directory Services Component OK

Installation completed. Results:

OpenText Directory Services Component OK

Installation finished.

[tomcat@otds-1 workspace_otds]$

… It worked …?

If the issue was really caused by the presence of “HEX” in the password, then replacing it with “HXE” should also work, right? Unfortunately, when I tried that, the issue came back… This indicates that the real problem is not the literal “HEX” string but maybe something related to password length, complexity, or how the installer processes and encrypts the password internally?

6. Conclusion

In the end, I reverted to the shorter 15-character password that worked and prepared all higher environments at this customer to use 15-character passwords. This approach worked without issue for five additional environments until, of course, it failed again, in Production…

Since it failed in another environment even with a 15-character password, the length alone does not seem to be the root cause. When reviewing previously installed environments across multiple customers, I found a few instances running with “otadmin” passwords of up to 19 characters long (about 111/120 bits of entropy according to a password manager like KeePass). This is significantly stronger than the 15-character password (96 bits) used in the Production environment where the issue occurred.

Therefore, since I couldn’t find any logical reasons why the issue happened on some environments but not with others, I opened a ticket with OpenText. I described everything and we went through several weeks of exchanges to try to find an explanation but without success. As of today, I still don’t know why ~10% of the OTDS Replicas that I installed faced an issue with the OT Admin password, but the fix, was simply to change the password in the UI and start the silent installation again. I don’t have an environment to test / debug that issue anymore, since it’s not easily reproducible. Guess I will need to wait for next time to get more debug logs from the OTDS installer (“-debug” option). In the meantime, I can only assume something is probably wrong in the way OTDS manages the password or its hash.

L’article OTDS – Installation of Replicas fail if the OT Admin password is too long? est apparu en premier sur dbi Blog.

Dctm – Another DM_LICENSE_E_INVALID_LICENSE error but caused by JMS this time

Morgan Patou — Sun, 22 Mar 2026 08:51:00 +0000

At the end of last year, I published a first blog about a DM_LICENSE_E_INVALID_LICENSE error in D2 SSO login through OTDS. The root cause in that previous post was a duplicate user with one lowercase and one uppercase user_login_name. However, I did mention that there can be several reasons for that error. In this blog, I will describe another such case.

1. Symptoms in D2 logs

The generated D2 logs associated with this new issue are almost exactly the same. The only difference is that the Repository returns “null” as the userid (user_name). See the message “Authentication failed for user null with docbase REPO_NAME“. This wasn’t the case in the other blog post:

[tomcat@d2-0 logs]$ cat D2.log
...
2025-12-08 12:21:14,784 UTC [INFO ] (https-jsse-nio-8080-exec-47) - c.emc.x3.portal.server.X3HttpSessionListener  : Created http session 8531D373A3EA12A398B158AF656E7D20
2025-12-08 12:21:14,784 UTC [DEBUG] (https-jsse-nio-8080-exec-47) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : OTDS : No user name on the Http session yet
2025-12-08 12:21:14,785 UTC [DEBUG] (https-jsse-nio-8080-exec-47) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : OTDS : No access_token found in Http request or Cookie Redirecting to OTDS Server
2025-12-08 12:21:14,786 UTC [DEBUG] (https-jsse-nio-8080-exec-47) - c.e.x.p.s.f.a.X3TrustHttpAuthenticationFilter : Identified scheme : https
2025-12-08 12:21:14,786 UTC [DEBUG] (https-jsse-nio-8080-exec-47) - c.e.x.p.s.f.a.X3TrustHttpAuthenticationFilter : Identified server name : d2.domain.com
2025-12-08 12:21:14,787 UTC [DEBUG] (https-jsse-nio-8080-exec-47) - c.e.x.p.s.f.a.X3TrustHttpAuthenticationFilter : Identified server port : 443
2025-12-08 12:21:14,787 UTC [DEBUG] (https-jsse-nio-8080-exec-47) - c.e.x.p.s.f.a.X3TrustHttpAuthenticationFilter : Built server host is : https://d2.domain.com:443
2025-12-08 12:21:14,788 UTC [DEBUG] (https-jsse-nio-8080-exec-47) - o.o.e.logging.slf4j.Slf4JLogLevelHandlers$4   : [EVENT SUCCESS -> /D2/HTTPUtilities] header name=Host, value=d2.domain.com
2025-12-08 12:21:14,789 UTC [DEBUG] (https-jsse-nio-8080-exec-47) - o.o.e.logging.slf4j.Slf4JLogLevelHandlers$4   : [EVENT SUCCESS -> /D2/HTTPUtilities] MaxHeaderValueSize: 8192
2025-12-08 12:21:14,792 UTC [DEBUG] (https-jsse-nio-8080-exec-47) - o.o.e.logging.slf4j.Slf4JLogLevelHandlers$4   : [EVENT SUCCESS -> /D2/HTTPUtilities] validating the input valued2.domain.com
2025-12-08 12:21:14,793 UTC [DEBUG] (https-jsse-nio-8080-exec-47) - c.e.x.p.s.f.a.X3TrustHttpAuthenticationFilter : Identified host : d2.domain.com
2025-12-08 12:21:14,794 UTC [DEBUG] (https-jsse-nio-8080-exec-47) - c.e.x.p.s.f.a.X3TrustHttpAuthenticationFilter : Overall base URL built : https://d2.domain.com/D2
2025-12-08 12:21:14,795 UTC [DEBUG] (https-jsse-nio-8080-exec-47) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : Redirection url post encoding - https%3A%2F%2Fd2.domain.com%2FD2%2Fd2_otds.html%3ForigUrl%3D%2FD2%2F
2025-12-08 12:21:14,797 UTC [DEBUG] (https-jsse-nio-8080-exec-47) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : OTDS : OAUTH final login sendRedirect URL : https://otds-mfa.domain.com/otdsws/oauth2/auth?response_type=token&client_id=dctm-ns-d2&redirect_uri=https%3A%2F%2Fd2.domain.com%2FD2%2Fd2_otds.html%3ForigUrl%3D%2FD2%2F&logon_appname=Documentum+Client+CE+23.4
2025-12-08 12:21:14,798 UTC [DEBUG] (https-jsse-nio-8080-exec-47) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : OTDS : Sending redirection as it's not a rpc call : https://otds-mfa.domain.com/otdsws/oauth2/auth?response_type=token&client_id=dctm-ns-d2&redirect_uri=https%3A%2F%2Fd2.domain.com%2FD2%2Fd2_otds.html%3ForigUrl%3D%2FD2%2F&logon_appname=Documentum+Client+CE+23.4
2025-12-08 12:21:15,018 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - o.o.e.logging.slf4j.Slf4JLogLevelHandlers$4   : [EVENT SUCCESS -> /D2/HTTPUtilities] MaxHeaderKeySize: 256
2025-12-08 12:21:15,018 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - o.o.e.logging.slf4j.Slf4JLogLevelHandlers$4   : [EVENT SUCCESS -> /D2/HTTPUtilities] MaxHeaderValueSize: 8192
2025-12-08 12:21:15,020 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : OTDS : No user name on the Http session yet
2025-12-08 12:21:15,021 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : OTDS : Found access_token on Http Cookie, invalidating the cookie by setting maxAge 0
2025-12-08 12:21:15,022 UTC [INFO ] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : OTDS : setting the cookie as secure as its a https request
2025-12-08 12:21:15,024 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : OTDS : OTDS responded with a oauth token
2025-12-08 12:21:15,025 UTC [INFO ] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : ------ Begin getUntrustedJwtHeader : eyJraWQiOiI1YjM4...oSD8Xh3vVmkekcA
2025-12-08 12:21:15,026 UTC [INFO ] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : getUntrustedJwtHeader oauthTokenWithoutSignature : eyJraWQiOiI1YjM4...i1xYWN0LWQyIn0.
2025-12-08 12:21:15,614 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : ------ Begin validateOTDSTokenClaims : MYUSERID
2025-12-08 12:21:15,615 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  :  validateOTDSTokenClaims for user : MYUSERID , OTDS : currenttime: 1765196475615 expirationtime: 1765200074000
2025-12-08 12:21:15,615 UTC [INFO ] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : ------ End validateOTDSTokenClaims : MYUSERID
2025-12-08 12:21:15,615 UTC [INFO ] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : PublicKey for Key id : 5b38b...bf487 exists
2025-12-08 12:21:15,617 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  :  OTDS Deafault Repository from shiro configured : REPO_NAME
2025-12-08 12:21:15,617 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : OTDS : generating DM_Ticket for user : MYUSERID in Repository : REPO_NAME
2025-12-08 12:21:16,522 UTC [ERROR] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : OTDS : OAuth Token Error occurred while generating a DCTM MultiUse Ticket for user  : MYUSERID
2025-12-08 12:21:16,522 UTC [ERROR] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : OTDS : OAuth Token Error please validate the OTDS Config of user exists in Repository
com.documentum.fc.client.DfAuthenticationException: [DM_SESSION_E_AUTH_FAIL]error:  "Authentication failed for user null with docbase REPO_NAME."
        at com.documentum.fc.client.impl.docbase.DocbaseExceptionMapper.newException(DocbaseExceptionMapper.java:52)
        at com.documentum.fc.client.impl.connection.docbase.MessageEntry.getException(MessageEntry.java:39)
        at com.documentum.fc.client.impl.connection.docbase.DocbaseMessageManager.getException(DocbaseMessageManager.java:137)
        at com.documentum.fc.client.impl.connection.docbase.netwise.NetwiseDocbaseRpcClient.checkForMessages(NetwiseDocbaseRpcClient.java:332)
        at com.documentum.fc.client.impl.connection.docbase.netwise.NetwiseDocbaseRpcClient.applyForObject(NetwiseDocbaseRpcClient.java:680)
        at com.documentum.fc.client.impl.connection.docbase.DocbaseConnection$8.evaluate(DocbaseConnection.java:1572)
        at com.documentum.fc.client.impl.connection.docbase.DocbaseConnection.evaluateRpc(DocbaseConnection.java:1272)
        at com.documentum.fc.client.impl.connection.docbase.DocbaseConnection.applyForObject(DocbaseConnection.java:1564)
        at com.documentum.fc.client.impl.docbase.DocbaseApi.authenticateUser(DocbaseApi.java:1894)
        at com.documentum.fc.client.impl.connection.docbase.DocbaseConnection.authenticate(DocbaseConnection.java:460)
        at com.documentum.fc.client.impl.connection.docbase.DocbaseConnection.open(DocbaseConnection.java:140)
        at com.documentum.fc.client.impl.connection.docbase.DocbaseConnection.(DocbaseConnection.java:109)
        at com.documentum.fc.client.impl.connection.docbase.DocbaseConnection.(DocbaseConnection.java:69)
        at com.documentum.fc.client.impl.connection.docbase.DocbaseConnectionFactory.newDocbaseConnection(DocbaseConnectionFactory.java:32)
        at com.documentum.fc.client.impl.connection.docbase.DocbaseConnectionManager.createNewConnection(DocbaseConnectionManager.java:202)
        at com.documentum.fc.client.impl.connection.docbase.DocbaseConnectionManager.getDocbaseConnection(DocbaseConnectionManager.java:132)
        at com.documentum.fc.client.impl.session.SessionFactory.newSession(SessionFactory.java:24)
        ...
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
        at java.base/java.lang.Thread.run(Thread.java:840)
2025-12-08 12:21:16,524 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : redirectToErrorPage : Redirecting to Error Page as Login failed for user : null and exception : {}
com.emc.x3.portal.server.filters.authc.X3OTDSAuthenticationFilter$1: Authentication failed for user null with repository REPO_NAME.
        at com.emc.x3.portal.server.filters.authc.X3OTDSAuthenticationFilter.validateTokenAndGetUserId(X3OTDSAuthenticationFilter.java:1167)
        at com.emc.x3.portal.server.filters.authc.X3OTDSAuthenticationFilter.onAccessDenied(X3OTDSAuthenticationFilter.java:293)
        at org.apache.shiro.web.filter.AccessControlFilter.onAccessDenied(AccessControlFilter.java:133)
        at org.apache.shiro.web.filter.AccessControlFilter.onPreHandle(AccessControlFilter.java:162)
        at org.apache.shiro.web.filter.PathMatchingFilter.isFilterChainContinued(PathMatchingFilter.java:223)
        at org.apache.shiro.web.filter.PathMatchingFilter.preHandle(PathMatchingFilter.java:198)
        ...
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
        at java.base/java.lang.Thread.run(Thread.java:840)
2025-12-08 12:21:16,524 UTC [INFO ] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : Adding the LicenseException to the Session : DM_SESSION_E_AUTH_FAIL
2025-12-08 12:21:16,526 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.a.X3TrustHttpAuthenticationFilter : Identified scheme : https
2025-12-08 12:21:16,526 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.a.X3TrustHttpAuthenticationFilter : Identified server name : d2.domain.com
2025-12-08 12:21:16,526 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.a.X3TrustHttpAuthenticationFilter : Identified server port : 443
2025-12-08 12:21:16,528 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.a.X3TrustHttpAuthenticationFilter : Built server host is : https://d2.domain.com:443
2025-12-08 12:21:16,529 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - o.o.e.logging.slf4j.Slf4JLogLevelHandlers$4   : [EVENT SUCCESS -> /D2/HTTPUtilities] header name=Host, value=d2.domain.com
2025-12-08 12:21:16,530 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - o.o.e.logging.slf4j.Slf4JLogLevelHandlers$4   : [EVENT SUCCESS -> /D2/HTTPUtilities] MaxHeaderValueSize: 8192
2025-12-08 12:21:16,531 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - o.o.e.logging.slf4j.Slf4JLogLevelHandlers$4   : [EVENT SUCCESS -> /D2/HTTPUtilities] validating the input valued2.domain.com
2025-12-08 12:21:16,532 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.a.X3TrustHttpAuthenticationFilter : Identified host : d2.domain.com
2025-12-08 12:21:16,533 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.a.X3TrustHttpAuthenticationFilter : Overall base URL built : https://d2.domain.com/D2
2025-12-08 12:21:16,534 UTC [DEBUG] (https-jsse-nio-8080-exec-5) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  :  D2 redirecting to errorPage JSP  : https://d2.domain.com/D2/errors/authenticationError.jsp
2025-12-08 12:21:16,567 UTC [DEBUG] (https-jsse-nio-8080-exec-26) - o.o.e.logging.slf4j.Slf4JLogLevelHandlers$4   : [EVENT SUCCESS -> /D2/HTTPUtilities] MaxHeaderKeySize: 256
2025-12-08 12:21:16,567 UTC [DEBUG] (https-jsse-nio-8080-exec-26) - o.o.e.logging.slf4j.Slf4JLogLevelHandlers$4   : [EVENT SUCCESS -> /D2/HTTPUtilities] MaxHeaderValueSize: 8192
2025-12-08 12:21:16,568 UTC [DEBUG] (https-jsse-nio-8080-exec-26) - c.e.x.p.s.f.authc.X3OTDSAuthenticationFilter  : No LicenseExcepton found on HttpSession hence not Redirectling to License ErrorPage
2025-12-08 12:21:16,571 UTC [DEBUG] (https-jsse-nio-8080-exec-26) - c.e.x.p.s.f.a.X3TrustHttpAuthenticationFilter :  Selected Repository : REPO_NAME
2025-12-08 12:21:16,573 UTC [DEBUG] (https-jsse-nio-8080-exec-26) - o.o.e.logging.slf4j.Slf4JLogLevelHandlers$4   : [EVENT SUCCESS -> /D2/HTTPUtilities] MaxHeaderKeySize: 256
2025-12-08 12:21:16,574 UTC [DEBUG] (https-jsse-nio-8080-exec-26) - o.o.e.logging.slf4j.Slf4JLogLevelHandlers$4   : [EVENT SUCCESS -> /D2/HTTPUtilities] MaxHeaderValueSize: 8192
2025-12-08 12:21:16,578 UTC [INFO ] (https-jsse-nio-8080-exec-26) - c.emc.x3.portal.server.X3HttpSessionListener  : Expired Http session id : 8531D373A3EA12A398B158AF656E7D20
2025-12-08 12:21:16,578 UTC [DEBUG] (https-jsse-nio-8080-exec-26) - com.emc.x3.server.context.ContextManager      : Create a new context manager
...
[tomcat@d2-0 logs]$

2. Checking the Repository authentication trace

As usual, the next step is to check the Repository logs with the authentication trace enabled:

[dmadmin@cs-0 ~]$ cat $DOCUMENTUM/dba/log/$DOCBASE_NAME.log
...
2025-12-08T12:21:16.235912      3567122[3567122]        0101234580c77e96        [AUTH]  Entering RPC AUTHENTICATE_USER
2025-12-08T12:21:16.236052      3567122[3567122]        0101234580c77e96        [AUTH]  Start Authentication : LOGON_NAME=MYUSERID, DOMAIN_NAME=, OS_LOGON_NAME=tomcat, OS_LOGON_DOMAIN=, ASSUME_USER=0, TRUSTED_LOGIN_ALLOWED=1, PRINCIPAL_AUTH=0, DO_SET_LOCALE=0, RECONNECT=0, CLIENT_TOKEN=[-36, 8, 66, 12, 89, 102, -85, -11, 6, -115, -34, -68, -123, 11, 100]
2025-12-08T12:21:16.236115      3567122[3567122]        0101234580c77e96        [AUTH]  Start Authenticate Client Instance
2025-12-08T12:21:16.236215      3567122[3567122]        0101234580c77e96        [AUTH]  Start Verify Signature, Client : dfc_327WHMY40Mglbp4taDgajZEM39Lc , Host : d2-0.d2.dctm-ns.svc.cluster.local
2025-12-08T12:21:16.244603      3567122[3567122]        0101234580c77e96        [AUTH]  End Verify Signature, Client : dfc_327WHMY40Mglbp4taDgajZEM39Lc , Host : d2-0.d2.dctm-ns.svc.cluster.local
2025-12-08T12:21:16.244657      3567122[3567122]        0101234580c77e96        [AUTH]  End Authenticate Client Instance
2025-12-08T12:21:16.303325      3567122[3567122]        0101234580c77e96        [AUTH]  Start-AuthenticateUser: ClientHost(d2-0.d2.dctm-ns.svc.cluster.local), LogonName(null), LogonOSName(tomcat), LogonOSDomain(), UserExtraDomain(), ServerDomain()
2025-12-08T12:21:16.303410      3567122[3567122]        0101234580c77e96        [AUTH]  Start-AuthenticateUserName:
2025-12-08T12:21:16.303442      3567122[3567122]        0101234580c77e96        [AUTH]  dmResolveNamesForCredentials: auth_protocol()
2025-12-08T12:21:16.305698      3567122[3567122]        0101234580c77e96        [AUTH]  [DM_USER_E_NOT_DOCUMENTUM_USER]error:  "User null does not exist in the docbase"
2025-12-08T12:21:16.305720      3567122[3567122]        0101234580c77e96        [AUTH]  End-AuthenticateUserName: dm_user.user_login_domain(), Result: 0
2025-12-08T12:21:16.305730      3567122[3567122]        0101234580c77e96        [AUTH]  Not Found dm_user.user_login_name(null), dm_user.user_login_domain()
2025-12-08T12:21:16.519331      3567122[3567122]        0101234580c77e96        [AUTH]  Final Auth Result=F, LOGON_NAME=null, AUTHENTICATION_LEVEL=1, OS_LOGON_NAME=tomcat, OS_LOGON_DOMAIN=, CLIENT_HOST_NAME=d2-0.d2.dctm-ns.svc.cluster.local, CLIENT_HOST_ADDR=172.1.1.1, USER_LOGON_NAME_RESOLVED=1, AUTHENTICATION_ONLY=0, USER_NAME=, USER_OS_NAME=null, USER_LOGIN_NAME=null, USER_LOGIN_DOMAIN=, USER_EXTRA_CREDENTIAL[0]=, USER_EXTRA_CREDENTIAL[1]=, USER_EXTRA_CREDENTIAL[2]=e2, USER_EXTRA_CREDENTIAL[3]=, USER_EXTRA_CREDENTIAL[4]=, USER_EXTRA_CREDENTIAL[5]=, SERVER_SESSION_ID=0101234580c77e96, AUTH_BEGIN_TIME=Mon Dec  8 12:21:16 2025, AUTH_END_TIME=Mon Dec  8 12:21:16 2025, Total elapsed time=0 seconds
2025-12-08T12:21:16.519359      3567122[3567122]        0101234580c77e96        [AUTH]  Exiting RPC AUTHENTICATE_USER
...
[dmadmin@cs-0 ~]$

There is one thing that is quite strange in these logs. If you look at the beginning, it traces the authentication for “MYUSERID“. But then, in the middle of the process, that user_name becomes “null“. I do not recall seeing that behavior before, so I started investigating what might have caused it.

The account “MYUSERID” existed in the Repository. This issue occurred on the same application as in the previous blog post, but this time in the TEST/QA environment (instead of DEV). The same OTDS and users were present, so my account was definitely there (without duplicates in TEST/QA).

3. Investigating OTDS authentication logs

Since the dm_user object had a “user_source” of OTDS, I then checked the OTDS Authentication log file from the JMS. For this Documentum 23.4 version, the log file was “$JMS_HOME/logs/otdsauth.log“. Starting from version 25.4, this log file is located inside “$DOCUMENTUM/dba/log” instead:

[dmadmin@cs-0 ~]$ cat $JMS_HOME/logs/otdsauth.log
...
2025-12-08 11:49:46,106 UTC ERROR [] (https-jsse-nio-9082-exec-36) Thread[https-jsse-nio-9082-exec-36,5,main] java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 502 Bad Gateway"
        at java.base/sun.net.www.protocol.http.HttpURLConnection.doTunneling0(HttpURLConnection.java:2311)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.doTunneling(HttpURLConnection.java:2181)
        at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1465)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1436)
        at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:220)
        at com.documentum.cs.otds.OTDSAuthenticationServlet.validatePassword(OTDSAuthenticationServlet.java:275)
        at com.documentum.cs.otds.OTDSAuthenticationServlet.doPost(OTDSAuthenticationServlet.java:175)
        at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:590)
        ...
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1740)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.base/java.lang.Thread.run(Thread.java:840)

2025-12-08 12:21:16,302 UTC ERROR [] (https-jsse-nio-9082-exec-50) Exception while fetching certificates from jwks url
[dmadmin@cs-0 ~]$

The first error message (11:49) occurred about 30 minutes before the authentication attempt. On the other hand, the last line (12:21) is directly linked to the problem according to its timestamp. This indicates that the Documentum Server was trying to fetch the JWKS certificate. This happens when the OTDS Authentication Servlet is configured with the “auto_cert_refresh=true” parameter (see the “otdsauth.properties” file).

This forces the Documentum Server to contact the OTDS Server in order to retrieve the correct or current SSL certificate to use. However, that request failed. Even though it is not explicitly written, it is easy to deduce that the first error, related to a proxy communication issue, is the root cause.

4. Checking newly added proxy and correcting it

As far as I knew, there should not have been any proxy configured on Documentum, since all components are internal to the customer and located within the same network. However, when checking the startup logs of the JMS, I noticed that a new proxy configuration had recently been added when the Tomcat process restarted less than two hours earlier:

[dmadmin@cs-0 ~]$ grep proxy $JMS_HOME/logs/catalina.out
...
2025-12-08 10:54:56,385 UTC INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dhttp.proxyHost=proxy.domain.com
2025-12-08 10:54:56,385 UTC INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dhttp.proxyPort=2010
2025-12-08 10:54:56,385 UTC INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dhttps.proxyHost=proxy.domain.com
2025-12-08 10:54:56,385 UTC INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dhttps.proxyPort=2011
...
[dmadmin@cs-0 ~]$

After checking with the relevant teams, it turned out that this issue was not really related to Documentum itself. Someone had simply restarted the JMS after adding proxy settings as new JVM parameters while testing an external service that required internet access. Yes, directly in TEST/QA without validating in DEV first – it happens apparently.

However, since no exceptions were configured through the no_proxy setting (“-Dhttp.nonProxyHosts” JVM parameter), it meant that 100% of the requests initiated by the JVM were forwarded to the proxy. That proxy had no knowledge of the OTDS server (which is expected), so the communication simply failed.

After correcting the proxy configuration (either by removing it or by adding all internal domains to the no_proxy setting), the JVM was able to communicate with OTDS again. As a consequence, the D2 SSO started working successfully and the environment was back “online” for all testers. These two blog posts clearly demonstrate that just because D2 displays an error, it doesn’t mean that the real root cause is obvious. Careful investigation and analysis of the log files is always essential.

L’article Dctm – Another DM_LICENSE_E_INVALID_LICENSE error but caused by JMS this time est apparu en premier sur dbi Blog.

Dctm – IDS Source 16.7.5 config.bin crash during execution

Morgan Patou — Tue, 10 Mar 2026 18:56:00 +0000

Around six months ago, I faced a confusing issue with IDS Source 16.7.5 where the “config.bin” executable always crashed when I tried to run it. The installation of IDS binaries itself completed successfully without any errors. However, the configurator, which is supposed to set up the required objects inside the Repository, consistently crashed.

1. Environment context and IDS upgrade

This Documentum environment had just been upgraded to 23.4. The next step was to upgrade the associated IDS component. The latest version of IDS compatible with recent Documentum versions is 16.7.5.

The execution of the “idsLinuxSuiteSetup.bin” installer properly extracted all binaries and deployed the WebCache application in its Tomcat server. To quickly verify that, you can check the version properties file and try starting/stopping the Tomcat instance of the IDS. On my side, there were no problems with that.

To verify the installed version of IDS and ensure that the configurator was also updated:

[dmadmin@cs-0 ~]$ cd $DM_HOME/webcache
[dmadmin@cs-0 webcache]$
[dmadmin@cs-0 webcache]$ cat version/version.properties
#Please don't remove this values
#Fri Oct 10 09:52:49 UTC 2025
INSTALLER_NAME=IDS
PRODUCT_VERSION=16.7.5
[dmadmin@cs-0 webcache]$
[dmadmin@cs-0 webcache]$ ls -l install/config.bin
-rwxrwxr-x 1 dmadmin dmadmin 54943847 Oct 19  2024 install/config.bin
[dmadmin@cs-0 webcache]$

The above confirms that WebCache was properly updated to version 16.7.5 on October 10. It also confirms that the “config.bin” is fairly recent (Q4 2024), i.e. much more recent that the old 16.7.4 file.

2. Running the IDS configurator in silent

My next step was therefore to execute the configurator, still in silent mode, as I have done for all previous IDS installations and configurations. I have not written a blog about IDS silent installation yet, but I have done so for several other components. For example, you can refer to this post for the latest one published.

The silent properties file for the IDS Source configurator is quite simple, as it only requires the Repository name to configure:

[dmadmin@cs-0 webcache]$ cat ${install_file}
### Silent installation response file for IDS configurator
INSTALLER_UI=silent
KEEP_TEMP_FILE=true

### Configuration parameters
DOC_BASE_NAME=REPO_01

[dmadmin@cs-0 webcache]$

Initially, I simply executed “config.bin“. Since it crashed and there was absolutely nothing in the logs, I had to run it again with the DEBUG flag enabled:

[dmadmin@cs-0 webcache]$ $DM_HOME/webcache/install/config.bin -DLOG_LEVEL=DEBUG -f ${install_file}
Preparing to install
Extracting the installation resources from the installer archive...
Configuring the installer for this system's environment...

Launching installer...

Picked up JAVA_TOOL_OPTIONS: -Djdk.util.zip.disableZip64ExtraFieldValidation=true -Djava.locale.providers=COMPAT,SPI --add-exports=java.base/sun.security.provider=ALL-UNNAMED --add-exports=java.base/sun.security.pkcs=ALL-UNNAMED --add-exports=java.base/sun.security.x509=ALL-UNNAMED --add-exports=java.base/sun.security.util=ALL-UNNAMED --add-exports=java.base/sun.security.tools.keytool=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED
[dmadmin@cs-0 webcache]$
[dmadmin@cs-0 webcache]$ echo $?
1
[dmadmin@cs-0 webcache]$

3. Investigating the logs

As shown above, the execution failed, as the return code was “1“. With DEBUG enabled and after checking the generated files, I found the following:

[dmadmin@cs-0 webcache]$ find . -type f -mmin -20 -ls
92475248907    380 -rw-rw-r--   1  dmadmin  dmadmin    384222 Oct 10 11:58 ./install/logs/install.log
92470810541      4 -rw-rw-r--   1  dmadmin  dmadmin       219 Oct 10 11:57 ./install/installer.properties
92475252084     12 -rwxrwxrwx   1  dmadmin  dmadmin     10564 Oct 10 11:57 ./install/config_log/OpenText_Documentum_Interactive_Delivery_Services_Configuration_Install_10_10_2025_11_57_42.log
[dmadmin@cs-0 webcache]$
[dmadmin@cs-0 webcache]$ grep -iE "_E_|_F_|ERROR|WARN|FATAL" install/logs/install.log
TYPE ERROR_TYPE 0000000000000000 0 0 0
13:24:12,192 DEBUG [main] com.documentum.install.shared.common.error.DiException - null/dba/config/GR_REPO/webcache.ini (No such file or directory)
13:24:12,193 DEBUG [main] com.documentum.install.shared.common.error.DiException - null
13:24:12,194 DEBUG [main] com.documentum.install.shared.common.error.DiException - null/dba/config/REPO_01/webcache.ini (No such file or directory)
13:24:12,194 DEBUG [main] com.documentum.install.shared.common.error.DiException - null
13:24:12,199 DEBUG [main] com.documentum.install.shared.common.error.DiException - null/dba/config/GR_REPO/webcache.ini (No such file or directory)
13:24:12,199 DEBUG [main] com.documentum.install.shared.common.error.DiException - null
13:24:12,199 DEBUG [main] com.documentum.install.shared.common.error.DiException - null/dba/config/REPO_01/webcache.ini (No such file or directory)
13:24:12,199 DEBUG [main] com.documentum.install.shared.common.error.DiException - null
13:24:12,200 DEBUG [main] com.documentum.install.shared.common.error.DiException - null/dba/config/REPO_01/webcache.ini (No such file or directory)
13:24:12,200 DEBUG [main] com.documentum.install.shared.common.error.DiException - null
TYPE ERROR_TYPE 0000000000000000 0 0 0
[dmadmin@cs-0 webcache]$

The DEBUG logs above might make it look like the “$DOCUMENTUM” environment variable is missing, since it complains about “null/dba/xxx” not being found. However, that is not the issue. I checked all parameters and environment variables, and everything was configured correctly. In addition, Documentum had just been successfully upgraded from version 20.2 to 23.4 from start to finish, which confirmed that there was no problem with the OS or environment configuration. So I checked the second file:

[dmadmin@cs-0 webcache]$ cat install/config_log/OpenText_Documentum_Interactive_Delivery_Services_Configuration_Install_10_10_2025_11_57_42.log

__________________________________________________________________________

Fri Oct 10 11:57:50 UTC 2025

Free Memory: 15947 kB
Total Memory: 49152 kB
...
Summary
-------

Installation: Successful with errors.

8 Successes
0 Warnings
1 NonFatalErrors
0 FatalErrors
...

Custom Action:            com.documentum.install.webcache.CustomActions.DiWAWebcsConfigureDocbase
                          Status: ERROR
                          Additional Notes: ERROR -     class com.documentum.install.webcache.CustomActions.DiWAWebcsConfigureDocbase.install() runtime exception:

...
====================STDERR ENTRIES==================

RepositoryManager: Trying fallback repository location...
8. final log file name=$DM_HOME/webcache/install/config_log/OpenText_Documentum_Interactive_Delivery_Services_Configuration_Install_10_10_2025_11_57_42.log
java.lang.NumberFormatException: For input string: ""
        at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
        at java.base/java.lang.Integer.parseInt(Integer.java:678)
        at java.base/java.lang.Integer.parseInt(Integer.java:786)
        at com.documentum.install.webcache.CustomActions.DiWAWebcsConfigureDocbase.configureDocbase(DiWAWebcsConfigureDocbase.java:329)
        at com.documentum.install.webcache.CustomActions.DiWAWebcsConfigureDocbase.install(DiWAWebcsConfigureDocbase.java:202)
        at com.zerog.ia.installer.actions.CustomAction.installSelf(Unknown Source)
        at com.zerog.ia.installer.InstallablePiece.install(Unknown Source)
        at com.zerog.ia.installer.InstallablePiece.install(Unknown Source)
        at com.zerog.ia.installer.GhostDirectory.install(Unknown Source)
        at com.zerog.ia.installer.InstallablePiece.install(Unknown Source)
        at com.zerog.ia.installer.Installer.install(Unknown Source)
        at com.zerog.ia.installer.LifeCycleManager.consoleInstallMain(Unknown Source)
        at com.zerog.ia.installer.LifeCycleManager.executeApplication(Unknown Source)
        at com.zerog.ia.installer.Main.main(Unknown Source)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at com.zerog.lax.LAX.launch(Unknown Source)
        at com.zerog.lax.LAX.main(Unknown Source)
Execute Custom Code
    class com.documentum.install.webcache.CustomActions.DiWAWebcsConfigureDocbase.install() runtime exception:
java.awt.HeadlessException:
No X11 DISPLAY variable was set,
but this program performed an operation which requires it.
        at java.desktop/java.awt.GraphicsEnvironment.checkHeadless(GraphicsEnvironment.java:164)
        at java.desktop/java.awt.Window.(Window.java:553)
        at java.desktop/java.awt.Frame.(Frame.java:428)
        at java.desktop/java.awt.Frame.(Frame.java:393)
        at java.desktop/javax.swing.JFrame.(JFrame.java:180)
        at com.documentum.install.webcache.CustomActions.DiWAWebcsConfigureDocbase.install(DiWAWebcsConfigureDocbase.java:215)
        at com.zerog.ia.installer.actions.CustomAction.installSelf(Unknown Source)
        at com.zerog.ia.installer.InstallablePiece.install(Unknown Source)
        at com.zerog.ia.installer.InstallablePiece.install(Unknown Source)
        at com.zerog.ia.installer.GhostDirectory.install(Unknown Source)
        at com.zerog.ia.installer.InstallablePiece.install(Unknown Source)
        at com.zerog.ia.installer.Installer.install(Unknown Source)
        at com.zerog.ia.installer.LifeCycleManager.consoleInstallMain(Unknown Source)
        at com.zerog.ia.installer.LifeCycleManager.executeApplication(Unknown Source)
        at com.zerog.ia.installer.Main.main(Unknown Source)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at com.zerog.lax.LAX.launch(Unknown Source)
        at com.zerog.lax.LAX.main(Unknown Source)
Retrying Installables deferred in pass 0
Deferral retries done because:
There were no deferrals in the last pass.
8. final log file name=$DM_HOME/webcache/install/config_log/OpenText_Documentum_Interactive_Delivery_Services_Configuration_Install_10_10_2025_11_57_42.log
====================STDOUT ENTRIES==================
...
[dmadmin@cs-0 webcache]$

That log file appeared to indicate that a certain “Number” value might be missing (“NumberFormatException“). Without access to the IDS source code (and I always avoid de-compiling Documentum source files), it was impossible to determine exactly what was missing. There were no additional details in the logs, so in the end I had to reach out to OpenText support to find out what was causing the issue.

4. Root cause: missing value for TOMCAT_PORT_SELECTED

After several back-and-forth exchanges and around 12 days of waiting for a solution, I finally received confirmation that this was a bug in the IDS Source 16.7.5 software. This version is the first one deployed on Tomcat instead of WildFly, so it was somewhat expected that some issues might appear.

When installing the IDS Source binaries, the silent installation properties file requires you to define the port that Tomcat will use. This parameter is “USER_PORT_CHOICE=6677“. You can of course change the port if needed, but 6677 was the default port used with previous IDS versions running on WildFly, so I kept the same value when installing IDS 16.7.5 on Tomcat.

The bug is that even though this value is used correctly during the Tomcat installation step, it is not properly written into the properties file that the configuration process later relies on. The IDS Source “config.bin” looks for the file “$DM_HOME/webcache/scs_tomcat.properties” and reads the port from the “TOMCAT_PORT_SELECTED” parameter.

However, in IDS 16.7.5 this file is not updated during installation. As a result, the port value remains empty, which corresponds to the missing number referenced in the logs and causes the configurator to crash.

5. Fix: updating scs_tomcat.properties

The solution is fairly simple: manually update that file and run the configurator again. In my case, I used the HTTPS port 6679, since my Tomcat was already in SSL (6677 + 2 = 6679):

[dmadmin@cs-0 webcache]$ port=6679
[dmadmin@cs-0 webcache]$ sed -i "s,\(TOMCAT_PORT_SELECTED=\).*,\1${port}," $DM_HOME/webcache/scs_tomcat.properties
[dmadmin@cs-0 webcache]$
[dmadmin@cs-0 webcache]$
[dmadmin@cs-0 webcache]$ $DM_HOME/webcache/install/config.bin -DLOG_LEVEL=DEBUG -f ${install_file}
Preparing to install
Extracting the installation resources from the installer archive...
Configuring the installer for this system's environment...

Launching installer...

Picked up JAVA_TOOL_OPTIONS: -Djdk.util.zip.disableZip64ExtraFieldValidation=true -Djava.locale.providers=COMPAT,SPI --add-exports=java.base/sun.security.provider=ALL-UNNAMED --add-exports=java.base/sun.security.pkcs=ALL-UNNAMED --add-exports=java.base/sun.security.x509=ALL-UNNAMED --add-exports=java.base/sun.security.util=ALL-UNNAMED --add-exports=java.base/sun.security.tools.keytool=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED
[dmadmin@cs-0 webcache]$
[dmadmin@cs-0 webcache]$ echo $?
0
[dmadmin@cs-0 webcache]$

As you can see above, the return code is now “0“, which indicates a successful execution. The logs generated during this new attempt are much cleaner, and there are no longer any exceptions or errors:

[dmadmin@cs-0 webcache]$ cat install/config_log/OpenText_Documentum_Interactive_Delivery_Services_Configuration_Install_10_22_2025_13_37_46.log
__________________________________________________________________________

Wed Oct 22 01:39:40 UTC 2025

Free Memory: 14800 kB
Total Memory: 49152 kB
...
Summary
-------

Installation: Successful.

9 Successes
0 Warnings
0 NonFatalErrors
0 FatalErrors
...

Custom Action:            com.documentum.install.webcache.CustomActions.DiWAWebcsConfigureDocbase
                          Status: SUCCESSFUL

...
====================STDERR ENTRIES==================

RepositoryManager: Trying fallback repository location...
8. final log file name=$DM_HOME/webcache/install/config_log/OpenText_Documentum_Interactive_Delivery_Services_Configuration_Install_10_22_2025_13_37_46.log
Retrying Installables deferred in pass 0
Deferral retries done because:
There were no deferrals in the last pass.
8. final log file name=$DM_HOME/webcache/install/config_log/OpenText_Documentum_Interactive_Delivery_Services_Configuration_Install_10_22_2025_13_37_46.log
====================STDOUT ENTRIES==================
...
[dmadmin@cs-0 webcache]$

As mentioned earlier, this configurator is responsible for installing components inside the Repository. It creates the required IDS objects or updates them if they already exist. The DAR files were also successfully installed:

[dmadmin@cs-0 webcache]$ iapi $DOCBASE_NAME -Udmadmin -Pxxx << EOC
> ?,c,select r_object_id, r_modify_date, object_name from dmc_dar order by r_modify_date asc;
> EOC

        OpenText Documentum iapi - Interactive API interface
        Copyright (c) 2023. OpenText Corporation
        All rights reserved.
        Client Library Release 23.4.0000.0180

Connecting to Server using docbase REPO_01
[DM_SESSION_I_SESSION_START]info:  "Session 011234568027fb88 started for user dmadmin."

Connected to OpenText Documentum Server running Release 23.4.0000.0143  Linux64.Oracle
1> 2>
r_object_id       r_modify_date              object_name
----------------  -------------------------  ---------------------------------
...               ...                        ...
08123456800c99a5  10/22/2025 13:38:32        SCSDocApp
08123456800c99be  10/22/2025 13:38:58        SCSWorkflow
08123456800c99e1  10/22/2025 13:39:29        icmRating

(43 rows affected)
1>
[dmadmin@cs-0 webcache]$

6. Another small bug

However, I later discovered another small bug. The “scs_admin_config.product_version” attribute in the Repository was not updated correctly. Previously installed version was 16.7.4, so it’s unclear whether the configurator updated the value (with 16.7.4 still) or not at all. In any case, the stored product version was incorrect.

This value is used by IDS to verify version compatibility during execution. For example, you can see this version referenced during the End-to-End tests. Therefore, I had to update the value manually. To correct the issue:

[dmadmin@cs-0 webcache]$ iapi $DOCBASE_NAME -Udmadmin -Pxxx << EOC
> ?,c,select product_version from scs_admin_config;
> ?,c,update scs_admin_config object set product_version='16.7.5' where product_version='16.7.4';
> ?,c,select product_version from scs_admin_config;
> exit
> EOC

        OpenText Documentum iapi - Interactive API interface
        Copyright (c) 2023. OpenText Corporation
        All rights reserved.
        Client Library Release 23.4.0000.0180

Connecting to Server using docbase REPO_01
[DM_SESSION_I_SESSION_START]info:  "Session 011234568027fd13 started for user dmadmin."

Connected to OpenText Documentum Server running Release 23.4.0000.0143  Linux64.Oracle
Session id is s0
API>
product_version
------------------------
16.7.4
(1 row affected)

API>
objects_updated
---------------
              1
(1 row affected)
[DM_QUERY_I_NUM_UPDATE]info:  "1 objects were affected by your UPDATE statement."

API>
product_version
------------------------
16.7.5
(1 row affected)

API> Bye
[dmadmin@cs-0 webcache]$

OpenText mentioned that both of these bugs should normally be fixed in a future update of the binaries. I have not checked in the last six months, but hopefully the issue has already been resolved. If not, at least you now have the information needed to fix it!

L’article Dctm – IDS Source 16.7.5 config.bin crash during execution est apparu en premier sur dbi Blog.

Dctm – Upgrade from 23.4 to 25.4 fails with DM_SERVER_E_SOCKOPT

Morgan Patou — Sat, 07 Mar 2026 11:43:02 +0000

Since its release in Q4 2025, I have worked on several upgrades from 23.4 to 25.4. The first one I worked on was for a customer using our own custom Documentum images. If you have followed my posts for some time, you might recall a previous blog post where I discussed a Documentum 20.2 to 23.4 upgrade and some issues related to IPv6 being disabled (c.f. this blog).

1. Environment details

The source Documentum 23.4 environment was configured exactly like the one from that previous blog post, with the fix to ensure the components could start and run on IPv4 only. Using the exact same source code, I simply rebuilt the image with the same OS version (RHEL8 for Dctm 23.4). One difference was that I was previously running on a vanilla Kubernetes cluster, while for this blog I used RKE2 (from SUSE), meaning the OS and Kubernetes cluster were not the same.

At the Documentum level, the Connection Broker was configured with “host=${Current-IPv4}” to force it to start with IPv4 only. The Repository had “ip_mode = V4ONLY” in its “server.ini” to achieve the same behavior. With these two configurations, the Documentum 23.4 environment was installed from scratch without any issues and was running properly. I performed several pod restarts over the next few days, and everything was running smoothly.

2. Upgrade to Documentum 25.4

Then it was time to upgrade the environment to 25.4. For that purpose, I built a new image on RHEL9 (switching from ubi8 to ubi9 for the base image). I had no problems with the preparation or the installation of the other binaries.

I then triggered the upgrade process, which completed successfully for the Connection Broker. However, when it reached the Repository part, things were not that simple. The Repository upgrade log file contained the following:

[dmadmin@cs-0 ~]$ cat $DM_HOME/install/logs/install.log
20:30:29,056  INFO [main] com.documentum.install.shared.installanywhere.actions.InitializeSharedLibrary - Log is ready and is set to the level - INFO
20:30:29,060  INFO [main] com.documentum.install.shared.installanywhere.actions.InitializeSharedLibrary - The product name is: UniversalServerConfigurator
20:30:29,060  INFO [main] com.documentum.install.shared.installanywhere.actions.InitializeSharedLibrary - The product version is: 25.4.0000.0143
20:30:29,060  INFO [main]  -
20:30:29,089  INFO [main] com.documentum.install.shared.installanywhere.actions.InitializeSharedLibrary - Done InitializeSharedLibrary ...
20:30:29,117  INFO [main] com.documentum.install.server.installanywhere.actions.DiWASilentCheckVaultStatus - Checking the vault status: Silent mode
20:30:29,117  INFO [main] com.documentum.install.server.installanywhere.actions.DiWASilentCheckVaultStatus - Checking whether vault enabled or not
20:30:29,121  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerInformation - Setting CONFIGURE_DOCBROKER value to TRUE for SERVER
20:30:29,121  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerInformation - Setting CONFIGURE_DOCBASE value to TRUE for SERVER
20:30:30,124  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerCheckEnvrionmentVariable - The installer was started using the dm_launch_server_config_program.sh script.
20:30:30,124  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerCheckEnvrionmentVariable - The installer will determine the value of environment variable DOCUMENTUM.
20:30:30,124  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerCheckEnvrionmentVariable - existingVersion : 25.4serverMajorversion : 25.4
20:30:33,125  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerCheckEnvrionmentVariable - The installer will determine the value of environment variable PATH.
20:30:33,125  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerCheckEnvrionmentVariable - existingVersion : 25.4serverMajorversion : 25.4
20:30:36,126  INFO [main]  - existingVersion : 25.4serverMajorversion : 25.4
20:30:36,136  INFO [main] com.documentum.install.server.installanywhere.actions.DiWASilentConfigurationInstallationValidation - Start to validate docbase parameters.
20:30:36,140  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerPatchExistingDocbaseAction - The installer will obtain all the DOCBASE on the machine.
20:30:38,146  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerDocAppFolder - The installer will obtain all the DocApps which could be installed for the repository.
20:30:38,148  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerLoadDocBaseComponentInfo - The installer will gather information about the component GR_REPO.
20:30:41,151  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerReadServerIniForLockbox - Lockbox disabled for the repository : GR_REPO.
20:30:41,153  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerCheckKeepAEKUnchanged - vaule of isVaultEnabledinPrevious : null
20:30:41,156  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerCheckKeystoreStatusForOld - The installer will check old AEK key status.
20:30:41,219  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerCheckKeystoreStatus - Executed dm_crypto_create -check command and the return code - 1
20:30:41,221  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerLoadValidAEKs - AEK key type provided in properties is  : Local
20:30:41,266  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerCheckKeystoreStatus - Executed dm_crypto_create -check command and the return code - 1
20:30:41,269  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerEnableLockBoxValidation - The installer will validate AEK fields.
20:30:41,272  INFO [main]  - Is Vault enabled :false
20:30:41,273  INFO [main]  - Is Vault enabled :false
20:30:41,326  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerValidateLockboxPassphrase - Installer will boot AEK key
20:31:11,376  INFO [main]  - Is Vault enabled :false
20:31:11,377  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerCheckKeystoreStatus - The installer will check keystore status.
20:31:11,419  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerCheckKeystoreStatus - Executed dm_crypto_create -check command and the return code - 1
20:31:11,419  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerCheckKeystoreStatus - The installer detected the keystore already exists and was created using user password.
20:31:11,425  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerQueryDatabaseInformation - The installer is gathering database connection information from the local machine.
20:31:11,427  INFO [main] com.documentum.install.appserver.services.DiAppServerUtil - appServer == null
20:31:11,427  INFO [main] com.documentum.install.appserver.services.DiAppServerUtil - isTomcatInstalled == true -- tomcat version is null
20:31:11,431  INFO [main] com.documentum.install.appserver.tomcat.TomcatApplicationServer - setApplicationServer sharedDfcLibDir is:$DOCUMENTUM/dfc
20:31:11,431  INFO [main] com.documentum.install.appserver.tomcat.TomcatApplicationServer - getFileFromResource for templates/appserver.properties
20:31:11,434  INFO [main] com.documentum.install.appserver.tomcat.TomcatApplicationServer - Tomcat Home = $DOCUMENTUM/tomcat
20:31:11,438  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerModifyDocbaseDirectory - The installer will create the folder structure for repository GR_REPO.
20:31:11,440  INFO [main]  - The installer will stop component process for GR_REPO.
20:31:11,479  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerUpdateTCSUnixServiceFile - The installer will check service entries for repository GR_REPO.
20:31:11,483  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerModifyDfcProperties - The installer will update dfc.properties file.
20:31:11,485  INFO [main] com.documentum.install.shared.common.services.dfc.DiDfcProperties - Installer is not adding connection broker information as it is already added.
20:31:13,488  INFO [main]  - The installer will update server.ini file for the repository.
20:31:13,493  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerDataIniGenerator - The installer will create data_dictionary.ini for the repository.
20:31:13,496  INFO [main]  - The installer will obtain database server name for database dctmdb.
20:31:13,497  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerLoadServerWebcacheInfo - The installer will obtain database information of dctmdb.
20:31:13,498  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerWebCacheIniGenerator - The installer will update webcache.ini file for the repository.
20:31:13,503  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerTestDatabaseConnection4Docbase_Upgrade - The installer is testing the database connection information
20:31:13,503  INFO [main] com.documentum.install.server.common.services.db.DiServerDbSvrOracleServer - The installer is validating the database version is supported.
20:31:13,638  INFO [main] com.documentum.install.server.common.services.db.DiServerDbSvrOracleServer - The installer is validating the database connection information in the server.ini file.
20:31:13,910  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerCheckIfDMSUpgraded - Check if upgrade of DMS tables is needed for current docbase.
20:31:13,911  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerCheckIfDMSUpgraded - The current repository doesn't need to upgrade DMS table.
20:31:13,922  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerUpdateDocbaseServiceScript - The installer will update start script for  repository GR_REPO.
20:31:13,929  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerUpdateDocbaseServiceScript - The installer will update stop script for  repository GR_REPO.
20:31:13,937  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerUpgradeAEKUtility - will not execute start and stop services
20:31:13,944  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerUpgradeAEKUtility - will not execute start and stop services
20:31:13,947  INFO [main]  - The installer will start component process for GR_REPO.
20:31:14,997  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAUnixServiceControl - logPath is $DOCUMENTUM/dba/log/GR_REPO.log
20:31:16,005  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAUnixServiceControl - logPath is $DOCUMENTUM/dba/log/GR_REPO.log
20:31:17,015  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAUnixServiceControl - logPath is $DOCUMENTUM/dba/log/GR_REPO.log
20:31:18,024  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAUnixServiceControl - logPath is $DOCUMENTUM/dba/log/GR_REPO.log
20:31:19,030  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAUnixServiceControl - logPath is $DOCUMENTUM/dba/log/GR_REPO.log
20:31:20,038  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAUnixServiceControl - logPath is $DOCUMENTUM/dba/log/GR_REPO.log
20:31:21,045  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAUnixServiceControl - logPath is $DOCUMENTUM/dba/log/GR_REPO.log
20:31:22,053  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAUnixServiceControl - logPath is $DOCUMENTUM/dba/log/GR_REPO.log
...
20:31:43,060  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAUnixServiceControl - logPath is $DOCUMENTUM/dba/log/GR_REPO.log
20:31:44,066  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAUnixServiceControl - logPath is $DOCUMENTUM/dba/log/GR_REPO.log
...
20:32:10,073  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAUnixServiceControl - logPath is $DOCUMENTUM/dba/log/GR_REPO.log
20:32:11,080  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAUnixServiceControl - logPath is $DOCUMENTUM/dba/log/GR_REPO.log
[dmadmin@cs-0 ~]$

3. Repository startup crash on DM_SERVER_E_SOCKOPT?

Everything looked fine at the beginning of the log file, at least until the start of the Repository process. After waiting a little less than one minute, there was still nothing happening, which was not normal. Therefore, I checked the OS processes and found none for the Repository “GR_REPO“:

[dmadmin@cs-0 ~]$ ps uxf | grep GR_REPO
dmadmin  2233289  0.0  0.0   3348  1824 pts/0    S+   20:32   0:00  \_ grep --color=auto GR_REPO
[dmadmin@cs-0 ~]$

Therefore, something happened to the Repository process, which means that the upgrade process failed or will remain stuck in that loop. When checking the Repository log file, I saw the following:

[dmadmin@cs-0 dba]$ cat $DOCUMENTUM/dba/log/GR_REPO.log

    OpenText Documentum Content Server (version 25.4.0000.0143 Linux64.Oracle)
    Copyright (c) 2025. OpenText Corporation
    All rights reserved.

2026-02-05T20:31:15.702051	2235323[2235323]	0000000000000000	[DM_SERVER_I_START_SERVER]info:  "Docbase GR_REPO attempting to open"

2026-02-05T20:31:15.802801	2235323[2235323]	0000000000000000	[DM_SERVER_I_START_KEY_STORAGE_MODE]info:  "Docbase GR_REPO is using database for cryptographic key storage"

2026-02-05T20:31:15.802878	2235323[2235323]	0000000000000000	[DM_SERVER_I_START_SERVER]info:  "Docbase GR_REPO process identity: user(dmadmin)"

2026-02-05T20:31:16.073327	2235323[2235323]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize Post Upgrade Processing."

2026-02-05T20:31:16.073843	2235323[2235323]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize Base Types."

2026-02-05T20:31:16.074476	2235323[2235323]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize dmRecovery."

2026-02-05T20:31:16.089286	2235323[2235323]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize dmACL."

...

2026-02-05T20:31:17.886348	2235323[2235323]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize Acs Config List."

2026-02-05T20:31:17.886487	2235323[2235323]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize dmLiteSysObject."

2026-02-05T20:31:17.886957	2235323[2235323]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize dmBatchManager."

2026-02-05T20:31:17.891417	2235323[2235323]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize Partition Scheme."

2026-02-05T20:31:17.891883	2235323[2235323]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize Critical Event Registry."

2026-02-05T20:31:17.891947	2235323[2235323]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize Transaction Tracking Event Registry."

2026-02-05T20:31:17.892014	2235323[2235323]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize Initialze External User Event Set."

2026-02-05T20:31:17.893580	2235323[2235323]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize Authentication Plugins."

2026-02-05T20:31:17.895070	2235323[2235323]	0000000000000000	[DM_SESSION_I_AUTH_PLUGIN_LOADED]info:  "Loaded Authentication Plugin with code 'dm_krb' ($DOCUMENTUM/dba/auth/libkerberos.so)."

2026-02-05T20:31:17.895090	2235323[2235323]	0000000000000000	[DM_SESSION_I_AUTH_PLUGIN_LOAD_INIT]info:  "Authentication plugin ( 'dm_krb' ) was disabled. This is expected if no keytab file(s) at location ($DOCUMENTUM/dba/auth/kerberos).Please refer the content server installation guide."

2026-02-05T20:31:17.896397	2235323[2235323]	0000000000000000	[DM_SERVER_I_START_SERVER]info:  "Docbase GR_REPO opened"

2026-02-05T20:31:17.896463	2235323[2235323]	0000000000000000	[DM_SERVER_I_SERVER]info:  "Setting exception handlers to catch all interrupts"

2026-02-05T20:31:17.896475	2235323[2235323]	0000000000000000	[DM_SERVER_I_START]info:  "Starting server using service name:  GR_REPO"

2026-02-05T20:31:17.933282	2235323[2235323]	0000000000000000	[DM_LICENSE_E_NO_LICENSE_CONFIG]error:  "Could not find dm_otds_license_config object."

2026-02-05T20:31:17.998180	2235323[2235323]	0000000000000000	[DM_SERVER_I_LAUNCH_MTHDSVR]info:  "Launching Method Server succeeded."

2026-02-05T20:31:17.999298	2235323[2235323]	0000000000000000	[DM_SERVER_E_SOCKOPT]error:  "setsockopt failed for (SO_REUSEADDR (client)) with error (88)"

[dmadmin@cs-0 dba]$

4. Investigation of the DM_SERVER_E_SOCKOPT

The Repository log file was also quite clean. Everything at the beginning was as expected, until the very last line. The Repository was about to become available when suddenly a “DM_SERVER_E_SOCKOPT” error appeared and crashed the Repository process.

I tried to start the Repository again, but without success. It always ended up with the exact same error as the last line. It was my first time encountering that error “DM_SERVER_E_SOCKOPT“, so I checked the official documentation and the OpenText support website. But I had no luck there either; there was not a single reference to that error anywhere.

So, what should I do next? As usual, I just tried things that could make sense based on the information available.

The error refers to SOCKOPT, which I assumed meant socket options, so possibly something related to networking or communications. I checked the “setsockopt” documentation (c.f. this man page), which indeed seemed related to networking protocols. I also reviewed the “SO_REUSEADDR” documentation (c.f. this other man page), which refers to address binding apparently.

5. Fix & Resolution

With that in mind, I remembered the IPv4 versus IPv6 issue I encountered back in 2024 and the blog post I wrote about it (linked earlier). Since there was no issue starting the Connection Broker with the “host=${Current-IPv4}” setting, I focused on the Repository configuration.

Therefore, I tried disabling what I had added for 23.4 to work. Specifically, I commented out the line “#ip_mode = V4ONLY” in “server.ini” so that it would return to the default value:

[dmadmin@cs-0 ~]$ grep ip_mode $DOCUMENTUM/dba/config/GR_REPO/server.ini
ip_mode = V4ONLY
[dmadmin@cs-0 ~]$
[dmadmin@cs-0 ~]$ sed -i 's/^\(ip_mode\)/#\1/' $DOCUMENTUM/dba/config/GR_REPO/server.ini
[dmadmin@cs-0 ~]$
[dmadmin@cs-0 ~]$ grep ip_mode $DOCUMENTUM/dba/config/GR_REPO/server.ini
#ip_mode = V4ONLY
[dmadmin@cs-0 ~]$

Then I tried starting the Repository again:

[dmadmin@cs-0 ~]$ $DOCUMENTUM/dba/dm_start_GR_REPO
starting Documentum server for repository: [GR_REPO]
with server log: [$DOCUMENTUM/dba/log/GR_REPO.log]
server pid: 2268601
[dmadmin@cs-0 ~]$
[dmadmin@cs-0 ~]$ cat $DOCUMENTUM/dba/log/GR_REPO.log

    OpenText Documentum Content Server (version 25.4.0000.0143 Linux64.Oracle)
    Copyright (c) 2025. OpenText Corporation
    All rights reserved.

2026-02-05T20:57:34.417095	2268601[2268601]	0000000000000000	[DM_SERVER_I_START_SERVER]info:  "Docbase GR_REPO attempting to open"

2026-02-05T20:57:34.517792	2268601[2268601]	0000000000000000	[DM_SERVER_I_START_KEY_STORAGE_MODE]info:  "Docbase GR_REPO is using database for cryptographic key storage"

2026-02-05T20:57:34.517854	2268601[2268601]	0000000000000000	[DM_SERVER_I_START_SERVER]info:  "Docbase GR_REPO process identity: user(dmadmin)"

2026-02-05T20:57:34.750986	2268601[2268601]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize Post Upgrade Processing."

2026-02-05T20:57:34.751390	2268601[2268601]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize Base Types."

2026-02-05T20:57:34.751810	2268601[2268601]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize dmRecovery."

2026-02-05T20:57:34.763115	2268601[2268601]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize dmACL."

...

2026-02-05T20:57:35.977776	2268601[2268601]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize Acs Config List."

2026-02-05T20:57:35.977873	2268601[2268601]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize dmLiteSysObject."

2026-02-05T20:57:35.978265	2268601[2268601]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize dmBatchManager."

2026-02-05T20:57:35.981015	2268601[2268601]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize Partition Scheme."

2026-02-05T20:57:35.981417	2268601[2268601]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize Critical Event Registry."

2026-02-05T20:57:35.981479	2268601[2268601]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize Transaction Tracking Event Registry."

2026-02-05T20:57:35.981543	2268601[2268601]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize Initialze External User Event Set."

2026-02-05T20:57:35.983046	2268601[2268601]	0000000000000000	[DM_SESSION_I_INIT_BEGIN]info:  "Initialize Authentication Plugins."

2026-02-05T20:57:35.984583	2268601[2268601]	0000000000000000	[DM_SESSION_I_AUTH_PLUGIN_LOADED]info:  "Loaded Authentication Plugin with code 'dm_krb' ($DOCUMENTUM/dba/auth/libkerberos.so)."

2026-02-05T20:57:35.984604	2268601[2268601]	0000000000000000	[DM_SESSION_I_AUTH_PLUGIN_LOAD_INIT]info:  "Authentication plugin ( 'dm_krb' ) was disabled. This is expected if no keytab file(s) at location ($DOCUMENTUM/dba/auth/kerberos).Please refer the content server installation guide."

2026-02-05T20:57:35.986125	2268601[2268601]	0000000000000000	[DM_SERVER_I_START_SERVER]info:  "Docbase GR_REPO opened"

2026-02-05T20:57:35.986189	2268601[2268601]	0000000000000000	[DM_SERVER_I_SERVER]info:  "Setting exception handlers to catch all interrupts"

2026-02-05T20:57:35.986199	2268601[2268601]	0000000000000000	[DM_SERVER_I_START]info:  "Starting server using service name:  GR_REPO"

2026-02-05T20:57:36.016709	2268601[2268601]	0000000000000000	[DM_LICENSE_E_NO_LICENSE_CONFIG]error:  "Could not find dm_otds_license_config object."

2026-02-05T20:57:36.067861	2268601[2268601]	0000000000000000	[DM_SERVER_I_LAUNCH_MTHDSVR]info:  "Launching Method Server succeeded."

2026-02-05T20:57:36.073641	2268601[2268601]	0000000000000000	[DM_SERVER_I_LISTENING]info:  "The server is listening on network address (Service Name: GR_REPO, Host Name: cs-0 :V4 IP)"

2026-02-05T20:57:36.082139	2268601[2268601]	0000000000000000	[DM_SERVER_I_LISTENING]info:  "The server is listening on network address (Service Name: GR_REPO_s, Host Name: cs-0 :V4 IP)"

2026-02-05T20:57:37.135498	2268601[2268601]	0000000000000000	[DM_WORKFLOW_I_AGENT_START]info:  "Workflow agent master (pid : 2268748, session 0101234580000007) is started sucessfully."
IsProcessAlive: Process ID 0 is not > 0
2026-02-05T20:57:37.136533	2268601[2268601]	0000000000000000	[DM_WORKFLOW_I_AGENT_START]info:  "Workflow agent worker (pid : 2268749, session 010123458000000a) is started sucessfully."
IsProcessAlive: Process ID 0 is not > 0
2026-02-05T20:57:38.137780	2268601[2268601]	0000000000000000	[DM_WORKFLOW_I_AGENT_START]info:  "Workflow agent worker (pid : 2268770, session 010123458000000b) is started sucessfully."
IsProcessAlive: Process ID 0 is not > 0
2026-02-05T20:57:39.139572	2268601[2268601]	0000000000000000	[DM_WORKFLOW_I_AGENT_START]info:  "Workflow agent worker (pid : 2268823, session 010123458000000c) is started sucessfully."
2026-02-05T20:57:40.139741	2268601[2268601]	0000000000000000	[DM_SERVER_I_START]info:  "Sending Initial Docbroker check-point "

2026-02-05T20:57:40.143121	2268601[2268601]	0000000000000000	[DM_MQ_I_DAEMON_START]info:  "Message queue daemon (pid : 2268845, session 0101234580000456) is started sucessfully."
2026-02-05T20:57:42.758073	2268844[2268844]	0101234580000003	[DM_DOCBROKER_I_PROJECTING]info:  "Sending information to Docbroker located on host (cs-0.cs.dctm-ns.svc.cluster.local) with port (1490).  Information: (Config(GR_REPO), Proximity(1), Status(Open), Dormancy Status(Active))."
[dmadmin@cs-0 ~]$

As you can see, it is working again. This means that what I previously had to add for the 23.4 environment to work is now causing an issue on 25.4. Obviously the OS is not the same (both the container and the real host), and the Kubernetes environment is also different. Still, it is interesting that something fixing an issue in a specific version can introduce a problem in a newer version.

As shown in the logs, the Repository still starts on IPv4, as it clearly states: “The server is listening on network address (Service Name: GR_REPO, Host Name: cs-0 :V4 IP)“. However, it does not accept the setting “ip_mode = V4ONLY“, somehow.

That’s pretty incredible, don’t you think? Anyway, once the Repository processes were running, I was able to restart the upgrade process properly by ensuring that ip_mode was not specified for version 25.4.

L’article Dctm – Upgrade from 23.4 to 25.4 fails with DM_SERVER_E_SOCKOPT est apparu en premier sur dbi Blog.

Dctm – Detect and analyze slow SQL queries

Morgan Patou — Wed, 25 Feb 2026 20:06:48 +0000

Documentum is a powerful but also complex software. Therefore, debugging performance issues often comes down to being able to implement and review various traces. In this blog, I will talk about how you can detect and analyze slow SQL queries from a Documentum Server. Several years ago, I wrote a blog post about DFC traces. It included the setup and investigation process, using the EMC Python/AWK parser scripts as a base. In addition, I also wrote another one about Tomcat slow/stuck threads. With these two posts, you could already cover the Documentum DFC clients (like JMS Apps, DA, D2, xPlore, etc.).

As you might imagine, SQL traces in Documentum can be quite large. A lot of queries are sent to the Database to retrieve, insert or update information. Things like document metadata, users, permissions, audittrail entries, document creation, etc. It is less intensive than DFC or SSL/Network traces, because those write dozens of lines for each and every action. On the other hand, SQL traces limit themselves to the query being executed, the execution time, and potentially some commits.

A customer recently provided me a Repository log file with SQL traces related to a problematic action in DA. I was expecting maybe a 10 or 20-minute log file at most. But what I received was a 4-hour log file, because the action took that long to complete in DA. I thought this would be a good use case to share in a blog, to explain how you can quickly extract interesting details from such SQL traces.

1. Defining variables

As often, I find it easier for both you and me to use predefined variables. This allows you to just change these details and, for the rest of the blog, simply copy/paste the exact same queries without having to wonder whether they will work.

I’m using my Mac and I have a folder which contains the Repository log file:

mac:~$ ls
dbi01.log
mac:~$
mac:~$ # Variables definition
mac:~$ source_repo_log_file="dbi01.log"
mac:~$ query_times="query_times.log"
mac:~$ extracted_session_log_file="extracted_session_log_file.log"

2. Extracting the SQL execution times

Once done, let’s proceed directly with extracting the SQL execution times. The durations will be sorted to easily understand whether there is a problem with some queries. In the initial log file, there are more than a hundred thousand SQL queries executed. Out of these, I selected only the 47 slowest “EXEC” entries below. That is not an arbitrary number; I initially retrieved the last 100 lines, but most of them were less than 0.1s. Therefore, I reduced that number to only execution times that might be interesting to investigate.

In the first command, I am selecting all “EXEC“, which will include the base “EXEC” but also other types like “EXECBATCH“, if any. There are also some “FETCH” entries in the SQL traces, but I will ignore these in this blog. The second and third commands simply show what was extracted:

mac:~$ ### Extracting all "EXEC" times from the Repo log
mac:~$ ### keeping the 47 slowest (>0.5s in this case)
mac:~$ grep "EXEC" ${source_repo_log_file} | \
         awk '{print $7}' | \
         sort --version-sort | \
         tail -47 > ${query_times}
mac:~$
mac:~$ wc -l ${query_times}
    47 query_times.log
mac:~$
mac:~$
mac:~$ ### Displaying the outcome sorted by asc time
mac:~$ cat ${query_times}
0.5566090000
1.0761300000
9.9041350000
11.2433810000
14.6858060000
...
786.4434090000
961.8524350000
1028.2339260000
1500.6862730000
1970.6185150000
mac:~$

The above command can be useful if you only want to extract the execution time (7th column). But most of the time, what is useful is the full line. It will display the date (1st column), of course as well as the OS PID (2nd column) that is executing this query, which can be useful in some cases. But most importantly, it will display the Session ID (3rd column) linked to the request. For that purpose, you can slightly modify the first command from above, to extract the full line for all “EXEC” and sort based on the 7th column:

mac:~$ ### Extracting all "EXEC" complete lines from the Repo log
mac:~$ ### keeping the 47 slowest (>0.5s in this case)
mac:~$ grep "EXEC" ${source_repo_log_file} | \
         sort -k 7 --version-sort | \
         tail -47
2025-12-19T12:25:33.056 102[102] 0112345682105da7 [SQL] 0 EXEC 0.5566090000
2025-12-19T11:09:40.772 104[104] 0112345682105d65 [SQL] 0 EXEC 1.0761300000
2025-12-19T12:01:28.042 110[110] 0112345682105d9b [SQL] 0 EXEC 9.9041350000
2025-12-19T12:01:13.943 105[105] 01123456821058fb [SQL] 0 EXEC 11.2433810000
2025-12-19T11:59:16.078 105[105] 01123456821058fb [SQL] 0 EXEC 14.6858060000
...
2025-12-19T11:03:02.001 105[105] 01123456821058fb [SQL] 0 EXEC 786.4434090000
2025-12-19T09:59:20.090 105[105] 01123456821058fb [SQL] 0 EXEC 961.8524350000
2025-12-19T11:59:01.360 105[105] 01123456821058fb [SQL] 0 EXEC 1028.2339260000
2025-12-19T10:49:55.553 105[105] 01123456821058fb [SQL] 0 EXEC 1500.6862730000
2025-12-19T09:29:55.637 105[105] 01123456821058fb [SQL] 0 EXEC 1970.6185150000
mac:~$

Another way to do (almost) the same thing is based on the previously exported execution times. If you loop over them with a certain command, you can retrieve the same lines again. The only difference is that this second method will be significantly slower (since it parses the log file for each execution time instead of only once as in the previous command) and slightly less accurate in some edge cases. Technically, if you have two lines with the exact same execution time, you would not get only 2 results, but 2 x 2 results (so there would be duplicates). In my case, I do not have duplicate execution times, so it produces 100% the same outcome:

mac:~$ ### Extracting all "EXEC" complete lines from the Repo log
mac:~$ ### based on previously extracted execution time
mac:~$ while read line; do \
         grep "EXEC.*[[:space:]]\+${line}$" ${source_repo_log_file}; \
       done < <(cat ${query_times})
2025-12-19T12:25:33.056 102[102] 0112345682105da7 [SQL] 0 EXEC 0.5566090000
2025-12-19T11:09:40.772 104[104] 0112345682105d65 [SQL] 0 EXEC 1.0761300000
2025-12-19T12:01:28.042 110[110] 0112345682105d9b [SQL] 0 EXEC 9.9041350000
2025-12-19T12:01:13.943 105[105] 01123456821058fb [SQL] 0 EXEC 11.2433810000
2025-12-19T11:59:16.078 105[105] 01123456821058fb [SQL] 0 EXEC 14.6858060000
...
2025-12-19T11:03:02.001 105[105] 01123456821058fb [SQL] 0 EXEC 786.4434090000
2025-12-19T09:59:20.090 105[105] 01123456821058fb [SQL] 0 EXEC 961.8524350000
2025-12-19T11:59:01.360 105[105] 01123456821058fb [SQL] 0 EXEC 1028.2339260000
2025-12-19T10:49:55.553 105[105] 01123456821058fb [SQL] 0 EXEC 1500.6862730000
2025-12-19T09:29:55.637 105[105] 01123456821058fb [SQL] 0 EXEC 1970.6185150000
mac:~$

3. How long is really spent on queries?

Based on the above results, the first thing that stood out was that 44 of the slowest queries appeared to be linked to the same Session ID. Therefore, except for the first 3 rows (0.5s, 1s, and 9.9s), all other queries came from a single source. This clearly indicates that the problematic action in DA is associated with that Session ID, and it is therefore what I need to investigate further. The next step was to extract all logs related to that Session ID:

mac:~$ ### Extracting logs related to the specific session ID
mac:~$ session_id="01123456821058fb"
mac:~$ grep "${session_id}" ${source_repo_log_file} \
         > ${extracted_session_log_file}
mac:~$
mac:~$ wc -l ${extracted_session_log_file}
    765 extracted_session_log_file.log
mac:~$

Compared to the size of the log file (a few hundred thousand lines), there are only 765 lines related to that Session ID. But how much time is actually spent on these few requests? For that purpose, you can compare the sum of all “EXEC” times for the Repository log file vs the Session ID log file vs the 44 slowest queries:

mac:~$ ### Checking the SQL execution time
mac:~$ ### for all SQL queries in the Repo log file
mac:~$ grep "EXEC" ${source_repo_log_file} | \
         awk '{print $7}' | \
         awk '{sum += $1}
         END {
           h = int(sum / 3600)
           m = int((sum % 3600) / 60)
           s = sum % 60
           printf "%02dh %02dm %.3fs\n", h, m, s
         }'
03h 11m 50.477s
mac:~$
mac:~$
mac:~$ ### Checking the SQL execution time
mac:~$ ### related to the specific session ID
mac:~$ grep "EXEC" ${extracted_session_log_file} | \
         awk '{print $7}' | \
         awk '{sum += $1}
         END {
           h = int(sum / 3600)
           m = int((sum % 3600) / 60)
           s = sum % 60
           printf "%02dh %02dm %.3fs\n", h, m, s
         }'
03h 09m 56.340s
mac:~$
mac:~$
mac:~$ ### Checking the SQL execution time
mac:~$ ### for the 44 slowest related to the specific session ID
mac:~$ cat ${query_times} | \
         tail -44 | \
         awk '{sum += $1}
         END {
           h = int(sum / 3600)
           m = int((sum % 3600) / 60)
           s = sum % 60
           printf "%02dh %02dm %.3fs\n", h, m, s
         }'
03h 09m 56.015s
mac:~$

The above means that inside that ~4-hour log file, the Documentum Server executed more than a hundred thousand queries for a total sum of 3h 11m 50.5s. Of course, that is not necessarily consecutive time, as some queries might be executed in parallel by different processes. Out of these 3h 11m 50s spent on Database queries, the time related to the problematic Session ID, which represents slightly less than 0.2% of the executed SQL queries, is 3h 9m 56.3s. And out of these 0.2%, the 44 slowest queries represent 3h 9m 56.0s. That means:

100% SQL queries >> 3h 11m 50.5s = 11’510.5s
- 99.8% SQL queries >> 1m 54s = 114s
- 0.2% SQL queries >> 3h 9m 56.3s = 11’396.3s
  - 0.165% SQL queries >> 0.3s
  - 0.035% SQL queries >> 3h 9m 56.0s = 11’396.0s

In other words, 0.035% of the SQL queries executed represent 99% of the time spent on the Database.

4. Finding the responsible queries

Now that we have clear evidence related to the execution times, the next step is to retrieve the associated queries that are the source of the performance issue. Below, I am simply retrieving the line containing the execution time, as well as the previous line, which is supposed to contain the triggered query.

This is normally safe because I am checking only the extracted Session ID logs. Since a session is most likely sequential, it is relatively straightforward to retrieve the query responsible for the long execution times. There might be parallel executions for some batch processing, but I am not sure whether that would use the same session ID or a dedicated one per batch/thread.

If you have long running SQLs for multiple sessions, then you might want/need to take care of that and do the check session by session.

If you try to use the full Repository log instead, you will probably end up with an incorrect N-1 line. In that case, you would need a more complex parsing mechanism based on the Session ID. You can check the blog I mentioned earlier for an example of DFC trace parsing using awk. For this blog, I am keeping it simple, but I might create another post with a more advanced parser that performs everything automatically. So, let’s extract the SQL queries related to the 44 slowest execution times:

mac:~$ ### Extracting all 44 slowest queries
mac:~$ ### from the specific session ID log file
mac:~$ while read line; do \
         grep -B1 "${line}" ${extracted_session_log_file}; \
       done < <(cat ${query_times})
2025-12-19T12:01:02.700 105[105] 01123456821058fb [SQL] 0 select all dm_audittrail.session_id, dm_audittrail.user_name, dm_audittrail.event_name, dm_audittrail.event_description, dm_audittrail.time_stamp, dm_audittrail.object_type from dm_audittrail_sp  dm_audittrail where (dm_audittrail.event_description like '[TRANSACTION_TRACKING_EVENTS]%')
2025-12-19T12:01:13.943 105[105] 01123456821058fb [SQL] 0 EXEC 11.2433810000
2025-12-19T11:59:01.393 105[105] 01123456821058fb [SQL] 0 select all dm_audittrail.user_name, dm_audittrail.session_id, dm_audittrail.event_name, dm_audittrail.string_1, dm_audittrail.string_2, dm_audittrail.string_3, dm_audittrail.string_4, dm_audittrail.time_stamp from dm_audittrail_sp  dm_audittrail where (dm_audittrail.event_name in ('dm_connect', 'dm_logon_failure'))
2025-12-19T11:59:16.078 105[105] 01123456821058fb [SQL] 0 EXEC 14.6858060000
2025-12-19T12:00:09.566 105[105] 01123456821058fb [SQL] 0 select all dm_audittrail.session_id, dm_audittrail.user_name, dm_audittrail.time_stamp from dm_audittrail_sp  dm_audittrail where ((dm_audittrail.event_name='dm_connect') and dm_audittrail.user_name in (select all dm_repeating.users_names from dm_group_sp  dm_group, dm_group_rp dm_repeating where ((dm_group.group_name='dm_occasional_user_role')) and dm_repeating.r_object_id=dm_group.r_object_id )) order by dm_audittrail.r_object_id desc
2025-12-19T12:01:02.696 105[105] 01123456821058fb [SQL] 0 EXEC 53.1301340000
2025-12-19T11:59:16.083 105[105] 01123456821058fb [SQL] 0 select all dm_audittrail.session_id, dm_audittrail.user_name, dm_audittrail.time_stamp from dm_audittrail_sp  dm_audittrail where ((dm_audittrail.event_name='dm_disconnect') and dm_audittrail.user_name in (select all dm_repeating.users_names from dm_group_sp  dm_group, dm_group_rp dm_repeating where ((dm_group.group_name='dm_occasional_user_role')) and dm_repeating.r_object_id=dm_group.r_object_id )) order by dm_audittrail.r_object_id desc
2025-12-19T12:00:09.563 105[105] 01123456821058fb [SQL] 0 EXEC 53.4798570000
2025-12-19T11:13:44.269 105[105] 01123456821058fb [SQL] 0 select all dmr_content.set_time from dmr_content_sp  dmr_content where (dmr_content.set_time=(select all max(dmr_content.set_time) from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000153')) ))
2025-12-19T11:15:01.033 105[105] 01123456821058fb [SQL] 0 EXEC 76.7641310000
2025-12-19T11:18:59.434 105[105] 01123456821058fb [SQL] 0 select all dmr_content.set_time from dmr_content_sp  dmr_content where (dmr_content.set_time=(select all max(dmr_content.set_time) from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000154')) ))
2025-12-19T11:20:16.256 105[105] 01123456821058fb [SQL] 0 EXEC 76.8221030000
2025-12-19T11:08:31.916 105[105] 01123456821058fb [SQL] 0 select all dmr_content.set_time from dmr_content_sp  dmr_content where (dmr_content.set_time=(select all max(dmr_content.set_time) from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000152')) ))
2025-12-19T11:09:49.809 105[105] 01123456821058fb [SQL] 0 EXEC 77.8924590000
2025-12-19T10:10:44.752 105[105] 01123456821058fb [SQL] 0 select all dmr_content.set_time from dmr_content_sp  dmr_content where (dmr_content.set_time=(select all max(dmr_content.set_time) from dmr_content_sp  dmr_content where ((dmr_content.storage_id='281234568000010b')) ))
2025-12-19T10:12:03.103 105[105] 01123456821058fb [SQL] 0 EXEC 78.3507420000
2025-12-19T11:03:02.009 105[105] 01123456821058fb [SQL] 0 select all dmr_content.set_time from dmr_content_sp  dmr_content where (dmr_content.set_time=(select all max(dmr_content.set_time) from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000151')) ))
2025-12-19T11:04:20.605 105[105] 01123456821058fb [SQL] 0 EXEC 78.5954180000
2025-12-19T11:36:05.711 105[105] 01123456821058fb [SQL] 0 select all dmr_content.set_time from dmr_content_sp  dmr_content where (dmr_content.set_time=(select all max(dmr_content.set_time) from dmr_content_sp  dmr_content where ((dmr_content.storage_id='28123456800001b4')) ))
2025-12-19T11:37:27.600 105[105] 01123456821058fb [SQL] 0 EXEC 81.8885210000
2025-12-19T11:24:26.958 105[105] 01123456821058fb [SQL] 0 select all dmr_content.set_time from dmr_content_sp  dmr_content where (dmr_content.set_time=(select all max(dmr_content.set_time) from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000155')) ))
2025-12-19T11:25:49.718 105[105] 01123456821058fb [SQL] 0 EXEC 82.7602900000
2025-12-19T11:30:19.107 105[105] 01123456821058fb [SQL] 0 select all dmr_content.set_time from dmr_content_sp  dmr_content where (dmr_content.set_time=(select all max(dmr_content.set_time) from dmr_content_sp  dmr_content where ((dmr_content.storage_id='28123456800001aa')) ))
2025-12-19T11:31:44.148 105[105] 01123456821058fb [SQL] 0 EXEC 85.0403360000
2025-12-19T09:32:33.679 105[105] 01123456821058fb [SQL] 0 select all dmr_content.set_time from dmr_content_sp  dmr_content where (dmr_content.set_time=(select all max(dmr_content.set_time) from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000102')) ))
2025-12-19T09:34:06.477 105[105] 01123456821058fb [SQL] 0 EXEC 92.7986000000
2025-12-19T10:03:31.061 105[105] 01123456821058fb [SQL] 0 select all dmr_content.set_time from dmr_content_sp  dmr_content where (dmr_content.set_time=(select all max(dmr_content.set_time) from dmr_content_sp  dmr_content where ((dmr_content.storage_id='281234568000010a')) ))
2025-12-19T10:05:07.569 105[105] 01123456821058fb [SQL] 0 EXEC 96.5074910000
2025-12-19T11:11:47.250 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000152') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is null) and (dmr_content.is_archived=0))
2025-12-19T11:13:44.264 105[105] 01123456821058fb [SQL] 0 EXEC 117.0139310000
2025-12-19T11:09:49.815 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000152') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is not null) and (dmr_content.is_archived=0))
2025-12-19T11:11:47.247 105[105] 01123456821058fb [SQL] 0 EXEC 117.4327020000
2025-12-19T11:17:00.411 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000153') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is null) and (dmr_content.is_archived=0))
2025-12-19T11:18:59.431 105[105] 01123456821058fb [SQL] 0 EXEC 119.0201020000
2025-12-19T11:15:01.040 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000153') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is not null) and (dmr_content.is_archived=0))
2025-12-19T11:17:00.406 105[105] 01123456821058fb [SQL] 0 EXEC 119.3658150000
2025-12-19T11:04:20.613 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000151') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is not null) and (dmr_content.is_archived=0))
2025-12-19T11:06:23.110 105[105] 01123456821058fb [SQL] 0 EXEC 122.4972100000
2025-12-19T11:22:23.775 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000154') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is null) and (dmr_content.is_archived=0))
2025-12-19T11:24:26.953 105[105] 01123456821058fb [SQL] 0 EXEC 123.1771960000
2025-12-19T10:12:03.114 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='281234568000010b') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is not null) and (dmr_content.is_archived=0))
2025-12-19T10:14:09.285 105[105] 01123456821058fb [SQL] 0 EXEC 126.1710020000
2025-12-19T10:14:09.290 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='281234568000010b') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is null) and (dmr_content.is_archived=0))
2025-12-19T10:16:15.546 105[105] 01123456821058fb [SQL] 0 EXEC 126.2559010000
2025-12-19T11:20:16.262 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000154') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is not null) and (dmr_content.is_archived=0))
2025-12-19T11:22:23.770 105[105] 01123456821058fb [SQL] 0 EXEC 127.5082070000
2025-12-19T11:33:56.973 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='28123456800001aa') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is null) and (dmr_content.is_archived=0))
2025-12-19T11:36:05.705 105[105] 01123456821058fb [SQL] 0 EXEC 128.7315320000
2025-12-19T11:06:23.115 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000151') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is null) and (dmr_content.is_archived=0))
2025-12-19T11:08:31.910 105[105] 01123456821058fb [SQL] 0 EXEC 128.7948250000
2025-12-19T11:37:27.606 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='28123456800001b4') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is not null) and (dmr_content.is_archived=0))
2025-12-19T11:39:39.265 105[105] 01123456821058fb [SQL] 0 EXEC 131.6585850000
2025-12-19T11:31:44.156 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='28123456800001aa') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is not null) and (dmr_content.is_archived=0))
2025-12-19T11:33:56.971 105[105] 01123456821058fb [SQL] 0 EXEC 132.8144410000
2025-12-19T11:39:39.270 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='28123456800001b4') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is null) and (dmr_content.is_archived=0))
2025-12-19T11:41:53.017 105[105] 01123456821058fb [SQL] 0 EXEC 133.7473340000
2025-12-19T11:25:49.725 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000155') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is not null) and (dmr_content.is_archived=0))
2025-12-19T11:28:03.796 105[105] 01123456821058fb [SQL] 0 EXEC 134.0701630000
2025-12-19T11:28:03.801 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000155') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is null) and (dmr_content.is_archived=0))
2025-12-19T11:30:19.104 105[105] 01123456821058fb [SQL] 0 EXEC 135.3037450000
2025-12-19T09:29:55.642 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000100') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is null) and (dmr_content.is_archived=0))
2025-12-19T09:32:33.673 105[105] 01123456821058fb [SQL] 0 EXEC 158.0316130000
2025-12-19T09:34:06.482 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000102') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is not null) and (dmr_content.is_archived=0))
2025-12-19T09:36:51.885 105[105] 01123456821058fb [SQL] 0 EXEC 165.4027790000
2025-12-19T10:07:57.988 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='281234568000010a') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is null) and (dmr_content.is_archived=0))
2025-12-19T10:10:44.749 105[105] 01123456821058fb [SQL] 0 EXEC 166.7611230000
2025-12-19T10:05:07.576 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='281234568000010a') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is not null) and (dmr_content.is_archived=0))
2025-12-19T10:07:57.984 105[105] 01123456821058fb [SQL] 0 EXEC 170.4081960000
2025-12-19T09:36:51.889 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000102') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is null) and (dmr_content.is_archived=0))
2025-12-19T09:39:43.525 105[105] 01123456821058fb [SQL] 0 EXEC 171.6363780000
2025-12-19T09:39:43.530 105[105] 01123456821058fb [SQL] 0 select all dmr_content.set_time from dmr_content_sp  dmr_content where (dmr_content.set_time=(select all max(dmr_content.set_time) from dmr_content_sp  dmr_content where ((dmr_content.storage_id='6d1234568000011e')) ))
2025-12-19T09:43:18.230 105[105] 01123456821058fb [SQL] 0 EXEC 214.6996070000
2025-12-19T09:59:20.094 105[105] 01123456821058fb [SQL] 0 select all count(*) "total_file_count" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='6d1234568000011e') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is null) and (dmr_content.is_archived=0))
2025-12-19T10:03:31.053 105[105] 01123456821058fb [SQL] 0 EXEC 250.9586420000
2025-12-19T08:51:17.559 105[105] 01123456821058fb [SQL] 0 select all dmr_content.set_time from dmr_content_sp  dmr_content where (dmr_content.set_time=(select all max(dmr_content.set_time) from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000100')) ))
2025-12-19T08:57:05.010 105[105] 01123456821058fb [SQL] 0 EXEC 347.4517930000
2025-12-19T10:16:15.551 105[105] 01123456821058fb [SQL] 0 select all dmr_content.set_time from dmr_content_sp  dmr_content where (dmr_content.set_time=(select all max(dmr_content.set_time) from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000150')) ))
2025-12-19T10:24:54.860 105[105] 01123456821058fb [SQL] 0 EXEC 519.3084910000
2025-12-19T10:49:55.557 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000150') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is null) and (dmr_content.is_archived=0))
2025-12-19T11:03:02.001 105[105] 01123456821058fb [SQL] 0 EXEC 786.4434090000
2025-12-19T09:43:18.238 105[105] 01123456821058fb [SQL] 0 select all count(*) "total_file_count" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='6d1234568000011e') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is not null) and (dmr_content.is_archived=0))
2025-12-19T09:59:20.090 105[105] 01123456821058fb [SQL] 0 EXEC 961.8524350000
2025-12-19T11:41:53.126 105[105] 01123456821058fb [SQL] 0 select all u.r_object_id, u.user_name, u.user_privileges, u.user_xprivileges, count(s.object_name) "document_count" from dm_user_sp  u, dm_sysobject_sp  s where ((u.user_name=s.owner_name)) and (s.i_has_folder = 1 and s.i_is_deleted = 0) group by u.user_name, u.user_privileges, u.user_xprivileges, u.r_object_id
2025-12-19T11:59:01.360 105[105] 01123456821058fb [SQL] 0 EXEC 1028.2339260000
2025-12-19T10:24:54.866 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000150') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is not null) and (dmr_content.is_archived=0))
2025-12-19T10:49:55.553 105[105] 01123456821058fb [SQL] 0 EXEC 1500.6862730000
2025-12-19T08:57:05.019 105[105] 01123456821058fb [SQL] 0 select all count(*) "totalsize" from dmr_content_sp  dmr_content where ((dmr_content.storage_id='2812345680000100') and  exists (select r_object_id from dmr_content_r where dmr_content.r_object_id = r_object_id and parent_id is not null) and (dmr_content.is_archived=0))
2025-12-19T09:29:55.637 105[105] 01123456821058fb [SQL] 0 EXEC 1970.6185150000
mac:~$

There are quite a few interesting things we can see above. The slowest queries (>76s) are all related to documents/contents:

For each dm_filestore that exists in the Repository (12 above):
- 1 x ‘select all dmr_content.set_time from dmr_content_sp …‘
- 2 x ‘select all count(*) “totalsize” from dmr_content_sp …‘ (objects with and without parent_id)
For each dm_ca_store that exists in the Repository (1 above):
- 1 x ‘select all dmr_content.set_time from dmr_content_sp …‘
- 2 x ‘select all count(*) “total_file_count” from dmr_content_sp …‘ (objects with and without parent_id)
1 x ‘select all u.r_object_id, u.user_name, u.user_privileges, u.user_xprivileges, count(s.object_name) “document_count” from dm_user_sp u, dm_sysobject_sp s …‘

This shows that the 40 slowest queries are:

retrieving the set_time for all documents in a storage
counting the number of documents in a storage
retrieving all users and their associated documents

The slightly faster (but still slow) queries (11s < x < 53s) are linked to the audittrail, retrieving entries related to ‘[TRANSACTION_TRACKING_EVENTS]‘, ‘dm_connect‘, ‘dm_logon_failure‘, or ‘dm_disconnect‘ events, as well as the ‘dm_occasional_user_role‘ group.

5. Linking queries to the initial issue

The final step, once you know all the problematic/slowest requests, is either to try to improve the queries or at least find a way to avoid them. This can take various forms, such as changing the query (if it comes from custom development/configuration), improving the execution plan at the DB level, or adding indexes.

In this case, as a reminder, the issue was a problematic action in DA. A user was trying to execute the “External Transaction Activity” report. If I am not mistaken, this is a relatively new System Report available since version 23.4 and backported to earlier versions through a HotFix. However, even in 23.4, it also requires a hotfix to work properly, as the OOTB version is unusable because of some logging errors.

This report is supposed to fetch and display:

the list of users considered “external” (e.g. part of dm_external_users or dm_extuser_data_access groups)
the number of transactions performed by these external users for the selected year and split by quarter
the total number of transactions for all external users

Nothing in this report should require document storage queries to be executed. Even worse, these long-running SQL queries are actually triggered twice. When you access the “External Transaction Activity” report page in DA, it immediately triggers all the report queries in the background, even if you have not selected a year or clicked any button. Then, when you select a year and click “Generate” or “Export“, it re-triggers the exact same queries a second time.

In the Repository traces discussed in this blog, it was only the initial access to the form page – and it already took 4 hours. That customer would therefore likely need around 8 hours to generate and display the report. And this was not even their PROD environment, but a smaller DEV one.

That is obviously a bug (and arguably a design issue). I am currently in discussion with OpenText to find a way to fix the problem in version 23.4. Please note that in version 25.4, several improvements were made to the System Report, and it appears to have fixed this specific issue from 25.4 onwards.

This blog and trace file were just an example, but one that really happened at a customer site. I hope it provided some useful insights and that it will help you in your investigations!

L’article Dctm – Detect and analyze slow SQL queries est apparu en premier sur dbi Blog.

Morgan Patou, auteur/autrice sur dbi Blog

M-Files BD – Trend widgets: line and area

1. Why these two are in the same post

2. A first example: monthly revenue trend

3. The aggregation types these widgets accept

4. display.smooth

5. display.decimals

6. display.thresholds in single-series mode

7. seriesProperty: multiple lines on the same chart

7.1. Use low-cardinality series only

8. includeEmptyResults: gap filling

9. The “no date-valued reducer” rule

10. Drill-through on trend widgets

11. Wrap-up

M-Files BD – Scalar widgets: kpiNumber and gauge

1. The summary aggregation in one paragraph

2. kpiNumber – the big number tile

2.1. display.unit

2.2. display.decimals

2.3. display.thresholds

2.4. Reducer types on kpiNumber

2.5. Drill-through on kpiNumber

3. gauge – the dial

3.1. Numeric mode

3.2. display.thresholds on a gauge

3.3. Date mode

3.4. When the reducer cannot produce a numeric value

4. Summary of reducers for scalar widgets

5. Closing thoughts

M-Files BD – Anatomy of a dashboard definition

1. The minimum valid dashboard definition

2. The dashboard-level fields

2.1. Identification and presentation

2.2. Auto-refresh

2.3. Exports and drill-through

2.4. Performance settings

2.5. Access control

2.6. Widgets

3. The widget object

4. The query object in one paragraph

5. The 12-column grid

6. Where the definition is stored

7. Quick overview of a full dashboard JSON definition

8. What comes next

M-Files BD – End-user experience

1. Where to find the dashboard pane

2. Selecting and switching between dashboards

3. The widget grid

4. Refreshing the data

5. Drill-through: exploring the objects behind the numbers

Why drill-through can potentially show more objects than the widget counts

6. Navigating to an object

7. Exporting to PDF

8. Exporting to CSV

9. Favorite dashboard

10. Sharing a dashboard link

11. Indicators worth knowing

12. What comes next

M-Files – Introducing the Business Dashboard module

1. The problem to solve

2. What the Business Dashboard module is

3. Concrete use cases

4. Where it runs

5. What this module is not

6. What comes next in the series

OTDS – Installation of Replicas fail if the OT Admin password is too long?

1. OTDS Replica installation failure

2. Going further into the installation logs

3. Status of the installer created/managed files

4. Attempts to debug the issue

5. A problem with the password length or content?

6. Conclusion

Dctm – Another DM_LICENSE_E_INVALID_LICENSE error but caused by JMS this time

1. Symptoms in D2 logs

2. Checking the Repository authentication trace

3. Investigating OTDS authentication logs

4. Checking newly added proxy and correcting it

Dctm – IDS Source 16.7.5 config.bin crash during execution

1. Environment context and IDS upgrade

2. Running the IDS configurator in silent