Here’s a walkthrough of how I added column filters with v-select and v-text-field embedded in v-menu components right inside the table headers.

The Problem

My users wanted to search not just by a global term but with more granularity—for example:

• Show only items where provider contains “aws”
• Filter out items with a specific state
• View all entries with a name that contains “master”

The default Vuetify Data Table doesn’t support per-column filtering out of the box, so I needed to get creative, even if you can override the default filtering used with the search prop.

The Approach

• Attach filter UI elements to each column header
• Bind filters to a filters object keyed by column title
• Apply the filters to the data using a computed property
• Change the icon and icon color on the header to indicate active filters

I. Injecting the Filter UI into Headers

In the v-slot:[header.key], I added a v-menu that displays a filter operator dropdown and a text input:

<template
  v-for="header in dataHeaders"
  v-slot:[header.${header.key}]="{ column }"
  :key="header.key"
>
  {{ column.title }}
  <v-menu :close-on-content-click="false">
    <template v-slot:activator="{ props }">
      <v-btn icon v-bind="props" color="transparent">
        <v-icon v-if="filters[column.title].value">mdi-filter</v-icon>
        <v-icon v-else>mdi-filter-outline</v-icon>
      </v-btn>
    </template>
    <div class="filter-menu">
      <v-select
        v-model="filters[column.title].operator"
        :items="['=', '!=']"
        label="Operator"
        variant="outlined"
        density="compact"
      ></v-select>
      <v-text-field
        v-model="filters[column.title].value"
        label="Search Term"
        variant="outlined"
        clearable
        density="compact"
      ></v-text-field>
    </div>
  </v-menu>
</template>

Each filter’s state is stored in a reactive object:

const filters = ref<Record<string, { value: string; operator: string }>>({});

II. Initializing Filters for Each Column

Once I loaded the table headers, I made sure to initialize the corresponding filters:

dataHeaders.value.forEach((header) => {
    if (!filters.value[header.title]) {
        filters.value[header.title] = { value: "", operator: "=" };
     }
});

This ensured every column could be filtered independently and to avoid errors related to empty/null data.

III. Applying the Filters to the Data

I used a computed property to transform the data on the fly:

const filteredData = computed(() => {
  return recordData.value.filter((item) => {
    return Object.entries(filters.value).every(([filterKey, filter]) => {
      if (!filter.value) return true;

      const matchingKey = Object.keys(item).find((key) => {
        const normalizedFilterKey = filterKey.toLowerCase();
        return key.toLowerCase() === normalizedFilterKey ||
               key.toLowerCase().endsWith("name") &&
               key.toLowerCase().startsWith(normalizedFilterKey);
      });

      if (!matchingKey) return true;

      const itemValue = String(item[matchingKey] || "").toLowerCase();
      const filterValue = filter.value.toLowerCase();

      return filter.operator === "="
        ? itemValue.includes(filterValue)
        : !itemValue.includes(filterValue);
    });
  });
});

This filtering happens client-side after the data is fetched.

IV. Adding UX Details

A few extra things I did for better UX:

• Colored icons: A regular filter icon shows when a filter is active.
• Clear filters button (optional): Could be added for each column.
• Auto-focus: Focuses on the text field when the menu opens.

<v-data-table-server
    v-model:items-per-page="itemsPerPage"
    :items-per-page-options="itemsPerPageOptions"
    :max-height="'50vh'"
    fixed-header
    v-model="selected"
    class="rounded-lg"
    :headers="dataHeaders"
    :items-length="recordLength"
    :items="filteredData"
    :loading="data.loading"
    :search="search"
    :select-strategy="'all'"
    @update:options="loadRecord"
    @click:row="onRowClick"
    show-select
  >
    <template
      v-for="dataKey in dataHeaders"
      v-slot:[`item.${dataKey.key}`]="{ item }"
    >
      <slot
        v-if="item"
        :name="dataKey.key"
        :value="item[dataKey.key]"
        :record="item"
        >{{ item[dataKey.key] }}</slot
      >
      <span :key="`${dataKey.key}-placeholder`" v-else>-</span>
    </template>

    <template
      v-for="header in dataHeaders"
      v-slot:[`header.${header.key}`]="{ column }"
      :key="header.key"
    >
      {{ column.title }}
      <v-menu :close-on-content-click="false">
        <template v-slot:activator="{ props }">
          {{ filters[header] }}
          <v-btn
            icon
            v-bind="props"
            color="rgba(255, 0, 0, 0.0)"
            style="box-shadow: none"
          >
            <v-icon v-if="filters[column!.title!].value == '' ||filters[column!.title!].value == null" color="white" size="small">mdi-filter-outline</v-icon>
            <v-icon v-else color="orange" size="small">mdi-filter</v-icon>
          </v-btn>
        </template>
        <div style="background-color: white; width: 200px; border-radius: 10px;">
          <v-select
            v-model="filters[column.title!].operator"
            :items="['=', '!=']"
            label="Select operator"
            class="pt-4 pl-4 pr-4"
            variant="outlined"
            density="compact"
          ></v-select>
          <v-text-field
            v-model="filters[column.title!].value"
            class="pl-4 pr-4"
            type="text"
            label="Enter the search term"
            :autofocus="true"
            variant="outlined"
            clearable
            density="compact"
          ></v-text-field>
        </div>
      </v-menu>
    </template>
  </v-data-table-server>

What’s Next?

After adding this new feature, the sort indicator next to each column header disappeared—although sorting still works as expected. I haven’t had time to look into it yet, but I’d like to find a solution. If you happen to figure it out, feel free to share it in the comments!

Where to find the complete code ?

You can check out the complete and latest code in the open-source project on GitLab:
👉 https://gitlab.com/yak4all/yak_frontend/yak_ui
The relevant logic lives in the YakGrid.vue component.

Conclusion

By embedding filter controls directly into column headers and managing state reactively, I was able to turn a basic Vuetify data table into a much more powerful data exploration tool.

Let me know if you try this pattern or come up with improvements—I’m always up for iterating! You can also suggest changes or contribute directly on the GitLab repo of the Yak project.

L’article How to add Column Filtering to Vuetify Data Table est apparu en premier sur dbi Blog.

The demo environment is not supposed to be used for production purpose (it’s just a demo and everything is scrubbed after 4 hours)
dbi services shall not be held liable for any data loss on this demo environment.

Wanna give it a try?

To do so, simply connect to https://yak4all.io/demo/, fill in the form and you will receive a confirmation email from no-reply@dbi-services.com.

Please allow 5 to 10 minutes for the environment to be provisioned and the YaK to spin up. Once up and running you’ll receive a second email from no-reply@demo.yak4all.io with the URL and credentials to connect to your YaK.

Check your inbox and junk mail to make sure you don’t miss out on your YaK experience!

Limitations

The demo environment we provision comes with the following restrictions (this will not be the case with a regular installation of YaK):

• AWS infrastructure:
  • Available shapes: t3.micro, t3.small, t3.medium
  • Available disk types: gp2, gp3
  • Disk sizes: max 50gb
  • Available component: PostgreSQL
  • AWS Region: eu-west-1

• Azure infrastructure:
  • Available shapes: Standard_B2s, Standard_B2ms
  • Available disk types: Standard_LRS, StandardSSD_LRS
  • Disk sizes: max 64gb
  • Available component: PostgreSQL

YaK Login Page

Open the link received and log into the YaK with the credentials from the email.

Where to start?

Alright, you’re all set, your environment is ready. Let’s give it a try!

We have pre-provisioned the YaK with a few things for you so that you can almost immediately deploy your servers and components:

• 2 infrastructures have been pre-provisioned:
  • An AWS demo infrastructure: “aws-demo-1”
  • A Microsoft Azure demo infrastructure: “azure-demo-1”
• Cloud providers secrets have been attached to these infrastructures
• A PostgreSQL component type is available to you

What you need to do now

Well, we’ve thought of a scenario, but feel free to explore the YaK on your own if you prefer.

Our use case here is to deploy a server to AWS or to Azure and then deploy a PostgreSQL database instance on it.

1. Declare a SSH key

In order to connect securely to the Linux servers once created, a pair of SSH keys must be generated and the private key must be transferred to the servers.

• Generate your SSH key:

SSH keys can be generated from any Linux-based machine running openSSH with the “ssh-keygen” command.

$ ssh-keygen -b 4096 -m PEM -t rsa -f my-ssh-key -q -N ""

Windows users can generate SSH keys using the PuTTYgen utility (follow this link)

• Copy the content of the private key file:

$ cat my-ssh-key
##This is just an example, generate your own private key

-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAACmFlczI1Ni1jdHIAAAAGYmNyeXB0AAAAGAAAABASDILVb1
MiZ78g848c6Dk6KYwkJZXloSIruLe+Sid0Xd+Lb1rAxbwzAhXWRMzzBT5k3WciXk2fMhUD
p4YSahpYvquZK950aXiZNDOJpKZP8wjf4g4nQyiJZ/0G0J9mIaI2h/fr8TA67KHYtu/Eyf
PN9bKJwII3aiGlu5/Viq2cgYT8jJ01AK8cY3tSqKdlrN2GyxllBFl42QplbxA2USPJ1E6v
VIjtbmJjdz[...........................................]qAc4HxP02t/N2ZN
q+dDNWeShfVz4zjApPQCzdiGqLAlzUy67KNxQgljhaOEaYTQgrLE8XNGsIFNZwGvsEo/xZ
XnN2p
-----END OPENSSH PRIVATE KEY-----

1. Feed the YaK with the SSH key:
   • From the menu, navigate to Configuration -> Secrets
   • Click “Declare“
   • Give a name to your secret (my-ssh-key)
   • Select “ssh key” as secret type
2. Paste the entirety of the private key previously copied, into the corresponding field of the form
3. Click “Save“

2. Attach your SSH key to the infrastructure

1. From the menu, navigate to “Infrastructures“
2. Select the infrastructure you would like to deploy to in the list (“aws-demo-1” or “azure-demo-1”)
3. Click the “Action” button
4. Click “Attach secret“
5. Select the SSH secret created in the previous step from the list
6. Click “Attach“

3. Declare a server

1. Navigate to the “Servers” menu
2. Click “Declare“
3. Give it a name
4. Select the Infrastructure to deploy your server to (“aws-demo-1” or “azure-demo-1”)
5. Select an Image to install : let’s take Debian 12 for example
6. Select a server Shape
7. Add custom tags as per your need (optional)
8. Network section:
   • Mode: automatic
   • IP: leave blank
   • Scope: public
   • Tick the “Admin access” checkbox: this defines the IP address used to connect to the server for configuration purpose.
9. Click Save

In Azure, VM names must be unique. Since we’re all sharing the same demo environment, avoid instance names that are too obvious and could conflict with other users’ machines, such as “srv01”, ‘server1’, “myserver”, etc. Be creative to avoid potential conflicts with other users

4. Deploy your server

1. Select your server in the list
2. Click “Deploy” from the “Actions” menu
3. Confirm “Deploy” in the pop up box
   • Follow the deployment steps from the Jobs menu to see what the YaK is doing behind the scene. If you’ve followed the steps it should succeed (if not you must have missed something )
4. Wait until the job status is “Successful”
5. Your server should be started automatically and will soon be in a “running” state
   • You should already be able to connect to it via SSH by selecting the server and retrieving the public IP from the server parameters. (see step “7. Connect to your server”)
   • But hold your horses! Let’s deploy a component on it! (if you can’t wait, skip the component deployment and come back to it later, that’s OK)

5. Declare your component

1. Navigate to the “Components” menu
2. Click “Declare“
3. Give it a name (ex: PG01)
4. Select your component type (postgresql_instance)
5. Select your subcomponent type (standalone)
   • HA requires 3 servers be deployed, in this example we will consider a standalone setup as we have declared 1 server only
6. Choose your Provider (here it will be aws or azure)
7. Select the PostgreSQL host where you want to deploy your database instance (the server created previously)
8. Fill in the “Instance name” field (ex: mypgdb)
9. Navigate to the “Storage” tab and fill in the required fields.
   • Set Max size of component disks for the 3 file systems:
     • /u01 -> 40GB
     • /u02 -> 20GB
     • /u90 -> 20GB
10. Click “Save“

6. Deploy your component

Your PostgreSQL database instance is configured, now let’s deploy it:

1. Select the component you’ve just configured from the list
2. Click “Actions“
3. Then click “Deploy a new PostgreSQL Standalone instance”
4. Confirm your choice in the pop-up menu
5. Seat back, relax, follow the deployment steps from the jobs menu and wait until your component is successfully installed and configured.

7. Connect to your server and play around

• Open a SSH connection to the public IP address of your server:

DEMOUSER@demo$ ssh -i my-ssh-key <user>@<PUBLIC_IP>

• Your PUBLIC_IP address can be retrieved from Servers -> Parameters
• User:
  • Azure default user is azureuser
  • AWS default users (see AWS doc for more details):
    • The default user for debian based AMI is “admin“, but this could be something else, based on the AMI you selected.
    • For an Amazon Linux AMI, the username is ec2-user.
    • For a CentOS AMI, the username is centos or ec2-user.
      • For a Debian AMI, the username is admin.
      • For a Fedora AMI, the username is fedora or ec2-user.
      • For a RHEL AMI, the username is ec2-user or root.
      • For a SUSE AMI, the username is ec2-user or root.
      • For an Ubuntu AMI, the username is ubuntu.
      • For an Oracle AMI, the username is ec2-user.
      • For a Bitnami AMI, the username is bitnami.

• Play with your PostgreSQL database instance!

admin@my-server1:~$ sudo su - postgres


 PostgreSQL Clusters up and running on this host:
 ----------------------------------------------------------------------------------------------
    mypgdb      ->  /u02/pgdata/17/mypgdb                                    ->  5432  ==> OPEN

 PostgreSQL Clusters NOT running on this host:
 ----------------------------------------------------------------------------------------------

16:39:13 postgres@my-server1:/home/postgres/ [pg17.2] psql -p 5432
psql (17.2 dbi services build)
Type "help" for help.

postgres=#

Thanks for reading this blog, for more info about the YaK, visit yak4all.io or contact us directly.

Read my other blog if you’d like to install your own RKE2 Kubernetes cluster and your own YaK at home!

Happy YaKing !

L’article Try YaK for free with YaK Demo est apparu en premier sur dbi Blog.

Let’s dive-in.

In this article, I will explain how I installed a single node Kubernetes cluster with SUSE RKE2 and deployed my own local YaK instance.

Here is my setup:

• Intel NUC 13 Pro (with Intel Core i5-1340P)
• 64GB of RAM DDR4
• 1TB SSD

I’m using Proxmox as a virtualisation platform on this mini PC, and I’ve chosen SUSE Leap 15.6 as the operating system for the virtual machine that will host my RKE2 installation.

Create my VM in Proxmox

First, I created a VM and dedicated some resources to my instance.

SUSE recommends at least 8GB of RAM and 4vCPU for RKE2 to run smoothly.

Let’s start with that, I can adjust later if the system struggles.

I added 50GB of storage, that should be more than enough (that’s a lot of storage, we really don’t need that much for the YaK itself, but I might reuse this VM for other purposes too)

I’m passing the installation steps of SUSE Leap in server mode, as they are straightforward.
But, just in case, I’m adding the links to Proxmox and to SUSE Leap install docs here.

Install packages

First of all, we need to install the following packages on the VM hosting the YaK:

• Tar
• Git
• Helm

Let’s check what is already installed by default with SUSE Leap 15.6:

localhost:~ # zypper se --installed-only tar
Loading repository data...
Reading installed packages...

S  | Name                    | Summary                                          | Type
---+-------------------------+--------------------------------------------------+--------
i  | cantarell-fonts         | Contemporary Humanist Sans Serif Font            | package
i  | deltarpm                | Tools to Create and Apply deltarpms              | package
i  | shared-python-startup   | Startup script shared by all Python interpreters | package
i  | star                    | POSIX.1-2001-Compliant Tar Implementation        | package
i  | star-rmt                | Magnetic tape control and server                 | package
i  | tar                     | GNU implementation of ((t)ape (ar)chiver)        | package
i  | tar-lang                | Translations for package tar                     | package
i  | tar-rmt                 | Remote tape drive control server by GNU          | package
i  | zypper-needs-restarting | needs-restarting compatibility with zypper       | package

localhost:~ # zypper se --installed-only git
Loading repository data...
Reading installed packages...
No matching items found.

localhost:~ # zypper se --installed-only helm
Loading repository data...
Reading installed packages...
No matching items found.

As you can see, Tar is already installed but not Git nor Helm. Let’s install them:

localhost:~ # zypper install git
localhost:~ # zypper install helm

Stop firewalld

As a pre-requisite, Firewalld must be stopped.
Let’s check its status:

localhost:~ # systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
     Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; preset: disabled)
     Active: active (running) since Wed 2025-04-30 12:12:43 CEST; 2h 7min ago
       Docs: man:firewalld(1)
   Main PID: 743 (firewalld)
      Tasks: 2 (limit: 4915)
        CPU: 349ms
     CGroup: /system.slice/firewalld.service
             └─743 /usr/bin/python3 /usr/sbin/firewalld --nofork --nopid

Apr 30 12:12:43 localhost systemd[1]: Starting firewalld - dynamic firewall daemon...
Apr 30 12:12:43 localhost systemd[1]: Started firewalld - dynamic firewall daemon.

Firewalld is active. I need to stop and disable the service so that it does not restart at system startup:

localhost:~ # systemctl stop firewalld
localhost:~ # systemctl disable firewalld

Install RKE2

Next, I installed RKE2. To do so, I simply followed the official procedure: https://docs.rke2.io/install/quickstart

localhost:~ # curl -sfL https://get.rke2.io | sh -
localhost:~ # systemctl enable rke2-server.service
localhost:~ # systemctl start rke2-server.service

As mentioned in the RKE2 documentation: “additional utilities will be installed at /var/lib/rancher/rke2/bin/. They include: kubectl, crictl, and ctr.
Note that these are not on your path by default.”

I’m adding the repository to my default PATH (this step can be done temporarily or permanently, by editing your ~/.bashrc file). I chose to make it permanent by editing my ~/.bashrc file, but it’s up to you.

export PATH="/var/lib/rancher/rke2/bin:$PATH"

RKE2 install writes a kubeconfig file to /etc/rancher/rke2/rke2.yaml.
Again, I’m adding this into my ~/.bashrc file

export KUBECONFIG=/etc/rancher/rke2/rke2.yaml

I check that the Kubernetes cluster is up and pods are running:

localhost:~ # kubectl get nodes
NAME        STATUS   ROLES                       AGE   VERSION
localhost   Ready    control-plane,etcd,master   22m   v1.31.7+rke2r1

localhost:~ # kubectl get pods -A
NAMESPACE     NAME                                                    READY   STATUS      RESTARTS      AGE
kube-system   cloud-controller-manager-localhost                      1/1     Running     1 (23m ago)   22m
kube-system   etcd-localhost                                          1/1     Running     0             22m
kube-system   helm-install-rke2-canal-nxphw                           0/1     Completed   0             23m
kube-system   helm-install-rke2-coredns-2whps                         0/1     Completed   0             23m
kube-system   helm-install-rke2-ingress-nginx-9mpwb                   0/1     Completed   0             23m
kube-system   helm-install-rke2-metrics-server-t8g27                  0/1     Completed   0             23m
kube-system   helm-install-rke2-runtimeclasses-nbq2v                  0/1     Completed   0             23m
kube-system   helm-install-rke2-snapshot-controller-crd-fmwqh         0/1     Completed   0             23m
kube-system   helm-install-rke2-snapshot-controller-nqq5g             0/1     Completed   1             23m
kube-system   kube-apiserver-localhost                                1/1     Running     0             22m
kube-system   kube-controller-manager-localhost                       1/1     Running     0             22m
kube-system   kube-proxy-localhost                                    1/1     Running     0             22m
kube-system   kube-scheduler-localhost                                1/1     Running     0             22m
kube-system   rke2-canal-qlk8b                                        2/2     Running     0             22m
kube-system   rke2-coredns-rke2-coredns-autoscaler-596dcdf688-dsjjz   1/1     Running     0             22m
kube-system   rke2-coredns-rke2-coredns-cf7df985b-zqwqn               1/1     Running     0             22m
kube-system   rke2-ingress-nginx-controller-b52nh                     1/1     Running     0             21m
kube-system   rke2-metrics-server-58ff89f9c7-dndfb                    0/1     Running     0             21m
kube-system   rke2-snapshot-controller-58dbcfd956-srw5z               1/1     Running     0             21m

Add a persistent volume

We need some storage, don’t we?
By default, RKE2 does not provide persistent volumes.
However, “Local Path Provisioner” provides a way for the Kubernetes users to utilize the local storage in each node (https://github.com/rancher/local-path-provisioner). Let’s use that.

localhost:~ # kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.31/deploy/local-path-storage.yaml
namespace/local-path-storage created
serviceaccount/local-path-provisioner-service-account created
role.rbac.authorization.k8s.io/local-path-provisioner-role created
clusterrole.rbac.authorization.k8s.io/local-path-provisioner-role created
rolebinding.rbac.authorization.k8s.io/local-path-provisioner-bind created
clusterrolebinding.rbac.authorization.k8s.io/local-path-provisioner-bind created
deployment.apps/local-path-provisioner created
storageclass.storage.k8s.io/local-path created
configmap/local-path-config created

Install YaK

Alright, I now have a fully functional Kubernetes cluster.
Let’s install the YaK on it now.

• Create a namespace:

kubectl create ns yak

• Add yak repo

helm repo add yak https://gitlab.com/api/v4/projects/63133284/packages/helm/stable && helm repo update

• Create the yak.values.yaml definition file

localhost:~ # vi yak.values.yaml

global:
  hostname: yak.lab
  admin_credentials:
    password: "ABCdef1234@" #default password to be changed to your need

yak-postgres:
  postgresPassword: "" # Randomly generated if not set
  persistence:
    size: 20Gi
    storageClassName: "local-path" # Uses default if empty

yak-graphile:
  # Archive storage stores the component types archives at import time
  persistence: &archives_storage
    size: 20Gi
    storageClassName: "local-path" # Uses default if empty
  # If using a multi-node cluster, uncomment these 2 lines:
    #accessModes:
     # - ReadWriteMany

  # If you want to expose the rest API at https://hostname/data/graphql and have the postgraphile UI at https://yak.my-domain.local/data/graphiql
  ingress:
    enabled: true
    className: "" # Uses default if empty
    tls:
     - secretName: yak-tls-secret
       hosts:
       - yak.lab

yak-runner:
  image:
    pullPolicy: Always
  persistence:
  # Runner shares the archives storage with graphile
    graphile: *archives_storage
    components:
      enabled: true
      storageClassName: "local-path" # Uses default if empty
      size: 8Gi
    sshconfig:
      enabled: true
      storageClassName: "local-path" # Uses default if empty
      size: 1Gi

yak-ui:
  # Mandatory if you want to access the UI through ingress
  ingress:
    enabled: true
    className: "" # Uses default if empty
    tls:
     - secretName: yak-tls-secret
       hosts:
       - yak.lab

• Run Helm install YaK

helm install yak yak/yak -f yak.values.yaml -n yak

• Check that YaK is properly deployed (all 4 pods should be Running after a few minutes)

localhost:~ # kubectl get pods -n yak
NAME                            READY   STATUS    RESTARTS   AGE
yak-graphile-5b5454fd64-hmk2c   1/1     Running   0          1m
yak-postgres-7bcf4f9897-7mrdg   1/1     Running   0          1m
yak-runner-6669948599-9ggwn     1/1     Running   0          1m
yak-ui-7f6d77c765-spgfn         1/1     Running   0          1m

Connect to the YaK with https

Finally, let’s connect to our web app!

• Update your local /etc/hosts file and add your YaK server, in my case:

192.168.10.185   yak.lab

• Open a web browser and navigate to your URL : https://yak.lab
  • Default credentials :
    • User: admin
    • Password: ABCdef1234@

Voilà!!

I’ve now got a local YaK on RKE2 running at home, ready to deploy infrastructures, servers and components wherever I want.

For more info about the YaK, visit yak4all.io

And if you can’t be bothered installing your own YaK, you can bring up a fully functional YaK demo environment, within minutes : yak4all.io/demo
Feel free to play around and explore our use cases

Happy YaKing 😉 !

L’article Install a single node Kubernetes cluster with SUSE RKE2 and deploy your own YaK instance est apparu en premier sur dbi Blog.

In this guide, we’ll walk you through the process of contributing to the YaK project by adding support for a new infrastructure provider, using OpenShift Virtualization as an example. We’ll cover the steps to fork the necessary repositories, implement the required changes, and integrate your contributions into the main project.

Overview of the YaK Project Architecture

YaK is organized into multiple GitLab repositories, each responsible for a specific part of the system, here the ones we will participate:

• yakenv: Environment container image with all the packages and dependencies
• yak_core: Is using yakenv as based image, core logic
• yak_runner: Is using yak_core as based image, backend logic for automation jobs
• yak_ui: Web-based frontend

These components work together to provide a modular, extensible framework for managing your different infrastructures and attached components.

Step 1: Fork the Repositories

Before starting development, fork each of the required repositories into your own GitLab account:

1. Go to each project page:
   • yakenv
   • yak_core
   • yak_runner
   • yak_ui
2. Click the Fork button at the top right of the page.
3. Choose your namespace (e.g., your GitLab username or organization).
4. Repeat for all repositories.

Step 2: Set Up Your Local Environment

Clone the forked repositories to your local machine:

git clone https://gitlab.com/<your-namespace>/yakenv.git
git clone https://gitlab.com/<your-namespace>/yak_core.git
git clone https://gitlab.com/<your-namespace>/yak_backend/yak_runner.git
git clone https://gitlab.com/<your-namespace>/yak_frontend/yak_ui.git

Then install dependencies (Docker or Podman) and ensure each project builds correctly. Refer to the README.md files for project-specific setup instructions.

Step 3: Implement the New Infrastructure Provider

To add OpenShift Virtualization as a new provider, you will need to make coordinated changes across the different components:

yakenv

• Dockerfile: Add to this file the commands to install Ansible collections and/or CLI packages that you will need to interract with your new provider.
  Here’s the lines we added for Openshift Virtualization:

# Install Kubevirt collection (OpenShift Virtualization)
RUN ansible-galaxy collection install kubevirt.core==2.1.0 --collections-path /etc/ansible/collections --force

# Install virtctl CLI (Kubevirt)
RUN curl -lL https://github.com/kubevirt/kubevirt/releases/download/v1.5.1/virtctl-v1.5.1-linux-amd64 -o /usr/bin/virtctl && chmod +x /usr/bin/virtctl

• requirements.txt: The python pip packages to install.
  Here’s the line we added for Openshift Virtualization:

kubernetes==22.6.0

yak_core

• collections/ansible_collections/yak/core/roles: This is the directory were you will find all the tasks to manage the infrastructures and the servers storage.
  In this directory you will find the infrastructures which is a template for a new infrastructures, you can copy it and rename it as the name of your infrastructure (replace white spaces by underscores _) here openshift_virtualization
• collections/ansible_collections/yak/core/roles/openshift_virtualization/defaults/main.yml: Define the variables defaults values:

---
state: present
openshift_namespace: default
custom_labels: {}
instance_type: u1.medium
kubeconfig_path: ~/yak/kubeconfig
preference_name: ""

rhel_activation_key: "{{ lookup('env','RHEL_ACTIVATION_KEY') }}"
rhel_org_id: "{{ lookup('env','RHEL_ORG_ID') }}"

• collections/ansible_collections/yak/core/roles/openshift_virtualization/tasks: the tasks files for your infrastructure. There is the minimum files:

1. main.yml: the default task file, it will redirect to the tasks depending of the state. Here is the content:

---
- include_tasks: check_credential.yml
- include_tasks: "{{ state }}.yml"

2. check_credentials.yml: Step to valid that the provided credentials are valid

---
- block:
  - name: List all VMs - Verifying credentials are valid
    delegate_to: localhost
    kubevirt.core.kubevirt_vm_info:
      namespace: "{{ openshift_namespace }}"
    retries: 3
    delay: 5
  rescue:
  - fail:
      msg: 'Please ensure your OpenShift credentials are valid.'

3. present.yml: Steps to create the server’s VM on your infrastructure. If there is some differences to create a linux or windows server, you can include sub-tasks files (here we developed create_linux.yml and create_windows.yml)

# Copyright: (c) 2025, dbi services, distributed without any warranty under the terms of the GNU General Public License v3
- ansible.builtin.include_tasks: 
    file: get_vm_state.yml

- delegate_to: localhost
  set_fact:
    number_of_vms: "{{ r_get_vm_state.resources | length }}"

- block:
  - include_tasks: "create_{{ os_type }}.yml"
  - include_tasks: post_config.yml
  when: number_of_vms | int == 0

- when: number_of_vms | int > 0
  block:
    - fail:
        msg: "Cannot Deploy as the VirtualMachine already exists."

4. started.yml: Steps to start the server’s VM

---
- name: Power On
  delegate_to: localhost
  ansible.builtin.shell: "virtctl --kubeconfig {{ openshift_virtualization_kubeconfig }} --namespace {{ openshift_namespace }} start {{ machine_name }}"
  register: r_openshift_virtualization_vm_started

- include_tasks: "post_config.yml"

5. stopped.yml: Steps to power off your server

---
- name: Power Off
  delegate_to: localhost
  ansible.builtin.shell: "virtctl --kubeconfig {{ openshift_virtualization_kubeconfig }} --namespace {{ openshift_namespace }} stop {{ machine_name }}"
  register: r_openshift_virtualization_vm_stopped

6. absent.yml: Steps to destroy the server

---
- name: Delete the VirtualMachine
  delegate_to: localhost
  kubevirt.core.kubevirt_vm:
    namespace: "{{ openshift_namespace }}"
    state: absent
    name: "{{ machine_name }}"
  register: r_openshift_virtualization_vm_absent

yak_runner

• sql/upgrade/2.2.0/02-infrastructures.sql: Add in the infrastructures jsonb specification schema, the new provider with its specific parameters

UPDATE jsonb_specifications
SET schema = 
'{
  "variableName": "variables",
  "dataType": "object",
  "niceName": "Variables",
  "isSelectable": true,
  "selectorType": "providerName",
  "selectorOption": [
    "aws",
    "azure",
    "oci",
    "openshift_virtualization"
  ],
  "aws": [
    ...
  ],
  "azure": [
    ...
  ],
  "oci": [
    ...
  ],
  "openshift_virtualization": [
    {
      "dataType": "string",
      "defaultValue": "default",
      "isOneOffSetting": false,
      "maxLength": 100,
      "mandatory": true,
      "variableName": "network_name",
      "usage": "Network name to connect",
      "regex": null,
      "niceName": "Network Name",
      "minLength": 1
    },
    {
      "dataType": "string",
      "defaultValue": "default",
      "isOneOffSetting": false,
      "maxLength": 100,
      "mandatory": true,
      "variableName": "domain_device_interface_name",
      "usage": "Domain device interface name",
      "regex": null,
      "niceName": "Domain Device Interface Name",
      "minLength": 1
    },
    { "variableName": "custom_labels", "niceName": "Infrastructure labels", "dataType": "dictionary", "mandatory": false }
  ]
}
'::jsonb
WHERE name = 'infrastructureVariables';

• sql/upgrade/2.2.0/05-server_secret_type_parameters.sql: Add in the according tables the new provider and if needed, new secret type with its parameters:

insert into providers values
(5,	'openshift_virtualization', true, '{}', 'OpenShift Virtualization');

insert into secret_types values
(7,	'kubeconfig',true,true,false,5);

-- Data: secret_type_parameters
insert into secret_type_parameters (secret_type_id, variable_name, nice_name, data_type_id, mandatory, options, min_length, max_length) values
-- Kubeconfig
(7, 'K8S_AUTH_KUBECONFIG', 'OpenShift Kubeconfig', 1, true, null, 1, 5120);

• sql/upgrade/2.2.0/06-provider_images.sql: Add new images for your new provider and allow admin to manage them:

insert into provider_images (name, provider_id, operating_system_type_id, ansible_user, variables, user_id) values
-- OpenShift Virtualization linux
('datasource_centos-stream9', 5, 1, 'admin', '{"dataSource": {"name": "centos-stream9", "namespace": "openshift-virtualization-os-images"}}', 1),
('datasource_centos-stream10', 5, 1, 'admin', '{"dataSource": {"name": "centos-stream10", "namespace": "openshift-virtualization-os-images"}}', 1),
('datasource_fedora', 5, 1, 'admin', '{"dataSource": {"name": "fedora", "namespace": "openshift-virtualization-os-images"}}', 1),
('datasource_rhel8', 5, 1, 'admin', '{"dataSource": {"name": "rhel8", "namespace": "openshift-virtualization-os-images"}}', 1),
('datasource_rhel9', 5, 1, 'admin', '{"dataSource": {"name": "rhel9", "namespace": "openshift-virtualization-os-images"}}', 1),
('datasource_ubuntu24-04', 5, 1, 'admin', '{"dataSource": {"name": "ubuntu24-04", "namespace": "openshift-virtualization-os-images"}}', 1),
('datasource_win2k22', 5, 1, 'admin', '{"dataSource": {"name": "win2k22", "namespace": "openshift-virtualization-os-images"}}', 1),
('datasource_win2k25', 5, 1, 'admin', '{"dataSource": {"name": "win2k25", "namespace": "openshift-virtualization-os-images"}}', 1),
('datasource_win10', 5, 1, 'admin', '{"dataSource": {"name": "win10", "namespace": "openshift-virtualization-os-images"}}', 1),
('datasource_win11', 5, 1, 'admin', '{"dataSource": {"name": "win11", "namespace": "openshift-virtualization-os-images"}}', 1);

insert into projects_provider_images (project_id, image_id, user_id) 
    select 1, id, 1 from provider_images where provider_id=5;

• sql/upgrade/2.2.0/07-servers.sql: Adding server parameters for the new provider :

UPDATE jsonb_specifications
SET schema = 
'{
  "variableName": "variables",
  "niceName": "Variables",
  "dataType": "object",
  "isSelectable": true,
  "selectorType": "providerName",
  "selectorOption": ["aws", "azure", "oci", "openshift_virtualization", "on_premises"],
  "aws": [
    ...
  ],
  "azure": [
    ...
  ],
  "oci": [
    ...
  ],
   "on_premises": [
    ...
  ],
  "openshift_virtualization": [
    {
      "variableName": "providerImageId",
      "niceName": "Provider Image ID",
      "dataType": "integer",
      "hidden": true,
      "mandatory": false,
      "isOneOffSetting": false
    },
    {
      "dataType": "string",
      "defaultValue": null,
      "isOneOffSetting": true,
      "maxLength": 100,
      "mandatory": true,
      "variableName": "providerImageName",
      "usage": "See documentation",
      "regex": null,
      "options": "values.vProviderImages",
      "niceName": "Provider Image Name",
      "minLength": 1
    },
    {
      "variableName": "providerShapeId",
      "niceName": "Provider Shape ID",
      "dataType": "integer",
      "hidden": true,
      "mandatory": false,
      "isOneOffSetting": false
    },
    {
      "dataType": "string",
      "defaultValue": null,
      "isOneOffSetting": true,
      "maxLength": 100,
      "mandatory": true,
      "variableName": "providerShapeName",
      "usage": "See documentation",
      "regex": null,
      "options": "values.vProviderShapes",
      "niceName": "Provider Shape Name",
      "minLength": 1
    },
    { 
      "variableName": "openshift_namespace",
      "niceName": "Namespace",
      "defaultValue": "default",
      "dataType": "string",
      "mandatory": true
    },
    { 
      "variableName": "preference_name",
      "niceName": "Preference Name",
      "defaultValue": "",
      "dataType": "string",
      "mandatory": true
    },
    { "variableName": "custom_labels", "niceName": "Server labels", "dataType": "dictionary", "mandatory": false }

  ]
}' :: jsonb
WHERE name = 'serverVariables';

• sql/upgrade/2.2.0/11-components-storage-jsonb.sql: Adding component disks parameters for the provider:

UPDATE jsonb_specifications
SET schema = 
    '{
      "dataType": "object",
      "on_premises": [],
      "isSelectable": true,
      "selectorType": "providerName",
      "variableName": "storageVariables",
      "selectorOption": [
        "aws",
        "azure",
        "oci",
        "on_premises",
        "openshift_virtualization"
      ],
      "aws": [
        ...
       ],
      "azure": [
        ...
      ],
      "oci": [
        ...
      ],
      "on_premises": [
       ...
      ],
      "openshift_virtualization": [
        {
          "dataType": "array",
          "isSelectable": true,
          "selectorType": "osType",
          "variableName": "filesystems",
          "niceName": "File systems",
          "selectorOption": [
            "Linux",
            "Windows"
          ],
          "Linux": [
            {
                "variableName": "mount_point",
                "niceName": "Mount point, e.g. /app",
                "dataType": "string",
                "defaultValue": null,
                "isOneOffSetting": true,
                "maxLength": 100,
                "mandatory": true,
                "minLength": 1
            },
            {
                "variableName": "filesystem_type",
                "niceName": "Filesystem type",
                "dataType": "string",
                "mandatory": true,
                "isOneOffSetting": false,
                "defaultValue": "xfs"
            },
            {
                "variableName": "size_gb",
                "niceName": "Mount point size in GB",
                "dataType": "integer",
                "mandatory": true,
                "isOneOffSetting": false,
                "defaultValue": 20
            },
            {
              "dataType": "integer",
              "isOneOffSetting": true,
              "mandatory": true,
              "variableName": "max_size_gb",
              "niceName": "Disk size",
              "min": 1,
              "defaultValue": -1,
              "usage": "Defines the size of the virtual disk at cloud provider end. 5% overhead is automatically added to each disk for metadata storage."
            },
            {
              "dataType": "string",
              "niceName": "Storage Class Name",
              "mandatory": false,
              "maxLength": 100,
              "minLength": 0,
              "defaultValue": "",
              "variableName": "volume_storageClassName",
              "isOneOffSetting": true
            }
          ],
          "Windows": [
            {
                "variableName": "drive_letter",
                "niceName": "Drive letter, e.g. F",
                "dataType": "string",
                "defaultValue": null,
                "isOneOffSetting": true,
                "maxLength": 100,
                "mandatory": true,
                "minLength": 1
            },
            {
                "variableName": "partition_label",
                "niceName": "Partition label",
                "dataType": "string",
                "mandatory": true,
                "isOneOffSetting": false,
                "defaultValue": "data"
            },
            {
                "variableName": "size_gb",
                "niceName": "Mount point size in GB",
                "dataType": "integer",
                "mandatory": true,
                "isOneOffSetting": false,
                "defaultValue": 20
            },
            {
              "dataType": "string",
              "niceName": "Storage Class Name",
              "mandatory": false,
              "maxLength": 100,
              "minLength": 0,
              "defaultValue": "",
              "variableName": "volume_storageClassName",
              "isOneOffSetting": true
            }
          ]
        }
      ]
    }' :: jsonb
  WHERE name = 'componentStorageVariables';

• sql/upgrade/2.2.0/12-provider_disks_parameters_root_jsonb.sql: Adding server disks parameters for the provider:

UPDATE jsonb_specifications
SET schema = 
    '{
      "dataType": "object",
      "on_premises": [],
      "isSelectable": true,
      "selectorType": "providerName",
      "variableName": "storageVariables",
      "selectorOption": [
        "aws",
        "azure",
        "oci",
        "on_premises",
        "openshift_virtualization"
      ],
      "aws": [
        ...
       ],
      "azure": [
        ...
      ],
      "oci": [
        ...
      ],
      "on_premises": [
       ...
      ],
      
  "openshift_virtualization": [
    {
      "Linux": [
        {
          "dataType": "integer",
          "niceName": "Size in GB",
          "mandatory": true,
          "defaultValue": 30,
          "variableName": "volume_size",
          "isOneOffSetting": true
        },
        {
          "dataType": "string",
          "niceName": "Storage Class Name",
          "mandatory": false,
          "maxLength": 100,
          "minLength": 0,
          "defaultValue": "",
          "variableName": "volume_storageClassName",
          "isOneOffSetting": true
        },
        {
          "options": [
            "Block",
            "Filesystem"
          ],
          "dataType": "string",
          "niceName": "Disk type",
          "mandatory": true,
          "maxLength": 100,
          "minLength": 1,
          "defaultValue": "Filesystem",
          "variableName": "volume_mode",
          "isOneOffSetting": true
        }
      ],
      "Windows": [
        {
          "dataType": "integer",
          "niceName": "Size in GB",
          "mandatory": true,
          "defaultValue": 30,
          "variableName": "volume_size",
          "isOneOffSetting": true
        },
        {
          "dataType": "string",
          "niceName": "Storage Class Name",
          "mandatory": false,
          "maxLength": 100,
          "minLength": 0,
          "defaultValue": "",
          "variableName": "volume_storageClassName",
          "isOneOffSetting": true
        },
        {
          "options": [
            "Block",
            "Filesystem"
          ],
          "dataType": "string",
          "niceName": "Disk type",
          "mandatory": true,
          "maxLength": 100,
          "minLength": 1,
          "defaultValue": "Filesystem",
          "variableName": "volume_mode",
          "isOneOffSetting": true
        }
      ],
      "dataType": "object",
      "isSelectable": true,
      "selectorType": "providerImageOsType",
      "variableName": "openshift_virtualization_root_disk_parameters",
      "selectorOption": [
        "Linux",
        "Windows"
      ]
    },
    {
      "variableName": "providerImageOsType",
      "niceName": "Operating System",
      "dataType": "string",
      "mandatory": true,
      "isOneOffSetting": true,
      "options": ["Linux", "Windows"],
      "readonly": true
    }
  ]
    }' :: jsonb
  WHERE name = 'componentStorageVariables';

• sql/upgrade/2.2.0/13-provider-shapes.sql: Adding shapes for the provider:

insert into provider_shapes (name, provider_id, variables, user_id) values
('openshift-virtualization-u1_medium-shape',   5, '{"instance_type": "u1.medium"}', 1),
('openshift-virtualization-u1_large-shape',   5, '{"instance_type": "u1.large"}', 1),
('openshift-virtualization-u1_xlarge-shape',   5, '{"instance_type": "u1.xlarge"}', 1);

insert into projects_provider_shapes (project_id, shape_id, user_id) 
    select 1, id, 1 from provider_shapes where provider_id=5;

yak_ui

• yak-ui/src/assets/providers/virt-icon.png: Adding the new provider logo in provider logos:
• yak-ui/src/utils/imageList.ts: Adding the new logo path in the ProviderLogos dict:

import OnPremiseLogo from "@/assets/providers/2860404-200-min.png";
import LinuxLogo from "@/assets/linux.png";
import WindowsLogo from "@/assets/windows.png";
import OpenShiftVirtualizationLogo from "@/assets/providers/virt-icon.png";

export const ProviderLogos: HashTable<any> = {
  aws: AwsLogo,
  oci: OciLogo,
  'on_premises': OnPremiseLogo,
  azure: AzureLogo,
  'openshift_virtualization': OpenShiftVirtualizationLogo,
};

...

• yak-ui/src/components/YakTabContainer.vue: Adding in containerClass.specialTabs and cardClass.specialTabs lists the new provider :

...
const containerClass = computed(() => {
  const specialTabs = ["aws", "oci", "azure", "on_premises", "openshift_virtualization"];
  return specialTabs.includes(tab.value)
    ? "container-grid-custom"
    : "container-custom";
});

const cardClass = computed(() => {
  const specialTabs = ["aws", "oci", "azure", "on_premises", "openshift_virtualization"];
  return specialTabs.includes(tab.value) ? "card-grid-custom" : "card-custom";
});
...

• yak-ui/src/components/customs/StorageEditor.vue: Adding in providerTypeField.options list the new provider :

...
const providerTypeField: Field = {
  variableName: "providerName",
  niceName: "providerName",
  dataType: "string",
  mandatory: true,
  isOneOffSetting: true,
  options: ["aws", "azure", "oci", "on_premises", "openshift_virtualization"],
};
...

• yak-ui/src/data_structures/vComponents.ts: Adding in children.options list parameter for allGroups.push call the new provider :

...
                    allGroups.push({
                      variableName: group.name,
                      niceName: "group " + group.name,
                      mandatory: true,
                      dataType: "object",
                      isOneOffSetting: false,
                      children: [{
                        variableName: "providerName",
                        niceName: "Provider",
                        dataType: "string",
                        mandatory: true,
                        isOneOffSetting: true,
                        options: ["aws", "azure", "oci", "on_premises", "openshift_virtualization"],
...

• yak-ui/src/data_structures/vSecrets.ts: Adding in vSecretTypeName list the new secret type :

...
              vSecretTypeName: [
                "aws credentials",
                "azure credentials",
                "oci credentials",
                "kubeconfig",
                "ssh key",
                "winrm",
              ],
...

Step 4: Test Your Changes

Each component has its own set of unit and integration tests. Be sure to:

• Run tests locally with each change.
  All sub-projects of the YaK have a Dockerfile on root directory. Build them in this order to respect the dependencies:
  1. yakenv
  2. yak_core
  3. yak_runner
  4. yak_postgres
  5. yak_graphile
  6. yak_ui

Once images are built, run them on Docker/Podman on your computer, or on a Container orchestrator (Kubernetes/Openshift/…)

• Validate that your provider appears correctly in the UI and can be selected.

New infrastructure declaration:

New infrastructure declared:

New secret declaration:

Provider images appeared well:

Provider shapes appeared well:

New server declaration:

• Test full cluster lifecycle operations (deploy, stop, start, destroy, delete) for OpenShift Virtualization.

Deploy server:

Result in Openshift:

Stop server:

Start server:

Destroy server:

Step 5: Contribute Your Changes

Once development is complete and tested:

1. Push your changes to your fork.
2. Create a Merge Request (MR) from your fork to the main project repository for each affected sub-project:
3. In your MR description, include:
   • A summary of the changes
   • A test plan and screenshots if applicable
   • Any known limitations or issues

Follow the contribution guidelines of the YaK project for naming conventions, commit messages, and code standards.

You can take my merge requests as an exemple:

• yakenv: https://gitlab.com/yak4all/yakenv/-/merge_requests/12
• yak_core: https://gitlab.com/yak4all/yak_core/-/merge_requests/20
• yak_runner: http://gitlab.com/yak4all/yak_backend/yak_runner/-/merge_requests/42
• yak_ui: https://gitlab.com/yak4all/yak_frontend/yak_ui/-/merge_requests/42

Step 6: Collaborate with the Community

After submitting your MR:

• Respond to any feedback or questions from project maintainers.
• Be open to revising your code based on review suggestions.
• Once approved, your changes will be merged into the main project!

Conclusion

Adding a new infrastructure provider to YaK, such as OpenShift Virtualization, is a powerful way to contribute to the project and support broader use cases. With a clear architecture and modular codebase, YaK makes it straightforward for developers to extend its capabilities.

YaK can also be integrated with various Red Hat products to enhance your automation workflows – check out our guide on Integrating YaK into Red Hat Ansible Automation Platform to learn more.

L’article How to Contribute to the YaK Project: Adding a New Infrastructure Provider (e.g., OpenShift Virtualization) est apparu en premier sur dbi Blog.

YaK core Multi-Platform open source Automation Tool simplifies the deployment of Ansible playbooks through a clean UI and API. It offers an intuitive interface where users can upload playbooks, configure parameters, and deploy them seamlessly across various platforms, and all managed through a centralized inventory stored in a PostgreSQL database. With YaK Core, developers can focus on writing application code without worrying about infrastructure setup or management.

YaK consists of two parts: YaK Core, which is open source, and YaK Components, which can be installed on top. These YaK Components are platform-agnostic service packages (e.g., PostgreSQL, Oracle DB, MongoDB, Kubernetes, etc.), written in Ansible by experts. They provide essential operational features such as backup, patching, upgrades, and high availability. If you’d like to learn more about the available YaK components, feel free to contact us!

But that’s not all. YaK Core also lets you create your own YaK Components . Once created, your component becomes immediately available for deployment across all platforms supported by YaK Core.

In this blog, I’ll show you how easy it is to create your own YaK Component using Ansible, upload it to YaK Core, and deploy it across any supported platform.

YaK Demo platform provisioning

To get started with YaK Core Multi-Platform open source solution, visit https://yak4all.io and provision your own YaK demo environment (take 5 minutes to be ready).

Build your YaK Component

To build a YaK Component, you need to declare at least the following three files

1. playbooks/create_linux_users.yml
2. manifest.yml
3. yak_variables_specifications/basic_variables_specifications.yml

1. The Ansible Playbook

playbooks/create_linux_users.yml
This file is simply your Ansible playbook, nothing more. The only requirement is that the code uses variables, which will be exposed in the UI for configuration. The example playbook below will create a user and optionally grant them sudo privileges.

---
- name: Create Linux users
  hosts: linux_hosts
  become: true
  gather_facts: true

  tasks:
    - debug:
        var: user

    - name: Create users
      ansible.builtin.user:
        name: "{{ item.username }}"
        create_home: "{{ item.create_home | default(true) }}"
        state: present
      loop: "{{ user }}"

    - name: Add users to sudoers
      community.general.sudoers:
        name: "yak-sudoer-{{ item.username }}"
        user: "{{ item.username }}"
        commands: ALL
        state: present
      loop: "{{ user }}"
      when: item.is_sudoer
        
  post_tasks:
    - name: Update component state
      delegate_to: localhost
      yak.core.yak_component_state_update:
        component_state_name: 'deployed'
...

2. Manifest file

manifest.yml
This file contains the basic information about your component and specifies which playbooks can be executed.

name: linux_users

version:
  major: 1
  minor: 0
  patch: 0

sub_component_types:
  - display_label: Linux users
    name: create_linux_users
    features:
      - display_label: Create Linux users
        name: create_linux_users
        playbook_name: playbooks/create_linux_users.yml

    inventory_maps:
      - group_name: linux_hosts
        group_nicename: Linux hosts
        group_description: Host on which the users will be created
        group_min_hosts: 1
        group_max_hosts: 100
        type: host
        os_type: Linux

3. Variable specification file

yak_variables_specifications/basic_variables_specifications.yml
Now you can define and provide all the specifications for the variables you want to make configurable, with all the required settings .

- variableName: user
  niceName: Users to create
  dataType: array
  children:
    - variableName: username
      niceName: Username
      dataType: string
      mandatory: true
      defaultValue: yak
      isOneOffSetting: false
      usage: Name of the user to create

    - variableName: create_home
      niceName: Create Home directory
      dataType: boolean
      mandatory: true
      defaultValue: true
      isOneOffSetting: false
      usage: Tick the box if you want to create a Home directory for the user (/home/<username>)

    - variableName: is_sudoer
      niceName: Grant sudo privileges
      dataType: boolean
      mandatory: true
      defaultValue: true
      isOneOffSetting: false
      usage: Tick the box if you want to grant "ALL" privileges escalation to the user

That’s it! You now have all the necessary files for your first YaK Component. Next, create a ZIP package and upload it to your deployed YaK Demo environment.

To make things easier, I’ve created a ZIP file that you can upload directly. : create_linux_user.zip

Setup a Server

For this task, simply follow the documentation below up to Step 4: Deploy your server https://dbi-services.gitbook.io/yak-user-doc/introduction/yak-demo

Declare and deploy your Component

You’re now ready to declare and deploy your component!

1. Declare

YaK UI -> Components -> Declare -> Component_type : linux_users -> Save

2 Deploy

YaK UI -> Components -> Select LinuxUser -> Action -> Create Linux User -> Confirm

Conclusion

This component can now be deployed on any cloud platform or integrated on On-Premises environment using the YaK UI, and can also be deployed in parallel on up to 100 servers, as specified in your Manifest file.

With this solution, you can provide your colleagues with an intuitive and efficient way to work with Ansible playbooks, enhancing their overall experience .

For more Information about YaK see the blogs available here : https://www.dbi-services.com/blog/yak

L’article YaK Core – The Holy Grail for Deploying Ansible Code Everywhere est apparu en premier sur dbi Blog.

Introduction to YaK

YaK is an open-source automation project developed by dbi services. Built on Ansible playbooks, YaK streamlines the deployment process for various components across any platform. It ensures adherence to best practices, maintains deployment quality, and significantly reduces time-to-deploy.

Initially created in response to the growing demand from dbi services’ consultants and clients, YaK simplifies and accelerates deployments across multi-technology infrastructures. Whether targeting cloud environments or on-premises systems, YaK drastically cuts down deployment effort, optimizing the overall time-to-market.

Find more informations on the YaK website

Why Integrate YaK into Red Hat Ansible Automation Platform (AAP)?

YaK Advantages:

• User-Friendly Interface: YaK simplifies configuration and deployment through an intuitive user interface, allowing teams to quickly manage servers and applications deployments.
• Centralized Metadata Database: It replaces traditional YAML configuration files with a centralized database to store deployment metadata, ensuring improved manageability and consistency.
• Comprehensive Reporting: YaK provides capabilities for generating detailed reports on all deployments, offering insights for continuous improvement.
• dbi services components: dbi services offering a range of subscriptions components readily deployable on any platform, further easing the accessibility, management of deployments. These components integrates all the expertise of dbi services’ expertise.
• Custom Application Integration: YaK supports creating custom components for your specific applications. Developers can easily add Ansible playbooks to deploy the application into the component template.

Why Red Hat Ansible Automation Platform (AAP) with YaK:

• Expert-Crafted Packages: YaK provides expertly maintained Ansible packages, ensuring reliability and built-in support for a wide range of scenarios, fully compatible with AAP.
• Unified Dynamic Inventory: A single dynamic Ansible inventory for all your infrastructures, supporting multi-platform environments.
• Platform-Agnostic Deployments: Seamless deployment across various platforms, enabling true platform independence.
• Deep Integration with AAP Features: Full integration with AAP’s scheduler, workflows, and other advanced features, simplifying automation of servers, components (databases, applications, etc..), and complex multi-component infrastructures. of servers/component and/or multi-component infrastructure automation.

Integration Steps

Generate a YaK API Token

To start integration, generate an API token from the YaK database pod. So you need:

1. To have access to the Kubernetes cluster (rke2 for exemple) on which is deployed your YaK instance, with the kubectl command
2. Know the namespace on which is deployed your YaK instance

Once you have access, you only have to type this command (replace <yak-namespace> by the namespace on which is deployed your YaK instance) :

$ kubectl -n <yak-namespace> exec -it deploy/yak-postgres -- psql -U postgres -d agoston -c 'select set_user_token(agoston_api.add_user()) as "token";'
                               token                               
-------------------------------------------------------------------
 <generated_token>
(1 row)

You can store the YaK API generated token for next steps.

AAP Resources Configuration

Access to Ansible Automation Platform with an administrator rôle.

• Execution Environment: Define a customized execution environment in AAP that includes YaK-specific dependencies and tools.
  In the left menu, go to Automation Execution ⟶ Infrastructure ⟶ Execution Environments, then click on Create execution environment button
  Fill the form like this:
  – Name: YaK EE
  – Image: registry.gitlab.com/yak4all/yak_core:ee-stable
  – Pull: Only pull the image if not present before running
  – Registry credential: <empty> (YaK images are publicly available on GitLab repository)
  – Description: Execution environment for YaK related jobs
  – Organization: Default (or any other if you have a specific policy)

• Job Settings: Update parameters to add persistency for YaK jobs.
  In the left menu, go to Settings ⟶ Job then click on Edit button
  update the parameter Paths to expose to isolated jobs, and add these lines at the end:

- /data/yak/component_types:/workspace/yak/component_types
- /data/yak/tmp:/tmp
- /data/yak/uploads:/uploads

• Credential Types: Create customs credential types to securely handle YaK specific credentials.
  In the left menu, go to Automation Execution ⟶ Infrastructure ⟶ Credential Types, then click on Create credential type button

1. YaK API:
   – Name: YaK API
   – Input configuration:

fields:
  - id: yak_ansible_transport_url
    type: string
    label: YaK API URL
  - id: yak_ansible_http_token
    type: string
    label: YaK API HTTP Token
    secret: true
  - id: yak_ansible_ssl_verify_certificate
    type: string
    label: Verify SSL certificate
    choices:
      - 'true'
      - 'false'
required:
  - yak_ansible_transport_url
  - yak_ansible_http_token
  - yak_ansible_ssl_verify_certificate

⠀⠀⠀⠀‎‎‎‎- Injector configuration:

env:
  YAK_ANSIBLE_DEBUG: 'false'
  YAK_ANSIBLE_HTTP_TOKEN: '{{ yak_ansible_http_token }}'
  YAK_ANSIBLE_TRANSPORT_URL: '{{ yak_ansible_transport_url }}'
  YAK_ANSIBLE_SSL_VERIFY_CERTIFICATE: '{{ yak_ansible_ssl_verify_certificate }}'

2. YaK API With Component:
   – Name: YaK API With Component
   – Input configuration:

fields:
  - id: yak_ansible_transport_url
    type: string
    label: YaK API URL
  - id: yak_ansible_http_token
    type: string
    label: YaK API HTTP Token
    secret: true
  - id: yak_ansible_ssl_verify_certificate
    type: string
    label: Verify SSL certificate
    choices:
      - 'true'
      - 'false'
  - id: yak_core_component
    type: string
    label: YaK Core Component (used for component deployment)
required:
  - yak_ansible_transport_url
  - yak_ansible_http_token
  - yak_ansible_ssl_verify_certificate

⠀⠀⠀⠀- Injector configuration:

env:
  YAK_ANSIBLE_DEBUG: 'true'
  YAK_CORE_COMPONENT: '{{ yak_core_component }}'
  YAK_ANSIBLE_HTTP_TOKEN: '{{ yak_ansible_http_token }}'
  YAK_ANSIBLE_TRANSPORT_URL: '{{ yak_ansible_transport_url }}'
  YAK_ANSIBLE_SSL_VERIFY_CERTIFICATE: '{{ yak_ansible_ssl_verify_certificate }}'

• Credentials: Set up credentials in AAP using the custom credential type to securely store and manage YaK API tokens.
  In the left menu, go to Automation Execution ⟶ Infrastructure ⟶ Credential, then click on Create credential button

1. YaK API Core:
   – Name: YaK API Core
   – Credential type: YaK API
   – YaK API URL: <url to your yak instance>/data/graphql
   – YaK API HTTP Token: <YaK API token generated previously>
   – Verify SSL certificate: depending if your YaK url have a valid SSL certificate (select true) or not (select false)

2. YaK API Component:
   – Name: YaK API Component – <component name set in YaK>
   – Credential type: YaK API Withe Component
   – YaK API URL: <url to your yak instance>/data/graphql
   – YaK API HTTP Token: <YaK API token generated previously>
   – YaK Core Component (used for component deployment): <component name set in YaK>
   – Verify SSL certificate: depending if your YaK url have a valid SSL certificate (select true) or not (select false)

• Project: Create an AAP project pointing to your YaK repository containing playbooks.
  In the left menu, go to Automation Execution ⟶ Project, then click on Create project button

1. YaK Core:
   – Name: YaK Core
   – Execution environment: YaK EE
   – Source control type: Git
   – Source control URL: https://gitlab.com/yak4all/yak_core.git
   – Source control branch/tag/commit: <select the same release version than your YaK deployed>
   You can find the YaK release version at the bottom of the YaK left menu:

2. YaK Component:
   – Name: YaK <component type> Component
   – Execution environment: YaK EE
   – Source control type: Git
   – Source control URL: <private git repository url to your component>
   – Source control branch/tag/commit: main
   – Source control credential: <your credential where stored your authentications to the git repository>

• Inventory: Configure the inventory, aligning it with YaK’s managed targets and deployment definitions.
  In the left menu, go to Automation Execution ⟶ Infrastructure ⟶ Inventories, then click on Create inventory button

1. YaK Inventory:
   – Name: YaK Inventory

⠀⠀From the YaK Inventory, go to Sources tab, then click on Create source button
⠀⠀- Name: YaK Core
⠀⠀- Execution environment: YaK EE
⠀⠀- Source: Sourced from a Project
⠀⠀- Credential: YaK API Core
⠀⠀- Project: YaK Core
⠀⠀- Inventory file: inventory/yak.core.db.yml
⠀⠀- Verbosity: 0
⠀⠀- Options: Overwrite, Overwrite variables, Update on launch

2. YaK Inventory for component (you will need to create one inventory by component you want to manage from AAP):
   – Name: YaK Inventory – <component name>

⠀⠀From the YaK Inventory – <component name>, go to Sources tab, then click on Create source button
⠀⠀- Name: YaK <component type>
⠀⠀- Execution environment: YaK EE
⠀⠀- Source: Sourced from a Project
⠀⠀- Credential: YaK API Component – <component name>
⠀⠀- Project: YaK <component type> Component
⠀⠀- Inventory file: inventory/yak.core.db.yml
⠀⠀- Verbosity: 0
⠀⠀- Options: Overwrite, Overwrite variables, Update on launch

• Template: Develop AAP templates leveraging YaK playbooks and workflows, enabling repeatable and consistent deployments.
  In the left menu, go to Automation Execution ⟶ Templates, then click on Create template button and select Create job template

1. Server – Deploy:
   – Name: Server – Deploy
   – Job type: Run
   – Inventory: YaK Inventory
   – Project: YaK Core
   – Playbook: servers/deploy.yml
   – Execution environment: YaK EE
   – Credentials: YaK API Core
   – Extra variables: target: ”
   Select the checkbox Prompt on launch for the Extra variables section. It will permit to set the server you want to deploy when you will run the job.

2. Your component – Deploy:
   – Name: <component name> – Deploy
   – Job type: Run
   – Inventory: YaK Inventory – <component name>
   – Project: YaK <component type> Component
   – Playbook: <path to your component deployment playbook>
   – Execution environment: YaK EE
   – Credentials: YaK API Component – <component name>

Creating an AAP Workflow for Full-Stack Deployment

Leveraging AAP workflows enables structured, automated deployments. In this chapter we will deploy a server named redhat-demo and the attached PostgreSQL component named pg-demo. These resources have already been created in the YaK, using the UI.

• In AAP, create a new workflow:
  In the left menu, go to Automation Execution ⟶ Templates, then click on Create template button and select Create workflow job template:
  – Name: Deploy Server and PG using YaK

• Add and connect job templates corresponding to each deployment stage using YaK inventories and playbooks, here the complete workflow to create:

1. YaK Core:
   After the Start, add a new step with the following infos:
   – Node type: Inventory Source Sync
   – Inventory source: YaK Core
   – Convergence: Any

2. Deploy redhat-demo server:
   After the YaK Core, add a new step with the following infos:
   – Node type: Job Template
   – Job template: Server – Deploy
   – Status: Run on success
   – Convergence: Any
   – Node alias: Deploy redhat-demo

⠀⠀⠀⠀After clicking on Next button, you will have to set the playbook extra variables:
⠀⠀⠀⠀- Variables:

target: redhat-demo

3. YaK Component inventory:
   After the Deploy redhat-demo, add a new step with the following infos:
   – Node type: Inventory Source Sync
   – Inventory source: YaK PostgreSQL
   – Status: Run on success
   – Convergence: Any

4. Deploy redhat-demo server:
   After the YaK PostgreSQL, add a new step with the following infos:
   – Node type: Job Template
   – Job template: PostgreSQL – Deploy PG demo
   – Status: Run on success
   – Convergence: Any
   – Node alias: Deploy pg-demo

• You can save your workflow template.
• Initiate the workflow manually or configure scheduled runs for fully automated deployments.

By integrating YaK into AAP workflows, teams can automate entire stack deployments efficiently, achieving unprecedented consistency and speed.

Conclusion

Integrating YaK with Red Hat Ansible Automation Platform combines YaK’s ease-of-use and powerful features with AAP’s comprehensive automation capabilities. This synergy ensures that deployment processes are more structured, faster, and consistently aligned with best practices, thus significantly enhancing overall efficiency and reducing time-to-market for businesses.

L’article Integrate YaK into Red Hat Ansible Automation Platform est apparu en premier sur dbi Blog.

Hello, dear tech enthusiasts and cloud aficionados!

We’ve got some news that’s about to make your life —or your deployments, at least— a whole lot easier. Meet YaK 2.0, the latest game-changer in the world of automated multi-cloud PaaS deployment. After months of development, testing, troubleshooting, a fair share of meetings and way too much coffee, YaK is officially launching today.

Automated multi-cloud PaaS deployment. What’s the deal with YaK 2.0?

Because we believe IT professionals should spend time on complex, value-added tasks, but not on repetitive setups, we have decided to develop the YaK.
YaK is a framework that allows anyone to deploy any type of component on any platform, while ensuring quality, cost efficiency and reducing deployment time.

YaK 2.0 is your new best friend when it comes to deploying infrastructure that’s not just efficient but also identical across every platform you’re working with – be it multi-cloud or on-premises. Originating from the need to deploy multi-technology infrastructures quickly and effortlessly, YaK ensures your setup is consistent, whether you’re working with AWS, Azure, Oracle Cloud, or your own on-prem servers.

In simpler terms, YaK makes sure your deployment process is consistent and reliable, no matter where. Whether you’re scaling in the cloud or handling things in-house, YaK’s got your back.

Why you should have a look at YaK 2.0?

Here’s why we think YaK is going to become your favorite pet:

• Flexibility: Deploy across AWS, Azure, OCI, or your own servers—YaK adapts to your infrastructure, making every platform feel like home.
• Automation: Eliminate repetitive setups with automated deployments, saving you time and headaches.
• Cost efficiency & speed: YaK cuts time-to-market, streamlining deployments for fast, standardized rollouts that are both cost-effective and secure.
• Freedom from vendor lock-In: YaK is vendor-neutral, letting you deploy on your terms, across any environment.
• Swiss software backed up by a consulting company (dbi services) with extensive expertise in deployments.

With this release, we’re excited to announce a major upgrade:

• Sleek new user interface: YaK 2.0 now comes with a user-friendly interface, making it easier than ever to manage your deployments. Say hello to intuitive navigation.

• Components: We’ve got components on our roadmap (available with an annual subscription), and we’ll be announcing them shortly : Oracle Database, PostgreSQL, MongoDB, and Kubernetes are already on the list and will be released soon.

Many more will follow… Stay tuned!

How does it work?

YaK Core is the open-source part and is the heart of our product, featuring Ansible playbooks and a custom plugin that provides a single inventory for all platforms, making your server deployments seamless across clouds like AWS, Azure, and OCI.
If you want to see for yourself, our GitLab project is available here!

YaK Components are the value-added part of the product and bring you expert-designed modules for deploying databases and application servers, with an annual subscription to dbi services.

Join the YaK pack

Explore the power of automated multi-cloud PaaS deployment with YaK 2.0 and experience a new level of efficiency and flexibility. We can’t wait for you to try it out and see just how much it can streamline your deployment process. Whether you’re a startup with big dreams or an established enterprise looking to optimize, YaK is here to make your life easier.

Our YaK deserved its own web page, check it out for more information, to contact us or to try it out (free demo environments will be available soon): yak4all.io

Wanna ride the YaK? Check out our user documentation to get started!
We promise it’ll be the smoothest ride you’ve had in a while.

We’re not just launching a product; we’re building a community. We’d love for you to chime in, share your experiences, and help us make YaK even better. Follow us on LinkedIn, join our community on GitLab, and let’s create something amazing together.

Feel free to reach out to us for more details or for a live presentation: info@dbi-services.com

Thanks for being part of this exciting journey. We can’t wait to see what you build with YaK.

The YaK Team

—

P.S. If you’re wondering about the name, well, yaks are known for being hardy, reliable, and able to thrive in any environment. Plus, they look pretty cool, don’t you think?

L’article Introducing YaK 2.0: The future of effortless PaaS deployments across Clouds and On-Premises est apparu en premier sur dbi Blog.

However, the diversification of processor architectures adds a new level of complexity to the creation of container images. Indeed, the construction has to cope with different instruction sets.

Docker buildx, the solution for building multi-architecture images

For the YaK project, we want to make amd64 (x86) and arm64 images available using GitLab CI/CD.

In order to create a build compatible with several architectures, I had to use “docker buildx” in my .gitlab-ci.yml file:

build:
  image: docker:latest
  stage: build
  services:
    - docker:dind
  before_script:
    - docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
    - docker buildx create --name yakbuilder --use
  script:
    - docker buildx build --pull --builder=yakbuilder --platform linux/amd64,linux/arm64 -t [IMG]:[TAG] --push .

How it works:

• In the "before_script" section , I initialize a QEMU container to emulate ARM architecture and to create a buildx context using the QEMU container
• In the "script" section itself, instead of a simple “docker build”, I use the "docker buildx build" command
• I also pass the buildx context created in the "before_script" with the --builder flag
• Finally, I add the list of architectures required for the build with the --platform flag

Build Result

With this method, the build is slower. That’s normal as several images are created (one per architecture) instead of just one.

The result can be seen in the GitLab container registry:

Now, below the image tag, a small “index” label is shown. This refers to the fact that several images are available for this tag. During the image pull, the container engine will choose the image version corresponding to its architecture.

Conclusion

With buildx and QEMU in GitLab CI/CD, building multi-architecture images is easy. You can manage different processor architectures and meet the needs of a wide range of users and ensure the compatibility of your container images.

L’article Building multi-architecture images with GitLab CI/CD est apparu en premier sur dbi Blog.

In this blog post, I will cover one of them and run different scenarios. We will also dig inside some internal mechanism used by Ansible.

SSH Connections

Ansible is connection intensive as it opens, and closes, many ssh connections to the targeted hosts.

I found two possible ways to count the amount of connections from control to agents nodes:

• Add -vvv option to the ansible-playbook command.
• grep audit.log file:

tail -f /var/log/audit/audit | grep USER_LOGIN

First option is really too much verbose, but I used it with the first playbook below to confirm the second option give the same count.

Simple Playbook

To demonstrate that, let’s start with a very minimal playbook without fact gathering:

---
- name: Test playbook
  hosts: all
  gather_facts: false
  pre_tasks:
    - name: "ping"
      ansible.builtin.ping:
...

This playbook triggered 8 ssh connections to the target host. If I enable facts gathering, count goes to 14 connections. Again, this is quiet a lot knowing that playbook does not do much beside check target is alive.

To summarize:

What are All These Connection For?

To determine what are these connections doing, we can analyze verbose (really verbose!!) output of Ansible playbook without fact gathering.

First Connection

First command of first connection is echo ~opc && sleep 0 which will return the home directory of ansible user.

Second

Second command is already scary:

( umask 77 && mkdir -p "` echo /home/opc/.ansible/tmp `"&& mkdir "` echo /home/opc/.ansible/tmp/ansible-tmp-1712570792.3350322-23674-205520350912482 `" && echo ansible-tmp-1712570792.3350322-23674-205520350912482="` echo /home/opc/.ansible/tmp/ansible-tmp-1712570792.3350322-23674-205520350912482 `" ) && sleep 0

1. It set the umask for the commands to follow.
2. Create tmp directory to store any python script on target
3. In that directory, create a directory to store the script for this specific task
4. Makes this ssh command return the temporary variable with full path to the task script directory
5. sleep 0

This one is mainly to ensure directory structure exists on the target.

Third

I will not paste this one here as it is very long and we can easily guess what it does with log just before:

Attempting python interpreter discovery

Roughly, what it does, it tries many versions of python.

Fourth

Next, it will run a python script with discovered python version to determine Operating System type and release.

Fifth

Fifth connection is actually a sftp command to copy module content (AnsiballZ_ping.py). AnsiballZ is a framework to embed module into script itself. This allows to be run modules with a single Python copy.

Seventh

This one is simply ensuring execution permission is set on temporary directory (ie. ansible-tmp-1712570792.3350322-23674-205520350912482) as well the python script (ie. AnsiballZ_ping.py).

Eighth and Last Connection

Lastly, the execution of the ping module itself:

/usr/bin/python3.9 /home/opc/.ansible/tmp/ansible-tmp-1712570792.3350322-23674-205520350912482/AnsiballZ_ping.py && sleep 0

Optimization

To reduce the amount of connection, there is one possible option: Pipelining

To enable that, I simply need to add following line in ansible.cfg:

pipelining = true

Or set ANSIBLE_PIPELINING environment variable to true.

How does it improve our playbook execution time:

As we can see there is a significant reduction on the amount of ssh connections as well as a reduction of the playbook duration.

In this configuration, only 3 connections are made:

• python interpreter discovery (connection #3)
• OS type discovery (connection #4)
• python module execution (connection #8). AnsiballZ data is piped to that process.

With the pipelining option, I also noticed that the Ansible temporary directory is not created.

Of course, we can’t expect such big speed-up on a real life playbook. So, we should do it now.

Deploy WebLogic Server Playbook

Let’s use the WebLogic YaK component to deploy a single WebLogic instance. It includes dbi service best practices, latest CPU patches and SSL configuration. The “normal” run takes 13 minutes 30 seconds when the pipelined run takes 12 minutes 13 seconds. This is 10% faster.

This is nice, but not as good as previous playbook. Why is that? Because most of the time is not spent in ssh connections, but with actual work (running WebLogic installer, starting services, patching with OPatch, etc).

What Next?

With such results, you might wonder why isn’t it enabled by default? As per documentation, there is a limitation:

This can conflict with privilege escalation (become). For example, when using sudo operations you must first disable ‘requiretty’ in the sudoers file for the target hosts, which is why this feature is disabled by default.

Ansible documentation

Until now, with all tests I have made, I never encountered that limitation. Did you?

L’article Faster Ansible est apparu en premier sur dbi Blog.

1. Using the OCI local connection feature

I started looking into the documentation of OCI. I didn’t have the necessary permissions to use the “Console Connection”, so I asked a colleague to grant me the needed IAM Policies in the correct compartment:

Allow group <group_name> to manage instance-console-connection in tenancy
Allow group <group_name> to read instance in tenancy

Once that was done, I was able to create a new “local connection” SSH access for my Mac. Once the connection was created (simply click on “Create local connection”; generate or paste your public key and finally click on “Create console connection”), I was able to copy the SSH command to connect to my unavailable instance:

This is the SSH command generated by OCI to be able to connect to the system (where XXX is a random ID for the console connection host and YYY is the random ID of your instance OCID):

ssh -o ProxyCommand='ssh -W %h:%p -p 443 ocid1.instanceconsoleconnection.oc1.eu-zurich-1.XXX@instance-console.eu-zurich-1.oci.oraclecloud.com' ocid1.instance.oc1.eu-zurich-1.YYY

As described in the documentation, since I’m not using the default SSH key or ssh-agent, then I needed to update that command with my custom private key, to be able to connect. Therefore, the previous command became (add 2 times the “-i /path/to/private/key“):

ssh -i ~/.ssh/ocilocalshell -o ProxyCommand='ssh -i ~/.ssh/ocilocalshell -W %h:%p -p 443 ocid1.instanceconsoleconnection.oc1.eu-zurich-1.XXX@instance-console.eu-zurich-1.oci.oraclecloud.com' ocid1.instance.oc1.eu-zurich-1.YYY

2. Using the local connection to connect to the instance?

At this point, you should see a login prompt, asking you to enter a username. I did have a public/private key used to connect to the default Ubuntu account of this server but no username/password. So, when I saw the login prompt, I thought I had a problem with the public/private key used by the local connection and I tried to change it to use the exact same used by my “ubuntu” OS account for this server, hoping for a seamless login. But of course, that didn’t work. I tried to alter the SSH command in a few ways to understand how this is supposed to work but no luck. The documentation contains this statement:

In the Oracle Cloud Infrastructure Console, reboot the instance. You do not need to enter a username or password. If the instance is functional and the connection is active, the serial output appears in your console. If serial output does not appear in the console, the instance operating system is not booting.

Therefore, even if this note only talk about displaying the serial output in the console, I still tried to reboot the instance and see how it behaved. Of course, since the SSH connection to the instance was impossible, the reboot command took some time. OCI will wait up to 15 minutes to perform the power-off. The accumulation of all timeouts I assume. In any cases, once the instance reboots, you will see the shutdown and then the startup information. And that’s all that is indicated in the documentation… Because yes, it’s a documentation to have this output displayed on the prompt, not to recover the access to the system!

So, what to do then? Well, we are one step further, since we can see the startup, that means that we might be able to enter the Boot Menu (GRUB), and indeed, there seemed to be a lot of blogs/videos talking about that on google. I tried that, but without success, multiple times… After some time, I looked at google again and found the Oracle Doc ID 2987985.1 and other notes/comments that appear to mention that it’s not possible for Ubuntu by default (might be dependent on some versions or something). In conclusion, we need to be able to login to the instance to change the GRUB settings to be able to open the GRUB to fix the issue to be able to login to the instance, nice infinite loop!

That’s not a viable solution. At that point, I had already spent an hour on this issue and still couldn’t see any utilities/features provided by OCI to easily get a local connection to the instance, for ubuntu.

3. No possible OOTB? Let’s do it the old way then

Since all ports were suddenly closed on this instance, I strongly suspected a human error while trying to update the firewall rules. On OCI, Ubuntu instances comes with iptables by default, but you can add/enable ufw as well. However, it would still use iptables in the background… It can happen that, if someone flushes the iptables rules, all ports would then be closed (assuming ufw was configured to defaults to the DROP policy). Please note that unless you save the iptables rules, a simple reboot would allow you to revert to the old set of rules and therefore, have your system back online. To verify if that was indeed the root cause, I only had one solution left: attach the root partition of my faulty instance to another running instance so I can check what’s going on inside the configuration files.

Let’s start with this plan.

3.1. Shutdown the current OCI instance #1

Just trigger a shutdown of the faulty OCI instance #1 and let it do its thing. As mentioned, it can take several minutes (up to 15 minutes) if it is waiting for some OS commands, like un-mounting disks. In this example, I will use an instance named “sbx-alf-ce” as #1:

3.2. Detach the root partition from OCI instance #1

On the faulty Instance details page, scroll down to the bottom and on the left side, select “Boot volume” (Compute > Instances > Instance details > Boot volume). Then copy the “resource OCID”, which is the ID of the Boot volume. Mine looked like that: “ocid1.bootvolume.oc1.eu-zurich-1.ab5heljrrmrkc5e34behkrde5ce3k2flezhh3lsl2trkrs55svtmcos3ymya”

A few seconds after clicking on “Detach boot volume”, the State should switch to “Detached”.

3.3. Create a new/empty OCI instance #2

For this example, I created a new instance “sbx2-alf-ce” as #2, still using our YaK. Once done, you can look at its details and make sure you can connect to it properly.

3.4. Attach the faulty root partition as a simple volume to the newly created OCI instance #2

On the newly created Instance details page, scroll down to the bottom and on the left side, select “Attached block volumes” (Compute > Instances > Instance details > Attached block volumes). In my case, there is already a 100Gb volume created and setup by the YaK for my Alfresco component, but in this case, I don’t really care about it since I will not use this OCI instance for anything else than just fixing my first one. Of course, I could have removed this, so it’s not created to begin with.

On this page, click on “Attach block volume”. Then you can either select the volume from the list, if you know which one to choose. Or you can just paste the “resource OCID” I asked you to copy in section “3.2.” above. Select the “Attachment type” as “Paravirtualized” and leave the “Access” to the default “Read/write”, then click on “Attach”.

When it completes, you should then have the #1 Boot Volume attached as Block Volume of #2:

3.5. Mount the disk on OCI instance #2

Now, we can connect through SSH to the OCI instance #2 and mount the #1 Boot Volume so we can access its content (again, here I’m using the YaK but you can connect to it as you prefer, as long as you have root access in the end):

yak@6073e16b8a87:~$ ssh oci/sbx2-alf-ce
Welcome to Ubuntu 22.04.2 LTS (GNU/Linux 5.15.0-1038-oracle x86_64)
...
Last login: Mon Jan 29 13:15:41 2024 from xxx.xxx.xxx.xxx
ubuntu@sbx2-alf-ce:~$
ubuntu@sbx2-alf-ce:~$ sudo su -
root@sbx2-alf-ce:~$
root@sbx2-alf-ce:~$ lsblk
NAME                  MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
loop0                   7:0    0  55.7M  1 loop /snap/core18/2785
loop1                   7:1    0 111.9M  1 loop /snap/lxd/24322
loop2                   7:2    0  63.4M  1 loop /snap/core20/1974
loop3                   7:3    0  38.7M  1 loop /snap/oracle-cloud-agent/58
loop4                   7:4    0  53.3M  1 loop /snap/snapd/19457
sda                     8:0    0  46.6G  0 disk
├─sda1                  8:1    0  46.5G  0 part /
├─sda14                 8:14   0     4M  0 part
└─sda15                 8:15   0   106M  0 part /boot/efi
sdb                     8:16   0   100G  0 disk
├─data-optalfresco    253:0    0    20G  0 lvm  /opt/alfresco
├─data-etcoptalfresco 253:1    0    10G  0 lvm  /etc/opt/alfresco
├─data-varoptalfresco 253:2    0    30G  0 lvm  /var/opt/alfresco
└─data-varlogalfresco 253:3    0    20G  0 lvm  /var/log/alfresco
sdc                     8:32   0  46.6G  0 disk
├─sdc1                  8:33   0  46.5G  0 part
├─sdc14                 8:46   0     4M  0 part
└─sdc15                 8:47   0   106M  0 part
root@sbx2-alf-ce:~$
root@sbx2-alf-ce:~$ ### --> Here it's sdc1 that needs to be mounted
root@sbx2-alf-ce:~$
root@sbx2-alf-ce:~$ mkdir /faulty_root_1
root@sbx2-alf-ce:~$ mount /dev/sdc1 /faulty_root_1
root@sbx2-alf-ce:~$

In case you don’t have an existing disk on your instance #2, then it would most probably be “sdb1” instead. To find it, just look at the “part” that isn’t “/” and that matches the size of the Boot volume we just attached.

3.6. Check the iptables configuration files inside the mounted disk on OCI instance #2 (not the OS files)

To check the iptables rules that have been saved, you can simply check for the files “/etc/iptables/rules.v*“. For the rules used on the OCI instance #1, the file would then be “/faulty_root_1/etc/iptables/rules.v4“:

root@sbx2-alf-ce:~$ cd /faulty_root_1
root@sbx2-alf-ce:/faulty_root_1$
root@sbx2-alf-ce:/faulty_root_1$ ls
bin  boot  dev  etc  home  lib  lib32  lib64  libx32  lost+found  media  mnt  opt  proc  root  run  sbin  snap  srv  sys  tmp  usr  var
root@sbx2-alf-ce:/faulty_root_1$
root@sbx2-alf-ce:/faulty_root_1$ cat /faulty_root_1/etc/iptables/rules.v4
# Generated by iptables-save v1.8.7 on Mon Jan 29 07:34:58 2024
*filter
:INPUT DROP [2:152]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [1:176]
:InstanceServices - [0:0]
:ufw-after-forward - [0:0]
:ufw-after-input - [0:0]
...
...
:ufw-user-logging-output - [0:0]
:ufw-user-output - [0:0]
COMMIT
# Completed on Mon Jan 29 07:34:58 2024
root@sbx2-alf-ce:/faulty_root_1$

3.7. If iptables was indeed the issue, fix it

As you can see above, it’s definitively this issue: the default policy is DROP and there are absolutely no rules present. Therefore, iptables would just reject all incoming requests. A similar issue could happen if your root partition is 100% full, in which case you would need to do some cleanup in this step, like removing old logs for example.

To “fix” the iptables, you can just modify the “rules.v4/6” files. You can look at the instance #2 file for an example but the minimal thing to add would be to re-enable at least port 22. You should just copy the whole “InstanceServices” chain section from the current instance #2 (/etc/iptables/rules.v4) and put it into the instance #1 file (/faulty_root_1/etc/iptables/rules.v4). If you aren’t sure about what you are doing, you can also switch the default policy from “DROP” to “ACCEPT” in this file. Here is an example of possible configuration:

root@sbx2-alf-ce:/faulty_root_1$ cat /faulty_root_1/etc/iptables/rules.v4
# Generated by iptables-save v1.8.7 on Mon Jan 29 07:34:58 2024
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [463:49013]
:InstanceServices - [0:0]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p udp --sport 123 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
-A OUTPUT -d 169.254.0.0/16 -j InstanceServices
-A InstanceServices -d 169.254.0.2/32 -p tcp -m owner --uid-owner 0 -m tcp --dport 3260 -m comment --comment "See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule" -j ACCEPT
-A InstanceServices -d 169.254.2.0/24 -p tcp -m owner --uid-owner 0 -m tcp --dport 3260 -m comment --comment "See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule" -j ACCEPT
...
...
-A InstanceServices -d 169.254.0.0/16 -p tcp -m tcp -m comment --comment "See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule" -j REJECT --reject-with tcp-reset
-A InstanceServices -d 169.254.0.0/16 -p udp -m udp -m comment --comment "See the Oracle-Provided Images section in the Oracle Cloud Infrastructure documentation for security impact of modifying or removing this rule" -j REJECT --reject-with icmp-port-unreachable
:ufw-after-forward - [0:0]
:ufw-after-input - [0:0]
:ufw-after-logging-forward - [0:0]
...
...
:ufw-user-logging-input - [0:0]
:ufw-user-logging-output - [0:0]
:ufw-user-output - [0:0]
COMMIT
# Completed on Mon Jan 29 07:34:58 2024
root@sbx2-alf-ce:/faulty_root_1$

Please note that OCI will add a lot of stuff inside the “Chain InstanceServices”, for its internal management (c.f. the “Oracle-Provided Images” section of the OCI documentation, here) and that’s also why it is recommended to not use UFW on OCI.

3.8. Un-mount the disk and detach it from OCI instance #2

Once you are satisfied with the status of the needed files (whether it’s iptables or logs for disk full cases), then the next step would be to go back into the initial situation. For that, we first unmount it:

root@sbx2-alf-ce:/faulty_root_1$ cd
root@sbx2-alf-ce:~$
root@sbx2-alf-ce:~$ umount /faulty_root_1
root@sbx2-alf-ce:~$
root@sbx2-alf-ce:~$ rmdir /faulty_root_1
root@sbx2-alf-ce:~$

And then we can detach it from the OCI instance #2:

3.9. Re-attach the root partition on OCI instance #1 and start again

The last step, if everything went well, is to re-attach the Boot volume to where it belonged at the beginning. So go back to the Instance details page of the OCI instance #1, scroll down to the bottom and on the left side, select “Boot volume” (Compute > Instances > Instance details > Boot volume). You should still see in the “Detached” State. Just click on “Attach boot volume” to get it back in the “Attached” State:

Once re-attached, you can start your OCI instance #1 again and you should now be able to login as you did before the issue started:

yak@6073e16b8a87:~$ ssh oci/sbx-alf-ce
Welcome to Ubuntu 22.04.2 LTS (GNU/Linux 5.15.0-1038-oracle x86_64)
...
Last login: Mon Jan 29 07:25:22 2024 from xxx.xxx.xxx.xxx
ubuntu@sbx-alf-ce:~$

If you still have some issues, there was probably some errors done on the configuration files. You can restart the process again (shutdown #1, detach, attach on #2, etc…).

Note that it is also possible to script all that and use the OCI CLI (command line) instead. For that purpose, you will need the Instance #1 and #2 OCIDs as well as the Boot volume OCID. Then you can use commands such as:

• Detach from OCI Instance #1: oci compute boot-volume-attachment detach –boot-volume-attachment-id ${instance_1_OCID}
• Attach to OCI Instance #2: oci compute volume-attachment attach –instance-id ${instance_2_OCID} –volume-id ${boot_volume_OCID} –type paravirtualized

L’article OCI – Recover lost SSH access to an Ubuntu instance est apparu en premier sur dbi Blog.