A few weeks ago I participated at the dbi services bootcamp. All new employees are gathered for a refreshment of dbi services best practices. This is also an opportunity to ask questions directly to our CEO David Hueber as well as provide feedback on our enrolment experience in the company. dbi services is very keen on continuously improving his integration process.

As part of this 1.5 days, we are grouped in teams and have to face some challenges like building famous monuments with LEGO with some weight limitations in the number of pieces we can pick. It is a good opportunity to be creative, work as a team and have a good laugh in the process!

The final part of this bootcamp is to work together on a last minute request from a fictitious client. This request is very challenging and the time is limited. We get the instructions around 5:00pm and have to be ready by the next morning at 9:00am to present our solution. It requires all the brain power of each freshly hired dbi services employee to work together as an emergency task force to tackle this new project. We split the work between us to match our specialty at the best and thus give us all the chances to present a working solution. I took care of building the Kubernetes infrastructure on which we had to create Windows Server 2019 containers.

The challenge

Yes you’ve read correctly: Windows in a container…really? I’ve never bothered to try it or even think about it. However the OMrun product (from OMIS that has been integrated in dbi services) of our colleagues is working on Windows Server 2019 and so we had the challenge to make it run in a container… so a Windows one! We also needed to connect it to a MySQL Server instance and provide some redundancy.

The time is short, so using a managed Kubernetes solution for orchestrating those Windows containers seemed right. Also as we needed Windows and MySQL Server then Azure AKS (the Microsoft managed Kubernetes solution) sounded like the right choice. Let’s see how to configure that, it’s kind of a LEGO but in a digitalised form!

AKS cluster creation with Windows workers

A quick research in the Microsoft documentation and you find a useful article for this very purpose. This is the configuration I’ve set for our project, directly from the Bash Azure Cloud Shell:

$ az group create --name BootCampOmrun --location westeurope

$ echo "Please enter the username to use as administrator credentials for Windows Server nodes on your cluster: " && read WINDOWS_USERNAME

The first command creates a resource group which is like a folder in Azure that will contain all the resources you will create in it for this project. The second command asks you for a Windows username that will be stored in the variable WINDOWS_USERNAME. We will use it in the next command:

$ az aks create \
    --resource-group BootCampOmrun \
    --name AKSBootCamp \
    --node-count 3 \
    --enable-addons monitoring \
    --generate-ssh-keys \
    --windows-admin-username $WINDOWS_USERNAME \
    --vm-set-type VirtualMachineScaleSets \
    --network-plugin azure

This command creates the AKS cluster in our resource group with 3 worker nodes. We are now asked to set the password for the parameter windows-admin-password. This username and password parameter set the administrator credentials for any Windows Server nodes on the cluster. Also the parameter generate-ssh-keys will create a private and public key used to connect to this AKS cluster.

Those workers created with the AKS cluster are Linux workers. Now we need to add a Windows node pool that will create Windows workers on which we can create Windows containers (our goal!):

$ az aks nodepool add \
    --resource-group BootCampOmrun \
    --cluster-name AKSBootCamp \
    --os-type Windows \
    --name npwin \
    --node-count 2 \
    --node-vm-size Standard_A4m_v2

So we choose the os-type Windows and the node-vm-size. Those workers (like the Linux ones) are virtual machines in Azure and you have to select the proper size according to your requirement. OMrun needs 32G of memory so I’ve checked the Microsoft documentation to choose the correct one at a decent price (yes cost budget is also a parameter to take into account for this project!).

We can now see all the worker nodes in our AKS cluster:

$ kubectl get nodes -owide
NAME                                STATUS   ROLES   AGE     VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                         KERNEL-VERSION     CONTAINER-RUNTIME
aks-nodepool1-25638203-vmss000000   Ready    agent   4h24m   v1.24.9   10.224.0.62    <none>        Ubuntu 18.04.6 LTS               5.4.0-1103-azure   containerd://1.6.15+azure-1
aks-nodepool1-25638203-vmss000001   Ready    agent   4h24m   v1.24.9   10.224.0.33    <none>        Ubuntu 18.04.6 LTS               5.4.0-1103-azure   containerd://1.6.15+azure-1
aks-nodepool1-25638203-vmss000002   Ready    agent   4h24m   v1.24.9   10.224.0.4     <none>        Ubuntu 18.04.6 LTS               5.4.0-1103-azure   containerd://1.6.15+azure-1
aksnpwin000000                      Ready    agent   172m    v1.24.9   10.224.0.91    <none>        Windows Server 2019 Datacenter   10.0.17763.4010    containerd://1.6.14+azure
aksnpwin000001                      Ready    agent   172m    v1.24.9   10.224.0.122   <none>        Windows Server 2019 Datacenter   10.0.17763.4010    containerd://1.6.14+azure

There are 3 Linux worker nodes and 2 Windows ones in our AKS cluster in version 1.24.9.

Creation of a Windows container on a Windows worker

That is the last step to do and I’ve used the information from this website in order to quickly create a Windows pod and container. I’ve used the following yaml file and saved it as podwin.yaml:

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    app: omrun-bc
  name: omrun-bc
spec:
  containers:
  - name: omrun-bc
    image: mcr.microsoft.com/windows/servercore:ltsc2019
    command:
     - powershell.exe
     - -command
     - "<#code used from https://gist.github.com/19WAS85/5424431#> ; $$listener = New-Object System.Net.HttpListener ; $$listener.Prefixes.Add('http://*:80/') ; $$listener.Start() ; $$callerCounts = @{} ; Write-Host('Listening at http://*:80/') ; while ($$listener.IsListening) { ;$$context = $$listener.GetContext() ;$$requestUrl = $$context.Request.Url ;$$clientIP = $$context.Request.RemoteEndPoint.Address ;$$response = $$context.Response ;Write-Host '' ;Write-Host('> {0}' -f $$requestUrl) ;  ;$$count = 1 ;$$k=$$callerCounts.Get_Item($$clientIP) ;if ($$k -ne $$null) { $$count += $$k } ;$$callerCounts.Set_Item($$clientIP, $$count) ;$$ip=(Get-NetAdapter | Get-NetIpAddress); $$header='<html><body><H1>Windows Container Web Server</H1>' ;$$callerCountsString='' ;$$callerCounts.Keys | % { $$callerCountsString+='<p>IP {0} callerCount {1} ' -f $$ip[1].IPAddress,$$callerCounts.Item($$_) } ;$$footer='</body></html>' ;$$content='{0}{1}{2}' -f $$header,$$callerCountsString,$$footer ;Write-Output $$content ;$$buffer = [System.Text.Encoding]::UTF8.GetBytes($$content) ;$$response.ContentLength64 = $$buffer.Length ;$$response.OutputStream.Write($$buffer, 0, $$buffer.Length) ;$$response.Close() ;$$responseStatus = $$response.StatusCode ;Write-Host('< {0}' -f $$responseStatus)  } ; "
  nodeSelector:
    kubernetes.io/os: windows
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

Now all is ready and we can apply this yaml file to be scheduled on one of our Windows worker and connect to this Windows container in command line:

$ kubectl apply -f podwin.yaml 

$ kubectl exec -it omrun-bc -- cmd

And voilà! I couldn’t believe my eyes to be connected to Windows CLI into a container!

How to install OMrun in it?

So this is all good but here comes the tricky part. We have to install OMrun executables in this Windows Server 2019 container and I’ve tried several options.

The first thing is to connect to this AKS cluster from my Mac laptop and install the Azure CLI:

% brew install azure-cli

It took a while and I could then copy the private and public key generated previously into my .ssh folder and test my connection:

% az login

% az aks get-credentials --resource-group BootCampOmrun --name AKSBootCamp

% kubectl get nodes
NAME                                STATUS   ROLES   AGE     VERSION
aks-nodepool1-25638203-vmss000000   Ready    agent   4h15m   v1.24.9
aks-nodepool1-25638203-vmss000001   Ready    agent   4h15m   v1.24.9
aks-nodepool1-25638203-vmss000002   Ready    agent   4h15m   v1.24.9
aksnpwin000000                      Ready    agent   163m    v1.24.9
aksnpwin000001                      Ready    agent   163m    v1.24.9

I’m now connected and can use kubectl (install the package first on your Mac) to interact with the AKS cluster.

From there it got messy because AKS is nice but it is meant to manage your cluster so you can’t easily connect to the workers and containers from your machine as you would with a vanilla Kubernetes. The idea was to upload the OMrun executables to the Windows worker then the container and launch the installation in it. There are ways to connect to the workers through a Linux VM or through RDP in a Windows VM (my colleague Pierre Thanasack configured it successfully) in the resource group but that was not the proper way to go. We sticked with the DevOps best practice and created a Docker image we pushed on Docker Hub. Then in the yaml file of our pod we simply referenced that image and voilà… again!

This is my colleague Ryan Badaï (one of our rising star in our DevOps Team) that took care of that part and installed all the OMrun executables over a Windows Server 2019 image and built the final docker image. So we had the complete OMrun product as an image that could be made available for our Windows container. Brilliant!

Conclusion

What a journey! Thanks to this challenge we were able to push our limits, learn something new and present a working solution for such projects. We are now ready to respond to requests of real clients having such requirement. Of course we will need to work out all the details and make this configuration clean. We will also have to try to build Windows Server containers on other Kubernetes platform to have several options to offer for this solution.

Real LEGO or containers are both fun and can be used to build great projects!

Note for future newcomers, the challenges and project are different at each Bootcamp so be prepared to be amazed by this unique experience! Thank you dbi services for organising such event! #greatplacetowork

Post Views: 1,565
Thumbnail [60x60]
by
Benoît Entzmann