Skip to content

Deploy awx-operator with Helm using images from a local registry

You want to deploy Ansible AWX on a Kubernetes cluster using awx-operator, but your security policy requires you to use a local registry to store the necessary container images ?

Okay, let’s see how to do this using Helm.

Default deployment

By reading the awx-operator documentation, you should have a values file similar to this one (with spec values according to your needs):

---
AWX:
  enabled: true
  name: awx-demo
  spec:
    service_type: ClusterIP

Then add awx-operator repository to Helm and deploy it:

$ helm repo add awx-operator https://ansible.github.io/awx-operator/
"awx-operator" has been added to your repositories

$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "awx-operator" chart repository
...Successfully got an update from the "harbor" chart repository
...Successfully got an update from the "komodorio" chart repository
Update Complete. ⎈Happy Helming!⎈

$ helm install -n awx --create-namespace awx-demo awx-operator/awx-operator -f awx.yaml
Release "awx-demo" does not exist. Installing it now.
NAME: awx-demo
LAST DEPLOYED: Fri Jun  9 15:23:27 2023
NAMESPACE: awx
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
AWX Operator installed with Helm Chart version 2.2.1

After a bit of time, AWX should be well deployed:

Questions about your own setup? A dbi services consultant can look at your case.

Talk to an expert (opens in a new tab)
$ kubectl get pods -n awx
NAME                                               READY   STATUS    RESTARTS   AGE
awx-operator-controller-manager-7695c6c799-9sh6x   2/2     Running   0          41m
awx-demo-postgres-13-0                             1/1     Running   0          40m
awx-demo-task-57ccc9c6d-c6qx7                      4/4     Running   0          40m
awx-demo-web-5f486b545-6vfm2                       3/3     Running   0          40m

Set components images

We just deployed our AWX with the default images, which one are in use?

$ kubectl get pods -n awx -o jsonpath='{range .items[*].spec.containers[*]}{.image}{"\n"}{end}'
gcr.io/kubebuilder/kube-rbac-proxy:v0.13.0
quay.io/ansible/awx-operator:2.2.1
postgres:13
docker.io/redis:7
quay.io/ansible/awx:22.3.0
quay.io/ansible/awx-ee:latest
quay.io/ansible/awx:22.3.0
docker.io/redis:7
quay.io/ansible/awx:22.3.0
quay.io/ansible/awx:22.3.0

Now we want to use images stored on a local registry, for example here, we use Harbor. Harbor is deployed on Kubernetes cluster and works as a remote registry replication.


In the awx-operator documentation, we can find values to set in the “spec” dictionary for various image locations. Here the new “awx.yaml” values file with these changes:

---
AWX:
  enabled: true
  name: "awx-demo"
  spec:
    hostname: awx-demo.dwi.local
    service_type: ClusterIP
    image: harbor.dwi.local/awx-demo/awx
    image_version: latest
    postgres_image: harbor.dwi.local/awx-demo/postgres
    postgres_image_version: "13"
    ee_images:
    - name: AWX Demo EE
      image: harbor.dwi.local/awx-demo/awx-ee:latest
    control_plane_ee_image: harbor.dwi.local/awx-demo/awx-ee:latest
    redis_image: harbor.dwi.local/awx-demo/redis
    redis_image_version: "7"

Now let’s delete and reinstall awx-operator completely to see what happens:

$ helm delete -n awx awx-demo
release "awx-demo" uninstalled
$ helm install -n awx --create-namespace awx-demo awx-operator/awx-operator -f awx.yaml
NAME: awx-demo
LAST DEPLOYED: Fri Jun  9 15:45:18 2023
NAMESPACE: awx
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
AWX Operator installed with Helm Chart version 2.2.1
$ kubectl get pods -n awx -o jsonpath='{range .items[*].spec.containers[*]}{.image}{"\n"}{end}'
gcr.io/kubebuilder/kube-rbac-proxy:v0.13.0
quay.io/ansible/awx-operator:2.2.1
harbor.dwi.local/awx-demo/postgres:13
harbor.dwi.local/awx-demo/redis:7
harbor.dwi.local/awx-demo/awx:latest
harbor.dwi.local/awx-demo/awx-ee:latest
harbor.dwi.local/awx-demo/awx:latest
harbor.dwi.local/awx-demo/redis:7
harbor.dwi.local/awx-demo/awx:latest
harbor.dwi.local/awx-demo/awx:latest

Now, most pods are using images from our local registry, with the exception of kube-rbac-proxy and awx-operator. Unfortunately, there are no values in the Helm to change them… So we’re going to rebuild the Helm chart 💪

Rebuild Helm chart


First we will clone the awx-operator project in /tmp/awx-operator

$ git clone https://github.com/ansible/awx-operator.git /tmp/awx-operator
Cloning into '/tmp/awx-operator'...
remote: Enumerating objects: 8796, done.
remote: Counting objects: 100% (1432/1432), done.
remote: Compressing objects: 100% (224/224), done.
remote: Total 8796 (delta 1251), reused 1306 (delta 1203), pack-reused 7364
Receiving objects: 100% (8796/8796), 2.35 MiB | 3.53 MiB/s, done.
Resolving deltas: 100% (5062/5062), done.
$ cd /tmp/awx-operator

The Helm chart is not directly sourced in the project, so we will use the Makefile that transforms the Kustomize into a Helm chart, and here’s some good news: we have two variables to define image and version of the awx-operator (IMAGE_TAG_BASE and VERSION).

$ IMAGE_TAG_BASE=harbor.dwi.local/awx-demo/awx-operator VERSION=2.2.1 make helm-chart
mkdir -p charts
== KUSTOMIZE: Set image and chart label ==
cd config/manager && /tmp/awx-operator/bin/kustomize edit set image controller=harbor.dwi.local/awx-demo/awx-operator:latest
cd config/manager && /tmp/awx-operator/bin/kustomize edit set label helm.sh/chart:awx-operator
cd config/default && /tmp/awx-operator/bin/kustomize edit set label helm.sh/chart:awx-operator
== Gather Helm Chart Metadata ==
[...]
Helm chart successfully configured for awx-operator version 2.2.1

Once this has been done, we will manually modify the template that uses the kube-rbac-proxy image. So, using vi (or another editor), edit the file /tmp/awx-operator/charts/awx-operator/templates/deployment-awx-operator-controller-manager.yaml
Modify line:

            image: gcr.io/kubebuilder/kube-rbac-proxy:v0.13.0

To match the image on your local registry

            image: harbor.dwi.local/awx-demo/kube-rbac-proxy:v0.13.0

We can finally delete the actual Helm release and deploy our custom awx-operator Helm chart:

$ helm delete -n awx awx-demo
release "awx-demo" uninstalled
$ helm install -n awx --create-namespace awx-demo /tmp/awx-operator/charts/awx-operator -f awx.yaml
NAME: awx-demo
LAST DEPLOYED: Fri Jun 23 01:46:27 2023
NAMESPACE: awx
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
AWX Operator installed with Helm Chart version 2.2.1

Wait a few seconds for all the pods to start up correctly, then you can list the images used:

$ kubectl get pods -n awx -o jsonpath='{range .items[*].spec.containers[*]}{.image}{"\n"}{end}'
harbor.dwi.local/awx-demo/kube-rbac-proxy:v0.13.0
harbor.dwi.local/awx-demo/awx-operator:2.2.1
harbor.dwi.local/awx-demo/postgres:13
harbor.dwi.local/awx-demo/redis:7
harbor.dwi.local/awx-demo/awx:latest
harbor.dwi.local/awx-demo/awx-ee:latest
harbor.dwi.local/awx-demo/awx:latest
harbor.dwi.local/awx-demo/redis:7
harbor.dwi.local/awx-demo/awx:latest
harbor.dwi.local/awx-demo/awx:latest

Congratulations ! You now have an AWX installed on your Kubernetes cluster that using only local images 🎉

Let’s work together

Ready to operate at
the next level?

Tell us about your challenge. We’ll bring the depth, the expertise, and the Swiss precision.

Donovan Winter

Donovan Winter

Donovan Winter has over eight years of experience in the IT industry, specializing in DevOps tools and web/software development, including Kubernetes, Ansible Automation Platform, GitLab, and HAProxy. Prior to dbi services, he worked as an industrial IT engineer at EES-Clemessy and as a software development engineer at Aubans Technologies. He holds a degree in electrical engineering and industrial computer science from the University of Belfort-Montbéliard, complemented by studies at ESTIA. His experience spans industrial supervision, aerospace, automotive, and defense sectors.View Author posts

8 posts since November 2022

Most read in Ansible

The Ansible posts our readers open most: the all-time favourites, and what is trending right now.

5 thoughts on “Deploy awx-operator with Helm using images from a local registry”

  1. I have managed to reroute awx operator 3.2.1 with these values-settings
    No changes in manifest needed
    —–

    image:
    repository: your-harbor-address.com/awx-operator/awx-operator

    AWX:
    enabled: true
    name: “awx”
    spec:
    hostname: your-awx-hostname
    service_type: ClusterIP
    image: your-harbor-address.com/awx-operator/awx
    image_version: 24.6.1
    postgres_image: your-harbor-address.com/awx-operator/postgresql-15-c9s
    postgres_image_version: “latest”
    redis_image: your-harbor-address.com/awx-operator/redis
    redis_image_version: “7”
    ee_images:
    – name: AWX Demo EE
    image: your-harbor-address.com/awx-operator/awx-ee:24.6.1
    control_plane_ee_image: your-harbor-address.com/awx-operator/awx-ee:24.6.1
    init_container_image: your-harbor-address.com/awx-operator/awx-ee
    init_container_image_version: 24.6.1
    init_projects_container_image: your-harbor-address.com/awx-operator/awx-ee:24.6.1

    operator-controller-containers:
    awx-manager:
    image: your-harbor-address.com/awx-operator/ansible/awx-operator:2.19.1
    kube-rbac-proxy:
    image: your-harbor-address.com/awx-operator/brancz/kube-rbac-proxy:v0.15.0

  2. With Operator 2.10.0 and AWX 23.6.0 sourced from a private Container Registry backed by STEP-CA, which I also added to my Minikube image (running rootless), it seems to work at first, but the awx-task resource has an error on the ReplicaSet, where the override does not work, which you do not notice at first … only when you do a kubectl describe after while to find it uses the default (quay.io/ansible/awx-ee:latest) as defined in https://github.com/ansible/awx-operator/blob/devel/roles/installer/defaults/main.yml. Other than that, your post is very complete.

  3. I followed your article it helped me but for some reason ansible task pods are not spinning up. Can you please guide us with this.

  4. Hi BB, thanks for your interest.
    I never used Fluxcd at the moment, but what I already did to implement it in a CD solution was to create an Ansible playbook which do exactly the steps described here:
    1. clone ansible-operator git project
    2. make with IMAGE_TAG_BASE and VERSION values
    3. modify the kube-rbac-proxy repository using sed command
    4. helm install

    Another solution will be to fork the ansible-operator on your git repository and modify the images paths in sources to pointing to your local container image registry.

  5. Hi Winter, it’s very good article . How to make changes for awx-operator and kube-rbac-proxy if we are using fluxcd.

Leave a Comment

Your email address will not be published. Required fields are marked *