Introduction

Keeping and using passwords in scripts is often an overlooked task, at least in the initial stages of development.
This behavior poses serious security problems, and quite often, especially in scripts, one can find plaintext passwords.
If the project uses Git, then the versions of git that contain the passwords are visible in the project history.
The use of a vault should be implemented in the first phase of every project.

This post describes how to install and use the HashiCorp Vault, in a Podman container.

Official container vault image:

https://github.com/hashicorp/docker-vault
https://hub.docker.com/_/vault

Installation

Install Podman

As these tests are made on a brand new cloud Ubuntu vm, let’s start from the beginning by :

Installing podman.

[email protected]:~# apt-get install podman
....

Create a vault user.

[email protected]:~# useradd vault --home /home/vault --create-home --shell /bin/bash
[email protected]:~# sudo su - vault
[email protected]:~$

[email protected]:~# id vault
uid=1002(vault) gid=1002(vault) groups=1002(vault)

[email protected]:~# loginctl enable-linger 1002

Get the podman vault container:

[email protected]:~$ podman login registry.connect.redhat.com
Username: your_username
Password: ********
Login Succeeded!

[email protected]:~$ podman pull registry.connect.redhat.com/hashicorp/vault:1.12.1-ubi
Trying to pull registry.connect.redhat.com/hashicorp/vault:1.12.1-ubi...
Getting image source signatures
Copying blob 4a6625bea753 done
Copying blob 7c43afe89fe5 done
Copying blob 66304ec43437 done
Copying blob 35f159681384 done
Copying blob baed4ec80d13 done
Copying blob c4a8cf07892e done
Copying blob 7ea744a06738 done
Copying config 11bd373e3f done
Writing manifest to image destination
Storing signatures
11bd373e3fd8c82d9e86c5143572ca79c90962ccc72cb15fab4449790bcdbd35

[email protected]:~$ podman images
REPOSITORY                                   TAG         IMAGE ID      CREATED       SIZE
registry.connect.redhat.com/hashicorp/vault  1.12.1-ubi  11bd373e3fd8  2 months ago  350 MB

Create the directories for the vault container external storage.

[email protected]:~$  mkdir -p $HOME/vault/logs
[email protected]:~$  mkdir -p $HOME/vault/file
[email protected]:~$  mkdir -p $HOME/vault/config

Create the vault configuration file.

[email protected]:~$  cat vault/config/local.json
{
  "storage": {
    "file": {
      "path": "/vault/file"
    }
  },
  "listener": [
    {
      "tcp": {
        "address": "0.0.0.0:8200",
        "tls_disable": true
      }
    }
  ],
  "default_lease_ttl": "168h",
  "max_lease_ttl": "720h",
  "ui": true
}

Start the vault podman container

[email protected]:~$ podman run --cap-add=IPC_LOCK \
-v $HOME/vault/logs:/vault/logs \
-v $HOME/vault/config:/vault/config \
-v $HOME/vault/file:/vault/file  \
-e 'SKIP_CHOWN=true' \
-e 'SKIP_SETCAP=true' \
-p 8200:8200 vault server

==> Vault server configuration:

                     Cgo: disabled
              Go Version: go1.19.2
              Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
               Log Level: info
                   Mlock: supported: true, enabled: true
           Recovery Mode: false
                 Storage: file
                 Version: Vault v1.12.1, built 2022-10-27T12:32:05Z
             Version Sha: e34f8a14fb7a88af4640b09f3ddbb5646b946d9c

==> Vault server started! Log data will stream in below:

2023-01-10T09:22:18.628Z [INFO]  proxy environment: http_proxy="" https_proxy="" no_proxy=""
2023-01-10T09:22:18.628Z [WARN]  no `api_addr` value specified in config or in VAULT_API_ADDR; falling back to detection if possible, but this value should be manually set
2023-01-10T09:22:18.651Z [INFO]  core: Initializing version history cache for core

Configure the vault

First init of the vault

Let’s init the vault and save the root_token in an env variable for easy use.

Save whole JSON output in a init_vault.json file.

[email protected]:~$  curl -s --request POST --data '{"secret_shares": 1, "secret_threshold": 1}' http://127.0.0.1:8200/v1/sys/init | jq .

{
  "keys": [
    "cc86645102f32c100ddb5239dd231457e4944d55139cc63d9e6eb73b23244f59"
  ],
  "keys_base64": [
    "zIZkUQLzLBAN21I53SMUV+SUTVUTnMY9nm63OyMkT1k="
  ],
  "root_token": "hvs.lkg7mE8OSF35JjR5va4Q6vwE"
}

[email protected]:~$  export VAULT_ROOT_TOKEN="hvs.lkg7mE8OSF35JjR5va4Q6vwE"
[email protected]:~$  export VAULT_UNSEAL_TOKEN="zIZkUQLzLBAN21I53SMUV+SUTVUTnMY9nm63OyMkT1k="

Unseal the vault using the VAULT_UNSEAL_TOKEN saved variable.

[email protected]:~$ curl -s --request POST --data '{"key": "'"$VAULT_UNSEAL_TOKEN"'}' http://127.0.0.1:8200/v1/sys/unseal | jq .
{
"type": "shamir",
"initialized": true,
"sealed": false,
"t": 1,
"n": 1,
"progress": 0,
"nonce": "",
"version": "1.12.1",
"build_date": "2022-10-27T12:32:05Z",
"migration": false,
"cluster_name": "vault-cluster-fa1e0851",
"cluster_id": "2dbcd016-63e7-82ac-e684-847d6584a508",
"recovery_seal": false,
"storage_type": "file"
}

Check the initialisation status:

[email protected]:~$ curl -s http://127.0.0.1:8200/v1/sys/init | jq .
{
   "initialized": true
}

Enable the AppRole auth method.

More on this AppRole authentification method: https://developer.hashicorp.com/vault/docs/auth/approle

[email protected]:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request POST \
--data '{"type": "approle"}' \
http://127.0.0.1:8200/v1/sys/auth/approle

 Get the AppRole information

[email protected]:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request GET \
http://127.0.0.1:8200/v1/sys/auth/approle | jq .

{
    "options": {},
    "plugin_version": "",
    "running_plugin_version": "v1.12.1+builtin.vault",
    "running_sha256": "",
    "type": "approle",
    "description": "",
    "external_entropy_access": false,
    "uuid": "f8c3835f-f6d0-cfab-34a0-6337aa3267a5",
    "deprecation_status": "supported",
    "config": {
        "default_lease_ttl": 0,
        "force_no_cache": false,
        "max_lease_ttl": 0,
        "token_type": "default-service"
    },
    "accessor": "auth_approle_f341e77c",
    "local": false,
    "seal_wrap": false,
    "request_id": "16755bed-2c14-f0ae-8dd4-5e4d16b530ae",
    "lease_id": "",
    "renewable": false,
    "lease_duration": 0,
    "data": {
        "accessor": "auth_approle_f341e77c",
        "config": {
            "default_lease_ttl": 0,
            "force_no_cache": false,
            "max_lease_ttl": 0,
            "token_type": "default-service"
            },
        "deprecation_status": "supported",
        "description": "",
        "external_entropy_access": false,
        "local": false,
        "options": {},
        "plugin_version": "",
        "running_plugin_version": "v1.12.1+builtin.vault",
        "running_sha256": "",
        "seal_wrap": false,
        "type": "approle",
        "uuid": "f8c3835f-f6d0-cfab-34a0-6337aa3267a5"
    },
    "wrap_info": null,
    "warnings": null,
    "auth": null
}

At this moment the vault is configured and the AppRole authentification method is activated.

Configure the vault for user API usage

Create a policy

Create the policy oci_policy with the rights read/update/create on the vault secret path.

[email protected]:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request PUT \
--data '{"policy":"\npath \"secret/data/\" {\n capabilities = [\"create\", \"update\", \"read\" ]\n}\n\npath \"secret/data/oci/\" {\n capabilities = [\"create\", \"update\", \"read\" ]\n}\n"}' \
http://127.0.0.1:8200/v1/sys/policies/acl/oci_policy

Get the create policy information

[email protected]:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request GET \
http://127.0.0.1:8200/v1/sys/policies/acl/oci_policy | jq .
{
  "request_id": "5ecd0b6d-1034-908d-3146-322ee531a6ea",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 0,
  "data": {
    "name": "oci_policy",
    "policy": "\npath \"secret/data/\" {\n capabilities = [\"create\", \"update\", \"read\" ]\n}\n\npath \"secret/data/oci/\" {\n capabilities = [\"read\"]\n}\n"
  },
  "wrap_info": null,
  "warnings": null,
"auth": null
}

Enable KV v2 secrets engine at secret

The KV secret engine is used to store arbitrary secrets.

We use the version 2, as this is the last one at this moment

[email protected]:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request POST \
--data '{ "type":"kv-v2" }' \
http://127.0.0.1:8200/v1/sys/mounts/secret

Associate the created policy with a role (oci_role)

[email protected]:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request POST \
--data '{"policies": ["oci_policy"]}' \
http://127.0.0.1:8200/v1/auth/approle/role/oci_role

Get oci_role id

va[email protected]:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
http://127.0.0.1:8200/v1/auth/approle/role/oci_role/role-id | jq -r ".data"
{
"role_id": "68c22d2b-adf0-2f88-ec07-d7c495c51e30"
}

# save this value

[email protected]:~$ export VAULT_ROLE_ID="68c22d2b-adf0-2f88-ec07-d7c495c51e30"

Creates a new SecretID using the oci_role

[email protected]:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request POST \
http://127.0.0.1:8200/v1/auth/approle/role/oci_role/secret-id | jq -r ".data"
{
  "secret_id": "d7602ddb-a2db-8e1d-47e3-4ee57e0a9140",
  "secret_id_accessor": "dd56e8ea-8207-88c8-6c1d-9ff6b1e2ab54",
  "secret_id_num_uses": 0,
  "secret_id_ttl": 0
}

# save the secret_id value
[email protected]:~$ export VAULT_SECRET_ID="d7602ddb-a2db-8e1d-47e3-4ee57e0a9140"

Call the login endpoint to fetch a new Vault token.

[email protected]:~$ curl -s --request POST --data '{"role_id": "'"$VAULT_ROLE_ID"'", "secret_id":"'"$VAULT_SECRET_ID"'"}' http://127.0.0.1:8200/v1/auth/approle/login | jq -r ".auth"
{
  "client_token": "hvs.CAESIDvS7Q9FG9tt_PWCFls0ya-vZzR9RwwmC4vT63fqZe0_Gh4KHGh2cy5RM0ZDeUpNM2VwbU51Qm9RVXo1ZGk4ZVQ",
  "accessor": "2wXPJf3Irw245R6jXaNmR60a",
  "policies": [
    "default",
    "oci_policy"
  ],
  "token_policies": [
    "default",
    "oci_policy"
  ],
  "metadata": {
    "role_name": "oci_role"
  },
  "lease_duration": 604800,
  "renewable": true,
  "entity_id": "dfb3bb81-6d8a-0fbb-1f7f-ce4c29e98019",
  "token_type": "service",
  "orphan": true,
  "mfa_requirement": null,
  "num_uses": 0
}

# save the client token id 

[email protected]:~$ export VAULT_CLIENT_TOKEN="vs.CAESIDvS7Q9FG9tt_PWCFls0ya-vZzR9RwwmC4vT63fqZe0_Gh4KHGh2cy5RM0ZDeUpNM2VwbU51Qm9RVXo1ZGk4ZVQ"

Create a version 1 of a secret with a password

Create the the key named password and the valus my_long_password

[email protected]:~$ curl -s \
--header "X-Vault-Token: $VAULT_CLIENT_TOKEN" \
--request POST \
--data '{ "data": {"password": "my-long-password"} }' \
http://127.0.0.1:8200/v1/secret/data/oci | jq -r ".data"

{
  "created_time": "2023-01-19T14:02:27.075843452Z",
  "custom_metadata": null,
  "deletion_time": "",
  "destroyed": false,
  "version": 1
}

Get the password

[email protected]:~$ curl -s \
--header "X-Vault-Token: $VAULT_CLIENT_TOKEN" \
--request GET \
http://127.0.0.1:8200/v1/secret/data/oci | jq -r ".data.data"

{
  "password": "my-long-password"
}

At this moment we are able to get a stored pair key:value using the fetched $VAULT_CLIENT_TOKEN

Restart everything

Restart the vault container to validate the retention of the vault.

Pay attention to shutdown traces: when the shutdown is triggered the vault is sealed

[email protected]:~$ ==> Vault shutdown triggered
2023-01-19T14:05:42.570Z [INFO] core: marked as sealed
2023-01-19T14:05:42.570Z [INFO] core: pre-seal teardown starting
2023-01-19T14:05:42.570Z [INFO] rollback: stopping rollback manager
2023-01-19T14:05:42.570Z [INFO] core: pre-seal teardown complete
2023-01-19T14:05:42.570Z [INFO] core: stopping cluster listeners
2023-01-19T14:05:42.570Z [INFO] core.cluster-listener: forwarding rpc listeners stopped
2023-01-19T14:05:42.609Z [INFO] core.cluster-listener: rpc listeners successfully shut down
2023-01-19T14:05:42.609Z [INFO] core: cluster listeners successfully shut down
2023-01-19T14:05:42.610Z [INFO] core: vault is sealed

Restart the podman container

[email protected]:~$ podman run --cap-add=IPC_LOCK -v $HOME/vault/logs:/vault/logs -v $HOME/vault/config:/vault/config -v $HOME/vault/file:/vault/file -e 'SKIP_CHOWN=true' -e 'SKIP_SETCAP=true' -p 8200:8200 vault server
==> Vault server configuration:
     
            Cgo: disabled
          Go Version: go1.19.2
          Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
           Log Level: info
               Mlock: supported: true, enabled: true
       Recovery Mode: false
             Storage: file
             Version: Vault v1.12.1, built 2022-10-27T12:32:05Z
         Version Sha: e34f8a14fb7a88af4640b09f3ddbb5646b946d9c
==> Vault server started! Log data will stream in below:

2023-01-19T14:06:04.376Z [INFO] proxy environment: http_proxy="" https_proxy="" no_proxy=""
2023-01-19T14:06:04.377Z [WARN] no api_addr value specified in config or in VAULT_API_ADDR; falling back to detection if possible, but this value should be manually set
2023-01-19T14:06:04.398Z [INFO] core: Initializing version history cache for core

Let’s save some variables

For the values of these variable see the initial configuration of the vault

[email protected]:~$ export VAULT_ROLE_ID="68c22d2b-adf0-2f88-ec07-d7c495c51e30"
[email protected]:~$ export VAULT_SECRET_ID="d7602ddb-a2db-8e1d-47e3-4ee57e0a9140"
[email protected]:~$ export VAULT_UNSEAL_TOKEN="zIZkUQLzLBAN21I53SMUV+SUTVUTnMY9nm63OyMkT1k="

Check the vault status

[email protected]:~$ curl -s http://127.0.0.1:8200/v1/sys/init | jq .
{
  "initialized": true
}

Unseal the vault

[email protected]:~$ curl -s --request POST --data '{"key": "'"$VAULT_UNSEAL_TOKEN"'"}' http://127.0.0.1:8200/v1/sys/unseal | jq .
{
  "type": "shamir",
  "initialized": true,
  "sealed": false,
  "t": 1,
  "n": 1,
  "progress": 0,
  "nonce": "",
  "version": "1.12.1",
  "build_date": "2022-10-27T12:32:05Z",
  "migration": false,
  "cluster_name": "vault-cluster-3e0948ec",
  "cluster_id": "1eb1ee65-f32c-7cf3-a5ec-7975e1f9240a",
  "recovery_seal": false,
  "storage_type": "file"
}

Get the client token

[email protected]:~$ export VAULT_CLIENT_TOKEN=$(curl -s --request POST --data '{"role_id": "'"$VAULT_ROLE_ID"'", "secret_id":"'"$VAULT_SECRET_ID"'"}' http://127.0.0.1:8200/v1/auth/approle/login | jq -r ".auth.client_token")

Finally fetch the password

[email protected]:~$ export MY_PASSWORD=$(curl -s --header "X-Vault-Token: $VAULT_CLIENT_TOKEN" --request GET http://127.0.0.1:8200/v1/secret/data/oci | jq -r ".data.data.password")

[email protected]:~$ echo $MY_PASSWORD
my-long-password

Get some help

All API endpoints can receive the ?help=1 parameter to output the end-point help.

[email protected]:~$ curl -s --header "X-Vault-Token:$VAUL_ROOT_TOKEN" http://127.0.0.1:8200/v1/secret?help=1 | jq .
{
"help": "Request: config\nMatching Route: ^config$\n\nConfigures settings for the KV store\n\n## PARAMETERS\n\n cas_required (bool)\n\n If true, the backend will require the cas parameter to be set for each write\n\n delete_version_after (duration (sec))\n\n 
….

Conclusion

Installing and using the Vault is quick and easy. Dealing with the problem of storing and using passwords later in a project is much more complicated and difficult to implement.

Happy scripting


Share on