dbi Blog

Introduction

Keeping and using passwords in scripts is often an overlooked task, at least in the initial stages of development.
This behavior poses serious security problems, and quite often, especially in scripts, one can find plaintext passwords.
If the project uses Git, then the versions of git that contain the passwords are visible in the project history.
The use of a vault should be implemented in the first phase of every project.

This post describes how to install and use the HashiCorp Vault, in a Podman container.

Official container vault image:

https://github.com/hashicorp/docker-vault
https://hub.docker.com/_/vault

Installation

Install Podman

As these tests are made on a brand new cloud Ubuntu vm, let’s start from the beginning by :

Installing `podman`.

root@blog-vault:~# apt-get install podman
....

Create a `vault` user.

root@blog-vault:~# useradd vault --home /home/vault --create-home --shell /bin/bash
root@blog-vault:~# sudo su - vault
vault@blog-vault:~$

root@blog-vault:~# id vault
uid=1002(vault) gid=1002(vault) groups=1002(vault)

root@blog-vault:~# loginctl enable-linger 1002

Get the `podman` vault container:

ubuntu@blog-vault:~$ podman login registry.connect.redhat.com
Username: your_username
Password: ********
Login Succeeded!

vault@blog-vault:~$ podman pull registry.connect.redhat.com/hashicorp/vault:1.12.1-ubi
Trying to pull registry.connect.redhat.com/hashicorp/vault:1.12.1-ubi...
Getting image source signatures
Copying blob 4a6625bea753 done
Copying blob 7c43afe89fe5 done
Copying blob 66304ec43437 done
Copying blob 35f159681384 done
Copying blob baed4ec80d13 done
Copying blob c4a8cf07892e done
Copying blob 7ea744a06738 done
Copying config 11bd373e3f done
Writing manifest to image destination
Storing signatures
11bd373e3fd8c82d9e86c5143572ca79c90962ccc72cb15fab4449790bcdbd35

vault@blog-vault:~$ podman images
REPOSITORY                                   TAG         IMAGE ID      CREATED       SIZE
registry.connect.redhat.com/hashicorp/vault  1.12.1-ubi  11bd373e3fd8  2 months ago  350 MB

Create the directories for the vault container external storage.

vault@blog-vault:~$  mkdir -p $HOME/vault/logs
vault@blog-vault:~$  mkdir -p $HOME/vault/file
vault@blog-vault:~$  mkdir -p $HOME/vault/config

Create the vault configuration file.

vault@blog-vault:~$  cat vault/config/local.json
{
  "storage": {
    "file": {
      "path": "/vault/file"
    }
  },
  "listener": [
    {
      "tcp": {
        "address": "0.0.0.0:8200",
        "tls_disable": true
      }
    }
  ],
  "default_lease_ttl": "168h",
  "max_lease_ttl": "720h",
  "ui": true
}

Start the vault `podman` container

vault@blog-vault:~$ podman run --cap-add=IPC_LOCK \
-v $HOME/vault/logs:/vault/logs \
-v $HOME/vault/config:/vault/config \
-v $HOME/vault/file:/vault/file  \
-e 'SKIP_CHOWN=true' \
-e 'SKIP_SETCAP=true' \
-p 8200:8200 vault server

==> Vault server configuration:

                     Cgo: disabled
              Go Version: go1.19.2
              Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
               Log Level: info
                   Mlock: supported: true, enabled: true
           Recovery Mode: false
                 Storage: file
                 Version: Vault v1.12.1, built 2022-10-27T12:32:05Z
             Version Sha: e34f8a14fb7a88af4640b09f3ddbb5646b946d9c

==> Vault server started! Log data will stream in below:

2023-01-10T09:22:18.628Z [INFO]  proxy environment: http_proxy="" https_proxy="" no_proxy=""
2023-01-10T09:22:18.628Z [WARN]  no `api_addr` value specified in config or in VAULT_API_ADDR; falling back to detection if possible, but this value should be manually set
2023-01-10T09:22:18.651Z [INFO]  core: Initializing version history cache for core

Configure the vault

First init of the vault

Let’s init the vault and save the root_token in an env variable for easy use.

Save whole JSON output in a init_vault.json file.

vault@blog-vault:~$  curl -s --request POST --data '{"secret_shares": 1, "secret_threshold": 1}' http://127.0.0.1:8200/v1/sys/init | jq .

{
  "keys": [
    "cc86645102f32c100ddb5239dd231457e4944d55139cc63d9e6eb73b23244f59"
  ],
  "keys_base64": [
    "zIZkUQLzLBAN21I53SMUV+SUTVUTnMY9nm63OyMkT1k="
  ],
  "root_token": "hvs.lkg7mE8OSF35JjR5va4Q6vwE"
}

vault@blog-vault:~$  export VAULT_ROOT_TOKEN="hvs.lkg7mE8OSF35JjR5va4Q6vwE"
vault@blog-vault:~$  export VAULT_UNSEAL_TOKEN="zIZkUQLzLBAN21I53SMUV+SUTVUTnMY9nm63OyMkT1k="

Unseal the vault using the `VAULT_UNSEAL_TOKEN` saved variable.

vault@blog-vault:~$ curl -s --request POST --data '{"key": "'"$VAULT_UNSEAL_TOKEN"'}' http://127.0.0.1:8200/v1/sys/unseal | jq .
{
"type": "shamir",
"initialized": true,
"sealed": false,
"t": 1,
"n": 1,
"progress": 0,
"nonce": "",
"version": "1.12.1",
"build_date": "2022-10-27T12:32:05Z",
"migration": false,
"cluster_name": "vault-cluster-fa1e0851",
"cluster_id": "2dbcd016-63e7-82ac-e684-847d6584a508",
"recovery_seal": false,
"storage_type": "file"
}

Check the initialisation status:

vault@blog-vault:~$ curl -s http://127.0.0.1:8200/v1/sys/init | jq .
{
   "initialized": true
}

Enable the `AppRole` auth method.

More on this AppRole authentification method: https://developer.hashicorp.com/vault/docs/auth/approle

vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request POST \
--data '{"type": "approle"}' \
http://127.0.0.1:8200/v1/sys/auth/approle

Get the `AppRole` information

vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request GET \
http://127.0.0.1:8200/v1/sys/auth/approle | jq .

{
    "options": {},
    "plugin_version": "",
    "running_plugin_version": "v1.12.1+builtin.vault",
    "running_sha256": "",
    "type": "approle",
    "description": "",
    "external_entropy_access": false,
    "uuid": "f8c3835f-f6d0-cfab-34a0-6337aa3267a5",
    "deprecation_status": "supported",
    "config": {
        "default_lease_ttl": 0,
        "force_no_cache": false,
        "max_lease_ttl": 0,
        "token_type": "default-service"
    },
    "accessor": "auth_approle_f341e77c",
    "local": false,
    "seal_wrap": false,
    "request_id": "16755bed-2c14-f0ae-8dd4-5e4d16b530ae",
    "lease_id": "",
    "renewable": false,
    "lease_duration": 0,
    "data": {
        "accessor": "auth_approle_f341e77c",
        "config": {
            "default_lease_ttl": 0,
            "force_no_cache": false,
            "max_lease_ttl": 0,
            "token_type": "default-service"
            },
        "deprecation_status": "supported",
        "description": "",
        "external_entropy_access": false,
        "local": false,
        "options": {},
        "plugin_version": "",
        "running_plugin_version": "v1.12.1+builtin.vault",
        "running_sha256": "",
        "seal_wrap": false,
        "type": "approle",
        "uuid": "f8c3835f-f6d0-cfab-34a0-6337aa3267a5"
    },
    "wrap_info": null,
    "warnings": null,
    "auth": null
}

At this moment the vault is configured and the AppRole authentification method is activated.

Configure the vault for user API usage

Create a policy

Create the policy oci_policy with the rights read/update/create on the vault secret path.

vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request PUT \
--data '{"policy":"\npath \"secret/data/\" {\n capabilities = [\"create\", \"update\", \"read\" ]\n}\n\npath \"secret/data/oci/\" {\n capabilities = [\"create\", \"update\", \"read\" ]\n}\n"}' \
http://127.0.0.1:8200/v1/sys/policies/acl/oci_policy

Get the create policy information

vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request GET \
http://127.0.0.1:8200/v1/sys/policies/acl/oci_policy | jq .
{
  "request_id": "5ecd0b6d-1034-908d-3146-322ee531a6ea",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 0,
  "data": {
    "name": "oci_policy",
    "policy": "\npath \"secret/data/\" {\n capabilities = [\"create\", \"update\", \"read\" ]\n}\n\npath \"secret/data/oci/\" {\n capabilities = [\"read\"]\n}\n"
  },
  "wrap_info": null,
  "warnings": null,
"auth": null
}

Enable KV v2 secrets engine at secret

The KV secret engine is used to store arbitrary secrets.

We use the version 2, as this is the last one at this moment

vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request POST \
--data '{ "type":"kv-v2" }' \
http://127.0.0.1:8200/v1/sys/mounts/secret

Associate the created policy with a role (`oci_role`)

vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request POST \
--data '{"policies": ["oci_policy"]}' \
http://127.0.0.1:8200/v1/auth/approle/role/oci_role

Get `oci_role` id

vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
http://127.0.0.1:8200/v1/auth/approle/role/oci_role/role-id | jq -r ".data"
{
"role_id": "68c22d2b-adf0-2f88-ec07-d7c495c51e30"
}

# save this value

vault@blog-vault:~$ export VAULT_ROLE_ID="68c22d2b-adf0-2f88-ec07-d7c495c51e30"

Creates a new SecretID using the `oci_role`

vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request POST \
http://127.0.0.1:8200/v1/auth/approle/role/oci_role/secret-id | jq -r ".data"
{
  "secret_id": "d7602ddb-a2db-8e1d-47e3-4ee57e0a9140",
  "secret_id_accessor": "dd56e8ea-8207-88c8-6c1d-9ff6b1e2ab54",
  "secret_id_num_uses": 0,
  "secret_id_ttl": 0
}

# save the secret_id value
vault@blog-vault:~$ export VAULT_SECRET_ID="d7602ddb-a2db-8e1d-47e3-4ee57e0a9140"

Call the login endpoint to fetch a new Vault token.

vault@blog-vault:~$ curl -s --request POST --data '{"role_id": "'"$VAULT_ROLE_ID"'", "secret_id":"'"$VAULT_SECRET_ID"'"}' http://127.0.0.1:8200/v1/auth/approle/login | jq -r ".auth"
{
  "client_token": "hvs.CAESIDvS7Q9FG9tt_PWCFls0ya-vZzR9RwwmC4vT63fqZe0_Gh4KHGh2cy5RM0ZDeUpNM2VwbU51Qm9RVXo1ZGk4ZVQ",
  "accessor": "2wXPJf3Irw245R6jXaNmR60a",
  "policies": [
    "default",
    "oci_policy"
  ],
  "token_policies": [
    "default",
    "oci_policy"
  ],
  "metadata": {
    "role_name": "oci_role"
  },
  "lease_duration": 604800,
  "renewable": true,
  "entity_id": "dfb3bb81-6d8a-0fbb-1f7f-ce4c29e98019",
  "token_type": "service",
  "orphan": true,
  "mfa_requirement": null,
  "num_uses": 0
}

# save the client token id 

vault@blog-vault:~$ export VAULT_CLIENT_TOKEN="vs.CAESIDvS7Q9FG9tt_PWCFls0ya-vZzR9RwwmC4vT63fqZe0_Gh4KHGh2cy5RM0ZDeUpNM2VwbU51Qm9RVXo1ZGk4ZVQ"

Create a version 1 of a secret with a password

Create the the key named password and the valus my_long_password

vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_CLIENT_TOKEN" \
--request POST \
--data '{ "data": {"password": "my-long-password"} }' \
http://127.0.0.1:8200/v1/secret/data/oci | jq -r ".data"

{
  "created_time": "2023-01-19T14:02:27.075843452Z",
  "custom_metadata": null,
  "deletion_time": "",
  "destroyed": false,
  "version": 1
}

Get the password

vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_CLIENT_TOKEN" \
--request GET \
http://127.0.0.1:8200/v1/secret/data/oci | jq -r ".data.data"

{
  "password": "my-long-password"
}

At this moment we are able to get a stored pair key:value using the fetched $VAULT_CLIENT_TOKEN

Restart everything

Restart the vault container to validate the retention of the vault.

Pay attention to shutdown traces: when the shutdown is triggered the vault is sealed

vault@blog-vault:~$ ==> Vault shutdown triggered
2023-01-19T14:05:42.570Z [INFO] core: marked as sealed
2023-01-19T14:05:42.570Z [INFO] core: pre-seal teardown starting
2023-01-19T14:05:42.570Z [INFO] rollback: stopping rollback manager
2023-01-19T14:05:42.570Z [INFO] core: pre-seal teardown complete
2023-01-19T14:05:42.570Z [INFO] core: stopping cluster listeners
2023-01-19T14:05:42.570Z [INFO] core.cluster-listener: forwarding rpc listeners stopped
2023-01-19T14:05:42.609Z [INFO] core.cluster-listener: rpc listeners successfully shut down
2023-01-19T14:05:42.609Z [INFO] core: cluster listeners successfully shut down
2023-01-19T14:05:42.610Z [INFO] core: vault is sealed

Restart the `podman` container

vault@blog-vault:~$ podman run --cap-add=IPC_LOCK -v $HOME/vault/logs:/vault/logs -v $HOME/vault/config:/vault/config -v $HOME/vault/file:/vault/file -e 'SKIP_CHOWN=true' -e 'SKIP_SETCAP=true' -p 8200:8200 vault server
==> Vault server configuration:
     
            Cgo: disabled
          Go Version: go1.19.2
          Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
           Log Level: info
               Mlock: supported: true, enabled: true
       Recovery Mode: false
             Storage: file
             Version: Vault v1.12.1, built 2022-10-27T12:32:05Z
         Version Sha: e34f8a14fb7a88af4640b09f3ddbb5646b946d9c
==> Vault server started! Log data will stream in below:

2023-01-19T14:06:04.376Z [INFO] proxy environment: http_proxy="" https_proxy="" no_proxy=""
2023-01-19T14:06:04.377Z [WARN] no api_addr value specified in config or in VAULT_API_ADDR; falling back to detection if possible, but this value should be manually set
2023-01-19T14:06:04.398Z [INFO] core: Initializing version history cache for core

Let’s save some variables

For the values of these variable see the initial configuration of the vault

vault@blog-vault:~$ export VAULT_ROLE_ID="68c22d2b-adf0-2f88-ec07-d7c495c51e30"
vault@blog-vault:~$ export VAULT_SECRET_ID="d7602ddb-a2db-8e1d-47e3-4ee57e0a9140"
vault@blog-vault:~$ export VAULT_UNSEAL_TOKEN="zIZkUQLzLBAN21I53SMUV+SUTVUTnMY9nm63OyMkT1k="

Check the vault status

vault@blog-vault:~$ curl -s http://127.0.0.1:8200/v1/sys/init | jq .
{
  "initialized": true
}

Unseal the vault

vault@blog-vault:~$ curl -s --request POST --data '{"key": "'"$VAULT_UNSEAL_TOKEN"'"}' http://127.0.0.1:8200/v1/sys/unseal | jq .
{
  "type": "shamir",
  "initialized": true,
  "sealed": false,
  "t": 1,
  "n": 1,
  "progress": 0,
  "nonce": "",
  "version": "1.12.1",
  "build_date": "2022-10-27T12:32:05Z",
  "migration": false,
  "cluster_name": "vault-cluster-3e0948ec",
  "cluster_id": "1eb1ee65-f32c-7cf3-a5ec-7975e1f9240a",
  "recovery_seal": false,
  "storage_type": "file"
}

Get the client token

vault@blog-vault:~$ export VAULT_CLIENT_TOKEN=$(curl -s --request POST --data '{"role_id": "'"$VAULT_ROLE_ID"'", "secret_id":"'"$VAULT_SECRET_ID"'"}' http://127.0.0.1:8200/v1/auth/approle/login | jq -r ".auth.client_token")

Finally fetch the password

vault@blog-vault:~$ export MY_PASSWORD=$(curl -s --header "X-Vault-Token: $VAULT_CLIENT_TOKEN" --request GET http://127.0.0.1:8200/v1/secret/data/oci | jq -r ".data.data.password")

vault@blog-vault:~$ echo $MY_PASSWORD
my-long-password

Get some help

All API endpoints can receive the ?help=1 parameter to output the end-point help.

vault@blog-vault:~$ curl -s --header "X-Vault-Token:$VAUL_ROOT_TOKEN" http://127.0.0.1:8200/v1/secret?help=1 | jq .
{
"help": "Request: config\nMatching Route: ^config$\n\nConfigures settings for the KV store\n\n## PARAMETERS\n\n cas_required (bool)\n\n If true, the backend will require the cas parameter to be set for each write\n\n delete_version_after (duration (sec))\n\n 
….

Conclusion

Installing and using the Vault is quick and easy. Dealing with the problem of storing and using passwords later in a project is much more complicated and difficult to implement.

Happy scripting

Post Views: 4,494

Hashicorp Vault. Install and quick first configuration

Introduction

Official container vault image:

Installation

Install Podman

Installing `podman`.

Create a `vault` user.

Get the `podman` vault container:

Create the directories for the vault container external storage.

Create the vault configuration file.

Start the vault `podman` container

Configure the vault

First init of the vault

Unseal the vault using the `VAULT_UNSEAL_TOKEN` saved variable.

Check the initialisation status:

Enable the `AppRole` auth method.

Get the `AppRole` information

Configure the vault for user API usage

Create a policy

Get the create policy information

Enable KV v2 secrets engine at secret

Associate the created policy with a role (`oci_role`)

Get `oci_role` id

Creates a new SecretID using the `oci_role`

Call the login endpoint to fetch a new Vault token.

Create a version 1 of a secret with a password

Get the password

Restart everything

Restart the `podman` container

Let’s save some variables

Check the vault status

Unseal the vault

Get the client token

Finally fetch the password

Get some help

Conclusion

Happy scripting

Alexandre Nestor

Leave a Reply:

Related blog articles

Introduction

Official container vault image:

Installation

Install Podman

Installing podman.

Create a vault user.

Get the podman vault container:

Create the directories for the vault container external storage.

Create the vault configuration file.

Start the vault podman container

Configure the vault

First init of the vault

Unseal the vault using the VAULT_UNSEAL_TOKEN saved variable.

Check the initialisation status:

Enable the AppRole auth method.

Get the AppRole information

Configure the vault for user API usage

Create a policy

Get the create policy information

Enable KV v2 secrets engine at secret

Associate the created policy with a role (oci_role)

Get oci_role id

Creates a new SecretID using the oci_role

Call the login endpoint to fetch a new Vault token.

Create a version 1 of a secret with a password

Get the password

Restart everything

Restart the podman container

Let’s save some variables

Check the vault status

Unseal the vault

Get the client token

Finally fetch the password

Get some help

Conclusion

Happy scripting

Alexandre Nestor

Leave a Reply:

Related blog articles

A first look at openSUSE Leap Micro 6.0

Another file system for Linux: bcachefs (3) – Mirroring/Replicas

Investigative look into cloud-native hyper-converged infrastructure with Harvester

Another file system for Linux: bcachefs (2) – multi device file systems

Installing `podman`.

Create a `vault` user.

Get the `podman` vault container:

Start the vault `podman` container

Unseal the vault using the `VAULT_UNSEAL_TOKEN` saved variable.

Enable the `AppRole` auth method.

Get the `AppRole` information

Associate the created policy with a role (`oci_role`)

Get `oci_role` id

Creates a new SecretID using the `oci_role`

Restart the `podman` container