Introduction
Keeping and using passwords in scripts is often an overlooked task, at least in the initial stages of development.
This behavior poses serious security problems, and quite often, especially in scripts, one can find plaintext passwords.
If the project uses Git, then the versions of git that contain the passwords are visible in the project history.
The use of a vault should be implemented in the first phase of every project.
This post describes how to install and use the HashiCorp Vault, in a Podman container.
Official container vault image:
https://github.com/hashicorp/docker-vault
https://hub.docker.com/_/vault
Installation
Install Podman
As these tests are made on a brand new cloud Ubuntu vm, let’s start from the beginning by :
Installing podman.
root@blog-vault:~# apt-get install podman
....
Create a vault user.
root@blog-vault:~# useradd vault --home /home/vault --create-home --shell /bin/bash
root@blog-vault:~# sudo su - vault
vault@blog-vault:~$
root@blog-vault:~# id vault
uid=1002(vault) gid=1002(vault) groups=1002(vault)
root@blog-vault:~# loginctl enable-linger 1002
Get the podman vault container:
ubuntu@blog-vault:~$ podman login registry.connect.redhat.com
Username: your_username
Password: ********
Login Succeeded!
vault@blog-vault:~$ podman pull registry.connect.redhat.com/hashicorp/vault:1.12.1-ubi
Trying to pull registry.connect.redhat.com/hashicorp/vault:1.12.1-ubi...
Getting image source signatures
Copying blob 4a6625bea753 done
Copying blob 7c43afe89fe5 done
Copying blob 66304ec43437 done
Copying blob 35f159681384 done
Copying blob baed4ec80d13 done
Copying blob c4a8cf07892e done
Copying blob 7ea744a06738 done
Copying config 11bd373e3f done
Writing manifest to image destination
Storing signatures
11bd373e3fd8c82d9e86c5143572ca79c90962ccc72cb15fab4449790bcdbd35
vault@blog-vault:~$ podman images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.connect.redhat.com/hashicorp/vault 1.12.1-ubi 11bd373e3fd8 2 months ago 350 MB
Create the directories for the vault container external storage.
vault@blog-vault:~$ mkdir -p $HOME/vault/logs
vault@blog-vault:~$ mkdir -p $HOME/vault/file
vault@blog-vault:~$ mkdir -p $HOME/vault/config
Create the vault configuration file.
vault@blog-vault:~$ cat vault/config/local.json
{
"storage": {
"file": {
"path": "/vault/file"
}
},
"listener": [
{
"tcp": {
"address": "0.0.0.0:8200",
"tls_disable": true
}
}
],
"default_lease_ttl": "168h",
"max_lease_ttl": "720h",
"ui": true
}
Start the vault podman container
vault@blog-vault:~$ podman run --cap-add=IPC_LOCK \
-v $HOME/vault/logs:/vault/logs \
-v $HOME/vault/config:/vault/config \
-v $HOME/vault/file:/vault/file \
-e 'SKIP_CHOWN=true' \
-e 'SKIP_SETCAP=true' \
-p 8200:8200 vault server
==> Vault server configuration:
Cgo: disabled
Go Version: go1.19.2
Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
Log Level: info
Mlock: supported: true, enabled: true
Recovery Mode: false
Storage: file
Version: Vault v1.12.1, built 2022-10-27T12:32:05Z
Version Sha: e34f8a14fb7a88af4640b09f3ddbb5646b946d9c
==> Vault server started! Log data will stream in below:
2023-01-10T09:22:18.628Z [INFO] proxy environment: http_proxy="" https_proxy="" no_proxy=""
2023-01-10T09:22:18.628Z [WARN] no `api_addr` value specified in config or in VAULT_API_ADDR; falling back to detection if possible, but this value should be manually set
2023-01-10T09:22:18.651Z [INFO] core: Initializing version history cache for core
Configure the vault
First init of the vault
Let’s init the vault and save the root_token in an env variable for easy use.
Save whole JSON output in a init_vault.json file.
vault@blog-vault:~$ curl -s --request POST --data '{"secret_shares": 1, "secret_threshold": 1}' http://127.0.0.1:8200/v1/sys/init | jq .
{
"keys": [
"cc86645102f32c100ddb5239dd231457e4944d55139cc63d9e6eb73b23244f59"
],
"keys_base64": [
"zIZkUQLzLBAN21I53SMUV+SUTVUTnMY9nm63OyMkT1k="
],
"root_token": "hvs.lkg7mE8OSF35JjR5va4Q6vwE"
}
vault@blog-vault:~$ export VAULT_ROOT_TOKEN="hvs.lkg7mE8OSF35JjR5va4Q6vwE"
vault@blog-vault:~$ export VAULT_UNSEAL_TOKEN="zIZkUQLzLBAN21I53SMUV+SUTVUTnMY9nm63OyMkT1k="
Unseal the vault using the VAULT_UNSEAL_TOKEN saved variable.
vault@blog-vault:~$ curl -s --request POST --data '{"key": "'"$VAULT_UNSEAL_TOKEN"'}' http://127.0.0.1:8200/v1/sys/unseal | jq .
{
"type": "shamir",
"initialized": true,
"sealed": false,
"t": 1,
"n": 1,
"progress": 0,
"nonce": "",
"version": "1.12.1",
"build_date": "2022-10-27T12:32:05Z",
"migration": false,
"cluster_name": "vault-cluster-fa1e0851",
"cluster_id": "2dbcd016-63e7-82ac-e684-847d6584a508",
"recovery_seal": false,
"storage_type": "file"
}
Check the initialisation status:
vault@blog-vault:~$ curl -s http://127.0.0.1:8200/v1/sys/init | jq .
{
"initialized": true
}
Enable the AppRole auth method.
More on this AppRole authentification method: https://developer.hashicorp.com/vault/docs/auth/approle
vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request POST \
--data '{"type": "approle"}' \
http://127.0.0.1:8200/v1/sys/auth/approle
Get the AppRole information
vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request GET \
http://127.0.0.1:8200/v1/sys/auth/approle | jq .
{
"options": {},
"plugin_version": "",
"running_plugin_version": "v1.12.1+builtin.vault",
"running_sha256": "",
"type": "approle",
"description": "",
"external_entropy_access": false,
"uuid": "f8c3835f-f6d0-cfab-34a0-6337aa3267a5",
"deprecation_status": "supported",
"config": {
"default_lease_ttl": 0,
"force_no_cache": false,
"max_lease_ttl": 0,
"token_type": "default-service"
},
"accessor": "auth_approle_f341e77c",
"local": false,
"seal_wrap": false,
"request_id": "16755bed-2c14-f0ae-8dd4-5e4d16b530ae",
"lease_id": "",
"renewable": false,
"lease_duration": 0,
"data": {
"accessor": "auth_approle_f341e77c",
"config": {
"default_lease_ttl": 0,
"force_no_cache": false,
"max_lease_ttl": 0,
"token_type": "default-service"
},
"deprecation_status": "supported",
"description": "",
"external_entropy_access": false,
"local": false,
"options": {},
"plugin_version": "",
"running_plugin_version": "v1.12.1+builtin.vault",
"running_sha256": "",
"seal_wrap": false,
"type": "approle",
"uuid": "f8c3835f-f6d0-cfab-34a0-6337aa3267a5"
},
"wrap_info": null,
"warnings": null,
"auth": null
}
At this moment the vault is configured and the AppRole authentification method is activated.
Configure the vault for user API usage
Create a policy
Create the policy oci_policy with the rights read/update/create on the vault secret path.
vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request PUT \
--data '{"policy":"\npath \"secret/data/\" {\n capabilities = [\"create\", \"update\", \"read\" ]\n}\n\npath \"secret/data/oci/\" {\n capabilities = [\"create\", \"update\", \"read\" ]\n}\n"}' \
http://127.0.0.1:8200/v1/sys/policies/acl/oci_policy
Get the create policy information
vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request GET \
http://127.0.0.1:8200/v1/sys/policies/acl/oci_policy | jq .
{
"request_id": "5ecd0b6d-1034-908d-3146-322ee531a6ea",
"lease_id": "",
"renewable": false,
"lease_duration": 0,
"data": {
"name": "oci_policy",
"policy": "\npath \"secret/data/\" {\n capabilities = [\"create\", \"update\", \"read\" ]\n}\n\npath \"secret/data/oci/\" {\n capabilities = [\"read\"]\n}\n"
},
"wrap_info": null,
"warnings": null,
"auth": null
}
Enable KV v2 secrets engine at secret
The KV secret engine is used to store arbitrary secrets.
We use the version 2, as this is the last one at this moment
vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request POST \
--data '{ "type":"kv-v2" }' \
http://127.0.0.1:8200/v1/sys/mounts/secret
Associate the created policy with a role (oci_role)
vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request POST \
--data '{"policies": ["oci_policy"]}' \
http://127.0.0.1:8200/v1/auth/approle/role/oci_role
Get oci_role id
vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
http://127.0.0.1:8200/v1/auth/approle/role/oci_role/role-id | jq -r ".data"
{
"role_id": "68c22d2b-adf0-2f88-ec07-d7c495c51e30"
}
# save this value
vault@blog-vault:~$ export VAULT_ROLE_ID="68c22d2b-adf0-2f88-ec07-d7c495c51e30"
Creates a new SecretID using the oci_role
vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_ROOT_TOKEN" \
--request POST \
http://127.0.0.1:8200/v1/auth/approle/role/oci_role/secret-id | jq -r ".data"
{
"secret_id": "d7602ddb-a2db-8e1d-47e3-4ee57e0a9140",
"secret_id_accessor": "dd56e8ea-8207-88c8-6c1d-9ff6b1e2ab54",
"secret_id_num_uses": 0,
"secret_id_ttl": 0
}
# save the secret_id value
vault@blog-vault:~$ export VAULT_SECRET_ID="d7602ddb-a2db-8e1d-47e3-4ee57e0a9140"
Call the login endpoint to fetch a new Vault token.
vault@blog-vault:~$ curl -s --request POST --data '{"role_id": "'"$VAULT_ROLE_ID"'", "secret_id":"'"$VAULT_SECRET_ID"'"}' http://127.0.0.1:8200/v1/auth/approle/login | jq -r ".auth"
{
"client_token": "hvs.CAESIDvS7Q9FG9tt_PWCFls0ya-vZzR9RwwmC4vT63fqZe0_Gh4KHGh2cy5RM0ZDeUpNM2VwbU51Qm9RVXo1ZGk4ZVQ",
"accessor": "2wXPJf3Irw245R6jXaNmR60a",
"policies": [
"default",
"oci_policy"
],
"token_policies": [
"default",
"oci_policy"
],
"metadata": {
"role_name": "oci_role"
},
"lease_duration": 604800,
"renewable": true,
"entity_id": "dfb3bb81-6d8a-0fbb-1f7f-ce4c29e98019",
"token_type": "service",
"orphan": true,
"mfa_requirement": null,
"num_uses": 0
}
# save the client token id
vault@blog-vault:~$ export VAULT_CLIENT_TOKEN="vs.CAESIDvS7Q9FG9tt_PWCFls0ya-vZzR9RwwmC4vT63fqZe0_Gh4KHGh2cy5RM0ZDeUpNM2VwbU51Qm9RVXo1ZGk4ZVQ"
Create a version 1 of a secret with a password
Create the the key named password and the valus my_long_password
vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_CLIENT_TOKEN" \
--request POST \
--data '{ "data": {"password": "my-long-password"} }' \
http://127.0.0.1:8200/v1/secret/data/oci | jq -r ".data"
{
"created_time": "2023-01-19T14:02:27.075843452Z",
"custom_metadata": null,
"deletion_time": "",
"destroyed": false,
"version": 1
}
Get the password
vault@blog-vault:~$ curl -s \
--header "X-Vault-Token: $VAULT_CLIENT_TOKEN" \
--request GET \
http://127.0.0.1:8200/v1/secret/data/oci | jq -r ".data.data"
{
"password": "my-long-password"
}
At this moment we are able to get a stored pair key:value using the fetched $VAULT_CLIENT_TOKEN
Restart everything
Restart the vault container to validate the retention of the vault.
Pay attention to shutdown traces: when the shutdown is triggered the vault is sealed
vault@blog-vault:~$ ==> Vault shutdown triggered
2023-01-19T14:05:42.570Z [INFO] core: marked as sealed
2023-01-19T14:05:42.570Z [INFO] core: pre-seal teardown starting
2023-01-19T14:05:42.570Z [INFO] rollback: stopping rollback manager
2023-01-19T14:05:42.570Z [INFO] core: pre-seal teardown complete
2023-01-19T14:05:42.570Z [INFO] core: stopping cluster listeners
2023-01-19T14:05:42.570Z [INFO] core.cluster-listener: forwarding rpc listeners stopped
2023-01-19T14:05:42.609Z [INFO] core.cluster-listener: rpc listeners successfully shut down
2023-01-19T14:05:42.609Z [INFO] core: cluster listeners successfully shut down
2023-01-19T14:05:42.610Z [INFO] core: vault is sealed
Restart the podman container
vault@blog-vault:~$ podman run --cap-add=IPC_LOCK -v $HOME/vault/logs:/vault/logs -v $HOME/vault/config:/vault/config -v $HOME/vault/file:/vault/file -e 'SKIP_CHOWN=true' -e 'SKIP_SETCAP=true' -p 8200:8200 vault server
==> Vault server configuration:
Cgo: disabled
Go Version: go1.19.2
Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
Log Level: info
Mlock: supported: true, enabled: true
Recovery Mode: false
Storage: file
Version: Vault v1.12.1, built 2022-10-27T12:32:05Z
Version Sha: e34f8a14fb7a88af4640b09f3ddbb5646b946d9c
==> Vault server started! Log data will stream in below:
2023-01-19T14:06:04.376Z [INFO] proxy environment: http_proxy="" https_proxy="" no_proxy=""
2023-01-19T14:06:04.377Z [WARN] no api_addr value specified in config or in VAULT_API_ADDR; falling back to detection if possible, but this value should be manually set
2023-01-19T14:06:04.398Z [INFO] core: Initializing version history cache for core
Let’s save some variables
For the values of these variable see the initial configuration of the vault
vault@blog-vault:~$ export VAULT_ROLE_ID="68c22d2b-adf0-2f88-ec07-d7c495c51e30"
vault@blog-vault:~$ export VAULT_SECRET_ID="d7602ddb-a2db-8e1d-47e3-4ee57e0a9140"
vault@blog-vault:~$ export VAULT_UNSEAL_TOKEN="zIZkUQLzLBAN21I53SMUV+SUTVUTnMY9nm63OyMkT1k="
Check the vault status
vault@blog-vault:~$ curl -s http://127.0.0.1:8200/v1/sys/init | jq .
{
"initialized": true
}
Unseal the vault
vault@blog-vault:~$ curl -s --request POST --data '{"key": "'"$VAULT_UNSEAL_TOKEN"'"}' http://127.0.0.1:8200/v1/sys/unseal | jq .
{
"type": "shamir",
"initialized": true,
"sealed": false,
"t": 1,
"n": 1,
"progress": 0,
"nonce": "",
"version": "1.12.1",
"build_date": "2022-10-27T12:32:05Z",
"migration": false,
"cluster_name": "vault-cluster-3e0948ec",
"cluster_id": "1eb1ee65-f32c-7cf3-a5ec-7975e1f9240a",
"recovery_seal": false,
"storage_type": "file"
}
Get the client token
vault@blog-vault:~$ export VAULT_CLIENT_TOKEN=$(curl -s --request POST --data '{"role_id": "'"$VAULT_ROLE_ID"'", "secret_id":"'"$VAULT_SECRET_ID"'"}' http://127.0.0.1:8200/v1/auth/approle/login | jq -r ".auth.client_token")
Finally fetch the password
vault@blog-vault:~$ export MY_PASSWORD=$(curl -s --header "X-Vault-Token: $VAULT_CLIENT_TOKEN" --request GET http://127.0.0.1:8200/v1/secret/data/oci | jq -r ".data.data.password")
vault@blog-vault:~$ echo $MY_PASSWORD
my-long-password
Get some help
All API endpoints can receive the ?help=1 parameter to output the end-point help.
vault@blog-vault:~$ curl -s --header "X-Vault-Token:$VAUL_ROOT_TOKEN" http://127.0.0.1:8200/v1/secret?help=1 | jq .
{
"help": "Request: config\nMatching Route: ^config$\n\nConfigures settings for the KV store\n\n## PARAMETERS\n\n cas_required (bool)\n\n If true, the backend will require the cas parameter to be set for each write\n\n delete_version_after (duration (sec))\n\n
….
Conclusion
Installing and using the Vault is quick and easy. Dealing with the problem of storing and using passwords later in a project is much more complicated and difficult to implement.
![Thumbnail [60x60]](https://www.dbi-services.com/blog/wp-content/uploads/2022/08/ALN_web-min-scaled.jpg)
![Thumbnail [90x90]](https://www.dbi-services.com/blog/wp-content/uploads/2022/08/ADE_WEB-min-scaled.jpg)
![Thumbnail [90x90]](https://www.dbi-services.com/blog/wp-content/uploads/2022/08/DWI_web-min-scaled.jpg)
![Thumbnail [90x90]](https://www.dbi-services.com/blog/wp-content/uploads/2022/08/NME_web-min-scaled.jpg)
![Thumbnail [90x90]](https://www.dbi-services.com/blog/wp-content/uploads/2024/03/AHI_web.jpg)