Manage SSH public keys with Oracle Key Vault (1/2)

In the first post these steps where performed:

  • Configure the REST API.
  • How to install the REST API on server.
  • Configure the REST API on server.
  • Declare and enroll the server endpoint using the REST API.

Let’s now upload the keys configure the ssh daemon and make a connection using ssh key from OKV.

Upload public key to OKV

First generate some ssh keys:

[opc@tstokvcli ~]$ ssh-keygen -t rsa -b 2048

If the public key is not in the format PKCS8, which is almost sure, we need to convert it first.

[opc@tstokvcli ~]$ ssh-keygen -e -m PKCS8 -f $HOME/.ssh/id_rsa.pub > $HOME/.ssh/id_rsa-pkcs8.pub

[opc@tstokvcli ~]$ cat $HOME/.ssh/id_rsa-pkcs8.pub
-----BEGIN PUBLIC KEY-----
MIIBojANBgkqhkiG9w0BAQEFAAOCAY8AMIIBigKCAYEAuLrgarG7OgvUkixlu25d
........
KhrUWpazjmhm9fR+RIec6fjgVbh/7Q6dKPzl0kY00yUawvejsLFChWi+V4Mk3PUk
2mbHWkxIFvMab4HXWfAwXyMTUXNv/Bs/jTaK4Z3eb87HAgMBAAE=
-----END PUBLIC KEY-----

Upload the key

[opc@tstokvcli ~]$ $OKV_HOME/bin/okv managed-object public-key register --generate-json-input > /tmp/t.json

[opc@tstokvcli log]$ cat /tmp/t.json
{
  "service" : {
    "category" : "managed-object",
    "resource" : "public-key",
    "action" : "register",
    "options" : {
      "object" : "/home/opc/.ssh/id_rsa-pkcs8.pub",
      "algorithm" : "RSA",
      "length" : "2048",
      "mask" : [ "ENCRYPT" ],
      "sshUser" : "oracle",
      "attributes" : {
        "name" : {
          "value" : "ORACLE-PUBKEY",
          "type" : "text"
        },
        "activationDate" : "NOW",
        "deactivationDate" : "2099-12-01 00:00:00"
      }
    }
  }
}

[opc@tstokvcli log]$ $OKV_HOME/bin/okv managed-object public-key register --from-json /tmp/t.json
{
  "result" : "Success",
  "value" : {
    "uuid" : "2AC6239C-C673-4FBC-BFF1-6D4A17990634"
  }
}

The UUID is to be kept, as this is the identifier in OKV for this key.

At this moment the key is loaded in OKV but is not a part of any wallet. In the previous post we create an ssh wallet oracle_ssh_wallet, and grant the rights to manage this wallet to the endpoint.

Now we are going to add this key to the wallet oracle_ssh_wallet:

[opc@tstokvcli log]$ $OKV_HOME/bin/okv managed-object wallet add-member --generate-json-input > /tmp/t.json

[opc@tstokvcli log]$ cat /tmp/t.json
{
  "service" : {
    "category" : "managed-object",
    "resource" : "wallet",
    "action" : "add-member",
    "options" : {
      "uuid" : "2AC6239C-C673-4FBC-BFF1-6D4A17990634",
      "wallet" : "oracle_ssh_wallet"
    }
  }
}

[opc@tstokvcli log]$ $OKV_HOME/bin/okv managed-object wallet add-member --from-json /tmp/t.json
{
  "result" : "Success"
}

As a verification we can list the wallet content by using the REST API commands:

[opc@tstokvcli log]$ $OKV_HOME/bin/okv manage-access wallet list-objects --generate-json-input > /tmp/t.json

[opc@tstokvcli log]$ cat /tmp/t.json
{
  "service" : {
    "category" : "manage-access",
    "resource" : "wallet",
    "action" : "list-objects",
    "options" : {
      "wallet" : "oracle_ssh_wallet"
    }
  }
}

[opc@tstokvcli log]$  $OKV_HOME/bin/okv manage-access wallet list-objects --from-json /tmp/t.json
{
  "result" : "Success",
  "value" : {
    "fetchedObjectCount" : "1",
    "managedObjects" : [ {
      "creatingEndpoint" : "TSTOKVCLI",
      "creationDate" : "2023-11-07 15:34:27",
      "deactivationDate" : "2099-12-01 00:00:00",
      "displayName" : "SSH Key for user: oracle, Fingerprint: SHA256:CalWw0m5Z3KnlXhCbFz3OmiYT2/ubfrk60DO/0oCTTY",
      "name" : "ORACLE-PUBKEY",
      "protectStopDate" : "",
      "state" : "Active",
      "type" : "Public Key",
      "uuid" : "2AC6239C-C673-4FBC-BFF1-6D4A17990634",
      "walletMembership" : [ "oracle_ssh_wallet" ]
    } ]
  }
}

or using the okvutil from the endpoint, which should work too as this will be the tool used by ssh daemon to retrieve the key from OKV:

[opc@tstokvcli ~]$ /opt/okvutil/bin/okvutil list
Unique ID                               Type            Identifier
2AC6239C-C673-4FBC-BFF1-6D4A17990634	Public Key	SSH Key for user: oracle, Fingerprint: SHA256:CalWw0m5Z3KnlXhCbFz3OmiYT2/ubfrk60DO/0oCTTY

Before configure ssh daemon let’s try to retrieve the key from OKV, and convert it back to an RSA key to be used in authorized_keys file:

[opc@tstokvcli ~]$ $OKV_HOME/bin/okv managed-object public-key get --output_format TEXT --uuid 2AC6239C-C673-4FBC-BFF1-6D4A17990634 > /tmp/ssh_pkcs8.pub 

[opc@tstokvcli ~]$ cat /tmp/ssh_pkcs8.pub 
-----BEGIN PUBLIC KEY-----
MIIBojANBgkqhkiG9w0BAQEFAAOCAY8AMIIBigKCAYEAuLrgarG7OgvUkixlu25d
MU9182jk+UEpCy2vGQdUvp65yp/NylkiBSeIzI35SA79vrJcy/1rXeovosxmryoQ
uDlGFUwCewXqogXSDuwXUlQGsNM2RcsfivTkZYtQ+8B7NGlzyXfDA5i/eGN8yv6U
DTFcsr6EEYgR42CtsbMxQDDeCNRjhKN0DP/41QMaWNFy5qB2zI+jeBgu4yvSmlKA
MH1OCXnRMmFFfkC8wLFRb6GFo1dqBWXAGY4VycGUqeYNfMX/fozb5ArsRCkh//ir
faH5R6gaga9W6UpOlsNWwQ69kaByeQq6Xsnd3s1o6BcVL6YgEH94AkENeUbS3Cs6
11FfgzLQBectgISitpDKtFSepENpcfsSoAEkj24mtPCZKkG1zxYIQyWfXR7y6GbF
KhrUWpazjmhm9fR+RIec6fjgVbh/7Q6dKPzl0kY00yUawvejsLFChWi+V4Mk3PUk
2mbHWkxIFvMab4HXWfAwXyMTUXNv/Bs/jTaK4Z3eb87HAgMBAAE=
-----END PUBLIC KEY-----

[opc@tstokvcli .ssh]$ ssh-keygen -i -f /tmp/ssh_pkcs8.pub  -m PKCS8 
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRmSoCFY7XflTUNoPBLLaLYfloGCPhbHewlo7niJ0xUnGHLzaHJ7DHbG9nmxRSFh+eZWzqDt2ZzKVWKLuAG53+3nJZ+/ksZZ80tjC6WtRA37nY3D+RBZ3wA4sxI5jCVqHxbpKQARlDlLMlLLxHEegJ1yXULHmIcJMf9p3HiBJwxSZKWBZyJAWMh6v671EK0RBez+bKDud7/VDhYeVcLOxNceC25NnfjJyftigN95Bk/miIJ49BGThdmcRY9txR2d9RrE7ZK1YEv+bAAKHRb9w8d/FWvo1XMGA+yNWa/5MV/V/EDA1OMcimyWT6YLM4zkmt9GUSOHgmmu/g8niE4HNl

The generated ssh key was:

[opc@tstokvcli .ssh]$ cat id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRmSoCFY7XflTUNoPBLLaLYfloGCPhbHewlo7niJ0xUnGHLzaHJ7DHbG9nmxRSFh+eZWzqDt2ZzKVWKLuAG53+3nJZ+/ksZZ80tjC6WtRA37nY3D+RBZ3wA4sxI5jCVqHxbpKQARlDlLMlLLxHEegJ1yXULHmIcJMf9p3HiBJwxSZKWBZyJAWMh6v671EK0RBez+bKDud7/VDhYeVcLOxNceC25NnfjJyftigN95Bk/miIJ49BGThdmcRY9txR2d9RrE7ZK1YEv+bAAKHRb9w8d/FWvo1XMGA+yNWa/5MV/V/EDA1OMcimyWT6YLM4zkmt9GUSOHgmmu/g8niE4HNl opc@tstokvcli

which is the same but the comment.

Now we are able to populate automatically the authorized_keys file, with a cron job for instance or by script.

Configure the ssh daemon get keys directly from OKV

Now we are going to configure the ssh to automatically get the public key when an ssh client try to connect.
For this we use the AuthorizedKeysCommand and AuthorizedKeysCommandUser parameters of sshd daemon sshd_config file.

AuthorizedKeysCommand specifies a program to be used to look up the user’s public keys.
The program must be owned by root, not writable by group or others and specified by an absolute path.

Otherwise you will have in /var/log/secure traces like:
error: Unsafe AuthorizedKeysCommand "... okv_ssh_ep_lookup_authorized_keys": bad ownership or modes for directory ... bin

For the demo purpose I will create an user oracle on the same server and make a connection from opc account to oracle account using ssh.

[root@tstokvcli ~]# cat /etc/ssh/sshd_config
....
AuthorizedKeysCommand /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys get_authorized_keys_for_user %u %f %k get_authorized_keys_for_user %u %f %k
AuthorizedKeysCommandUser root
....

[root@tstokvcli ~]# systemctl restart sshd

For debug purpose, I started sshd daemon in debug mode. The traces show the call of /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys program when connection as oracle (ssh oracle@tstokvcli) user are made.

[root@tstokvcli ~]# tail -f /var/log/secure
.....
Nov  8 14:59:56 tstokvcli sshd[21835]: AuthorizedKeysCommand /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys get_authorized_keys_for_user oracle SHA256:o6r9tlURDNyA104Ei7RWGUBmXYDdy8jJFSfbWGFKtd0 AAAAB3NzaC1yc2EAAAADAQABAAABAQDRmSoCFY7XflTUNoPBLLaLYfloGCPhbHewlo7niJ0xUnGHLzaHJ7DHbG9nmxRSFh+eZWzqDt2ZzKVWKLuAG53+3nJZ+/ksZZ80tjC6WtRA37nY3D+RBZ3wA4sxI5jCVqHxbpKQARlDlLMlLLxHEegJ1yXULHmIcJMf9p3HiBJwxSZKWBZyJAWMh6v671EK0RBez+bKDud7/VDhYeVcLOxNceC25NnfjJyftigN95Bk/miIJ49BGThdmcRY9txR2d9RrE7ZK1YEv+bAAKHRb9w8d/FWvo1XMGA+yNWa/5MV/V/EDA1OMcim

and from the debug traces:

debug1: restore_uid: 0/0
debug1: temporarily_use_uid: 0/0 (e=0/0)
debug1: /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys get_authorized_keys_for_user %u %f %k:14: matching key found: RSA SHA256:o6r9tlURDNyA104Ei7RWGUBmXYDdy8jJFSfbWGFKtd0
debug1: /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys get_authorized_keys_for_user %u %f %k:14: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
Accepted key RSA SHA256:o6r9tlURDNyA104Ei7RWGUBmXYDdy8jJFSfbWGFKtd0 found at /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys get_authorized_keys_for_user %u %f %k:14
debug1: restore_uid: 0/0
Postponed publickey for oracle from 172.168.1.181 port 51386 ssh2 [preauth]
.....

debug1: /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys get_authorized_keys_for_user %u %f %k:14: matching key found: RSA SHA256:o6r9tlURDNyA104Ei7RWGUBmXYDdy8jJFSfbWGFKtd0
debug1: /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys get_authorized_keys_for_user %u %f %k:14: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
Accepted key RSA SHA256:o6r9tlURDNyA104Ei7RWGUBmXYDdy8jJFSfbWGFKtd0 found at /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys get_authorized_keys_for_user %u %f %k:14
debug1: restore_uid: 0/0
.....
Accepted publickey for oracle from 172.168.1.181 port 51386 ssh2: RSA SHA256:o6r9tlURDNyA104Ei7RWGUBmXYDdy8jJFSfbWGFKtd0

In a high secure environement sshd can be configured to disable the .ssh/authorized_keys at all so the only keys to be used will be those get from OKV.

IMPORTANT.

As I spent some time to figure out what happens….

/opt/bin/okvutil use java. So I had to pass SELINUX to PERMISSIVE. Otherwise java cannot be executed from systemd. The error is:

Nov 09 13:38:28 tstokvcli sshd[22452]: + CURRENT_JAVA_VERSION='Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00007f89584bc000, 2555904, 1) failed; error='\''Permission denied'\'' (errno=13)

To output this error in the /var/log/messages I added the bash debug option set -x to /opt/okvutil/bin/okvcli shell.

In PERMISSIVE mode, SELINUX will output in /var/log/messages all informations and commands to execute to keep the ENFORCED mode, but on this test platform I didn’t implement all this stuff.

Conclusion

The post treat only the the SSH public keys management. OKV propose also to manage the private KEYS and also to generate them. This is for another post.

Is OKV the simplest method to concentrate all public keys ? The answer is maybe no.

But as OKV is the best solution to manage the Oracle TDE keys, it is nice to have, only one tool to manage all keys, SSH included.


Share on