by Alexandre Nestor
Manage SSH public keys with Oracle Key Vault (1/2)
In the first post these steps where performed:
- Configure the REST API.
- How to install the REST API on server.
- Configure the REST API on server.
- Declare and enroll the server endpoint using the REST API.
Let’s now upload the keys configure the ssh daemon and make a connection using ssh key from OKV.
Upload public key to OKV
First generate some ssh keys:
[opc@tstokvcli ~]$ ssh-keygen -t rsa -b 2048
If the public key is not in the format PKCS8, which is almost sure, we need to convert it first.
[opc@tstokvcli ~]$ ssh-keygen -e -m PKCS8 -f $HOME/.ssh/id_rsa.pub > $HOME/.ssh/id_rsa-pkcs8.pub
[opc@tstokvcli ~]$ cat $HOME/.ssh/id_rsa-pkcs8.pub
-----BEGIN PUBLIC KEY-----
MIIBojANBgkqhkiG9w0BAQEFAAOCAY8AMIIBigKCAYEAuLrgarG7OgvUkixlu25d
........
KhrUWpazjmhm9fR+RIec6fjgVbh/7Q6dKPzl0kY00yUawvejsLFChWi+V4Mk3PUk
2mbHWkxIFvMab4HXWfAwXyMTUXNv/Bs/jTaK4Z3eb87HAgMBAAE=
-----END PUBLIC KEY-----
Upload the key
[opc@tstokvcli ~]$ $OKV_HOME/bin/okv managed-object public-key register --generate-json-input > /tmp/t.json
[opc@tstokvcli log]$ cat /tmp/t.json
{
"service" : {
"category" : "managed-object",
"resource" : "public-key",
"action" : "register",
"options" : {
"object" : "/home/opc/.ssh/id_rsa-pkcs8.pub",
"algorithm" : "RSA",
"length" : "2048",
"mask" : [ "ENCRYPT" ],
"sshUser" : "oracle",
"attributes" : {
"name" : {
"value" : "ORACLE-PUBKEY",
"type" : "text"
},
"activationDate" : "NOW",
"deactivationDate" : "2099-12-01 00:00:00"
}
}
}
}
[opc@tstokvcli log]$ $OKV_HOME/bin/okv managed-object public-key register --from-json /tmp/t.json
{
"result" : "Success",
"value" : {
"uuid" : "2AC6239C-C673-4FBC-BFF1-6D4A17990634"
}
}
The UUID is to be kept, as this is the identifier in OKV for this key.
At this moment the key is loaded in OKV but is not a part of any wallet. In the previous post we create an ssh wallet oracle_ssh_wallet
, and grant the rights to manage this wallet to the endpoint.
Now we are going to add this key to the wallet oracle_ssh_wallet
:
[opc@tstokvcli log]$ $OKV_HOME/bin/okv managed-object wallet add-member --generate-json-input > /tmp/t.json
[opc@tstokvcli log]$ cat /tmp/t.json
{
"service" : {
"category" : "managed-object",
"resource" : "wallet",
"action" : "add-member",
"options" : {
"uuid" : "2AC6239C-C673-4FBC-BFF1-6D4A17990634",
"wallet" : "oracle_ssh_wallet"
}
}
}
[opc@tstokvcli log]$ $OKV_HOME/bin/okv managed-object wallet add-member --from-json /tmp/t.json
{
"result" : "Success"
}
As a verification we can list the wallet content by using the REST API commands:
[opc@tstokvcli log]$ $OKV_HOME/bin/okv manage-access wallet list-objects --generate-json-input > /tmp/t.json
[opc@tstokvcli log]$ cat /tmp/t.json
{
"service" : {
"category" : "manage-access",
"resource" : "wallet",
"action" : "list-objects",
"options" : {
"wallet" : "oracle_ssh_wallet"
}
}
}
[opc@tstokvcli log]$ $OKV_HOME/bin/okv manage-access wallet list-objects --from-json /tmp/t.json
{
"result" : "Success",
"value" : {
"fetchedObjectCount" : "1",
"managedObjects" : [ {
"creatingEndpoint" : "TSTOKVCLI",
"creationDate" : "2023-11-07 15:34:27",
"deactivationDate" : "2099-12-01 00:00:00",
"displayName" : "SSH Key for user: oracle, Fingerprint: SHA256:CalWw0m5Z3KnlXhCbFz3OmiYT2/ubfrk60DO/0oCTTY",
"name" : "ORACLE-PUBKEY",
"protectStopDate" : "",
"state" : "Active",
"type" : "Public Key",
"uuid" : "2AC6239C-C673-4FBC-BFF1-6D4A17990634",
"walletMembership" : [ "oracle_ssh_wallet" ]
} ]
}
}
or using the okvutil
from the endpoint, which should work too as this will be the tool used by ssh daemon to retrieve the key from OKV:
[opc@tstokvcli ~]$ /opt/okvutil/bin/okvutil list
Unique ID Type Identifier
2AC6239C-C673-4FBC-BFF1-6D4A17990634 Public Key SSH Key for user: oracle, Fingerprint: SHA256:CalWw0m5Z3KnlXhCbFz3OmiYT2/ubfrk60DO/0oCTTY
Before configure ssh daemon let’s try to retrieve the key from OKV, and convert it back to an RSA key to be used in authorized_keys
file:
[opc@tstokvcli ~]$ $OKV_HOME/bin/okv managed-object public-key get --output_format TEXT --uuid 2AC6239C-C673-4FBC-BFF1-6D4A17990634 > /tmp/ssh_pkcs8.pub
[opc@tstokvcli ~]$ cat /tmp/ssh_pkcs8.pub
-----BEGIN PUBLIC KEY-----
MIIBojANBgkqhkiG9w0BAQEFAAOCAY8AMIIBigKCAYEAuLrgarG7OgvUkixlu25d
MU9182jk+UEpCy2vGQdUvp65yp/NylkiBSeIzI35SA79vrJcy/1rXeovosxmryoQ
uDlGFUwCewXqogXSDuwXUlQGsNM2RcsfivTkZYtQ+8B7NGlzyXfDA5i/eGN8yv6U
DTFcsr6EEYgR42CtsbMxQDDeCNRjhKN0DP/41QMaWNFy5qB2zI+jeBgu4yvSmlKA
MH1OCXnRMmFFfkC8wLFRb6GFo1dqBWXAGY4VycGUqeYNfMX/fozb5ArsRCkh//ir
faH5R6gaga9W6UpOlsNWwQ69kaByeQq6Xsnd3s1o6BcVL6YgEH94AkENeUbS3Cs6
11FfgzLQBectgISitpDKtFSepENpcfsSoAEkj24mtPCZKkG1zxYIQyWfXR7y6GbF
KhrUWpazjmhm9fR+RIec6fjgVbh/7Q6dKPzl0kY00yUawvejsLFChWi+V4Mk3PUk
2mbHWkxIFvMab4HXWfAwXyMTUXNv/Bs/jTaK4Z3eb87HAgMBAAE=
-----END PUBLIC KEY-----
[opc@tstokvcli .ssh]$ ssh-keygen -i -f /tmp/ssh_pkcs8.pub -m PKCS8
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRmSoCFY7XflTUNoPBLLaLYfloGCPhbHewlo7niJ0xUnGHLzaHJ7DHbG9nmxRSFh+eZWzqDt2ZzKVWKLuAG53+3nJZ+/ksZZ80tjC6WtRA37nY3D+RBZ3wA4sxI5jCVqHxbpKQARlDlLMlLLxHEegJ1yXULHmIcJMf9p3HiBJwxSZKWBZyJAWMh6v671EK0RBez+bKDud7/VDhYeVcLOxNceC25NnfjJyftigN95Bk/miIJ49BGThdmcRY9txR2d9RrE7ZK1YEv+bAAKHRb9w8d/FWvo1XMGA+yNWa/5MV/V/EDA1OMcimyWT6YLM4zkmt9GUSOHgmmu/g8niE4HNl
The generated ssh key was:
[opc@tstokvcli .ssh]$ cat id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRmSoCFY7XflTUNoPBLLaLYfloGCPhbHewlo7niJ0xUnGHLzaHJ7DHbG9nmxRSFh+eZWzqDt2ZzKVWKLuAG53+3nJZ+/ksZZ80tjC6WtRA37nY3D+RBZ3wA4sxI5jCVqHxbpKQARlDlLMlLLxHEegJ1yXULHmIcJMf9p3HiBJwxSZKWBZyJAWMh6v671EK0RBez+bKDud7/VDhYeVcLOxNceC25NnfjJyftigN95Bk/miIJ49BGThdmcRY9txR2d9RrE7ZK1YEv+bAAKHRb9w8d/FWvo1XMGA+yNWa/5MV/V/EDA1OMcimyWT6YLM4zkmt9GUSOHgmmu/g8niE4HNl opc@tstokvcli
which is the same but the comment.
Now we are able to populate automatically the authorized_keys
file, with a cron
job for instance or by script.
Configure the ssh daemon get keys directly from OKV
Now we are going to configure the ssh to automatically get the public key when an ssh client try to connect.
For this we use the AuthorizedKeysCommand
and AuthorizedKeysCommandUser
parameters of sshd
daemon sshd_config
file.
AuthorizedKeysCommand
specifies a program to be used to look up the user’s public keys.
The program must be owned by root, not writable by group or others and specified by an absolute path.
Otherwise you will have in /var/log/secure
traces like:error: Unsafe AuthorizedKeysCommand "... okv_ssh_ep_lookup_authorized_keys": bad ownership or modes for directory ... bin
For the demo purpose I will create an user oracle
on the same server and make a connection from opc
account to oracle
account using ssh.
[root@tstokvcli ~]# cat /etc/ssh/sshd_config
....
AuthorizedKeysCommand /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys get_authorized_keys_for_user %u %f %k get_authorized_keys_for_user %u %f %k
AuthorizedKeysCommandUser root
....
[root@tstokvcli ~]# systemctl restart sshd
For debug purpose, I started sshd daemon in debug mode. The traces show the call of /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys
program when connection as oracle
(ssh oracle@tstokvcli
) user are made.
[root@tstokvcli ~]# tail -f /var/log/secure
.....
Nov 8 14:59:56 tstokvcli sshd[21835]: AuthorizedKeysCommand /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys get_authorized_keys_for_user oracle SHA256:o6r9tlURDNyA104Ei7RWGUBmXYDdy8jJFSfbWGFKtd0 AAAAB3NzaC1yc2EAAAADAQABAAABAQDRmSoCFY7XflTUNoPBLLaLYfloGCPhbHewlo7niJ0xUnGHLzaHJ7DHbG9nmxRSFh+eZWzqDt2ZzKVWKLuAG53+3nJZ+/ksZZ80tjC6WtRA37nY3D+RBZ3wA4sxI5jCVqHxbpKQARlDlLMlLLxHEegJ1yXULHmIcJMf9p3HiBJwxSZKWBZyJAWMh6v671EK0RBez+bKDud7/VDhYeVcLOxNceC25NnfjJyftigN95Bk/miIJ49BGThdmcRY9txR2d9RrE7ZK1YEv+bAAKHRb9w8d/FWvo1XMGA+yNWa/5MV/V/EDA1OMcim
and from the debug traces:
debug1: restore_uid: 0/0
debug1: temporarily_use_uid: 0/0 (e=0/0)
debug1: /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys get_authorized_keys_for_user %u %f %k:14: matching key found: RSA SHA256:o6r9tlURDNyA104Ei7RWGUBmXYDdy8jJFSfbWGFKtd0
debug1: /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys get_authorized_keys_for_user %u %f %k:14: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
Accepted key RSA SHA256:o6r9tlURDNyA104Ei7RWGUBmXYDdy8jJFSfbWGFKtd0 found at /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys get_authorized_keys_for_user %u %f %k:14
debug1: restore_uid: 0/0
Postponed publickey for oracle from 172.168.1.181 port 51386 ssh2 [preauth]
.....
debug1: /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys get_authorized_keys_for_user %u %f %k:14: matching key found: RSA SHA256:o6r9tlURDNyA104Ei7RWGUBmXYDdy8jJFSfbWGFKtd0
debug1: /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys get_authorized_keys_for_user %u %f %k:14: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
Accepted key RSA SHA256:o6r9tlURDNyA104Ei7RWGUBmXYDdy8jJFSfbWGFKtd0 found at /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys get_authorized_keys_for_user %u %f %k:14
debug1: restore_uid: 0/0
.....
Accepted publickey for oracle from 172.168.1.181 port 51386 ssh2: RSA SHA256:o6r9tlURDNyA104Ei7RWGUBmXYDdy8jJFSfbWGFKtd0
In a high secure environement sshd can be configured to disable the .ssh/authorized_keys
at all so the only keys to be used will be those get from OKV.
IMPORTANT.
As I spent some time to figure out what happens….
/opt/bin/okvutil
use java
. So I had to pass SELINUX to PERMISSIVE
. Otherwise java
cannot be executed from systemd
. The error is:
Nov 09 13:38:28 tstokvcli sshd[22452]: + CURRENT_JAVA_VERSION='Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00007f89584bc000, 2555904, 1) failed; error='\''Permission denied'\'' (errno=13)
To output this error in the /var/log/messages
I added the bash debug option set -x
to /opt/okvutil/bin/okvcli
shell.
In PERMISSIVE
mode, SELINUX will output in /var/log/messages
all informations and commands to execute to keep the ENFORCED
mode, but on this test platform I didn’t implement all this stuff.
Conclusion
The post treat only the the SSH public keys management. OKV propose also to manage the private KEYS and also to generate them. This is for another post.
Is OKV the simplest method to concentrate all public keys ? The answer is maybe no.
But as OKV is the best solution to manage the Oracle TDE keys, it is nice to have, only one tool to manage all keys, SSH included.
[email protected]
16.01.2025Thank you for the great post. I have completed all the steps in post1 and post2. When I try to connect ssh oracle@host_name I am getting the following error:
Jan 15 18:15:47 ol951 sshd[11699]: debug1: userauth-request for user oracle service ssh-connection method none [preauth]
Jan 15 18:15:47 ol951 sshd[11699]: debug1: attempt 0 failures 0 [preauth]
Jan 15 18:15:47 ol951 sshd[11699]: debug1: PAM: initializing for "oracle"
Jan 15 18:15:47 ol951 sshd[11699]: debug1: PAM: setting PAM_RHOST to "192.168.1.122"
Jan 15 18:15:47 ol951 sshd[11699]: debug1: PAM: setting PAM_TTY to "ssh"
Jan 15 18:15:47 ol951 sshd[11699]: debug1: userauth-request for user oracle service ssh-connection method publickey [preauth]
Jan 15 18:15:47 ol951 sshd[11699]: debug1: attempt 1 failures 0 [preauth]
Jan 15 18:15:47 ol951 sshd[11699]: debug1: userauth_pubkey: test pkalg rsa-sha2-256 pkblob RSA SHA256:Uu3gOL+cjZrvQEUcOmkwsynwGYUbC87nDRNA/H807uE [preauth]
Jan 15 18:15:47 ol951 sshd[11699]: debug1: temporarily_use_uid: 54321/54322 (e=0/0)
Jan 15 18:15:47 ol951 sshd[11699]: debug1: trying public key file /home/oracle/.ssh/authorized_keys
Jan 15 18:15:47 ol951 sshd[11699]: debug1: Could not open authorized keys '/home/oracle/.ssh/authorized_keys': No such file or directory
Jan 15 18:15:47 ol951 sshd[11699]: debug1: restore_uid: 0/0
Jan 15 18:15:47 ol951 sshd[11699]: debug1: temporarily_use_uid: 0/0 (e=0/0)
Jan 15 18:15:47 ol951 sshd[11699]: debug1: restore_uid: 0/0
Jan 15 18:15:47 ol951 sshd[11699]: debug1: temporarily_use_uid: 0/0 (e=0/0)
Jan 15 18:15:47 ol951 sshd[11699]: AuthorizedKeysCommand /opt/okvutil/bin/okv_ssh_ep_lookup_authorized_keys get_authorized_keys_for_user oracle SHA256:Uu3gOL+cjZrvQEUcOmkwsynwGYUbC87nDRNA/H807uE AAAAB3NzaC1yc2EAAAADAQABAAABAQCuTKm2t4ChCdfZGO4uLOWw7BHQAj5dwlbKIYehJ5t70Ayod+OBwvChibj3syNHV6unQt/VQbUELP7UJGQkrizlTGpzDDh2J9ceFgBARI0UUi8opSs5Zua1QZuZYWCVGFtsMw5VvIX5LwhTaxo4NdTVX72CxRO6iJ2a8XZ9T1qEf1+AhhJwE+RneXOi27j/ekU5ySmuV0ECHhlqT4ZfuVoDMATjs5tUQ3Dv2RnqvhU/euNixxHfePMsIYuEsyOWmBgaJBTnWm98KQu1gjP157LSJF/pUYGY5op0xO6Sj3rqpeD05WkiHtCR
Jan 15 18:15:47 ol951 sshd[11699]: debug1: restore_uid: 0/0
Jan 15 18:15:47 ol951 sshd[11699]: Failed publickey for oracle from 192.168.1.122 port 41174 ssh2: RSA SHA256:Uu3gOL+cjZrvQEUcOmkwsynwGYUbC87nDRNA/H807uE
Your help would be greatly appreciated!
Thank you