By Franck Pachot
.
Google Cloud, Open Source and Oracle Databases… what seems to be a paradox is possible, thanks to cloud providers who contribute to open infrastructure. The idea is to use Operators (custom resource controllers on Kubernetes) to automate the Oracle Database operations in a standard, open and portable way. If you ever attempted to run Oracle Database on containers, trying to keep up with the DevOps approach, you know that it requires a bit of complexity and careful orchestration.
The public announce was on the Google Open Source Blog: Modernizing Oracle operations with Kubernetes and El Carro. This is an Open Source project where we can contribute: https://github.com/GoogleCloudPlatform/elcarro-oracle-operator. I’ve tried the simplest thing: install Oracle XE – the free edition of Oracle, because it is the only one that you can deploy without cross-checking, with your lawyers, the license contracts and the “educational purpose only” documents about Oracle audit policies. But running Oracle on Kubernetes applies the same rules as virtualization: count the vCPU or the physical processors (depending on the hypervisor isolation accepted by Oracle). Basically the “installed or running” terms apply where the image is pulled.
Download El Carro software and install Oracle 18c XE
I’ll run all this from the Cloud Shell but of course you can do it from any configured gcloud CLI.
franck@cloudshell:~ (google-cloud.424242)$ gcloud alpha iam service-accounts list
DISPLAY NAME EMAIL DISABLED
Compute Engine default service account [email protected] False
I take note of my service account from there.
Installing Oracle is 3 lines only:
mkdir -p $HOME/elcarro-oracle-operator
gsutil -m cp -r gs://elcarro/latest $HOME/elcarro-oracle-operator
bash $HOME/elcarro-oracle-operator/latest/deploy/install-18c-xe.sh --service_account [email protected]
I’m following the instructions from https://github.com/GoogleCloudPlatform/elcarro-oracle-operator/blob/main/docs/content/quickstart-18c-xe.md here.
This takes a while the first time (45 minutes) because it has to create the image, built from oracle-database-xe-18c-1.0-1.x86_64.rpm RPM. The image is nearly 6GB and is stored in your Container Registry. Then it creates the cluster. The default cluster name is “gkecluster”, the CDB name is GCLOUD and the defaut zone is us-central1-a but you can pass the -c -k -z option on the install-18c-xe.sh to change those defaults. It creates a 2 nodes, total 4 vCPUs, 15 GB RAM, 200GB persistent storage. The namespace is “db”.
...
kubeconfig entry generated for gkecluster.
NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS
gkecluster us-central1-a 1.19.9-gke.1900 34.67.217.61 n1-standard-2 1.19.9-gke.1900 2 RUNNING
storageclass.storage.k8s.io/csi-gce-pd created
volumesnapshotclass.snapshot.storage.k8s.io/csi-gce-pd-snapshot-class created
namespace/operator-system created
...
Waiting for startup, statuses: InstanceReady=, InstanceDatabaseReady=, DatabaseReady=
Waiting for startup, statuses: InstanceReady=, InstanceDatabaseReady=, DatabaseReady=CreatePending
...
Waiting for startup, statuses: InstanceReady=CreateInProgress, InstanceDatabaseReady=, DatabaseReady=CreatePending
...
Waiting for startup, statuses: InstanceReady=CreateComplete, InstanceDatabaseReady=CreateInProgress, DatabaseReady=CreatePending
...
Waiting for startup, statuses: InstanceReady=CreateComplete, InstanceDatabaseReady=CreateComplete, DatabaseReady=CreatePending
Waiting for startup, statuses: InstanceReady=CreateComplete, InstanceDatabaseReady=CreateComplete, DatabaseReady=CreateComplete
Oracle Operator is installed. Database connection command:
> sqlplus scott/[email protected]:6021/pdb1.gke
franck@cloudshell:~ (google-cloud.424242)$
Be patient… it is Oracle, it has a pre-DevOps installation timing… And this is why it is really good to have a standardized way for automation. Building your own is a lot of effort because each iteration takes time to validate.
So, all is installed, with a service endpoint exposed to the public internet on port 6021:
[opc@a ~]$ ~/sqlcl/bin/sql scott/[email protected]:6021/pdb1.gke
SQLcl: Release 21.1 Production on Fri May 14 10:40:03 2021
Copyright (c) 1982, 2021, Oracle. All rights reserved.
Connected to:
Oracle Database 18c Express Edition Release 18.0.0.0.0 - Production
Version 18.4.0.0.0
SQL> select * from v$session_connect_info;
SID SERIAL# AUTHENTICATION_TYPE OSUSER NETWORK_SERVICE_BANNER CLIENT_CHARSET CLIENT_CONNECTION CLIENT_OCI_LIBRARY CLIENT_VERSION CLIENT_DRIVER CLIENT_LOBATTR CLIENT_REGID CON_ID
______ __________ ______________________ _________ _________________________________________________________________________ _________________ ____________________ _____________________ _________________ ________________________ _________________________ _______________ _________
283 20073 DATABASE opc TCP/IP NT Protocol Adapter for Linux: Version 18.0.0.0.0 - Production Unknown Heterogeneous Unknown 21.16.0.0.0 jdbcthin : 21.1.0.0.0 Client Temp Lob Rfc On 0 3
283 20073 DATABASE opc Encryption service for Linux: Version 18.0.0.0.0 - Production Unknown Heterogeneous Unknown 21.16.0.0.0 jdbcthin : 21.1.0.0.0 Client Temp Lob Rfc On 0 3
283 20073 DATABASE opc Crypto-checksumming service for Linux: Version 18.0.0.0.0 - Production Unknown Heterogeneous Unknown 21.16.0.0.0 jdbcthin : 21.1.0.0.0 Client Temp Lob Rfc On 0 3
SQL> select initcap(regexp_replace(reverse('El Carro'),'(.)\1+| ','\1')) "K8s Operator for" from dual;
K8s Operator for
__________________
Oracle
SQL>
Now you see where the “El Carro” name comes from, right? 🤣
I can check the pods, from the Web Console, or CLI, remember the namespace is ‘db’:
franck@cloudshell:~ (google-cloud.424242)$ kubectl get pods -n db
NAME READY STATUS RESTARTS AGE
mydb-agent-deployment-6c8b7647fb-d4lkf 2/2 Running 0 77m
mydb-sts-0 4/4 Running 0 77m
franck@cloudshell:~ (google-cloud.424242)$ kubectl get services -n db
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
mydb-agent-svc ClusterIP 10.59.242.135 3202/TCP,9161/TCP 107m
mydb-dbdaemon-svc ClusterIP 10.59.244.15 3203/TCP 107m
mydb-svc LoadBalancer 10.59.245.142 35.224.235.49 6021:30156/TCP,3307:32007/TCP 107m
mydb-svc-node NodePort 10.59.250.249 6021:32512/TCP,3307:31243/TCP 107m
franck@cloudshell:~ (google-cloud.424242)$
The service is exposed externally by the LoadBalancer on the Secure Listener port
I can connect to the container to look at what is running there.
franck@cloudshell:~ (google-cloud.424242)$ kubectl exec -it -n db mydb-sts-0 -c oracledb -- bash -i
bash-4.2$ grep ":[YN]" /etc/oratab
GCLOUD:/opt/oracle/product/18c/dbhomeXE:N
bash-4.2$ . oraenv <<<GCLOUD
ORACLE_SID = [] ? The Oracle base remains unchanged with value /opt/oracle
bash-4.2$ ps -fp $(pgrep tnslsnr)
UID PID PPID C STIME TTY TIME CMD
oracle 488 1 0 09:32 ? 00:00:00 /opt/oracle/product/18c/dbhomeXE/bin/tnslsnr SECURE -inherit
bash-4.2$ lsnrctl status SECURE
LSNRCTL for Linux: Version 18.0.0.0.0 - Production on 14-MAY-2021 10:18:11
Copyright (c) 1991, 2018, Oracle. All rights reserved.
TNS-01101: Could not find listener name or service name SECURE
bash-4.2$ lsnrctl status //localhost:6021
LSNRCTL for Linux: Version 18.0.0.0.0 - Production on 14-MAY-2021 10:17:42
Copyright (c) 1991, 2018, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(CONNECT_DATA=(SERVICE_NAME=))(ADDRESS=(PROTOCOL=TCP)(HOST=127.0.0.1)(PORT=6021)))
STATUS of the LISTENER
------------------------
Alias SECURE
Version TNSLSNR for Linux: Version 18.0.0.0.0 - Production
Start Date 14-MAY-2021 09:32:52
Uptime 0 days 0 hr. 44 min. 49 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u02/app/oracle/oraconfig/network/SECURE/listener.ora
Listener Log File /u02/app/oracle/diag/tnslsnr/mydb-sts-0/secure/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=REGLSNR_6021)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=mydb-sts-0)(PORT=6021)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=127.0.0.1)(PORT=5500))(Security=(my_wallet_directory=/opt/oracle/admin/GCLOUD_uscentral1a/xdb_wallet))(Presentation=HTTP)(Session=RAW))
Services Summary...
Service "GCLOUD.gke" has 1 instance(s).
Instance "GCLOUD", status UNKNOWN, has 1 handler(s) for this service...
Service "GCLOUDXDB.gke" has 1 instance(s).
Instance "GCLOUD", status READY, has 1 handler(s) for this service...
Service "GCLOUD_uscentral1a.gke" has 1 instance(s).
Instance "GCLOUD", status READY, has 1 handler(s) for this service...
Service "PDB1.gke" has 2 instance(s).
Instance "GCLOUD", status UNKNOWN, has 1 handler(s) for this service...
Instance "GCLOUD", status READY, has 1 handler(s) for this service...
Service "c246feca2ab003e8e0530901380a0e21.gke" has 1 instance(s).
Instance "GCLOUD", status READY, has 1 handler(s) for this service...
The command completed successfully
bash-4.2$
Here is the Oracle XE listener, with TNS_ADMIN in /u02/app/oracle/oraconfig/network/SECURE
You can have a look at the available operations from the samples (that you can customize and tun with `kubectl apply -n db -f`):
franck@cloudshell:~ (google-cloud.424242)$ ls ./elcarro-oracle-operator/latest/samples
v1alpha1_backup_rman1.yaml v1alpha1_database_pdb1.yaml
v1alpha1_backup_rman2.yaml v1alpha1_database_pdb2.yaml
v1alpha1_backup_rman3.yaml v1alpha1_database_pdb3.yaml
v1alpha1_backup_rman4.yaml v1alpha1_database_pdb4.yaml
v1alpha1_backupschedule.yaml v1alpha1_export_dmp1.yaml
v1alpha1_backup_snap1.yaml v1alpha1_export_dmp2.yaml
v1alpha1_backup_snap2.yaml v1alpha1_import_pdb1.yaml
v1alpha1_backup_snap_minikube.yaml v1alpha1_instance_18c_XE_express.yaml
v1alpha1_config_bm1.yaml v1alpha1_instance_18c_XE.yaml
v1alpha1_config_bm2.yaml v1alpha1_instance_custom_seeded.yaml
v1alpha1_config_gcp1.yaml v1alpha1_instance_express.yaml
v1alpha1_config_gcp2.yaml v1alpha1_instance_gcp_ilb.yaml
v1alpha1_config_gcp3.yaml v1alpha1_instance_minikube.yaml
v1alpha1_config_minikube.yaml v1alpha1_instance_standby.yaml
v1alpha1_cronanything.yaml v1alpha1_instance_unseeded.yaml
v1alpha1_database_pdb1_express.yaml v1alpha1_instance_with_backup_disk.yaml
v1alpha1_database_pdb1_gsm.yaml v1alpha1_instance.yaml
v1alpha1_database_pdb1_unseeded.yaml
franck@cloudshell:~ (google-cloud.424242)$
Storage snapshots (v1alpha1_backup_snap2.yaml), backups (v1alpha1_backup_rman3.yaml), exports (v1alpha1_export_dmp1.yaml)
All is documented: https://github.com/GoogleCloudPlatform/elcarro-oracle-operator and will probably evolve.