TFA (Trace File Analyzer) is part of AHF (Autonomous Health Framework). Those tools are preinstalled and part of ODA (Oracle Database Appliance). As you might know patching and upgrading are normally always going through ODA global Bundle patches. AHF can, without any problem, be upgraded independently. In this blog I wanted to share with you how I upgraded TFA with latest v21.4 version. The upgrade is performed with root user. This version addresses CVE-2021-45105/CVE-2021-44228/CVE-2021-45046. For reminder Apache Log4j Vulnerabilities are covered by CVE-2021-44228 and CVE-2021-45046.

Check current version of TFA

First we can check if TFA is up and running and which version is currrently used.

[root@ODA01 ~]# /opt/oracle/dcs/oracle.ahf/bin/tfactl status
WARNING - TFA Software is older than 180 days. Please consider upgrading TFA to the latest version.

.------------------------------------------------------------------------------------------------.
| Host      | Status of TFA | PID  | Port | Version    | Build ID             | Inventory Status |
+-----------+---------------+------+------+------------+----------------------+------------------+
| ODA01     | RUNNING       | 5388 | 5000 | 20.1.3.0.0 | 20130020200429161658 | COMPLETE         |
'-----------+---------------+------+------+------------+----------------------+------------------'


As we can see we are currently running TFA/AHF 20.1.3.0.0 version.

Check running processes

We can check as well the TFA running processes.

[root@ODA01 ~]# ps -ef | grep -i tfa | grep -v grep
root      4536     1  0 Oct18 ?        00:18:06 /bin/sh /etc/init.d/init.tfa run >/dev/null 2>&1 </dev/null
root      5388     1  0 Oct18 ?        02:55:07 /opt/oracle/dcs/oracle.ahf/jre/bin/java -server -Xms512m -Xmx1024m -Djava.awt.headless=true -Ddisable.checkForUpdate=true -XX:HeapDumpPath=/opt/oracle/dcs/oracle.ahf/data/ODA01/diag/tfa -XX:ParallelGCThreads=5 oracle.rat.tfa.TFAMain /opt/oracle/dcs/oracle.ahf/tfa
[root@ODA01 ~]#


Check the location of AHF

It is important to check in which directory AHF is currently installed in order to provide appropriate directory in the upgrade command option. Thus the setup script will be able to see that there is a current version installed and will suggest to upgrade it. Otherwise another new AHF installation will be performed.

[root@ODA01 ~]# cat /etc/oracle.ahf.loc
/opt/oracle/dcs/oracle.ahf


AHF is installed on the ODA in the /opt/oracle/dcs/oracle.ahf directory.

Backup of the current AHF version

Before doing any modification it is important to backup current AHF version for fallback if needed. I have been doing a tar of the currrent installation directory.

[root@ODA01 ~]# cd /opt/oracle/dcs

[root@ODA01 dcs]# ls -ltrh
total 25M
drwxr-xr-x.  3 root   root     4.0K Jul  2  2019 rdbaas
drwxr-xr-x.  3 root   root     4.0K Jul  2  2019 scratch
drwx------   2 root   root     4.0K Jul  2  2019 dcsagent_wallet
drwxr-xr-x   2 root   root     4.0K Jul  4  2019 ft
drwxr-xr-x   2 root   root     4.0K Aug 11  2019 Inventory
drwxr-xr-x   4 root   root     4.0K May 17  2020 dcs-ui
-rwxr-xr-x   1 root   root     6.8K May 21  2020 configuredcs.pl
-rw-r--r--   1 root   root      25M May 21  2020 dcs-ui.zip
drwxr-xr-x   4 root   root     4.0K Sep  2  2020 repo
-rw-r--r--   1 root   root        0 Sep  2  2020 dcscontroller-stderr.log
-rw-r--r--   1 root   root     6.7K Sep  3  2020 dcscontroller-stdout.log
drwxr-xr-x   6 oracle oinstall  32K Sep  3  2020 commonstore
drwxr-xr-x  12 root   root     4.0K Sep  3  2020 oracle.ahf
drwxr-xr-x.  2 root   root     4.0K Sep  3  2020 agent
drwxr-xr-x.  2 root   root     4.0K Sep  3  2020 sample
drwxr-xr-x   4 root   root     4.0K Sep  3  2020 java
drwxr-xr-x.  3 root   root     4.0K Sep  3  2020 conf
drwxr-xr-x.  3 root   root     4.0K Sep  3  2020 dcscli
drwxr-xr-x.  2 root   root     4.0K Sep  3  2020 bin
drwx------.  5 root   root      20K Dec 21 00:00 log

[root@ODA01 dcs]# mkdir /root/backup_ahf_for_upgrade/

[root@ODA01 dcs]# tar -czf /root/backup_ahf_for_upgrade/oracle.ahf.20.1.3.0.0.tar ./oracle.ahf

[root@ODA01 dcs]# ls -ltrh /root/backup_ahf_for_upgrade
total 1.3G
-rw-r--r-- 1 root root 1.3G Dec 21 14:26 oracle.ahf.20.1.3.0.0.tar


Download new AHF version

You can download latest AHF version through my oracle support portal. Download patch 30166242 :
Patch 30166242: PLACEHOLDER – DOWNLOAD LATEST AHF (TFA and ORACHK/EXACHK)

I have created a directory on the ODA to upload the patch :

[root@ODA01 dcs]# mkdir /u01/app/patch/TFA


Upgrade AHF on the ODA

In this part we will see the procedure to upgrade AHF on the ODA. We first need to unzip the AHF-LINUX_v21.4.0.zip file and run ahf_setup. The installation script will recognise the existing 20.1.3 version and suggest to upgrade it.

root@ODA01 dcs]# cd /u01/app/patch/TFA

[root@ODA01 TFA]# ls -ltrh
total 394M
-rw-r--r-- 1 root root 393M Dec 21 10:16 AHF-LINUX_v21.4.0.zip

[root@ODA01 TFA]# unzip -q AHF-LINUX_v21.4.0.zip

[root@ODA01 TFA]# ls -ltrh
total 792M
-r-xr-xr-x 1 root root 398M Dec 20 19:28 ahf_setup
-rw-r--r-- 1 root root  384 Dec 20 19:30 ahf_setup.dat
-rw-r--r-- 1 root root 1.5K Dec 20 19:31 README.txt
-rw-r--r-- 1 root root  625 Dec 20 19:31 oracle-tfa.pub
-rw-r--r-- 1 root root 393M Dec 21 10:16 AHF-LINUX_v21.4.0.zip

[root@ODA01 TFA]# ./ahf_setup -ahf_loc /opt/oracle/dcs -data_dir /opt/oracle/dcs

AHF Installer for Platform Linux Architecture x86_64

AHF Installation Log : /tmp/ahf_install_214000_58089_2021_12_21-14_30_06.log

Starting Autonomous Health Framework (AHF) Installation

AHF Version: 21.4.0 Build Date: 202112200745

AHF is already installed at /opt/oracle/dcs/oracle.ahf

Installed AHF Version: 20.1.3 Build Date: 202004291616

Do you want to upgrade AHF [Y]|N : Y

Upgrading /opt/oracle/dcs/oracle.ahf

Shutting down AHF Services
Stopped OSWatcher
Nothing to do !
Shutting down TFA
/etc/init.d/init.tfa: line 661: /sbin/stop: No such file or directory
. . . . .
Killing TFA running with pid 5388
. . .
Successfully shutdown TFA..

Starting AHF Services
Starting TFA..
Waiting up to 100 seconds for TFA to be started..
. . . . .
. . . . .
. . . . .
Successfully started TFA Process..
. . . . .
TFA Started and listening for commands


Do you want AHF to store your My Oracle Support Credentials for Automatic Upload ? Y|[N] : N

AHF is successfully upgraded to latest version

.-----------------------------------------------------------------.
| Host      | TFA Version | TFA Build ID         | Upgrade Status |
+-----------+-------------+----------------------+----------------+
| ODA01     |  21.4.0.0.0 | 21400020211220074549 | UPGRADED       |
'-----------+-------------+----------------------+----------------'

Moving /tmp/ahf_install_214000_58089_2021_12_21-14_30_06.log to /opt/oracle/dcs/oracle.ahf/data/ODA01/diag/ahf/

[root@ODA01 TFA]#


Check new AHF version

We can check that the new version of AHF is 21.4.

[root@ODA01 TFA]# /opt/oracle/dcs/oracle.ahf/bin/ahfctl version

AHF version: 21.4.0


Check TFA running processes

We can check that TFA is up and running.

[root@ODA01 TFA]# ps -ef | grep -i tfa | grep -v grep
root      4536     1  0 Oct18 ?        00:18:06 /bin/sh /etc/init.d/init.tfa run >/dev/null 2>&1 </dev/null
root     61938     1 62 14:31 ?        00:01:36 /opt/oracle/dcs/oracle.ahf/jre/bin/java -server -Xms512m -Xmx1024m -Djava.awt.headless=true -Ddisable.checkForUpdate=true -XX:HeapDumpPath=/opt/oracle/dcs/oracle.ahf/data/ODA01/diag/tfa -XX:ParallelGCThreads=5 oracle.rat.tfa.TFAMain /opt/oracle/dcs/oracle.ahf/tfa
[root@ODA01 TFA]#


After the upgrade script is completed there might still be some TFA processes running in order to rebuild the inventory :

root     15469 15077  0 14:58 ?        00:00:00 sh -c /opt/oracle/dcs/oracle.ahf/tfa/bin/tfactl rediscover -mode full > /dev/null 2>&1
root     15470 15469  0 14:58 ?        00:00:00 /bin/sh /opt/oracle/dcs/oracle.ahf/tfa/bin/tfactl rediscover -mode full
root     15505 15500  0 14:58 ?        00:00:00 /bin/sh /opt/oracle/dcs/oracle.ahf/tfa/bin/tfactl.tfa rediscover -mode full
root     15524 15505  1 14:58 ?        00:00:00 /u01/app/19.0.0.0/grid/perl/bin/perl /opt/oracle/dcs/oracle.ahf/tfa/bin/tfactl.pl rediscover -mode full


Make sure all those processes are completed successfully (not existing any more) before stopping AHF. Otherwise your inventory status will end with a STOPPED status.

Check status of AHF

We can check AHF status.

[root@ODA01 TFA]# /opt/oracle/dcs/oracle.ahf/bin/ahfctl statusahf


.-------------------------------------------------------------------------------------------------.
| Host      | Status of TFA | PID   | Port | Version    | Build ID             | Inventory Status |
+-----------+---------------+-------+------+------------+----------------------+------------------+
| ODA01     | RUNNING       | 61938 | 5000 | 21.4.0.0.0 | 21400020211220074549 | COMPLETE         |
'-----------+---------------+-------+------+------------+----------------------+------------------'


No scheduler for any ID

orachk daemon is not running

[root@ODA01 TFA]#


TFA is running. No AHF scheduler. No orachk daemon.

Stop AHF and TFA

To check all is working as expected, let’s stop AHF.

[root@ODA01 TFA]# /opt/oracle/dcs/oracle.ahf/bin/ahfctl stopahf

Stopping TFA from the Command Line
Stopped OSWatcher
Nothing to do !
Please wait while TFA stops
Please wait while TFA stops
TFA-00002 Oracle Trace File Analyzer (TFA) is not running
TFA Stopped Successfully
Successfully stopped TFA..

orachk scheduler is not running


There is still one process for TFA, the one from init.d :

[root@ODA01 TFA]# ps -ef | grep -i tfa | grep -v grep
root      4536     1  0 Oct18 ?        00:18:06 /bin/sh /etc/init.d/init.tfa run >/dev/null 2>&1 </dev/null
[root@ODA01 TFA]#


We are going to stop it :

[root@ODA01 TFA]# /etc/init.d/init.tfa stop
Stopping TFA from init for shutdown/reboot
Nothing to do !
TFA Stopped Successfully
Successfully stopped TFA..


And there is no more TFA processes up and running :

[root@ODA01 TFA]# ps -ef | grep -i tfa | grep -v grep
[root@ODA01 TFA]#


Start AHF

[root@ODA01 TFA]# /opt/oracle/dcs/oracle.ahf/bin/ahfctl startahf

Starting TFA..
Waiting up to 100 seconds for TFA to be started..
. . . . .
. . . . .
Successfully started TFA Process..
. . . . .
TFA Started and listening for commands

INFO: Starting orachk scheduler in background. Details for the process can be found at /opt/oracle/dcs/oracle.ahf/data/ODA01/diag/orachk/compliance_start_211221_143845.log


We can check TFA status :

[root@ODA01 TFA]# /opt/oracle/dcs/oracle.ahf/bin/tfactl status

.-------------------------------------------------------------------------------------------------.
| Host      | Status of TFA | PID   | Port | Version    | Build ID             | Inventory Status |
+-----------+---------------+-------+------+------------+----------------------+------------------+
| ODA01     | RUNNING       | 87371 | 5000 | 21.4.0.0.0 | 21400020211220074549 | COMPLETE         |
'-----------+---------------+-------+------+------------+----------------------+------------------'


We can check AHF status as well and see that we have now scheduler and orachk daemon up and running :

[root@ODA01 TFA]# /opt/oracle/dcs/oracle.ahf/bin/ahfctl statusahf


.-------------------------------------------------------------------------------------------------.
| Host      | Status of TFA | PID   | Port | Version    | Build ID             | Inventory Status |
+-----------+---------------+-------+------+------------+----------------------+------------------+
| ODA01     | RUNNING       | 87371 | 5000 | 21.4.0.0.0 | 21400020211220074549 | COMPLETE         |
'-----------+---------------+-------+------+------------+----------------------+------------------'

------------------------------------------------------------

Master node = ODA01

orachk daemon version = 21.4.0

Install location = /opt/oracle/dcs/oracle.ahf/orachk

Started at = Tue Dec 21 14:38:57 CET 2021

Scheduler type = TFA Scheduler

Scheduler PID:  87371

------------------------------------------------------------
ID: orachk.autostart_client_oratier1
------------------------------------------------------------
AUTORUN_FLAGS  =  -usediscovery -profile oratier1 -dball -showpass -tag autostart_client_oratier1 -readenvconfig
COLLECTION_RETENTION  =  7
AUTORUN_SCHEDULE  =  3 2 * * 1,2,3,4,5,6
------------------------------------------------------------
------------------------------------------------------------
ID: orachk.autostart_client
------------------------------------------------------------
AUTORUN_FLAGS  =  -usediscovery -tag autostart_client -readenvconfig
COLLECTION_RETENTION  =  14
AUTORUN_SCHEDULE  =  3 3 * * 0
------------------------------------------------------------

Next auto run starts on Dec 22, 2021 02:03:00

ID:orachk.AUTOSTART_CLIENT_ORATIER1


We can also check TFA processes :

[root@ODA01 TFA]#  ps -ef | grep -i tfa | grep -v grep
root     86989     1  0 14:38 ?        00:00:00 /bin/sh /etc/init.d/init.tfa run >/dev/null 2>&1 </dev/null
root     87371     1 19 14:38 ?        00:00:13 /opt/oracle/dcs/oracle.ahf/jre/bin/java -server -Xms512m -Xmx1024m -Djava.awt.headless=true -Ddisable.checkForUpdate=true -XX:HeapDumpPath=/opt/oracle/dcs/oracle.ahf/data/ODA01/diag/tfa -XX:ParallelGCThreads=5 oracle.rat.tfa.TFAMain /opt/oracle/dcs/oracle.ahf/tfa
root     92789 87371 38 14:39 ?        00:00:00 /u01/app/19.0.0.0/grid/perl/bin/perl /opt/oracle/dcs/oracle.ahf/tfa/bin/tfactl.pl availability product Europe/Zurich
[root@ODA01 TFA]#


Stop AHF and TFA

We will stop AHF and TFA again.

[root@ODA01 TFA]# /opt/oracle/dcs/oracle.ahf/bin/ahfctl stopahf

Stopping TFA from the Command Line
Nothing to do !
Please wait while TFA stops
Please wait while TFA stops
TFA-00002 Oracle Trace File Analyzer (TFA) is not running
TFA Stopped Successfully
Successfully stopped TFA..

Stopping orachk scheduler ...
Removing orachk cache discovery....
No orachk cache discovery found.



Unable to send message to TFA



Removed orachk from inittab


Stopped orachk


AHF status checks :

[root@ODA01 TFA]# /opt/oracle/dcs/oracle.ahf/bin/ahfctl statusahf

TFA-00002 Oracle Trace File Analyzer (TFA) is not running


No scheduler for any ID

orachk daemon is not running


TFA status checks :

[root@ODA01 TFA]# /opt/oracle/dcs/oracle.ahf/bin/tfactl status
TFA-00002 Oracle Trace File Analyzer (TFA) is not running


Check processes and stop TFA init.d :

[root@ODA01 TFA]#  ps -ef | grep -i tfa | grep -v grep
root     86989     1  0 14:38 ?        00:00:00 /bin/sh /etc/init.d/init.tfa run >/dev/null 2>&1 </dev/null

[root@ODA01 TFA]# /etc/init.d/init.tfa stop
Stopping TFA from init for shutdown/reboot
Nothing to do !
TFA Stopped Successfully
Successfully stopped TFA..

[root@ODA01 TFA]#  ps -ef | grep -i tfa | grep -v grep
[root@ODA01 TFA]#


Restart only TFA

Finally we only want to keep TFA up and running. No AHF scheduling or orachk daemon. So we are only going to start TFA.

[root@ODA01 TFA]# /etc/init.d/init.tfa start
Starting TFA..
Waiting up to 100 seconds for TFA to be started..
. . . . .
Successfully started TFA Process..
. . . . .
TFA Started and listening for commands


Final checks

TFA running processes :

[root@ODA01 TFA]#  ps -ef | grep -i tfa | grep -v grep
root      5344     1  0 14:43 ?        00:00:00 /bin/sh /etc/init.d/init.tfa run >/dev/null 2>&1 </dev/null
root      5732     1 77 14:43 ?        00:00:11 /opt/oracle/dcs/oracle.ahf/jre/bin/java -server -Xms512m -Xmx1024m -Djava.awt.headless=true -Ddisable.checkForUpdate=true -XX:HeapDumpPath=/opt/oracle/dcs/oracle.ahf/data/ODA01/diag/tfa -XX:ParallelGCThreads=5 oracle.rat.tfa.TFAMain /opt/oracle/dcs/oracle.ahf/tfa
[root@ODA01 TFA]#


TFA status :

[root@ODA01 TFA]# /opt/oracle/dcs/oracle.ahf/bin/tfactl status

.------------------------------------------------------------------------------------------------.
| Host      | Status of TFA | PID  | Port | Version    | Build ID             | Inventory Status |
+-----------+---------------+------+------+------------+----------------------+------------------+
| ODA01     | RUNNING       | 5732 | 5000 | 21.4.0.0.0 | 21400020211220074549 | COMPLETE         |
'-----------+---------------+------+------+------------+----------------------+------------------'
[root@ODA01 TFA]#


AHF status :

[root@ODA01 TFA]# /opt/oracle/dcs/oracle.ahf/bin/ahfctl statusahf


.------------------------------------------------------------------------------------------------.
| Host      | Status of TFA | PID  | Port | Version    | Build ID             | Inventory Status |
+-----------+---------------+------+------+------------+----------------------+------------------+
| ODA01     | RUNNING       | 5732 | 5000 | 21.4.0.0.0 | 21400020211220074549 | COMPLETE         |
'-----------+---------------+------+------+------------+----------------------+------------------'


No scheduler for any ID

orachk daemon is not running

[root@ODA01 TFA]#


Cleanup

We can still keep previous AHF version backup a few days just in case and remove it later.

The AHF installation files can be deleted :

[root@ODA01 ~]# cd /u01/app/patch/

[root@ODA01 patch]# ls -l TFA
total 810144
-rw-r--r-- 1 root root 411836201 Dec 21 10:16 AHF-LINUX_v21.4.0.zip
-r-xr-xr-x 1 root root 416913901 Dec 20 19:28 ahf_setup
-rw-r--r-- 1 root root       384 Dec 20 19:30 ahf_setup.dat
-rw-r--r-- 1 root root       625 Dec 20 19:31 oracle-tfa.pub
-rw-r--r-- 1 root root      1525 Dec 20 19:31 README.txt

[root@ODA01 patch]# rm -rf TFA

[root@ODA01 patch]# ls
[root@ODA01 patch]#