{"id":16073,"date":"2021-04-01T08:29:09","date_gmt":"2021-04-01T06:29:09","guid":{"rendered":"https:\/\/www.dbi-services.com\/blog\/temboard-on-sles15-1-the-webui\/"},"modified":"2021-04-01T08:29:09","modified_gmt":"2021-04-01T06:29:09","slug":"temboard-on-sles15-1-the-webui","status":"publish","type":"post","link":"https:\/\/www.dbi-services.com\/blog\/temboard-on-sles15-1-the-webui\/","title":{"rendered":"temBoard on SLES15 – 1 – The WebUI"},"content":{"rendered":"

During our last SwissPUG<\/a> online meeting there was a presentation about monitoring PostgreSQL with temBoard<\/a>. A question that came up afterwards was: How to install that on SLES 15<\/a> as there are only packages for Debian<\/a> and CentOS\/RHEL<\/a>. As temBoard is written in Python you can also install it using pip<\/a>. That requires a bit more work compared to the package method, but it is perfectly fine. The only downside (but this is true for the packaged version as well) is, that you need Python 2.7<\/a> for the WebUI, and this is already end of life. So, lets give it a try on a fresh SLES 15 minimal installation.<\/p>\n

<\/p>\n

As mentioned just before we’ll start with a minimal installation, as this is intended to be a step by step guide:<\/p>\n

\nsles15webui:~ $ cat \/etc\/os-release \nNAME=\"SLES\"\nVERSION=\"15-SP2\"\nVERSION_ID=\"15.2\"\nPRETTY_NAME=\"SUSE Linux Enterprise Server 15 SP2\"\nID=\"sles\"\nID_LIKE=\"suse\"\nANSI_COLOR=\"0;32\"\nCPE_NAME=\"cpe:\/o:suse:sles:15:sp2\"\n<\/pre>\n
The minimal installation of SLES 15 comes with Python 3.6.13 and this is not what we need:<\/p>\n
\nsles15webui:~ $ python3 --version\nPython 3.6.13\n<\/pre>\n
For getting Python 2.7 onto the system, the corresponding repository needs to be enabled:<\/p>\n
\nsles15webui:~ $ SUSEConnect --product sle-module-python2\/15.2\/x86_64\nRegistering system to SUSE Customer Center\n\nUpdating system details on https:\/\/scc.suse.com ...\n\nActivating sle-module-python2 15.2 x86_64 ...\n-> Adding service to system ...\n-> Installing release package ...\n\nSuccessfully registered system\n<\/pre>\n
Having that ready, we can install the requirements for temBoard:<\/p>\n
\nsles15webui:~ $ zypper in python2 python2-pip\nRefreshing service 'Basesystem_Module_15_SP2_x86_64'.\nRefreshing service 'Python_2_Module_15_SP2_x86_64'.\nRefreshing service 'SUSE_Linux_Enterprise_Server_15_SP2_x86_64'.\nRefreshing service 'Server_Applications_Module_15_SP2_x86_64'.\n...\nsles15webui:~ $ python2 --version\nPython 2.7.18\n<\/pre>\n
temBoard requires a PostgreSQL instance up and running as a repository database, and SLES 15 comes with PostgreSQL 13 already packaged:<\/p>\n
\nsles15webui:~ $ zypper search postgres | grep 13\n  | postgresql13                 | Basic Clients and Utilities for PostgreSQL                              | srcpackage\n  | postgresql13                 | Basic Clients and Utilities for PostgreSQL                              | package\n  | postgresql13-contrib         | Contributed Extensions and Additions to PostgreSQL                      | package\n  | postgresql13-devel           | PostgreSQL client development header files and libraries                | package\n  | postgresql13-docs            | HTML Documentation for PostgreSQL                                       | package\n  | postgresql13-plperl          | The PL\/Tcl, PL\/Perl, and  PL\/Python procedural languages for PostgreSQL | package\n  | postgresql13-plpython        | The PL\/Python Procedural Languages for PostgreSQL                       | package\n  | postgresql13-pltcl           | PL\/Tcl Procedural Language for PostgreSQL                               | package\n  | postgresql13-server          | The Programs Needed to Create and Run a PostgreSQL Server               | package\n  | postgresql13-server-devel    | PostgreSQL server development header files and utilities                | package\n<\/pre>\n
Either use that to install PostgreSQL and create a cluster, or install PostgreSQL from source code<\/a>. I’ll skip these steps here and my PostgreSQL instance is already running:<\/p>\n
\npostgres@sles15webui:\/home\/postgres\/ [tb] psql\npsql (13.2)\nType \"help\" for help.\n\npostgres=# select version();\n                                      version                                       \n------------------------------------------------------------------------------------\n PostgreSQL 13.2 on x86_64-pc-linux-gnu, compiled by gcc (SUSE Linux) 7.5.0, 64-bit\n(1 row)\n<\/pre>\n
The installation of temBoard is quite simple, all you have to do is this:<\/p>\n
\npostgres@sles15webui:\/home\/postgres\/ [tb] sudo pip2 install temboard psycopg2-binary\nDEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https:\/\/pip.pypa.io\/en\/latest\/development\/release-process\/#python-2-support\nCollecting temboard\n...\n    Running setup.py install for tornado ... done\n  WARNING: The script mako-render is installed in '\/usr\/local\/bin' which is not on PATH.\n  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.\n  WARNING: The script alembic is installed in '\/usr\/local\/bin' which is not on PATH.\n  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.\n  WARNING: The scripts temboard and temboard-migratedb are installed in '\/usr\/local\/bin' which is not on PATH.\n  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.\nSuccessfully installed Mako MarkupSafe alembic backports-abc configparser contextlib2 futures importlib-metadata pathlib2 psycopg2-binary python-dateutil python-editor scandir singledispatch sqlalchemy temboard tornado zipp\n<\/pre>\n
As you can see from the last lines above, temBoard gets installed into “\/usr\/local\/bin”:<\/p>\n
\npostgres@sles15webui:\/home\/postgres\/ [tb] ls \/usr\/local\/bin\/\nalembic  mako-render  temboard  temboard-migratedb\n<\/pre>\n
The other important directory is “\/usr\/local\/share\/temboard\/”, this one contains all the scripts:<\/p>\n
\npostgres@sles15webui:\/home\/postgres\/ [tb] ls \/usr\/local\/share\/temboard\/\nauto_configure.sh  create_repository.sh  purge.sh  quickstart  sql\n<\/pre>\n
The “auto_configure.sh” script is the one which does all the work. You can either go with the default or adjust the parameters as you like, e.g.:<\/p>\n
\npostgres@sles15webui:\/home\/postgres\/ [tb] sudo su -\nsles15webui:~ $ export ETCDIR=\/u01\/app\/postgres\/local\/dmk\/etc\nsles15webui:~ $ export VARDIR=\/u01\/app\/postgres\/local\/dmk\/bin\nsles15webui:~ $ export LOGDIR=\/u01\/app\/postgres\/local\/dmk\/log\nsles15webui:~ $ export LOGFILE=\/u01\/app\/postgres\/local\/dmk\/log\/temboard-auto-configure.log\n<\/pre>\n
The auto configuration script needs to connect to your PostgreSQL instance, so make sure that this works before starting:<\/p>\n
\nsles15webui:~ $ export PATH=\/u01\/app\/postgres\/product\/13\/db_2\/bin:$PATH\nsles15webui:~ $ export PGHOST=\/tmp\nsles15webui:~ $ sudo -Eu postgres psql\ncould not change directory to \"\/root\": Permission denied\npsql (13.2)\nType \"help\" for help.\n<\/pre>\n
If that works just execute the script:<\/p>\n
\nsles15webui:~ $ \/usr\/local\/share\/temboard\/auto_configure.sh\nCreating system user temBoard.\nConfiguring temboard in \/u01\/app\/postgres\/local\/dmk\/etc.\nGenerating self-signed certificate.\nCreating Postgres user, database and schema.\nFailure. See \/u01\/app\/postgres\/local\/dmk\/log\/temboard-auto-configure.log for details.\n<\/pre>\n
… and that fails. Checking the log file, the reason is that “temboard-migratedb” is not found:<\/p>\n
\nCREATE DATABASE\n.\/create_repository.sh: line 17: temboard-migratedb: command not found\n.\/create_repository.sh: line 18: temboard-migratedb: command not found\n<\/pre>\n
The easy fix is to update the create_repository.sh script:<\/p>\n
\nif ! \/usr\/local\/bin\/temboard-migratedb check ; then\n    \/usr\/local\/bin\/temboard-migratedb upgrade\n    psql=\"psql -aw --set ON_ERROR_STOP=on --pset pager=off\"\n    if [ -n \"${DEV-}\" ] ; then\n        $psql -f $SQLDIR\/dev-fixture.sql\n    fi\nfi\n<\/pre>\n
Running the auto configuration once more, and it succeeds:<\/p>\n
\nsles15webui:~ $ \/usr\/local\/share\/temboard\/auto_configure.sh\nConfiguring temboard in \/u01\/app\/postgres\/local\/dmk\/etc.\nCreating Postgres user, database and schema.\n\nSuccess. You can now start temboard using:\n\n    systemctl start temboard\n\nRemember to replace default admin user!!!\n<\/pre>\n
A systemd service was created automatically:<\/p>\n
\nsles15webui:~ $ systemctl status temboard\n\u25cf temboard.service - temBoard Web UI\n   Loaded: loaded (\/usr\/local\/lib\/systemd\/system\/temboard.service; enabled; vendor preset: disabled)\n   Active: inactive (dead)\nsles15webui:~ $ cat \/usr\/local\/lib\/systemd\/system\/temboard.service\n[Unit]\nDescription=temBoard Web UI\nAfter=network.target\n\n[Service]\nType=simple\nUser=temboard\nGroup=temboard\nExecStart=\/usr\/bin\/env SYSTEMD=1 temboard -c \/etc\/temboard\/temboard.conf\n\n[Install]\nWantedBy=multi-user.target\n<\/pre>\n
The issue with that service is, that the configuration file does not exist:<\/p>\n
\nsles15webui:~ $ cat \/etc\/temboard\/temboard.conf\ncat: \/etc\/temboard\/temboard.conf: No such file or directory\n<\/pre>\n
Because we’ve set the environment variables before doing the temBoard installation, our configuration file is here:<\/p>\n
\nsles15webui:~ $ cat \/u01\/app\/postgres\/local\/dmk\/etc\/temboard.conf \n# Configuration initiated by \/usr\/local\/share\/temboard\/auto_configure.sh on Thu Apr  1 09:17:58 CEST 2021\n#\n# See https:\/\/temboard.rtfd.io\/ for details on configuration\n# possibilities.\n\n[temboard]\nssl_cert_file = \/etc\/ssl\/certs\/temboard.pem\nssl_key_file = \/etc\/ssl\/private\/temboard.key\ncookie_secret = d11bcef48a3f6bc6de09cb3e39b88b99b2fae98b4d3dbfe501019efdcac681044be29eed3907b0cbe12dc64d49923f50f262b33fe1d154cf9b8609f2770edaef\nhome = \/u01\/app\/postgres\/local\/dmk\/bin\n\n[repository]\nhost = \/tmp\nport = 5432\nuser = temboard\npassword = 4ebe5cd9715718780702420436cbd268\ndbname = temboard\n\n[logging]\nmethod = stderr\nlevel = INFO\n\n[monitoring]\n# purge_after = 365\n\n[statements]\n# purge_after = 7\n<\/pre>\n
Let’s try to start temBoard using this configuration manually as root:<\/p>\n
\nsles15webui:~ $ \/usr\/bin\/env SYSTEMD=1 temboard -c \/u01\/app\/postgres\/local\/dmk\/etc\/temboard.conf\n INFO: Starting temboard 7.6.\n INFO: Running on SLES 15-SP2.\n INFO: Using Python 2.7.18 (\/usr\/bin\/python2).\n INFO: Using Psycopg2 2.8.6 (dt dec pq3 ext lo64), Tornado 5.1.1 and SQLAlchemy 1.4.4\nLoaded plugin 'activity'.\nLoaded plugin 'dashboard'.\nLoaded plugin 'monitoring'.\nLoaded plugin 'pgconf'.\n...\n<\/pre>\n
This works and accessing the UI over https:\/\/[IP]:8888 works as well:
\n\"\"<\/a><\/p>\n
So, we need to fix the systemd unit file to use the correct temBoard configuration file like this:<\/p>\n
\nsles15webui:~ $ cat \/usr\/local\/lib\/systemd\/system\/temboard.service\n[Unit]\nDescription=temBoard Web UI\nAfter=network.target\n\n[Service]\nType=simple\nUser=temboard\nGroup=temboard\nExecStart=\/usr\/bin\/env SYSTEMD=1 temboard -c \/u01\/app\/postgres\/local\/dmk\/etc\/temboard.conf\n\n[Install]\nWantedBy=multi-user.target\nsles15webui:~ $ systemctl daemon-reload\n<\/pre>\n
Lets try to start with systemd:<\/p>\n
\nsles15webui:~ $ systemctl start temboard\nsles15webui:~ $ systemctl status temboard\n\u25cf temboard.service - temBoard Web UI\n   Loaded: loaded (\/usr\/local\/lib\/systemd\/system\/temboard.service; enabled; vendor preset: disabled)\n   Active: failed (Result: exit-code) since Thu 2021-04-01 09:49:17 CEST; 3s ago\n  Process: 28629 ExecStart=\/usr\/bin\/env SYSTEMD=1 temboard -c \/u01\/app\/postgres\/local\/dmk\/etc\/temboard.conf (code=exited, status=1\/FAILURE)\n Main PID: 28629 (code=exited, status=1\/FAILURE)\n\nApr 01 09:49:16 sles15webui systemd[1]: Started temBoard Web UI.\nApr 01 09:49:17 sles15webui env[28629]:  INFO: Starting temboard 7.6.\nApr 01 09:49:17 sles15webui env[28629]: ERROR: Invalid temboard_ssl_key_file from config: \/etc\/ssl\/private\/temboard.key: File not found...\nApr 01 09:49:17 sles15webui env[28629]: CRITI: Failed to load configuration.\nApr 01 09:49:17 sles15webui systemd[1]: temboard.service: Main process exited, code=exited, status=1\/FAILURE\nApr 01 09:49:17 sles15webui systemd[1]: temboard.service: Unit entered failed state.\nApr 01 09:49:17 sles15webui systemd[1]: temboard.service: Failed with result 'exit-code'.\n<\/pre>\n
Next issue, the key file can not be found (the service uses the tembaord user to start tembaord). The file actually is there, but the temboard user does not have access to it:<\/p>\n
\nsles15webui:~ $ ls -la \/etc\/ssl\/private\/temboard.key\n-rw------- 1 root root 1704 Apr  1 08:29 \/etc\/ssl\/private\/temboard.key\n<\/pre>\n
As I did not want to open permissions to the SSL configuration in \/etc, I’ve copied that over:<\/p>\n
\nsles15webui:~ $ mv \/etc\/ssl\/private\/temboard.key \/u01\/app\/postgres\/local\/dmk\/etc\/\nsles15webui:~ $ chown temboard:temboard \/u01\/app\/postgres\/local\/dmk\/etc\/temboard.key \n<\/pre>\n
Adjusted the temboard configuration with the new location of the key file:<\/p>\n
\nsles15webui:~ $ cat \/u01\/app\/postgres\/local\/dmk\/etc\/temboard.conf | grep key\nssl_key_file = \/u01\/app\/postgres\/local\/dmk\/etc\/temboard.key\n<\/pre>\n
Started again:<\/p>\n
\nsles15webui:~ $ systemctl start temboard\nsles15webui:~ $ systemctl status temboard\n\u25cf temboard.service - temBoard Web UI\n   Loaded: loaded (\/usr\/local\/lib\/systemd\/system\/temboard.service; enabled; vendor preset: disabled)\n   Active: failed (Result: exit-code) since Thu 2021-04-01 09:54:50 CEST; 2s ago\n  Process: 28792 ExecStart=\/usr\/bin\/env SYSTEMD=1 temboard -c \/u01\/app\/postgres\/local\/dmk\/etc\/temboard.conf (code=exited, status=1\/FAILURE)\n Main PID: 28792 (code=exited, status=1\/FAILURE)\n\nApr 01 09:54:50 sles15webui env[28792]:     now=datetime.utcnow()\nApr 01 09:54:50 sles15webui env[28792]:   File \"\/usr\/local\/lib\/python2.7\/site-packages\/temboardui\/toolkit\/tasklist\/sqlite3_engine.py\", line 253, in recover\nApr 01 09:54:50 sles15webui env[28792]:     raise StorageEngineError(\"Could not recover tasks.\")\nApr 01 09:54:50 sles15webui env[28792]: StorageEngineError: Could not recover tasks.\nApr 01 09:54:50 sles15webui env[28792]: temboardui version is 7.6.\nApr 01 09:54:50 sles15webui env[28792]: This is a bug!\nApr 01 09:54:50 sles15webui env[28792]: Please report traceback to https:\/\/github.com\/dalibo\/temboard\/issues! Thanks!\nApr 01 09:54:50 sles15webui systemd[1]: temboard.service: Main process exited, code=exited, status=1\/FAILURE\nApr 01 09:54:50 sles15webui systemd[1]: temboard.service: Unit entered failed state.\nApr 01 09:54:50 sles15webui systemd[1]: temboard.service: Failed with result 'exit-code'.\n<\/pre>\n
… and it again fails. Lets try to start manually with the temBoard user:<\/p>\n
\nsles15webui:~ $ su - temboard\nThis account is currently not available.\nsles15webui:~ $ usermod -s \/bin\/bash temboard\nsles15webui:~ $ su - temboard\nattempt to write a readonly database\nTraceback (most recent call last):\n  File \"\/usr\/local\/lib\/python2.7\/site-packages\/temboardui\/toolkit\/tasklist\/sqlite3_engine.py\", line 244, in recover\n    (st_aborted, datetime_to_epoch(now), st_doing)\nOperationalError: attempt to write a readonly database\nUnhandled error:\nTraceback (most recent call last):\n  File \"\/usr\/local\/lib\/python2.7\/site-packages\/temboardui\/toolkit\/app.py\", line 298, in entrypoint\n    retcode = self.main(argv, environ)\n  File \"\/usr\/local\/lib\/python2.7\/site-packages\/temboardui\/__main__.py\", line 395, in main\n    self.scheduler.apply_config()\n  File \"\/usr\/local\/lib\/python2.7\/site-packages\/temboardui\/__main__.py\", line 143, in apply_config\n    super(SchedulerService, self).apply_config()\n  File \"\/usr\/local\/lib\/python2.7\/site-packages\/temboardui\/toolkit\/taskmanager.py\", line 724, in apply_config\n    self.scheduler.setup_task_list()\n  File \"\/usr\/local\/lib\/python2.7\/site-packages\/temboardui\/toolkit\/taskmanager.py\", line 482, in setup_task_list\n    self.task_list.recover()\n  File \"\/usr\/local\/lib\/python2.7\/site-packages\/temboardui\/toolkit\/taskmanager.py\", line 196, in recover\n    now=datetime.utcnow()\n  File \"\/usr\/local\/lib\/python2.7\/site-packages\/temboardui\/toolkit\/tasklist\/sqlite3_engine.py\", line 253, in recover\n    raise StorageEngineError(\"Could not recover tasks.\")\nStorageEngineError: Could not recover tasks.\ntemboardui version is 7.6.\nThis is a bug!\nPlease report traceback to https:\/\/github.com\/dalibo\/temboard\/issues! Thanks!\n<\/pre>\n
The issue is this: “attempt to write a readonly database”. As there is no information where that database (seems to be SQLite) is located, lets use strace to spot the location:<\/p>\n
\ntemboard@sles15webui:~> strace -f \/usr\/bin\/env SYSTEMD=1 temboard -c \/u01\/app\/postgres\/local\/dmk\/etc\/temboard.conf \n<\/pre>\n
Digging through the strace output, the database is this one: \/u01\/app\/postgres\/local\/dmk\/bin\/server_tasks.db. Because we started as root before, this file has the woring permissions (this is not the fault of temBoard):<\/p>\n
\nsles15webui:~$  ls -la \/u01\/app\/postgres\/local\/dmk\/bin\/server_tasks.db\n-rw-r--r-- 1 root root 12288 Apr  1 09:45 \/u01\/app\/postgres\/local\/dmk\/bin\/server_tasks.db\n<\/pre>\n
Lets fix this, and try again:<\/p>\n
\nsles15webui:~ $ chown temboard:temboard \/u01\/app\/postgres\/local\/dmk\/bin\/server_tasks.db\nsles15webui:~ $ systemctl status temboard\n\u25cf temboard.service - temBoard Web UI\n   Loaded: loaded (\/usr\/local\/lib\/systemd\/system\/temboard.service; enabled; vendor preset: disabled)\n   Active: active (running) since Thu 2021-04-01 10:15:22 CEST; 3s ago\n Main PID: 1064 (temboard)\n    Tasks: 5\n   CGroup: \/system.slice\/temboard.service\n           \u251c\u25001064 temboard: web\n           \u251c\u25001071 temboard: worker pool\n           \u2514\u25001072 temboard: scheduler\n\nApr 01 10:15:22 sles15webui env[1064]: Loaded plugin 'pgconf'.\nApr 01 10:15:22 sles15webui env[1064]: Loaded plugin 'maintenance'.\nApr 01 10:15:22 sles15webui env[1064]: Loaded plugin 'statements'.\nApr 01 10:15:22 sles15webui env[1064]: temBoard database is up-to-date.\nApr 01 10:15:22 sles15webui env[1064]: Starting web.\nApr 01 10:15:22 sles15webui env[1064]: Serving temboardui on https:\/\/0.0.0.0:8888\nApr 01 10:15:22 sles15webui env[1064]: Starting scheduler.\nApr 01 10:15:22 sles15webui env[1064]: Starting worker pool.\nApr 01 10:15:23 sles15webui env[1064]: Starting collector scheduler worker.\nApr 01 10:15:23 sles15webui env[1064]: End of collector scheduler worker.\n<\/pre>\n
Now we are fine, temBoard is started with the correct user with systemd, login (admin\/admin) works as well:
\n\"\"<\/a><\/p>\n
This was kind of a brute force method for getting it installed, but if you plan that well, it should not be an issue if you follow these steps:<\/p>\n