Log Rotation in JBoss-EAP

Log rotation is an important topic while configuring any software. If it is not tuned as it should it could lead to full partitions, service outages or even configuration corruptions.
JBoss-EAP is not different, thus I will explain what are our best practices.

JBoss-EAP Log Rotation Capabilities

Out of the box, JBoss-EAP profiles are configured with two loggers:

• CONSOLE: Used to log when run from script directly
• FILE: Used to log into server.log

FILE is configured with a suffix “.yyyy-MM-dd”. This indicate to JBoss-EAP to rotate it every day, at midnight.

This is a good start, but it omits an important part: Purge of rotated log files.

Purging Log Files

There is another handler that is taking care of oldest rotated logs, size-rotating-file-handler, as explained in documentation:

Defines a handler which writes to a file, rotating the log after the size of the file grows beyond a certain point and keeping a fixed number of backups.

This is nice, but as log files are rotated based on size and not on time, this will make search painful. So we have to find another option.

Nevertheless, here is an example of jboss-cli command to set it:

[[email protected]:9990 /] /subsystem=logging/size-rotating-file-handler=SRFH:add(named-formatter=COLOR-PATTERN,file={"path"=>"server3.log","relative-to"=>"jboss.server.log.dir"},append=true,autoflush=true,level=INFO)
{"outcome" : "success"}

External Log Rotation

As logrotate is available on all main Linux distributions, we will go for that option. This is as simple as creating a file in /etc/logrotate.d/ with following content:

/data/JBoss-EAP/log/*.log {
        daily
        missingok
        rotate 30
        dateext
        compress
        delaycompress
        notifempty
        copytruncate
}

Here are the meaning of each line:

• daily: All .log files are rotated
• missingok: Missing file is ignored. Next file will be processed.
• rotate: Keeping 30 days.
  • This could be decreased based on available disk space and verbosity.
  • This could be increased, for example, for regulatory needs.
• dateext adds date to the rotated log file instead of a simple index. This will make search in logs easier.
• compress rotated log files.
• delay that compression by one rotation cycle. The next day, in configuration above. This also makes search easier as if I am looking for a log in previous day, I don’t need to uncompress the rotated file.
• notifempty will prevent empty file rotation.
• copytruncate will keep existing file handler is the safest way to rotate log file.

What About Access Logs?

access log is not enabled by default. Enabling it is just one command:

/subsystem=undertow/server=default-server/host=default-host/setting=access-log:add(pattern="%h %l %u %t \"%r\" %s %b \"%{i,Referer}\" \"%{i,User-Agent}\" Cookie: \"%{i,COOKIE}\" Set-Cookie: \"%{o,SET-COOKIE}\" SessionID: %S Thread: \"%I\" TimeTaken: %T",rotate=true)

As you might have seen, there is a rotate attribute which I set to true, but nothing indicates when it will occur. So, here is a small tip to have more information on a field: read-resource-description

This can be called by changing directory first or with absolute path:

/subsystem=undertow/server=default-server/host=default-host/setting=access-log:read-resource-description

Here is a the section concerning rotate field:

...
"rotate" : {
	"type" : {
		"TYPE_MODEL_VALUE" : "BOOLEAN"
	},
	"description" : "Rotate the access log every day.",
	"expressions-allowed" : true,
	"required" : false,
	"nillable" : true,
	"default" : true,
	"access-type" : "read-write",
	"storage" : "configuration",
	"restart-required" : "all-services"
},
...

This means that we have very limited control on how access logs are rotated. For example, hourly rotation is not possible.

Outcome

Despite JBoss-EAP log rotation options, it is highly recommended to delegate log rotation and purge to an external tool like logrotate because JBoss-EAP:

• Can not manage time named files and purge with one file handler
• Can not rotate access log files beside daily rotation