logrotate

Linux daemons write log files. Left to itself, these log files would grow with time and take up the disk space. It is necessary to purge the log files periodically so that the disk space is conserved and is available for smooth running of the system. logrotate is a utility for rotation, compression, removal and mailing of log files. The compression and mailing activities are optional. logrotate does this work daily, weekly, monthly, or when the log file becomes bigger than a predefined limit. The syntax of using the logrotate command is,

logrotate [OPTION ...] <config-file>

logrotate is run as a cron job. There is a script logrotate in the /etc/cron.daily directory, which is run by anacron, or by cron, if anacron is not installed. The line which calls logrotate in the script /etc/cron.daily/logrotate is,

/usr/sbin/logrotate /etc/logrotate.conf

where /etc/logrotate.conf is the logrotate configuration file. There is a state file, which is, by default, /var/lib/logrotate/status. logrotate keeps the date of file rotation for each log file in the state file. When logrotate is invoked, it checks up the last rotation date for a log file from the state file, looks at the rotation criteria and works out whether the log file is due for rotation.

For using logrotate, one has to add directives to the configuration file, /etc/logrotate.conf.

/etc/logrotate.conf

A sample /etc/logrotate.conf is given below:

# rotate log files weekly weekly # keep 4 weeks worth of backlogs rotate 4 # create new (empty) log files after rotating old ones create # uncomment this if you want your log files compressed #compress # packages drop log rotation information into this directory include /etc/logrotate.d # no packages own wtmp, or btmp -- we'll rotate them here /var/log/wtmp { missingok monthly create 0664 root utmp rotate 1 } /var/log/btmp { missingok monthly create 0660 root utmp rotate 1 } # system-specific logs may be configured here

Lines with the first non-whitespace character as # are treated as comments. The configuration file contains global directives and definition of log files to be rotated. Directives specific to a log file are given inside braces after the log file name. Local directives inside the braces override the global directives. Also, directives coming later on in the configuration file override the earlier ones. It is possible to give multiple configuration file names to logrotate on the command line. However, this is not recommended and single configuration file name should be given on the command line and other configuration files should be included in the main configuration file using the include directive. For example, in the above /etc/logrotate.conf configuration file, we have the include directive,

include /etc/logrotate.d

The logrotate configuration files for the individual packages are located in the directory, /etc/logrotate.d.

$ ls -ls /etc/logrotate.d total 72 4 -rw-r--r-- 1 root root 194 2012-02-14 23:25 apache2 4 -rw-r--r-- 1 root root 126 2011-10-09 13:01 apport 4 -rw-r--r-- 1 root root 173 2011-10-06 21:00 apt 4 -rw-r--r-- 1 root root 330 2011-01-28 01:20 atop 4 -rw-r--r-- 1 root root 135 2011-05-21 14:42 consolekit 4 -rw-r--r-- 1 root root 248 2011-09-27 18:46 cups 4 -rw-r--r-- 1 root root 232 2011-10-06 13:34 dpkg 4 -rw-r--r-- 1 root root 146 2011-10-06 10:07 jockey-common 4 -rw-r--r-- 1 root root 880 2012-03-29 00:58 mysql-server 4 -rw-r--r-- 1 root root 302 2012-04-23 20:17 nginx 4 -rw-r--r-- 1 root root 157 2011-07-09 05:21 pm-utils 4 -rw-r--r-- 1 root root 94 2011-02-04 14:13 ppp 4 -rw-r--r-- 1 root root 1061 2011-01-28 01:20 psaccs_atop 4 -rw-r--r-- 1 root root 512 2011-01-28 01:20 psaccu_atop 4 -rw-r--r-- 1 root root 515 2011-10-04 02:51 rsyslog 4 -rw-r--r-- 1 root root 513 2011-06-06 07:59 speech-dispatcher 4 -rw-r--r-- 1 root root 178 2011-07-19 04:06 ufw 4 -rw-r--r-- 1 root root 114 2010-08-02 13:05 unattended-upgrades

And, as an example, /etc/logrotate.d/rsyslog configuration file is,

$ cat /etc/logrotate.d/rsyslog /var/log/syslog { rotate 7 daily missingok notifempty delaycompress compress postrotate reload rsyslog >/dev/null 2>&1 || true endscript } /var/log/mail.info /var/log/mail.warn /var/log/mail.err /var/log/mail.log /var/log/daemon.log /var/log/kern.log /var/log/auth.log /var/log/user.log /var/log/lpr.log /var/log/cron.log /var/log/debug /var/log/messages { rotate 4 weekly missingok notifempty compress delaycompress sharedscripts postrotate reload rsyslog >/dev/null 2>&1 || true endscript }

If there are multiple files before a block of configuration directives inside braces, the block applies to all the files named before the block.

The following paragraphs give details about individual directives.

File rotation

weekly rotate, if the current day of the week is less that the weekday of the last rotation
daily rotate files daily.
monthly files are rotated the first time logrotate is run in the month.
yearly files are rotated if the current year is different from the year of the last rotation.
rotate count files are rotated count times before being removed or mailed.
maxage count If the log file is to be rotated, all old logs, older than count days are removed or mailed.
size size Rotate files once they become bigger than size. size can be in plain bytes or it can be suffixed with a 'K' 'M' or 'G', indicating kilobytes, megabytes and gigabytes respectively.
minsize size Rotate files once they become bigger than size and also once the time limit imposed by a directive like daily, weekly, monthly or yearly, has passed.
ifempty Rotate even if the log file is empty. This is the default.
notifempty Do not rotate if the log file is empty.

Create new files

create [mode][owner][group] Create the new log file with the given mode, owner and group after rotation. If any of the attributes (mode, owner, group) are omitted, use the corresponding attribute of the old log file.
copy Makes a copy of the log file without changing the original. The original log file stays as it is.
copytruncate copies the log file and then truncates the (original) log file.
nocreate Do not create a new log file after rotation.
nocopy Do not create a copy of the original log file and leave it after rotation.
nocopytruncate Do not truncate a copy of the log file after rotation.

Compression

compress compress old versions of log files with gzip, the default compression command
compresscmd The command to use for compression. Default is gzip.
uncompresscmd The command to use for uncompression. Default is gunzip.
compressext extension for compressed log files. Defaults to the extension provided by the configured compression command.
compressoptions Options for the compression command. For the default compression command, gzip, the default option id -9 (best compression).
delaycompress Delay the compression of the log file to next rotation cycle.
nocompress Do not compress the older versions of log files.
nodelaycompress Do not delay the compression of the log file to next rotation cycle.

Include

include <file|directory> Include the given file or directory inline in the configuration file at the place of the include directive. Include is a global directive. If a directory is included, files in that directory are included in the alphabetical order and non-regular files in the directory likes sub-directories and pipes are not included. Also files with taboo extensions, as defined with tabooext directive, are not included.
tabooext [+] list The default taboo list of extensions is .rpmorig, .rpmsave, .v, .swp, .rpmnew, .~, .cfsaved, .rhn-cfg-tmp-*, .dpkg-dist, .dpkg-old, .dpkg-new, .disabled. If the include directive has a '+' then the original list is augmented with the given extensions. If '+' is omitted from the include directive, the taboo list is replaced.

File extensions

start count Start the rotated file extension number with the number count. For example, with start 2 directive, the most recently rotated file has extension .2, previous to that, .3, and so on.
dateext Use system date as YYYYMMDD as extension for the rotated log files instead of a number.
dateformat format-string If dateext directive is used, specify the format for date, using strftime (3)-like notation. Supported specifiers are %Y, %m, %d and %s. Default format-string is -%Y%m%d. If you wish to have hyphens between year, month and day, the format-string would be -%Y-%m-%d.
extension ext In the rotated log file, preserve the extension, ext. For example, file myfile.foo would be rotated as myfile.1.foo.gz. The ext specification contains the period. That is, the extension in myfile.foo is .foo.
nodateext Do not use the date extension in the rotated log files.

Scripts

prerotate/endscript If the file is to be rotated, the command lines between prerotate and endscript are executed before the rotation. Both prerotate and endscript tags must each occupy separate lines, above and below the script respectively. prerotate/endscript script can only appear inside a log file definition.
postrotate/endscript If the file is to be rotated, the command lines between postrotate and endscript are executed after the rotation. Both postrotate and endscript tags must each occupy separate lines, above and below the script respectively. postrotate/endscript script can only appear inside a log file definition.
firstaction/endscript This relates to wildcard in the log file name, thus matching multiple log files. Both, firstaction and endscript must occupy separate lines by themselves and the commands between them are executed only if at least one file would be rotated. The script is executed before all log files that match the wildcard are rotated and also before any prerotate script. If the script exits with an error, the processing aborts and no further action is taken.
lastaction/endscript This relates to wildcard in the log file name, thus matching multiple log files. Both, lastaction and endscript must occupy separate lines by themselves and the commands between them are executed only if at least one file is rotated. The script is executed after all log files that match the wildcard are rotated and also after any postrotate script. If the script exits with an error, the error message is displayed.
sharedscripts By default, prerotate and postrotate scripts are run for each file that is rotated. Since wild cards are used and the same block can be defined for multiple log files, this essentially means that the same script would be executed multiple times, once for each file. By using the sharedscripts directive, it is ensured that the script is run only once for all the log files for which the block is defined. Also, the scripts are run only if at least one file requires rotation. If any script aborts with an error, no further action is taken.
nosharedscripts The scripts are executed for each log file which is rotated. This is the default. If the script exits with an error, no further processing is done for the concerned log file. Processing proceeds for other log files.

Mail

mail address Mail the log file, which is being deleted, to the given address.
mailfirst Mail the log file which has just been rotated the first time instead of the oldest log file.
maillast Mail the the oldest log file which is being removed. This is the default.
nomail Do not mail log files.

Miscellaneous

missingok It is OK if a log file is missing. Do not generate error messages for the missing log files.
nomissingok If a log file is missing, generate an error message. This is the default.
olddir directory Move the old versions of log file to directory. directory needs to be on the same physical device as the log file and is assumed to be relative to the directory of log file, unless absolute path is given. This results in all the old versions of log file getting placed in the given directory.
noolddir The older rotated log files are kept in the same directory as the original log file.
shred The oldest logfile is deleted by shred -u instead of unlink.
shredcycles count Tells shred to overwrite the file to be deleted count times before removing it.
noshred Do not use shred for deleting the oldest logfile.

logrotate options

The above discussion looked at the various directives in the logrotate configuration file. Now we look at the command line options to the logrotate command. The command syntax is,

logrotate [-d|--debug] [-f|--force] [-m|--mail=command] [-s|--state=statefile] [-v|--verbose] [-?|--help] [--usage] <configfile>

logrotate - Command Line Options
Option Description
-d,
--debug
Work in the debug mode. -d implies the -v option. Actual rotation is not done and no changes are made log files or the state file.
-f, --force Regardless of the information in the state file, force the rotation.
-m|
--mail=
command
Specifies the command to be used for sending mail. This command should accept the Subject and mail address as first two parameters and take message from the standard input. The default command is /usr/bin/mail -s.
-s|
--state=
statefile
Specifies the state file to be used. The default state file is /var/lib/logrotate/status.
-v|
--verbose
Work in the verbose mode. Print diagnostic messages.
-?|--help Prints the detailed help message for using the command.
--usage Prints the command syntax.

The -d, -f and -s options are quite useful in debugging the configuration files (global and those for individual packages), by running logrotate from the command line. The -d option only prints the diagnostic messages but does not rotate the log files. If actual rotation is required along with the diagnostic messages, the -v option should be used.

Software: