Normally, there's an ad running in this spot, but you're using an ad blocker. To help support our blog, which provides free tutorials for everybody, we just ask that you whitelist or follow us on facebook, twitter, or subscribe using the form to the leftabove. Thank you!

    Log Rotation with Logrotate

    Log rotation, an automated archiving process, is designed to alleviate the maintenance of large numbers of log files on any given server. On Linux, this process is carried out by a utility called logrotate, and allows for the automatic rotation, compression, deletion, and transmission of logs. Parameters may be set to ensure that logs are processed either periodically or once a threshold has been reached.

    It is absolutely critical that you rotate your logs or you run the risk of running out of disk space. It will also keep you from having to swim through millions of log entries just to track down a particular event.

    System requirements

    • A 32-bit or 64-bit computer with at least 8GB of RAM. 16GB or more is recommended.
    • At least 50GB of free disk space.

    Prerequisites

    1. Prior to setting up Linux Ubuntu, you'll first need either a virtual machine or a physical or cloud-based server.

    Logrotate Configuration

    Most systems will already have a daily log rotation script at either /etc/cron.daily/logrotate or /etc/cron.daily/logrotate.cron. You'll notice there are also designations for hourly, weekly, and monthly in the same location. All of these scripts start by reading the configuration file at /etc/logrotate.conf. Take the time to scan through this file, as it's well commented and fairly straightforward.

    # see "man logrotate" for details
    # rotate log files weekly
    weekly
    
    # use the syslog group by default, since this is the owning group
    # of /var/log/syslog.
    su root syslog
    
    # keep 4 weeks worth of backlogs
    rotate 4
    
    # create new (empty) log files after rotating old ones
    create
    
    # uncomment this if you want your log files compressed
    #compress
    
    # packages drop log rotation information into this directory
    include /etc/logrotate.d
    
    # no packages own wtmp, or btmp -- we'll rotate them here
    /var/log/wtmp {
        missingok
        monthly
        create 0664 root utmp
        rotate 1
    }
    
    /var/log/btmp {
        missingok
        monthly
        create 0660 root utmp
        rotate 1
    }
    # system-specific logs may be configured here
    

    So in this configuration file, you'll notice we're rotating our logs weekly, but holding onto a month of backlogs. By rotation, we mean to log line-by-line entries to the same file for up to a week, then backlog it and create a new file.

    Compression is off by default. We tell logrotate to store specific logs in /etc/logrotate.d. So Nginx would store its log configuration in /etc/logrotate.d/nginx. Finally, we tell logrotate what to do with wtmp and btmp logs. These are just classifications of utmp files that keep track of logins and logouts to the system. utmp tracks the current login state per user, wtmp records login and logout history, and btmp records failed login attempts.

    Now, let's move onto the logrotate.d files. Run ls /etc/logrotate.d to get a list of all the specific package logs that go through rotation.

    Understanding Logrotate Log Files

    Here are the contents of /etc/logrotate.d/nginx. In the following sections, we'll be breaking this down, line by line.

    /var/log/nginx/*.log {
      weekly
      missingok
      rotate 52
      compress
      delaycompress
      notifempty
      create 0640 www-data adm
      sharedscripts
      prerotate
        if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
          run-parts /etc/logrotate.d/httpd-prerotate; \
        fi \
      endscript
      postrotate
        [ -s /run/nginx.pid ] && kill -USR1 `cat /run/nginx.pid`
      endscript
    }
    

    Log file paths

    This is indicated by the top level block, which gives logrotate a wildcard(*) or space-separated values to target specific logs with this configuration.

    /var/log/nginx/*.log /var/log/nginx/log.txt {
    ...
    }
    

    Other parameters

    You then have the ability to indicate rotation interval, rotation count, and compression.

    weekly # rotate weekly
    missingok # move onto next log file without error message if log file is missing
    rotate 52 # delete after 52 log files have been created
    compress # compress after rotation...
    delaycompress # ...but compress on the next rotation cycle
    notifempty # don't rotate empty logs
    create 0640 www-data adm # file creation and permissions
    

    Prerotate, Postrotate and Shared Scripts

    prerotate designates scripts to be run before rotating the file, postrotate runs after, and sharedscripts ensures that the script only runs once rather than every rotated log file.

    sharedscripts
    prerotate
      if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
        run-parts /etc/logrotate.d/httpd-prerotate; \
      fi \
    endscript
    postrotate
      [ -s /run/nginx.pid ] && kill -USR1 `cat /run/nginx.pid`
    endscript
    

    You should have a good overview of logrotate at this point. Make sure to also check out this article on Nginx error logging, as well.

    Did you like this tutorial? Help us pay for server costs by following us on Facebook, Twitter, and subscribing below, where you'll get post notifications, training webinar invites, and free bundles.