Logrotate: The Art of the Clean Exit

Imagine you have a high-traffic web server. Every time someone visits, a line is written to access.log. After a few months, that file could be 50 Gigabytes. If your server only has a 60GB hard drive, the system will crash.

logrotate is the background janitor that prevents this.

It automatically takes the big access.log, renames it to access.log.1, compresses it to save space, and eventually deletes it once it gets too old. In this lesson, we will learn to configure rotation policies that balance "Saving Space" with "Keeping History."

---

2. The Rotation Cycle

How logrotate works:

1. Today: access.log (Current data).
2. Tomorrow: access.log.1 (Yesterday's data).
3. Next Week: access.log.2.gz (Compressed data from 2 weeks ago).
4. Conclusion: Eventually, the oldest file is deleted permanently.

---

3. Mastering the Configuration

The main config is in /etc/logrotate.conf, but you should put your custom app rules in /etc/logrotate.d/.

A standard rule looks like this:

/var/log/myapp/*.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 0640 myapp_user myapp_group
    postrotate
        systemctl reload myapp
    endscript
}

Key Directives:

• daily / weekly / monthly: How often to rotate.
• rotate 7: Keep 7 old files. On the 8th day, the oldest is deleted.
• compress: Turn old logs into .gz files (reduces size by 90%!).
• copytruncate: Use this if your app is "Stingy" and won't let go of the log file. It copies the content and then empties the original file without closing it.
• postrotate: A script to run after rotating (usually used to tell the app: "Hey, I changed the file, start writing to the new one!").

---

4. Practical: The "Stuck" Janitor

logrotate is usually run once a day by a Cron job. If you just added a new rule and don't want to wait until midnight to see if it works, you can force it.

# Force a rotation now and show what is happening (verbose)
sudo logrotate -vf /etc/logrotate.d/myapp

---

5. Troubleshooting: Why didn't it rotate?

If a log file is massive but hasn't rotated:

1. Check the size trigger: Did you use size 1G? If the file is only 900MB, it won't rotate yet.
2. Check the status file: /var/lib/logrotate/status records the last time every file was touched.
3. Check permissions: If logrotate (running as root) cannot write to the folder where the log is, it will silently fail.

---

6. Example: A Log Storage Predictor (Python)

If you know your log grows by 100MB a day and you have 10GB of space, how many days of history can you keep? Here is a Python script that helps you calculate a safe rotate number for your configuration.

import os
import math

def calculate_retention(log_path, partition_free_gb, daily_mb_estimate):
    """
    Suggests a safe rotation count based on available space.
    """
    # Assuming compress reduces logs to 10% of their size
    compressed_mb = daily_mb_estimate * 0.1
    
    available_mb = partition_free_gb * 1024
    
    # Let's say we only want to use 20% of free space for logs
    budget_mb = available_mb * 0.2
    
    # Days = Budget / Daily compressed size
    safe_days = math.floor(budget_mb / compressed_mb)
    
    print(f"--- Log Retention Strategy ---")
    print(f"Est. Daily Growth: {daily_mb_estimate} MB")
    print(f"Retaining {safe_days} days of history will use ~{budget_mb/1024:.2f} GB.")
    print(f"\nRecommended Config: 'rotate {safe_days}'")

if __name__ == "__main__":
    # Example: 200MB daily growth, 50GB free space
    calculate_retention("/var/log/nginx", 50, 200)

---

7. Professional Tip: Use 'size' for Aggressive Apps

Don't rely on daily for apps that produce random bursts of data. Use the size directive. This tells logrotate: "I don't care if it's 2 PM on a Tuesday—if this file reaches 1GB, rotate it NOW."

---

8. Summary

logrotate is the survival mechanism for your server's storage.

• daily and rotate define your retention policy.
• compress is non-negotiable for large logs.
• postrotate prevents "Split Logs" where half the data goes to an old file.
• copytruncate is the fallback for poorly written applications.

In the next lesson, we will move from looking at the past (logs) to looking at the present: Visualizing Performance with Prometheus and Grafana.

Quiz Questions

1. What is the difference between rotate 5 and rotate 0?
2. Why is the delaycompress option often used for service logs?
3. How do you verify your logrotate configuration without actually deleting or moving any files?

---

Continue to Lesson 5: Visualization—Prometheus and Grafana (Intro).