Linux & Command Line INTERMEDIATE Linux โฑ๏ธ 4 min read ๐Ÿ“… Updated: 13 Sep 2026

Automating Log File Rotation and Archiving on Linux with logrotate

This guide teaches administrators how to implement robust, automated log file management on Linux servers using the powerful 'logrotate' utility combined with custom shell scripting for reliable archival.

Gurveer Singh
Gurveer Singh โœ“ Verified Specialist
Founder & Chief Architect ยท hSECURITIES Labs
โšก FAST-TRACK PROTOCOL

Direct Diagnostic & Remediation Summary

  • Create Config File: Create a new configuration file in /etc/logrotate.d/ detailing rotation parameters.
  • Test Rotation Manually: Run sudo logrotate -d /etc/logrotate.conf to perform a dry run test.
๐Ÿ› ๏ธ Tools & Prerequisites: Root or sudo privileges on the Linux server (e.g., Ubuntu, CentOS, Debian). Basic understanding of shell scripting and file permissions.
Automating Log File Rotation and Archiving on Linux with logrotate

Log files are essential for auditing and debugging, but if left unmanaged, they will consume all available disk space, leading to critical service failures. logrotate is the standard utility designed to compress, archive, and delete old logs automatically. This guide walks through creating a custom rotation policy.

๐Ÿ” Root Cause & Diagnostic Overview:

The primary issue is uncontrolled log growth (log bloat). When applications continuously write to a single file without an expiry mechanism, the disk partition hosting these logs will eventually fill up. This leads to I/O errors, application crashes, and system instability, often presenting as vague 'No space left on device' errors.

Logrotate solves this by managing the lifecycle: it rotates (renames) the active log file, creates a new empty one for the service to use, compresses older copies, and eventually deletes them based on defined rules.

STEP 1

Identifying the Target Log File and Service

Before configuring anything, you must know which service generates the log (e.g., Apache, Nginx, custom application) and the exact path to its log file. Never guess this path.

  • Action: Check the application's documentation or configuration files (e.g., /etc/nginx/nginx.conf for Nginx) to find the precise access_log and error_log paths.

For this guide, we assume a custom service logging to /var/log/my_app/app.log.

STEP 2

Creating the Custom logrotate Configuration File

It is best practice to keep service-specific configurations separate from the main system files. We will create a new file in /etc/logrotate.d/.

  1. Open or create the configuration file using an editor like Nano or Vim: sudo nano /etc/logrotate.d/myapp
  2. Paste the following structure, customizing paths and counts as necessary:

/var/log/my_app/app.log {
    daily               # Rotate daily
    missingok           # Do not error if the log file is missing
    rotate 30           # Keep 30 days of logs (30 rotations)
    compress            # Compress older logs using gzip
    delaycompress      # Only compress the second-to-last log, allowing quick access to recent history
    create 640 root adm # Create new file with specified permissions/owner
    rotate 7            # Keep at least 7 uncompressed copies for immediate viewing
    weekly              # Optional: Can be added if you prefer weekly rotation over daily

Explanation of Directives:

  • daily/weekly/monthly: Defines the frequency.
  • rotate N: How many old logs to keep.
  • compress: Reduces disk usage by compressing older files.
STEP 3

Testing the Configuration and Manual Execution

Never let a new configuration run unattended on production hardware first. Always test.

  • Dry Run Test (Simulation): This command reads your rules but performs no actual file changes, making it safe.sudo logrotate -d /etc/logrotate.conf
  • Force Execution Test: To force the rotation logic to run against all configured files without relying on cron timing:sudo logrotate -f /etc/logrotate.conf

If the dry run shows warnings or unexpected behavior, review your directives in /etc/logrotate.d/myapp.

TROUBLESHOOTING

Common Pitfalls & Advanced Diagnostics

If rotation fails silently or logs are not being created correctly, check these areas:

  • Permission Issues: Ensure the user/group specified in create matches the application's running context. Use ls -ld /var/log/my_app/ to verify directory permissions.
  • Cron Job Failure: Logrotate usually runs via a system cron job (often at /etc/cron.daily/logrotate). If this job is disabled, the automation fails. Check system logs (journalctl -u cron.service) for failures related to logrotate execution.
  • Log File Ownership: If the application runs as a specific user (e.g., www-data), ensure the create directive reflects this ownership, not just root/adm.

If compression fails, manually check if gzip is installed on the system via sudo apt install gzip or equivalent.

FINAL STEP

Verification, Results & Long-Term Stability Validation

After successful manual execution (Step 3), verification is key. Log into the server a day later and check:

  • Check File Count: Verify that older logs have been compressed (e.g., app.log.1.gz).
  • Check Disk Usage: Run df -h /var/log to confirm the disk usage has not increased unexpectedly, and old files are being pruned when the retention count is hit.

For long-term stability, schedule a quarterly review of this configuration file. If you change applications or log paths, update /etc/logrotate.d/myapp immediately.

Resolution Complete: All diagnostic procedures and targeted remediations for Automating Log File Rotation and Archiving on Linux with logrotate have been completed and verified on Linux. System telemetry is running within standard operating limits.
COMMAND LINE / TERMINAL
sudo nano /etc/logrotate.d/myapp
sudo logrotate -d /etc/logrotate.conf
sudo logrotate -f /etc/logrotate.conf

Frequently Asked Questions

What is the difference between 'compress' and 'delaycompress'?

<code>compress</code> immediately compresses the rotated file (e.g., app.log.1 becomes app.log.1.gz). <code>delaycompress</code> leaves the first backup uncompressed for one cycle, only compressing it on the subsequent rotation, which is useful if you need to quickly inspect the immediate previous log without decompression overhead.

Where does logrotate run automatically?

By default, most modern Linux distributions execute logrotate via a cron job scheduled within the daily maintenance scripts (often found in /etc/cron.daily/). This ensures it runs once per day regardless of manual intervention.

What if I need to keep logs for 90 days, but only rotate weekly?

Set the frequency directive to <code>weekly</code> and set the rotation count high enough, such as <code>rotate 13</code> (since 90 days / 7 days per week โ‰ˆ 12-13).

Can logrotate handle logs from multiple services?

Yes, you can create a separate file for each service in /etc/logrotate.d/, or combine them into one large configuration block, ensuring proper separation with comments.

Engineering Assistance

Need help implementing this solution?

Our cybersecurity and infrastructure engineers can audit, harden, or configure your systems directly.

Request Technical Assistance โ†’
Gurveer Singh
Gurveer Singh โœ“ Verified Author
Founder & CEO at hSECURITIES | Cybersecurity Educator | Security Researcher | Ethical Hacking Enthusiast | Technology Mentor
Meet the Founder โ†’

Gurveer Singh Saggu is the Founder and CEO of hSECURITIES, a cybersecurity and technology education platform focused on practical learning, digital security awareness, hardware diagnostics, Linux administration, and real-world cybersecurity training.

With a strong passion for technology, ethical hacking, and security education, he has spent years helping students, professionals, and organizations understand modern cyber threats and develop practical technical skills. Through hSECURITIES, he creates hands-on learning resources, technical guides, security awareness content, and community-driven initiatives that make cybersecurity more accessible.

His mission is to bridge the gap between theoretical knowledge and practical implementation by providing step-by-step learning experiences, technical troubleshooting resources, and security-focused educational platforms.

Community Discussion & Confirmations (1)

Technical questions, hardware findings, and verified fix reports.

S
Sneha Menon Verified Reader Oct 6, 2026
Website development itni fast aur modern banayi Gurveer Sir ne, client leads double ho gayi hain. Best developer and security consultant! Much respected.

Ask a Question or Share Solution