Automate Your File Cleanup with a Python Script

Introduction:
Maintaining clean file directories is crucial for developers, IT admins, and anyone managing large volumes of data. Over time, logs, temporary files, and large media assets can clutter your system, wasting storage and slowing operations. In this post, we’ll create a Python script that automatically scans directories and deletes files based on age or size thresholds, using the os and datetime modules. By the end, you’ll have a fully functional cleanup automation tool.

1. Setting Up the Project

Before diving into code, make sure you have Python installed (version 3.6 or higher is recommended). We’ll work with built-in modules, so there’s no need for external packages.

Let’s start by importing the necessary modules and defining our constants:

import os
import time
from datetime import datetime, timedelta

# Directory to scan
directory = '/path/to/cleanup'
# Delete files older than N days
max_age_days = 30
# Delete files larger than N megabytes
max_size_mb = 100

Here, max_age_days sets a threshold for file age, and max_size_mb represents the size limit. The script will target any file that meets either condition. This configuration-based approach allows flexibility and reuse across different cleanup tasks.

2. Scanning and Gathering File Metadata

We’ll need to inspect every file in the given directory (including subdirectories). The os.walk() method is ideal for recursive traversal. For each file, we’ll collect information about its modification time and size.

def get_file_details(base_dir):
    files = []
    for root, dirs, filenames in os.walk(base_dir):
        for name in filenames:
            path = os.path.join(root, name)
            try:
                last_modified = os.path.getmtime(path)
                size_bytes = os.path.getsize(path)
                files.append({
                    'path': path,
                    'last_modified': last_modified,
                    'size_bytes': size_bytes
                })
            except PermissionError:
                print(f'Skipping {path} (permission denied)')
    return files

This function includes a safe handling mechanism for potential PermissionError exceptions, which are common when traversing system or restricted directories. Using this defensive design ensures your automation won’t fail mid-scan.

3. Filtering Files for Deletion

Now that we have metadata for each file, we’ll determine which ones to delete. We’ll filter based on both last modification date and size.

def filter_files(files, max_age_days, max_size_mb):
    cutoff_date = time.time() - (max_age_days * 86400)  # Convert days to seconds
    max_size_bytes = max_size_mb * 1024 * 1024

    to_delete = []
    for f in files:
        if f['last_modified'] < cutoff_date or f['size_bytes'] > max_size_bytes:
            to_delete.append(f['path'])
    return to_delete

Here, we calculate a cutoff_date representing the timestamp for our age limit. Any file with a modification time older than this value—or exceeding the max size—is marked for deletion. Combining both conditions allows flexible cleanup based on multiple criteria.

4. Deleting the Files Safely

Now we can implement the deletion logic. It’s best practice to preview the deletion list before removing anything, as file removal is irreversible. Let’s add a confirmation prompt for user safety.

def delete_files(file_paths):
    print(f"Preparing to delete {len(file_paths)} files...")
    confirm = input("Proceed? (y/n): ")
    if confirm.lower() != 'y':
        print("Operation canceled.")
        return

    for path in file_paths:
        try:
            os.remove(path)
            print(f"Deleted: {path}")
        except Exception as e:
            print(f"Failed to delete {path}: {e}")

This version provides safety and reliability. In production automation scenarios, you could log deletions to a file instead of printing, or use environment-based prompts (i.e., skip interactivity when running via cron or CI/CD).

5. Putting It All Together with Automation

Now we can orchestrate the process and execute the cleanup automatically. Here’s the final runnable script:

if __name__ == '__main__':
    files = get_file_details(directory)
    files_to_delete = filter_files(files, max_age_days, max_size_mb)

    if not files_to_delete:
        print("No files meet the cleanup criteria.")
    else:
        delete_files(files_to_delete)

Running this script regularly (via task scheduler or cron job) ensures continuous cleanup. For instance, you could run it daily using a simple cron entry like:

0 3 * * * /usr/bin/python3 /path/to/cleanup_script.py

This will execute the cleanup script every day at 3 AM, keeping your directories optimized and clutter-free.

6. Optimization Tips

When dealing with massive directories, you can improve performance by batching deletions or running checks asynchronously using concurrent.futures. Additionally, consider integrating logging and email notifications to track activity.

import logging
logging.basicConfig(filename='cleanup.log', level=logging.INFO)

# Example log usage in delete_files():
logging.info(f'Deleted {path} at {datetime.now()}')

This adds traceability and allows easy debugging in long-term automation setups. Another optimization is skipping specific file types (e.g., .config, .env) or directories using conditional checks inside get_file_details().

Final Thoughts:
Automating file cleanup not only keeps your systems lean and efficient but also reduces manual maintenance overhead. This script serves as a foundation for more complex system hygiene automations—like rotating logs, archiving old backups, or clearing temporary cache directories. With a few enhancements, it can scale to enterprise-level maintenance use cases.

Useful links: