Your cron jobs were supposed to run overnight, but when you check the logs, nothing happened. The backup didn't execute, the cleanup script didn't run, and now you need to figure out why.
Understanding Cron
Cron is a time-based job scheduler in Unix-like operating systems. Users schedule jobs (commands or scripts) to run periodically at fixed times, dates, or intervals. The daemon cron reads crontab files and executes commands at the specified times.
Typical Symptoms
- Scripts run manually but not via cron
- No output or logs from cron jobs
- Cron daemon not running
- Jobs running at wrong times
- Permission denied errors
- Environment variables missing
Diagnosing Cron Issues
Step 1: Verify Cron Daemon is Running
```bash # Check if cron service is running systemctl status cron # Debian/Ubuntu systemctl status crond # RHEL/CentOS/Fedora
# Check running process ps aux | grep cron pgrep -a cron
# Start cron if not running systemctl start cron # Debian/Ubuntu systemctl start crond # RHEL/CentOS/Fedora
# Enable cron to start at boot systemctl enable cron ```
Step 2: Check Cron Logs
```bash # Check system logs for cron messages grep CRON /var/log/syslog # Debian/Ubuntu grep CROND /var/log/cron # RHEL/CentOS
# View recent cron logs journalctl -u cron -n 50 # Debian/Ubuntu journalctl -u crond -n 50 # RHEL/CentOS
# Follow cron logs in real-time journalctl -u cron -f
# Check for cron execution messages grep "CMD" /var/log/syslog ```
Step 3: Check Crontab Syntax
```bash # View current user's crontab crontab -l
# View root's crontab sudo crontab -l
# View system-wide cron jobs ls -la /etc/crontab ls -la /etc/cron.d/
# Edit crontab crontab -e
# Crontab syntax reference: # ┌───────────── minute (0 - 59) # │ ┌───────────── hour (0 - 23) # │ │ ┌───────────── day of month (1 - 31) # │ │ │ ┌───────────── month (1 - 12) # │ │ │ │ ┌───────────── day of week (0 - 6) (Sunday = 0) # │ │ │ │ │ # * * * * * command
# Common patterns: # 0 * * * * - Every hour at minute 0 # 0 2 * * * - Every day at 2:00 AM # 0 0 * * 0 - Every Sunday at midnight # 0 0 1 * * - First day of every month # */15 * * * * - Every 15 minutes # 0 9-17 * * 1-5 - Hourly from 9-5, Mon-Fri ```
Step 4: Check for Syntax Errors
```bash # Validate crontab syntax before saving # When using crontab -e, cron will report errors on save
# For system crontabs, validate manually # Check for common syntax errors: # - Missing newline at end of file # - Wrong minute/hour values (e.g., 60 for minutes) # - Using spaces instead of tabs (tabs preferred in /etc/crontab) # - Missing username in /etc/crontab entries
# Common error - forgetting to add newline at end # Ensure there's a blank line after the last entry
# Check crontab ownership ls -la /var/spool/cron/crontabs/ # Debian/Ubuntu ls -la /var/spool/cron/ # RHEL/CentOS ```
Common Issues and Solutions
Issue 1: Script Works Manually But Not in Cron
This is the most common cron problem. The issue is almost always the environment.
```bash # Cron jobs run with a minimal environment # Check what environment cron sees * * * * * env > /tmp/cron-env.txt
# Compare with your shell environment env > /tmp/shell-env.txt diff /tmp/shell-env.txt /tmp/cron-env.txt
# Key differences: # - PATH is minimal (/usr/bin:/bin) # - HOME is set to user's home # - SHELL is /bin/sh (not bash) # - No user-defined environment variables ```
Solution: Set the PATH in your crontab or script
```bash # Option 1: Set PATH at top of crontab PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
# Then your jobs 0 2 * * * /path/to/script.sh
# Option 2: Use full paths in commands 0 2 * * * /usr/bin/python3 /home/user/script.py
# Option 3: Source environment in script # Add to your script: source /home/user/.bashrc # or source /home/user/.profile
# Option 4: Set environment in crontab entry 0 2 * * * . $HOME/.profile; /path/to/script.sh ```
Issue 2: Permission Denied
```bash # Check script permissions ls -la /path/to/script.sh
# Make script executable chmod +x /path/to/script.sh
# Check ownership chown user:user /path/to/script.sh
# Verify cron user can access the path sudo -u user ls -la /path/to/script.sh
# Check directory permissions along the path namei -l /path/to/script.sh ```
Issue 3: No Output Captured
Cron by default mails output to the user. If mail isn't configured, output is lost.
```bash # Redirect output to log file 0 2 * * * /path/to/script.sh >> /var/log/script.log 2>&1
# Redirect to syslog 0 2 * * * /path/to/script.sh 2>&1 | logger -t myscript
# Redirect both stdout and stderr 0 2 * * * /path/to/script.sh >> /var/log/script.log 2>&1
# Discard all output (not recommended) 0 2 * * * /path/to/script.sh > /dev/null 2>&1
# Check if mail is configured which mail dpkg -l mailutils # Debian/Ubuntu rpm -qa | grep mailx # RHEL/CentOS ```
Issue 4: Wrong Timing
```bash # Check system timezone timedatectl date cat /etc/timezone # Debian/Ubuntu
# Cron uses the system timezone # For system crontab (/etc/crontab), check the CRON_TZ variable grep CRON_TZ /etc/crontab
# Set timezone for user crontab CRON_TZ=America/New_York 0 2 * * * /path/to/script.sh
# For specific timezone in user crontab (not all cron versions support this) # Instead, calculate the UTC time for your desired local time ```
Issue 5: Cron Job Running as Wrong User
```bash # User crontabs are stored in: /var/spool/cron/crontabs/username # Debian/Ubuntu /var/spool/cron/username # RHEL/CentOS
# Edit crontab as specific user crontab -u username -e
# For root jobs sudo crontab -e
# System-wide cron jobs (/etc/crontab) require username # /etc/crontab format: # m h dom mon dow user command 0 2 * * * root /path/to/script.sh
# Files in /etc/cron.d/ also require username # /etc/cron.d/myjob: 0 2 * * * user /path/to/script.sh ```
Issue 6: Script Using Relative Paths
```bash # Cron jobs run in user's home directory # A script using relative paths may fail
# Bad: Uses relative path * * * * * ./script.sh # Won't work * * * * * cd project && ./run.sh # May not work as expected
# Good: Use absolute paths * * * * * /home/user/project/run.sh
# Or cd first, then run * * * * * cd /home/user/project && ./run.sh
# In your script, set working directory #!/bin/bash cd /home/user/project || exit 1 # rest of script ```
Issue 7: Special Characters Not Escaped
```bash # Percent signs (%) are newlines in cron # Escape them with backslash
# Bad: % will be treated as newline 0 2 * * * date +%Y-%m-%d
# Good: Escape the % 0 2 * * * date +\%Y-\%m-\%d
# This applies to all % characters in the command 0 2 * * * echo "Progress: 50\%" >> /var/log/output.log ```
Issue 8: Cron Allow/Deny Restrictions
```bash # Check for cron access restrictions cat /etc/cron.allow # If exists, only listed users can use cron cat /etc/cron.deny # If exists, listed users cannot use cron
# If cron.allow exists, user must be in it echo "username" | sudo tee -a /etc/cron.allow
# If only cron.deny exists, ensure user is not in it grep username /etc/cron.deny && sudo sed -i '/username/d' /etc/cron.deny
# Permissions for cron directories ls -la /etc/cron.d ls -la /etc/cron.daily # Directories must not be writable by non-root for security ```
Testing Cron Jobs
Quick Testing Method
```bash # For quick testing, schedule job 2 minutes in the future # Get current time date
# Add 2 minutes and create entry # If it's 14:30, schedule for 14:32 32 14 * * * /path/to/test-script.sh >> /tmp/cron-test.log 2>&1
# Watch for output tail -f /tmp/cron-test.log ```
Test with Minimal Script
```bash # Create a simple test script cat > /tmp/cron-test.sh << 'EOF' #!/bin/bash echo "Cron test started at $(date)" echo "Current user: $(whoami)" echo "Current directory: $(pwd)" echo "PATH: $PATH" echo "Environment:" env echo "Cron test completed" EOF
chmod +x /tmp/cron-test.sh
# Add to crontab for testing (runs every minute) * * * * * /tmp/cron-test.sh >> /tmp/cron-test-output.log 2>&1
# Watch the output tail -f /tmp/cron-test-output.log ```
Special Cron Directories
System cron directories for periodic tasks:
```bash # These directories contain scripts that run at specified intervals /etc/cron.daily/ # Runs daily (usually around 6:25 AM) /etc/cron.hourly/ # Runs hourly /etc/cron.weekly/ # Runs weekly /etc/cron.monthly/ # Runs monthly
# Scripts in these directories must be executable chmod +x /etc/cron.daily/my-script
# And must not have dots in the filename (run-parts skips them) # Bad: my-script.sh # Good: my-script mv /etc/cron.daily/my-script.sh /etc/cron.daily/my-script ```
Debugging Techniques
```bash # Run the cron command exactly as cron would # Use the minimal environment env -i HOME=$HOME SHELL=/bin/sh PATH=/usr/bin:/bin /path/to/script.sh
# Run with the same shell cron uses /bin/sh -c "/path/to/script.sh"
# Check if the command exists which python which python3
# Test the full command from crontab /usr/bin/python3 /home/user/script.py >> /tmp/test.log 2>&1 ```
Verification
After fixing issues, verify:
```bash # Check cron is running systemctl status cron
# Verify crontab syntax crontab -l
# Watch cron logs journalctl -u cron -f
# Check your job ran grep CRON /var/log/syslog | grep your-script
# Verify output cat /var/log/your-script.log ```
Cron Best Practices
```bash # 1. Always use absolute paths 0 2 * * * /usr/bin/rsync -av /src/ /dest/
# 2. Redirect output 0 2 * * * /path/to/script.sh >> /var/log/script.log 2>&1
# 3. Set PATH in crontab PATH=/usr/local/bin:/usr/bin:/bin 0 2 * * * script.sh
# 4. Use lock files to prevent overlapping runs 0 2 * * * flock -n /var/lock/myjob.lock /path/to/script.sh
# 5. Add comments to your crontab # Daily backup at 2 AM 0 2 * * * /usr/local/bin/backup.sh
# 6. Include error handling in scripts #!/bin/bash set -euo pipefail # Script continues only if all commands succeed
# 7. Test jobs manually first /path/to/script.sh ```