Cron jobs are the unsung heroes of unattended system tasks, and automating FTP transfers is a classic use case.
Let’s see a simple transfer happen. We’ll use curl for this example, as it’s widely available and handles FTP well.
#!/bin/bash
# Define variables
LOCAL_FILE="/home/user/data/report.txt"
REMOTE_DIR="/upload/reports"
REMOTE_HOST="ftp.example.com"
REMOTE_USER="myuser"
REMOTE_PASS="mypassword" # In a real scenario, avoid hardcoding passwords!
# Upload the file using curl
curl -T "$LOCAL_FILE" "ftp://$REMOTE_USER:$REMOTE_PASS@$REMOTE_HOST$REMOTE_DIR/"
# Check exit status
if [ $? -eq 0 ]; then
echo "FTP transfer of $LOCAL_FILE to $REMOTE_HOST:$REMOTE_DIR successful."
else
echo "FTP transfer of $LOCAL_FILE to $REMOTE_HOST:$REMOTE_DIR failed."
fi
To schedule this, we’d first make it executable: chmod +x /home/user/scripts/upload_report.sh. Then, we’d add it to cron. Open your crontab with crontab -e and add a line like this to run it every day at 3:00 AM:
0 3 * * * /home/user/scripts/upload_report.sh
This setup lets us push files to a remote server without manual intervention. The curl command, with its -T flag, tells it to upload the specified local file to the target FTP directory. We’re passing credentials directly in the URL, which is convenient but insecure for production.
The power here is in the simplicity of the script and the reliability of cron. Cron wakes up at the scheduled time and executes the command. If the script runs successfully, the file is uploaded. If it fails, the script reports it.
The core problem this solves is repetitive, manual file transfers. Imagine needing to send daily sales reports to a partner or collect logs from remote servers. Doing this manually is tedious and error-prone. Automating it frees up human time and ensures consistency.
Internally, cron is a daemon (crond) that reads a configuration file (the crontab) for each user. It maintains a list of scheduled jobs and, when the system clock matches a job’s schedule, it executes that job by forking a new process and running the specified command.
The levers you control are the schedule itself (the five asterisks in the crontab line) and the script’s logic. You can add more complex logic to the script: check if the local file exists, perform transformations before uploading, or even implement retry mechanisms if the FTP server is temporarily unavailable.
What most people don’t realize is how granular cron scheduling can be. Beyond the common * * * * * (every minute), you can specify ranges (0-5 * * * * for the first 6 minutes of every hour), lists (0,15,30,45 * * * * for every 15 minutes), or even specific days of the week combined with times. For example, to run a job only on weekdays at 7 PM: 0 19 * * 1-5.
The next step in automating data movement often involves more complex orchestration, like ensuring a file is ready to be uploaded before the cron job even runs, or handling bidirectional syncs.