Building a Folder Sync with lftp and cron

  • Posted on: 4 May 2014
  • By: Administrator

The goal of this post is to track my progress in creating a sync between an FTP site and my home server.  The Server is running CentOS 6.5 minimal with apache2, PHP, mysql to host my test websites.  I also have a Netgear router running readyshare for an anonomoys FTP server for students to upload video files for me.  I want to make sure that I get any files put on the FTP onto a physical drive in the house so I'm creating a cron job to check the FTP folder every 5 minutes and sync with a local folder on the server.  

First I yum install lftp software:

[jimusik@asrock ~]$ sudo yum install lftp
...truncated...
  Installing : gnutls-2.8.5-13.el6_5.x86_64                                                                     1/2 
  Installing : lftp-4.0.9-1.el6_5.1.x86_64                                                                      2/2 
  Verifying  : lftp-4.0.9-1.el6_5.1.x86_64                                                                      1/2 
  Verifying  : gnutls-2.8.5-13.el6_5.x86_64                                                                     2/2 
Installed:
  lftp.x86_64 0:4.0.9-1.el6_5.1                                                                                    
Dependency Installed:
  gnutls.x86_64 0:2.8.5-13.el6_5                                                                                    
Complete!

 Then I created a folder under my home folder to store scripts and created a script.

[jimusik@asrock ~]$ mkdir scripts
[jimusik@asrock scripts]$ sudo nano syncftp.sh

Then I create a file with the following instructions based on This Website:

#!/bin/bash
login="anonymous"
pass=""
host="<link to the FTP>"
remote_dir="/shares/USB_Storage"
local_dir="/home/jimusik/testfiles"
trap "rm -f /tmp/synctorrent.lock" SIGINT SIGTERM
if [ -e /tmp/synctorrent.lock ]
then
echo "Synctorrent is running already."
exit 1
else
touch /tmp/synctorrent.lock
lftp -u $login,$pass $host << EOF
set ftp:ssl-allow no
set mirror:use-pget-n 5
mirror -c -P5 --log=synctorrents.log $remote_dir $local_dir
  quit
EOF
rm -f /tmp/synctorrent.lock
trap - SIGINT SIGTERM
exit 0
fi

Then I made it executable:

 [jimusik@asrock scripts]$ sudo chmod 700 syncftp.sh

Then I tested it:

 [jimusik@asrock scripts]$ sudo ./syncftp.sh

The inital run ran for some time but copied the whole ftp site properly.  Next, I setup a cron job by switching to root, moving to the cron folder in var/spool and creating a cron job named cronsyn.cron:

[jimusik@asrock scripts]$ sudo su
[root@asrock scripts]# cd /var/spool/cron
[root@asrock cron]# nano cronsync.cron

The cron file has the following:

 */5 * * * * /home/jimusik/scripts/syncftp.sh >> /home/jimusik/sync_cron.log 2>&1

This runs the above script every 5 minutes syncing the ftp site with the proper folder.  

Once created you must link the file to the user:

[root@asrock cron]# crontab -u root cronsync.cron

To make sure it was implemnted I ran:

[root@asrock cron]# crontab -l
*/5 * * * * /home/jimusik/scripts/syncftp.sh >> /home/jimusik/sync_cron.log 2>&1

After testing, any file put on the FTP server for 5 minutes showsup on the synced folder.