Hi!

Long time since my last post! I want to say happy new year to everyone who visits my blog and I hope all your wishes come true in this year. I have been very busy with my start-up company the lasts months (I even had to work in my 3 days annual vacation ) so that’s why I didn’t upload any new article. So today I will show you a little script that will upload your Virtualmin backups to S3. I have a customer that have backups larger than 5Gb and I can’t upload them directly to S3 since the limit per file in S3 is 5 Gbs so I used the split command to have 1Gb pieces of the backup and then upload all to S3. To use this script you have to configure your Virtualmin to backup to a local folder (In this case /bkps) and then have a daily script that can be placed in /etc/cron.daily for example with the following content:

#!/bin/bash

# Clean backup folder for old files
echo “Cleaning old backups…”
/usr/bin/find /bkps -type f -mtime +5 -print
/usr/bin/find /bkps -type f -mtime +5 -exec rm -f {} \;

for bkpfile in `ls -1 /bkps/*.bkp`
do
echo “Generating MD5SUM for the original backup file $bkpfile…”
md5sum $bkpfile >> $bkpfile-md5sums
echo “Splitting backup file $bkpfile…”
split -a 1 -d -b 1073741824 –verbose $bkpfile $bkpfile-part
for splitbkpfile in `ls -1 $bkpfile-part*`
do
echo “Generating MD5SUM for split backup file $splitbkpfile…”
md5sum $splitbkpfile >> $bkpfile-md5sums
done
rm $bkpfile
done

# Upload backups to S3
su -l -s /bin/bash -c “s3cmd –delete-removed -v sync /bkps s3://YOURBUCKET”

As you can see above, my backups are created with the extension .bkp also I have used the s3cmd to have access to S3. The installation of s3cmd is very easy and it haves packages for almost all the Linux distributions.

I hope you enjoy this script and later I will share with you some of my experiences with AWS cloud services for a large project I worked the lasts months including Load Balancing, Auto Scaler, etc. I have created some nice scripts (I think) to make my life with the AWS, so stay tuned! (I promise that you will not have to wait 2 months to have news from me )

Thank you very much for your time.

Cheers!