Backups are only there for when everything else fails and you need to be able to go back in time. Backing up everything is often problematic, because it requires several times more storage than you need to just run the application.
With snapshots, we have the means to easily and very quickly take a snapshot of a volume. You can later restore this snapshot to another volume. Snapshots are incremental, not at the file level, but at the block level. This means you don’t need 100 times as much storage for 100 backups—you probably need just a couple of times the size of your volume. However, this is difficult to verify, because we have never found out where to see how much storage we use for our snapshots.
To finish our backup/restore mechanism, we need a way to clean up the old snapshots. If we can take snapshots for which we can set the expiration date to one week or one month in the future, we have enough. We created a couple of scripts that use SimpleDB for snapshot administration, and the EC2 command-line utilities to take and delete snapshots.
For our backup mechanism, we use SimpleDB to administer expiration
dates. And we want our solution to be no more than two scripts, one for
taking a snapshot and one for expiring snapshots. In Linux, there are some
powerful tools like date that we use to calculate
dates. Use the command
man date if you want to know about calculating dates. Furthermore, we need the EC2 command-line tools and a Perl command-line tool for ...