Backtory Revived

Posted by Jesco Freund at Jan. 23, 2010 9:43 p.m.

Almost one year ago, I started a project named “Backtory”. Its aim was to create a backup solution suitable for my internet servers. However, the whole idea was much too bloated, and my attention to the project wasn't as steady as intended. So for quite a long time, I didn't write a single line of code. Some weeks ago (at Christmas, to be precise) I moved the project to Google Code. After some deeper looks at what I had done so far, I decided to throw away all the old stuff and to wipe the slate clean.

The idea of the reborn Backtory project is a CLI application instead of a heavy client server infrastructure. The requirements however remained nearly untouched:

• Create differential (not incremental!) backups
• Encrypt and sign backup archives
• Store backup data remotely (at least FTP support is required)
• Make use of ZFS (and optionally LVM2) snapshot capabilities
• Generate procedures (i. e. POSIX shell scripts) for a bare metal recovery
• Support pre- and post-backup actions
• Support point-in-time recovery

Maybe this requirement catalogue illustrates why I started a new project instead of using one of the existing solutions. On the one hand, big solutions like Bacula or Amanda are too bloated (and too much focused on LAN setups for my taste). On the other hand, smaller solutions (mostly shell or Perl scripts) do not meet my requirements.

Duplicity is what I'd call a close miss, but its fixation on incremental backups and its lacking support for snapshot operations would have meant at least a 50% reimplementation (plus the pain of having to endure the restrictions of GPLv2). Furthermore, I'm not sure about using librsync for my implementation. Its development seems to have ceded some years ago, so no wonder some of its implementation details are no longer state of the art (e. g. using MD4 to verify the equity of file chunks).

So what's the plan to go ahead? Well, so far I wrote some Python extensions to build interfaces towards OpenSSL's RIPEMD-160 implementation, the POSIX lstat() function, VMAC (which could become an MD4 replacement in my implementation of the rsync algorithm) and ZFS (works for FreeBSD so far, can create snapshots and destroy them). The next steps will be to implement further Python extensions (I still need interfaces to OpenSSL's enc, genrsa and rsautl for signing and encrypting the archives) and to make my ZFS extension work on Solaris and OpenSolaris. After this, I hopefully will be able to start implementing the full backup functionality; differential backups (including the still-missing rsync algorithm) and the restore functions will follow.

As soon as I will have implemented FTP remote storage functionality, I'll be thinking about a first release. LVM2 support and bare metal recovery are not so urgent for me, so maybe I'll skip them for the first release and implement them later. And of course I'll keep you informed by blogging about Backtory from time to time…

No comments | Defined tags for this entry: backtory

Comments

No comments