Synctory Stepping towards Release

Posted by Jesco Freund at June 19, 2011 8:36 p.m.

Synctory (or libsynctory) is the project I'm currently spending most of my leisure time on. Basically, it implements algorightms similar to librsync. Originally, I developed libsynctory as part of a backup software project, but later decided to continue it as a separate library. The first code was written during my last summer holiday in Denmark, and since then, a lot (of the code) has changed.

Now, after almost one year of rather less steady development, libsynctory is stepping towards release maturity. Today I finally managed to commit a performance test to evaluate how fast libsynctory actually works. Though I immediately discovered a memory leak bug which is not resolved yet, the performance seems to be somewhat satisfactory for a first approach (average runtime in seconds):

+-------------+---------+---------+---------+---------+
|             | small   | big     | huge    | giant   |
+-------------+---------+---------+---------+---------+
| fingerprint |    0.01 |    6.67 |   33.94 |   67.94 |
+-------------+---------+---------+---------+---------+
| diff        |    0.04 |   31.99 |  144.99 |  303.08 |
+-------------+---------+---------+---------+---------+
| synth       |    0.02 |   14.11 |   81.01 |  152.82 |
+-------------+---------+---------+---------+---------+

The test was taken by a constant chunk size (512 bytes) for all four file sizes (0.5 MiB, 0.5 GiB, 2.5 GiB, 5.0 GiB). For me, the results illustrate two main conclusions: First, the algorithms of libsynctory seem to scale linearly with the file size (however I would have been surprised if they didn't), and secondly, bigger files require larger chunk sizes to allow a runtime-friendly processing. Therefore, I think the decision to allow individual chunk size adjustmens was a good one.

The way forward seems to be clear by now – some bugs to fix, some man pages to write, and then I'll hopefully be able to produce a first release of libsynctory. As (almost) always in open source projects, any help is appreciated, particularly I am looking for testers on Linux and OpenSolaris. So if you have some spare time, don't hesitate to contact me!

No comments | Defined tags for this entry: development, open source, synctory

Comments

No comments