Jeff Abrahamson on 2 Aug 2006 20:14:42 -0000


[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

Re: [PLUG] Large data transfer


On Sat, Jul 29, 2006 at 09:03:27PM -0400, Aaron Mulder wrote:
> Also, as a side note, the one time I ever had a real problem with
> rsync was when transferring a file larger than the available drive
> space.  It took forever to update an existing version of the file --
> way longer than to copy it in the first place -- and then crapped
> out after a couple hours claiming that it was out of disk space.  So
> I guess it tried to create a whole separate copy of the destination
> file, and couldn't manage to do the free space calculation at the
> beginning of the process.  Grrr!

The rsync algorithm does not work in place: it creates a new copy
based on diffs and the old copy.

I saw a paper at Usenix ATC in 2003 about in-place rsync.  Cool graph
theoretic stuff so that things were copied only as they could be to
update the file, probably useful for handhelds with limited memory.
The problem, of course, is that if the transfer dies mid-stream,
you've corrupted the file pending a new rsync, instead of having a
usable but out-of-date file.

-- 
 Jeff

 Jeff Abrahamson  <http://jeff.purple.com/>          +1 215/837-2287
 GPG fingerprint: 1A1A BA95 D082 A558 A276  63C6 16BF 8C4C 0D1D AE4B

Attachment: signature.asc
Description: Digital signature

___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug