Stephen Gran on 2 Aug 2006 22:40:23 -0000


[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

Re: [PLUG] Large data transfer


On Wed, Aug 02, 2006 at 04:13:11PM -0400, Jeff Abrahamson said:
> On Sat, Jul 29, 2006 at 09:03:27PM -0400, Aaron Mulder wrote:
> > Also, as a side note, the one time I ever had a real problem with
> > rsync was when transferring a file larger than the available drive
> > space.  It took forever to update an existing version of the file --
> > way longer than to copy it in the first place -- and then crapped
> > out after a couple hours claiming that it was out of disk space.  So
> > I guess it tried to create a whole separate copy of the destination
> > file, and couldn't manage to do the free space calculation at the
> > beginning of the process.  Grrr!
> 
> The rsync algorithm does not work in place: it creates a new copy
> based on diffs and the old copy.
> 
> I saw a paper at Usenix ATC in 2003 about in-place rsync.  Cool graph
> theoretic stuff so that things were copied only as they could be to
> update the file, probably useful for handhelds with limited memory.
> The problem, of course, is that if the transfer dies mid-stream,
> you've corrupted the file pending a new rsync, instead of having a
> usable but out-of-date file.

This is the option:
            --inplace               update destination files in-place

Not recommended if you care about the data, as Jeff pointed out, but an
option if it's just backups.
-- 
 --------------------------------------------------------------------------
|  Stephen Gran                  | If only Dionysus were alive!  Where     |
|  steve@lobefin.net             | would he eat?   -- Woody Allen          |
|  http://www.lobefin.net/~steve |                                         |
 --------------------------------------------------------------------------

Attachment: signature.asc
Description: Digital signature

___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug