Re: [PLUG] inotify

[Rich Freeman <r-plug@thefreemanclan.net>]

On Tue, Aug 14, 2012 at 11:05 PM, Julien Vehent <julien@linuxwall.info> wrote:
> During the meeting tonight I mentionned inotify and how it can be used to
> keep two directory trees in sync. I used it to keep a local directory tree,
> that contained huge amounts of small images, in sync with a bucket in S3
> (via s3fs).

An alternative to s3fs is s3cmd.  It has a sync function which
utilizes the fact that s3 maintains an accurate list of hashes to
reduce file transfer.  It does still need to read the local file to
determine the local hash. You could potentially use it in combination
with inotify to reduce local IO, while avoiding copying files if they
haven't actually changed.

One risk of the inotify approach is that if somehow the tress do get
out of sync they'll tend to stay that way, unless you periodically run
a full sync.  That could happen if something is modified while it
isn't running, or if something directly modifies the s3 tree.

Rich
___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug