Re: [PLUG] Rsync advice wanted

[Fred Stluka <fred@bristle.com>]

Michael,

Sorry for the delay.  Been busy...

Yes, this sounds reasonable.  Rsync is very good at keeping 2
directories in sync, so you should be able to set it up to do
exactly what you need.

You're using --remove-source-files to remove files from the
source after copying them to the target.  That gives you an
empty source tree most of the time, indicating that they have
all been successfully copied to the target tree.

Do you give the receiving vendor a way to delete them from
target after STFPing them out?  If so, do you want a way to
keep copies in source until the vendor has deleted them from
target?  You can do that via the --delete option when rsyncing
in the opposite direction.

See my rsync tips for more ideas:
- http://bristle.com/Tips/Unix.htm#rsync

--Fred
------------------------------------------------------------------------
Fred Stluka -- mailto:fred@bristle.com -- http://bristle.com/~fred/
Bristle Software, Inc -- http://bristle.com -- Glad to be of service!
Open Source: Without walls and fences, we need no Windows or Gates.
------------------------------------------------------------------------

On 12/16/13 1:12 PM, Michael Leone wrote:
> I have a setup now where I have a RH server in my DMZ, and my vendors
> use SFTP to copy files to us. I also have a Ubuntu VM on my trusted
> LAn that will reach into that box (via SSH; the DMZ has copies of it's
> keys); run a script to sweep all new files into a single ZIP file;
> rsync that back into the trusted LAN; and deletes the source on the
> DMZ. I had a whole lot of help from this list with that script close
> to 3.5 years ago now (thanks again!) and it's been running fine.
>
> Now, I have a need to go in the opposite direction. We want to send
> files to the DMZ, for other vendors to pick up (also over SFTP).
> Ideally, Here's what I am thinking, please comment.
>
> These new vendors will have home directories in an entirely different
> tree from the vendors that send in files. Reason: I can then easily
> duplicate this tree on the trusted side (we're talking less than 10
> user directories, total).
>
> When we have a file to send to User-1, we write it in the "home
> directory" of User-1 on the trusted side. Then, on a schedule, a
> script will make an rsync connection over SSH to the DMZ; replicate
> any files on the trusted side to the DMZ side (ignoring any files on
> the DMZ side that happen to already exist), and then deleting the file
> from the trusted side.
>
> Rsync should be able to do this, with these options?
>
> rsync -rvz  --ignore-existing --remove-source-files
> <trusted-side-tree>  <DMZ-user@DMZ-IP:/DMZ-side-tree>
>
> DMZ-user can be given rights into the user home directory, so it can
> write files there.
>
> This way, the script cleans out the trusted side (the "outgoing
> files"), while not touching any files that the vendor may have not yet
> SFTPed out.
>
> Thoughts? I know I've missed something, or made some stupid assumption.
>
> I have very very little experience writing Linux scripts, but have
> written many on Windows (CMD, Powershell) so it's more the specific
> implementation on Linux that I am weak on.
> ___________________________________________________________________________
> Philadelphia Linux Users Group         --        http://www.phillylinux.org
> Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
> General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug
___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug