Re: [PLUG] check for a file being transmitted via ftp

On Tue, Mar 25, 2014 at 2:34 PM, Eric at Lucii.org <eric@lucii.org> wrote:

Thanks for the link.

I don't have control over the ftp process on either end so I'm going to stick with lsof or the "im_done" file.

Thanks,
Eric

On 03/25/2014 02:16 PM, Amir Tahvildaran wrote:
> I have a different approach to FTP operations, but these guys have some good tips:
> http://stackoverflow.com/questions/7241978/how-to-determine-wheter-a-file-is-still-being-transferred-via-ftp
>
> briefly: try to get a lock on the file seems like the most straightforward
>
> My approach would be to sub out the filesystem for FTP operations when processing is involved. Use an FTP server library (nodejs or your favorite language) that handles events as they happen as opposed to polling. If you're using FTP for more than just uploading to be processed, than nevermind.
>
> -Amir
>
>

> On Tue, Mar 25, 2014 at 2:05 PM, Eric at Lucii.org <eric@lucii.org <mailto:eric@lucii.org>> wrote:
>
> I expect a file to be routinely transmitted to me via ftp. A bash script,
> invoked as a cron job, will process and then archive that ftp file.
>
> All of that is easy/routine.
>
> My question is: how do I know that the file is complete? I don't want to
> start processing the file without being sure that the ftp process is done.
>
> I thought about checking the size of the file in bytes and then comparing that
> to it's size 1 minute earlier... if it's not growing then it's probably done.
> But, that's just *probably* as ftp could be stalled temporarily or worse.
>
> I have also considered about using inotify cron (incron) but I expect that it
> would be triggered at the file's creation - not when it's time for me to
> process it.
>
> Thanks
> Eric

You could also handle it in a similar way to things like bittorrent... transmit the file as $filename.part, and when the transmit is finished, have the sender use the rename FTP command to change the name to $filename.

--Matt

___________________________________________________________________________ Philadelphia Linux Users Group -- http://www.phillylinux.org Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce General Discussion -- http://lists.phillylinux.org/mailman/listinfo/plug