|
[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]
|
RE: [PLUG] perplexing file transfer problem
|
Eric,
My suggestion would be to look at the network, the NIC on the new box,
router/switch, etc. if you changed subnets. You may also want to do some
packet capturing to see what is happening when this failure occurs.
----------------------------------------------------------------------------
---
Scott Ziegler
Research Associate
Merck Research Laboratories
Imaging Research
Ph. 215-652-7253
Mail - WP44C-2
-----Original Message-----
From: plug-bounces@lists.phillylinux.org
[mailto:plug-bounces@lists.phillylinux.org] On Behalf Of Eric
Sent: Monday, July 10, 2006 10:03 AM
To: PLUG
Subject: [PLUG] perplexing file transfer problem
I've encountered a file transfer problem that is quite perplexing -
and I'm at a loss to identify the cause. Part of the problem is
that it's random and it's complex (a number of variables are
involved) and therefore hard to narrow down to a single failure
point.
The basic scenario is this: I have a shell script that uses curl
commands to retrieve multiple files from two ftp servers. The
host for this is a rented box that I do not have root or physical
access to. I do not have root or physical access to the ftp
server either for that matter... I'm just a user/developer
tryin' to bring the data home :-)
The script ran on a shared box for several months but was moved to
it's own box with a new IP on Thursday. On Friday the random
failures began.. some initial portion of the file would download at
normal speed then the rate would drop to zero bytes per second and
stay there. If the script is re-run it may download that file just
fine only to fail on the next download. Also, the size of the
initial download appears random too - one was 292 kB, the next was 1.9
Meg... etc. There is a log fragment at the bottom of this email
showing the relevant part (the output of the curl command).
Other factors I've considered:
The transfer randomly fails if we use ftp from the command line too
- it's not just curl
For security reasons, the ftp server must have the IP of the
requestor on file - and it does. Otherwise, we could not
even log it.
The failure can be from either of the ftp servers - although they
are on the same domain they are separate host names.
Now if anybody has seen something similar or has any ideas on how to
further troubleshoot this I'd be very grateful. Keep in mind that it's
been working like clockwork since March or April so it's not the
programmer's fault :-D
Thanks.
Eric
Our host:
# uname -a
Linux host.xxxxxxxxx.com 2.6.9-34.EL #1 Wed Mar 8 00:07:35 CST 2006
i686 athlon i386 GNU/Linux
Log fragment showing how it fails:
-----
curl -n $DATAURL1 > $DATAFILE1
retrieving BR_20060710_013000__dat_full_.zip
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
0 24.1M 0 1460 0 0 1283 0 5:29:00 0:00:01 5:28:59
1283
0 24.1M 0 168k 0 0 106k 0 0:03:53 0:00:01 0:03:52
371k
2 24.1M 2 684k 0 0 264k 0 0:01:33 0:00:02 0:01:31
470k
4 24.1M 4 1166k 0 0 319k 0 0:01:17 0:00:03 0:01:14
462k
4 24.1M 4 1199k 0 0 219k 0 0:01:52 0:00:05 0:01:47
277k
4 24.1M 4 1199k 0 0 185k 0 0:02:13 0:00:06 0:02:07
225k
4 24.1M 4 1199k 0 0 160k 0 0:02:33 0:00:07 0:02:26
175k
4 24.1M 4 1199k 0 0 141k 0 0:02:54 0:00:08 0:02:46
89742
4 24.1M 4 1199k 0 0 126k 0 0:03:15 0:00:09 0:03:06
5781
4 24.1M 4 1199k 0 0 114k 0 0:03:35 0:00:10 0:03:25
0
4 24.1M 4 1199k 0 0 104k 0 0:03:56 0:00:11 0:03:45
0
4 24.1M 4 1199k 0 0 98511 0 0:04:17 0:00:12 0:04:05
0
4 24.1M 4 1199k 0 0 91188 0 0:04:37 0:00:13 0:04:24
0
4 24.1M 4 1199k 0 0 84879 0 0:04:58 0:00:14 0:04:44
0
.
. [five hours later we see.... Current Speed is stuck at zero]
.
4 24.1M 4 1199k 0 0 67 0 4d 09h 5:03:55 99:56:12
0
4 24.1M 4 1199k 0 0 67 0 4d 09h 5:03:56 99:56:11
0
4 24.1M 4 1199k 0 0 67 0 4d 09h 5:03:57 99:56:10
0
4 24.1M 4 1199k 0 0 67 0 4d 09h 5:03:58 99:56:09
0
4 24.1M 4 1199k 0 0 67 0 4d 09h 5:03:59 99:56:08
0
4 24.1M 4 1199k 0 0 67 0 4d 09h 5:04:00 99:56:07
0
4 24.1M 4 1199k 0 0 67 0 4d 09h 5:04:01 99:56:06
0
4 24.1M 4 1199k 0 0 67 0 4d 09h 5:04:02 99:56:05
0
4 24.1M 4 1199k 0 0 67 0 4d 09h 5:04:03 99:56:04
0
4 24.1M 4 1199k 0 0 67 0 4d 09h 5:04:04 99:56:03
0
--
------------------------------------------------------------------------
# Eric A Lucas
# ------------
# "Oh, I have slipped the surly bond of earth
# and danced the skies on laughter-silvered wings...
# -- John Gillespie Magee Jr.
___________________________________________________________________________
Philadelphia Linux Users Group -- http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion -- http://lists.phillylinux.org/mailman/listinfo/plug
------------------------------------------------------------------------------
Notice: This e-mail message, together with any attachments, contains information of Merck & Co., Inc. (One Merck Drive, Whitehouse Station, New Jersey, USA 08889), and/or its affiliates (which may be known outside the United States as Merck Frosst, Merck Sharp & Dohme or MSD and in Japan, as Banyu) that may be confidential, proprietary copyrighted and/or legally privileged. It is intended solely for the use of the individual or entity named on this message. If you are not the intended recipient, and have received this message in error, please notify us immediately by reply e-mail and then delete it from your system.
------------------------------------------------------------------------------
___________________________________________________________________________
Philadelphia Linux Users Group -- http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion -- http://lists.phillylinux.org/mailman/listinfo/plug
|
|