gabriel rosenkoetter on Thu, 4 Apr 2002 19:48:48 +0200


[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

Re: [PLUG] Moveing large files


On Thu, Apr 04, 2002 at 10:29:47AM -0500, ian reinhart geiser wrote:
> i was hopeing for a way to do with with ssh.  I cannot remember how you do a 
> stream over ssh from one system to another.  failing that i will install nc 
> on the other box, because it is lacking there.

You don't ever want to move files this large with ssh. It's not made
for that. ftp or rsh over a private network. (rsh is the absolute
minimum of overhead of all the network protocols out there, so it's
probably what you want to use. netcat may come close, but it sends
more control BS than rsh does. Anyway, rsh is a standard utility.)

If you must, you can do something like this:

dd if=<big file> | ssh <host> dd of=<out file> ibs=<bs on source host> obs=1024k

But don't do that. sshd will go through the roof on processor usage
on the destination host and it'll take a year and a half to
transfer. (Yes, you probably really do have to specify the block
sizes. Don't trust that dd will just figure it out. If it gets it
wrong, it'll scramble your file and waste your time.)

If you really must not transfer this file in the clear, PGP encrypt
it before you send it.

Also, if your basic file system utilities can't handle 2 GB files,
you *really* need to upgrade them. 2 GB limits are so 1999. Note
that ext2fs probably has been upgraded, but that tar, dd, du, so
forth also need to be.

(Be aware that the problem may be your shell. bash couldn't deal
with running >2GB files through pipes, named or otherwise, for quite
some time. And I don't think Midnight Commander can deal with 2 GB
files yet, which is totally ridiculous.)

-- 
gabriel rosenkoetter
gr@eclipsed.net

Attachment: pgpFN1LZdHkr8.pgp
Description: PGP signature