Re: [PLUG] gnu parallel and tar

[Jason Harlow <jharlow1@gmail.com>]

Not sure about the tar bit, but I've used pigz ( http://www.zlib.net/pigz/ ) for the compression piece on many-core servers for fairly significant performance improvements.

On Fri, Apr 1, 2011 at 9:26 PM, Julien Vehent <julien@linuxwall.info> wrote:

On Fri, 1 Apr 2011 19:39:20 -0400, K.S. Bhaskar wrote:

Switch from bz2 to gzip and you may get home before Monday morning!
bzip2 is terribly slow for large byte streams - I once found it faster
move data uncompressed than to use bz2 (gzip was faster yet).  bzip2
is great if you have to download a disk image over a 110 Baud modem.

Sorry, I don't know about parallelizing tar.

well. I compressed 2GB of log files into a 10MB archive, and be able to send it by email, so bzip2 was worth it in the end, even on one core.

___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug

___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug