[nflug] speed imaging netcat

[Bradley A. Llewellyn]

What kind of performance are you getting, relative to your NIC speed?  I 
just did this the other day, and I was moving the image at 75% the speed 
of my NIC.

Install the tool called "pv", and add it to your pipe stream to see how 
fast the image is being sent and/or received.

For instance:

dd if=/dev/sda | pv | netcat {host} {port}

Also, when using DD, the block size can also determine performance...not 
sure how much, since the NIC will be a bottleneck, but you can at least 
play around with it to get the best possible read/write speeds.

ex:

dd if=/dev/sda bs=16K

Imaging from a SCSI drive, it made a big difference which block size I 
picked.  Set the "of" = /dev/null and try a couple of different numbers.

It may or may not help to add gzip to stream on the way out the door:

dd ... | gzip ... | pv ... | netcat ....

then on the receiving end...

netcat... > whatever.img.gz

Makes sense to compress the stream on the way out the door, although it 
doesn't always make a big impact.  It probably would in this case, 
because you probably have long sequences of zeros on your hard drive.  
You may want to fill your hard drive with zero's ahead of time too.  Use 
dd to take input from /dev/zero and write it to a file that is big 
enough to almost fill up the drive...that's assuming you can write to 
the filesystem(s) on the drive:

ex.
dd if=/dev/zero of=/tmp/delete.me bs=1M count=1000

As always, be VERY CAREFUL with DD...

--Brad



Eric Benoit wrote:
> *I'm trying to image a ssd drive across the network to my hard drive, 
> I used the directions below and everything is working fine, however it 
> is slow as time next to a blackhole ;)
>
> does anyone know a way I could speed things up a bit or byte?
>
>
> based on the following instructions:
> *
>
> This method uses dd and nc provided by the Debian Installer and 
> another computer located on your local network:
>
>    * First, boot the debian installer on your USB disk
>    * Then be sure to activate network and configure it, same thing for
>      the disk (Do not partition it)
>    * Open a console shell (Alt+F2 or from the Debian Installer menu)
>    * On the backup computer:
>          o get its IP adress (ifconfig)
>          o start a listening nc session on port 9000:
>
> nc -l -p 9000 | gzip -1 -c > ./eeepc.img.gz
>
>    * With netcat-openbsd installed this line should read
>
> nc -l 9000 | gzip -1 -c > ./eeepc.img.gz
>
>    * On the EeePC shell:
>
> dd if=/dev/sda | nc -w 5 computer_ip_adress 9000
>
> Wait some minutes (about 35)... and you get a eeepc.img.gz with about 
> 900Mb
>
> from the website:
> *
> http://tinyurl.com/5hozkf
>
>
> Thank you,
> Eric
> *
> _______________________________________________
> nflug mailing list
> nflug at nflug.org
> http://www.nflug.org/mailman/listinfo/nflug