Mdthbs

I like to clone a whole partition or a whole hard drive onto a larger external disk but like to create a sparse file. I often use dd for cloning, but it doesn't support sparse files. As a workaround I used something like:

cp --sparse=always <(dd if=/dev/sda1 bs=8M) /mount/external/backup/sda1.raw

However this is a little too tricky for my taste and doesn't allow me to resume the process if aborted. It is funny that there is a NTFS tool for this (ntfsclone) but no such tool exists for the native file systems of Linux (EXT2-4).

Is there some better tool for this, e.g. a dd variant with sparse support?
I do not look for some proprietary software for disk backups but simply want to make a sparse clone copy which I can mount as loop device if required.

You want dd_rescue.

dd_rescue -a -b 8M /dev/sda1 /mount/external/backup/sda1.raw

Just for completeness the call for ddrescue. The --sparse or -S flag allows the destination to be written sparsely:

$ ddrescue -S -b8M /dev/sda1 /mount/external/backup/sda1.raw

Or with long option:

$ ddrescue --sparse --block-size 8M /dev/sda1 /mount/external/backup/sda1.raw

Or if you prefer MiBs:

$ ddrescue -S -b8Mi /dev/sda1 /mount/external/backup/sda1.raw

To allow the rescue to be interrupted and resumed, you can also make use of a logfile:

$ ddrescue -S -b8Mi /dev/sda1 /mount/external/backup/sda1.raw ~/sda1.rescue.log

Note that GNU ddrescue and dd_rescue are different programs. But GNU ddrescue seems to be more widespread. For example it is already packaged with GRML.

There was a patch offered in 2007 to provide sparse file support in GNU dd, but it looks to have not made it into coreutils (at least not as of 8.4). I doubt dd has changed too much since then, the patch might apply against the current version without a lot of work.

I'm also really impressed by the creative use of cp in your question, and it got me on the track of using it to accomplish resuming (here resuming from ~80M into the source):

cp --sparse=always 

  <(dd if=/dev/sda1 bs=8M skip=10) /dev/stdout 

  | dd bs=8M seek=10 of=/mount/external/backup/sda1.raw

Edit: scratch that. The second dd would of course be seeking to the wrong position in the output file, since it's not the same length as the input.

Just adding my 2 cents. Another way to create a sparse file from a raw disk is with qemu-img using something like:

qemu-img convert -f raw /dev/sda /tmp/sda.raw

You can use this on a single partition as well. Also, you have the option to convert the raw disk/partition to any other format that qemu-img supports (QCOW2, VHD[x], vmdk, etc)

Note: this doesn't work for the reasons described in the comments, I'm leaving it here for reference.

Monitor the statistics of dd by using kill -USR1:

$ cp --sparse=always <(dd if=/dev/urandom bs=8M) 

    /mount/external/backup/sda1.raw&

$ watch kill -USR1 `pidof -s /bin/dd`

Resume by using skip/seek:

$ i_bytes= # get from the last dd statistic

$ o_bytes=`du -b /mount/external/backup/sda1.raw | cut -f 1`   

$ cp --sparse=always <(dd if=/dev/urandom bs=8M skip=$i_bytes 

    seek=$o_bytes) /mount/external/backup/sda1.raw&

$ watch kill -USR1 `pidof -s /bin/dd`

Without $i_bytes it would be more difficult to resume. It's probably easiest to log the dd statistics to a file in case the machine crashes or whatever.