How to make space/storage efficient backups of Raspberry Pi SD-cards

RaspberryPi mini computers are great for all kinds of home automation, media centers, home servers. Yet it should be known that the draw back for permanent operation can be the SD-card. Especially when applications on the RasperrbyPi cause excessive read/writes or if it is used outsides where temperatures can reach the SD-Card operating limits, IO errors should be expected that can just f* up the operating system.

The easiest way to be prepared is to have a full SD-card image of the RaspberryPi at hand! The easiest way is to clone the SD card for example using Win32 Disk Imager.

The resulting image is as large as the SD-card. As the card is usually not 100% in use, the image size should considerably decrease by compression. Yet it is not uncommon that a 16 GB image with 3 GB data cannot be compressed to less than 10 GB! The reason is that deleting files only removes the file handles and leaves the data unreferenced. So: the longer the SD-card was in use (=the more files were written / deleted on it), the worse the compression factor.

How to make a well compressable SD-card image

The trick simply to wipe the free space! An easy way is to fill the free space with zeros. When the image is compressed later, all zeros can be compressed to almost nothing.

  1. Open a shell: dd if=/dev/zero of=/tmp/zero1.tmp bs=1048576. This creates a single file, filled with just Zeros.
  2. Delete the zero-file rm /tmp/zero1.tmp and shut down the Raspi. The free space is now wiped!
  3. Read (Win32 Disk Imager) and compress the image.

The compressed image can be up to 10x smaller than without wiping the free space!

4 thoughts on “How to make space/storage efficient backups of Raspberry Pi SD-cards”

  1. Nice, but,

    For step 2 – We do not need to stop the dd command before completing all the disk space. Just run the command:
    dd if=/dev/zero of=/tmp/zero1.tmp bs=1048576

    Step 3 – Is not necessary.

    Step 4 – Just run the command (only one large file):
    $ sudo rm -R /tmp/zero1.tmp

    (Commands tested via SSH with PuTTY.exe)

  2. Hello my friend! Thanks for all the help! how did you compress your image? which compressor did you use?

    1. Hey,
      actually I switch between fast ZIP, GZIP or 7z – each time with a “fast” option. Otherwise it takes WAYS longer with just very small improvement in size.

Comments are closed.