Raspberry Pi Image Compression and Distribution

Overview

Creating an optimized and minimized image for the Raspberry Pi 5 is a critical step when moving a system from development to deployment. A 1:1 clone of a 64 GB SD card or NVMe SSD is impractical as it contains empty space and is unwieldy to transfer. A multi-step process is recommended to keep the image as small and professional as possible.

Step 1: Clean the System (Preparation)

Before taking the image, the running system must be cleared of unnecessary ballast and system-specific data. This reduces the image size and prevents security issues (e.g., identical SSH keys on all distributed devices).

Preparation Script (`prepare_image.sh`)

Execute the following script on the Raspberry Pi to clean logs, caches, temporary files, and remove system-specific IDs and SSH host keys.

#!/bin/bash

# ==============================================================================
# Script:   prepare_image.sh
# Purpose:  Clean and prepare a Raspberry Pi 5 system for image distribution
#           (removes caches, logs, SSH keys, and system IDs).
# Note:     Must be run with sudo/root privileges!
# ==============================================================================

# Colors for output
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
RED='\033[0;31m'
NC='\033[0m' # No Color

# Check if script is run as root
if [ "$EUID" -ne 0 ]; then
    echo -e "${RED}Error: This script must be run with sudo or as root!${NC}"
    exit 1
fi

echo -e "${YELLOW}=== Starting system cleanup for image preparation ===${NC}"

# 1. Clean package manager
echo -e "${GREEN}[1/5] Cleaning APT package cache...${NC}"
apt-get clean
apt-get autoremove -y

# 2. Delete temporary files and logs
echo -e "${GREEN}[2/5] Deleting temporary files and logfiles...${NC}"
# Empty/delete system logs
find /var/log -type f -exec truncate -s 0 {} \;
rm -rf /var/log/*.gz /var/log/*.[0-9] /var/log/*-????????

# Empty temporary directories
rm -rf /tmp/* /tmp/.* 2>/dev/null
rm -rf /var/tmp/* /var/tmp/.* 2>/dev/null

# 3. Remove SSH host keys (Security relevant step!)
echo -e "${GREEN}[3/5] Removing SSH host keys (will be regenerated on next boot)...${NC}"
rm -f /etc/ssh/ssh_host_*

# 4. Reset unique system IDs (Machine-ID)
echo -e "${GREEN}[4/5] Resetting Machine-ID...${NC}"
if [ -f /etc/machine-id ]; then
    truncate -s 0 /etc/machine-id
fi
if [ -f /var/lib/dbus/machine-id ]; then
    rm -f /var/lib/dbus/machine-id
    # Restore symbolic link if necessary (often expected by systemd)
    ln -sf /etc/machine-id /var/lib/dbus/machine-id
fi

# 5. Clear bash history for all users
echo -e "${GREEN}[5/5] Clearing Bash history...${NC}"
# History of current root user
cat /dev/null > ~/.bash_history
history -c

# History for normal users
for user_home in /home/*; do
    if [ -d "$user_home" ]; then
        if [ -f "$user_home/.bash_history" ]; then
            cat /dev/null > "$user_home/.bash_history"
        fi
    fi
done

echo -e "${YELLOW}======================================================${NC}"
echo -e "${GREEN}Cleanup completed successfully!${NC}"
echo -e "${YELLOW}The Raspberry Pi will SHUT DOWN in 10 seconds.${NC}"
echo -e "${YELLOW}Afterwards, you can remove the SD card/SSD to pull the image.${NC}"
echo -e "${YELLOW}======================================================${NC}"

# Countdown before shutdown
for i in {10..1}; do
    echo -n "$i... "
    sleep 1
done
echo ""

# Safely shut down system
shutdown -h now

Important: Do not reboot the Raspberry Pi before creating the image, as the temporary IDs and SSH keys will be regenerated immediately.

Step 2: Create the Raw Image

Remove the SD card or NVMe SSD from the Raspberry Pi 5 and connect it to a Linux machine (or Mac/Windows with WSL). Create a 1:1 clone using dd:

sudo dd if=/dev/sdX of=my_system.img bs=4M status=progress

*(Replace /dev/sdX with the correct path to your drive).*

This image will be the exact size of the physical drive, regardless of actual data usage.

Step 3: Shrink the Image (PiShrink)

Use PiShrink to cut off empty space. It shrinks the filesystem and partition to the absolute minimum and injects a script that automatically resizes the partition on the first boot on the target device.

1. Download PiShrink

wget https://raw.githubusercontent.com/Drewsif/PiShrink/master/pishrink.sh
chmod +x pishrink.sh
sudo mv pishrink.sh /usr/local/bin

2. Shrink and Compress

The -z flag compresses the image directly with gzip after shrinking.

sudo pishrink.sh -z my_system.img

Result: A 64 GB image with 8 GB of actual data will be converted into a my_system.img.gz file of roughly 2 to 3 GB, which is ideal for distribution via Raspberry Pi Imager or BalenaEtcher.

Alternative: Infrastructure as Code (Best Practice)

For complex software systems, constant cloning and cleaning can become error-prone. A long-term alternative is to build the image automatically from scratch:

Pi-Gen: The official tool from the Raspberry Pi Foundation. You can define custom "Stages" where complex software is installed during the build process.
Ansible or Docker: Distribute a minimal Raspberry Pi OS "Lite" along with a script that automatically pulls required Docker containers or executes an Ansible playbook on the first boot.