Featured image

Table of Contents Link to heading

Compression Fundamentals Link to heading

Compression reduces the size of data by encoding it more efficiently. The two categories have fundamentally different use cases:

Lossless compression preserves every bit of the original data. Compressing and decompressing produces a bit-for-bit identical copy of the original. All Linux compression tools (gzip, bzip2, xz) are lossless — critical for configuration files, source code, binaries, and any data where exact reproduction is required. Lossless formats include GIF, PNG, and ZIP.

Lossy compression permanently discards information to achieve higher compression ratios. Decompressing a lossy file produces an approximation of the original. JPEG images and MP3 audio are lossy — two similar shades of green in an image might be encoded as identical to save space. Lossy compression is never appropriate for text files, binaries, or data that must survive a round-trip intact.

gzip: Standard Linux Compression Link to heading

gzip is the standard compression tool on Linux, using the Lempel-Ziv (LZ77) algorithm. It is fast, widely supported, and the default compression format for most Linux log rotation and distribution packages.

By default, gzip compresses a file in-place, replacing it with a .gz version and removing the original:

# Compress a file (replaces original)
gzip file.txt                         # creates file.txt.gz, removes file.txt

# Compress recursively
gzip -r /path/to/directory/

# Keep the original file
gzip -k file.txt                      # creates file.txt.gz, keeps file.txt

# Compress to a specific output file
gzip -c file.txt > compressed.gz

# Set compression level (1=fastest/worst, 9=slowest/best, default=6)
gzip -9 -c large_file.txt > archive.gz

# Show compression statistics
gzip -l archive.gz
# compressed  uncompressed  ratio uncompressed_name
#      12345         45000  72.6% large_file.txt

gunzip and Decompression Link to heading

gunzip decompresses .gz files. It is functionally a wrapper around gzip --decompress:

# Decompress (replaces .gz file with original)
gunzip file.txt.gz

# Decompress and keep the archive
gunzip -k file.txt.gz

# Decompress to stdout (pipe to another command)
gunzip -c archive.gz | grep "error"
cat archive.gz | gunzip | wc -l

# List contents of compressed file
gunzip -l file.txt.gz

# Decompress a .tar.gz archive (outputs .tar file)
gunzip archive.tar.gz

For reading compressed log files without decompressing to disk:

zcat /var/log/syslog.1.gz            # decompress to stdout
zless /var/log/syslog.1.gz           # page through compressed file
zgrep "error" /var/log/syslog.1.gz   # grep inside compressed file

Compression Algorithm Comparison Link to heading

Linux offers three main compression algorithms, each with different speed/ratio trade-offs:

Tool Algorithm Extension Speed Compression Ratio Use Case
gzip LZ77 .gz Fast Good Default choice; logs, software distribution
bzip2 / bunzip2 Burrows-Wheeler .bz2 Slower Better When ratio matters more than speed
xz / unxz LZMA .xz Slowest Best Kernel/software source archives; maximum compression

xz achieves the best compression ratios of the three — it is the format used for distributing the Linux kernel source (linux-6.x.tar.xz). However, it is significantly slower than gzip for both compression and decompression. For frequently accessed compressed files or when CPU time is constrained, gzip is the better choice.

# bzip2 usage
bzip2 file.txt                        # compress → file.txt.bz2
bzip2 -d file.txt.bz2                 # decompress
bzip2 -k file.txt                     # compress, keep original

# xz usage
xz file.txt                           # compress → file.txt.xz
xz -d file.txt.xz                     # decompress
xz -k -9 file.txt                     # best compression, keep original
xz -T0 large_file.txt                 # use all available CPU threads

Archiving with tar Link to heading

tar (tape archive) combines multiple files and directories into a single archive file, preserving directory structure, permissions, and ownership. It does not compress by default — compression is applied separately or combined in a single step via flags.

Note
On Linux, archiving (combining files) and compression (reducing size) are separate operations. tar handles the archiving; gzip/bzip2/xz handle compression. The common .tar.gz (or .tgz) format combines both in one step. This separation means you can use tar alone to bundle files without compression, or use compression tools alone on single files.

Key behaviour: tar traverses subdirectories recursively by default — no -r flag needed.

Create Mode Link to heading 

# Create an uncompressed archive
tar cf archive.tar file1 file2 dir1/
tar --create --file archive.tar file1 file2 dir1/

# Create a gzip-compressed archive (.tar.gz or .tgz)
tar czf archive.tar.gz file1 file2 dir1/
tar --create --gzip --file archive.tar.gz dir1/

# Create a bzip2-compressed archive (.tar.bz2)
tar cjf archive.tar.bz2 dir1/

# Create an xz-compressed archive (.tar.xz)
tar cJf archive.tar.xz dir1/

# Create from a directory, using relative paths (portable)
tar czf archive.tar.gz -C /path/to/parent dir_to_archive/

# Auto-detect compression from filename extension
tar caf archive.tar.xz dir1/         # -a: auto-compress based on suffix

# Exclude files matching a pattern
tar czf archive.tar.gz dir1/ --exclude="*.log" --exclude=".git"

# Verify what would be archived without creating (dry run with verbose)
tar czf /dev/null -v dir1/

Extract Mode Link to heading 

# Extract to current directory
tar xf archive.tar.gz

# Extract verbosely (show files as they are extracted)
tar xvf archive.tar.gz

# Extract to a specific directory
tar xf archive.tar.gz -C /destination/path/

# Extract a specific file from an archive
tar xf archive.tar.gz path/to/specific/file.txt

# Extract files matching a pattern
tar xf archive.tar.gz --wildcards "*.conf"

# Preserve permissions and ownership during extraction (requires root for ownership)
tar xpf archive.tar.gz
Tip
tar xf works for .tar, .tar.gz, .tar.bz2, and .tar.xz archives — tar auto-detects the compression format. You do not need to specify -z, -j, or -J when extracting; only when creating.

List Mode Link to heading 

# List archive contents without extracting
tar tf archive.tar.gz
tar --list --verbose --file archive.tar.gz    # with details (permissions, sizes)

Quick Reference Link to heading

Operation Command
Create .tar.gz tar czf archive.tar.gz files/
Create .tar.bz2 tar cjf archive.tar.bz2 files/
Create .tar.xz tar cJf archive.tar.xz files/
Extract any tar archive tar xf archive.tar.*
Extract to directory tar xf archive.tar.gz -C /dest/
List contents tar tf archive.tar.gz

ZIP Files: Cross-Platform Archives Link to heading

ZIP is the dominant archive format on Windows and macOS. While not the standard on Linux, ZIP archives are commonly encountered when exchanging files across platforms or when password-protected archives are required.

Unlike tar + gzip, ZIP compresses each file individually, which means individual files can be extracted without decompressing the entire archive — useful for large archives where you need only a subset of files.

Creating ZIP Archives Link to heading 

# Create a zip archive (recursive)
zip -r archive.zip directory/
zip -r archive.zip file1 file2 dir1/

# Set compression level (1=fastest, 9=best, default=6)
zip -r -9 archive.zip directory/

# Exclude files
zip -r archive.zip directory/ -x "*.log" "*.tmp"

# Create encrypted archive (password prompt)
zip -r -e secure.zip confidential/

# Create split (multi-part) archive (e.g., 100MB parts)
zip -r -s 100m large_archive.zip bigdirectory/

# List contents
zip -sf archive.zip

# Remove a file from an existing archive
zip -d archive.zip path/inside/archive.txt

Extracting ZIP Archives Link to heading 

# Extract to current directory
unzip archive.zip

# Extract to a specific directory
unzip archive.zip -d /destination/

# Extract a specific file
unzip archive.zip path/to/file.txt

# List contents without extracting
unzip -l archive.zip

# Extract to stdout (useful for piping)
unzip -p archive.zip path/to/file.txt | grep "pattern"

# Handle archives with different encodings (e.g., created on Windows)
unzip -O cp936 archive.zip        # Chinese Windows encoding

When to Use Each Tool Link to heading

Scenario Recommended Tool
Log archiving, software distribution on Linux tar czf (.tar.gz)
Maximum compression for infrequently accessed data tar cJf (.tar.xz)
Single file compression gzip, bzip2, or xz
Cross-platform sharing (Windows/macOS/Linux) zip
Password-protected archive zip -e
Splitting large archives across media zip -s
Streaming to a tape device or over a network tar (uncompressed or piped through gzip) 
# Stream a directory over SSH to a remote system (no intermediate file)
tar czf - /source/directory/ | ssh user@remote "tar xzf - -C /destination/"