Checkpoint

A powerful cross-platform utility for creating, managing, and restoring timestamped snapshots of directories. Designed specifically for developers, system administrators, and DevOps engineers who need reliable recovery points during iterative development and system configuration changes.

Overview

Checkpoint bridges the gap between informal backup practices and enterprise-grade snapshot management by providing simple commands for complex operations. Create recovery points before risky changes, track development progress through organized snapshots, and quickly restore when needed—all while maintaining security and automation compatibility.

Key Benefits

• Development Safety: Create quick recovery points before risky code changes
• Visual Change Tracking: Compare differences between snapshots to understand evolution
• Flexible Recovery: Restore entire directories or specific file patterns
• Linux-Native: Targets Bash 5.2+ with GNU coreutils (Ubuntu/Debian and similar)
• Automation Ready: Works reliably in CI/CD pipelines and scripts
• Non-Root Friendly: Works seamlessly for regular users with smart directory defaults

Core Features

• Smart Snapshots: Creates timestamped backups with automatic exclusions
• Intelligent Defaults: Automatically selects appropriate backup directories based on user privileges
• Atomic Operations: Ensures backup integrity with temporary directories and atomic rename
• Concurrency Protection: Lockfile mechanism prevents data corruption from parallel operations
• Powerful Comparison: Visualizes differences between snapshots with color-coded output
• Flexible Restoration: Supports complete or selective file recovery with preview mode
• Space Optimization: Automatic hardlinking between versions (when hardlink is installed) to minimize disk usage
• Backup Rotation: Manages history by count or age for automatic cleanup
• Automation Support: Non-interactive operation with timeout safeguards
• Smart Privilege Handling: Detects whether the backup directory is writable and advises sudo when needed (never escalates on its own)

Installation

One-Line Install

# Install with automatic dependency installation
curl -fsSL https://raw.githubusercontent.com/Open-Technology-Foundation/checkpoint/main/install.sh | bash

# Or with wget
wget -qO- https://raw.githubusercontent.com/Open-Technology-Foundation/checkpoint/main/install.sh | bash

# Install to custom location (no sudo required)
curl -fsSL https://raw.githubusercontent.com/Open-Technology-Foundation/checkpoint/main/install.sh | INSTALL_DIR=~/.local/bin bash

The installer also:

• Creates a chkpoint symlink for convenience
• Installs the man page (man checkpoint)
• Installs bash completion for tab completion

Manual Install

# Download script and make executable
curl -fsSL https://raw.githubusercontent.com/Open-Technology-Foundation/checkpoint/main/checkpoint -o checkpoint
chmod +x checkpoint
sudo cp checkpoint /usr/local/bin/

# Install man page
curl -fsSL https://raw.githubusercontent.com/Open-Technology-Foundation/checkpoint/main/checkpoint.1 -o checkpoint.1
sudo cp checkpoint.1 /usr/local/share/man/man1/
sudo mandb

# Install bash completion
curl -fsSL https://raw.githubusercontent.com/Open-Technology-Foundation/checkpoint/main/checkpoint.bash_completion -o checkpoint.bash_completion
sudo cp checkpoint.bash_completion /usr/share/bash-completion/completions/checkpoint

# Optional: Install hardlink for space efficiency
sudo apt install hardlink  # Ubuntu/Debian
# or
brew install hardlink       # macOS

Requirements

Core Dependencies (required):

• rsync - File synchronization
• find - File discovery
• stat - File metadata

Optional Dependencies:

• hardlink - Space-efficient backup storage
• delta or colordiff - Enhanced diff visualization

Quick Start

For Non-Root Users

# Create checkpoint of current directory (backs up to ~/.checkpoint/)
checkpoint

# Backup a specific directory
checkpoint ~/my-project

# Use custom backup location
checkpoint -d ~/backups/project ~/my-project

# Set default backup directory for all operations
export CHECKPOINT_BACKUP_DIR=~/my-backups
checkpoint ~/my-project

Basic Operations

# Create checkpoint of current directory
checkpoint

# Create checkpoint with descriptive name
checkpoint -s "before-refactor"

# List all checkpoints
checkpoint --list

# Restore latest checkpoint
checkpoint --restore

# Compare current files with checkpoint
checkpoint --restore --diff

Usage Examples

Development Workflow

# Before major changes
checkpoint -s "pre-api-refactor"

# Compare with previous state
checkpoint --restore --diff

# Restore if needed
checkpoint --restore --from 20250430_091429

System Administration

# Backup configuration before updates
sudo checkpoint -d /var/backups/system /etc

# Web server configuration checkpoint
checkpoint -s "ssl-optimization" /etc/nginx

# Compare configuration changes
checkpoint --from 20250430_091429 --compare-with 20250430_101530 --detailed

Selective Operations

# Restore only specific files
checkpoint --restore --files "*.js" --files "docs/*.md"

# Dry run to preview changes
checkpoint --restore --dry-run

# Custom backup location
checkpoint -d ~/backups/myproject

# Exclude specific patterns
checkpoint --exclude "node_modules/" --exclude "*.log"

Backup Management

# Automatic rotation: keep only 5 most recent
checkpoint --keep 5

# Age-based rotation: remove backups older than 30 days
checkpoint --age 30

# Prune without creating new backup
checkpoint --prune-only --keep 3

Concurrency Protection

Checkpoint includes a lockfile mechanism to prevent data corruption from concurrent operations:

# Normal operation (locking enabled by default)
checkpoint

# Disable locking (DANGEROUS - allows concurrent operations)
checkpoint --no-lock

# Set custom lock timeout (default: 300 seconds)
checkpoint --lock-timeout 60

# Force removal of stale locks
checkpoint --force-unlock

The locking mechanism:

• Prevents multiple checkpoint instances from operating on the same backup directory
• Automatically detects and removes stale locks from crashed processes
• Can be disabled for special use cases (use with caution)

Command Reference

Core Options

Option	Description
-d, --backup-dir DIR	Custom backup location (default: context-dependent)
-s, --suffix SUF	Add descriptive suffix to checkpoint name
--no-hardlink	Do not hardlink to previous backup
--hardlink	Hardlink to previous backup (default if available)
-q, --quiet	Minimal output (just backup path)
-v, --verbose	Detailed output with progress (default)
-l, --list	List existing checkpoints with sizes
-x, --exclude PATTERN	Additional exclusion pattern (repeatable)
-V, --version	Print version and exit
-h, --help	Display help

Backup Management

Option	Description
-k, --keep N	Keep only N most recent backups
--age DAYS	Remove backups older than DAYS days
-p, --prune-only	Only prune backups without creating new one
--no-sudo	Hard-error on an unwritable backup dir instead of advising sudo
--no-lock	Disable lockfile mechanism (DANGEROUS)
--lock-timeout N	Lock acquisition timeout in seconds (default: 300)
--force-unlock	Force removal of stale locks

Restore and Compare

Option	Description
-r, --restore	Restore from checkpoint
-f, --from ID	Source checkpoint (defaults to most recent)
-t, --to DIR	Target restore directory (defaults to original)
-n, --dry-run	Preview changes without making them
--diff	Show differences between current files and checkpoint
--compare-with ID	Compare two checkpoints
--detailed	Show file content differences in comparison
--files PATTERN	Select specific files/patterns (repeatable)

Automation Integration

Environment Variables

# Set default backup directory for all operations
export CHECKPOINT_BACKUP_DIR=~/my-backups

# Skip interactive prompts
export CHECKPOINT_AUTO_CONFIRM=1

CI/CD Examples

# GitHub Actions / GitLab CI
- name: Create Checkpoint
  run: CHECKPOINT_AUTO_CONFIRM=1 checkpoint -s "build-${GITHUB_RUN_NUMBER}"

# Jenkins Pipeline
stage('Backup') {
    steps {
        sh 'CHECKPOINT_AUTO_CONFIRM=1 checkpoint -s "build-${BUILD_NUMBER}"'
    }
}

# Cron job for regular backups
0 2 * * * CHECKPOINT_AUTO_CONFIRM=1 /usr/local/bin/checkpoint -d /var/backups/nightly /home/user/project

Timeout Protection

All interactive prompts have built-in timeouts (auto-answer "no"/"N" on expiry):

• Directory creation: 30 seconds
• Restore overwrite confirmation: 30 seconds

Backup Directory Locations

Smart Directory Selection

Checkpoint intelligently selects backup directories based on your user context:

User Type	Default Location	Example
Root/sudo	/var/backups/FULL/PATH/	/var/backups/home/user/myproject/
Regular user	~/.checkpoint/FULL/PATH/	~/.checkpoint/home/user/myproject/
Custom	$CHECKPOINT_BACKUP_DIR/FULL/PATH/	~/backups/home/user/myproject/

The full canonical source path (with leading / stripped) is used as the subdirectory, preventing collisions when backing up different directories with the same basename.

Privilege Management

Checkpoint never runs sudo itself. It checks whether the backup directory is writable and acts accordingly:

• Writability Detection: Determines whether the current user can write to the backup directory
• Non-Root Friendly: Regular users can backup to any writable directory without sudo
• Advice, Not Escalation: When the directory needs root, checkpoint exits with guidance to re-run under sudo (or pick a writable directory)
• Explicit Control: Use --no-sudo to turn an unwritable backup directory into an immediate error instead of advising sudo

# Force non-root operation
checkpoint --no-sudo ~/myproject

# Let checkpoint decide (recommended)
checkpoint ~/myproject

Default Exclusions

These patterns are automatically excluded from all backups:

• Backup directory itself (prevents recursion)
• .gudang/, temp/, .temp/, tmp/ directories
• Temporary files: *~ and ~*
• .tmp.* directories (atomic operation temporaries)
• .checkpoint.lock/ directories (concurrency locks)

Storage and Performance

Space Efficiency

With hardlinking enabled, checkpoint can achieve 90%+ space savings between similar versions by sharing identical files. Example:

# First backup: 100MB
checkpoint -s "v1.0"

# Second backup: Only changed files use additional space
checkpoint -s "v1.1"  # Might only use 5MB additional space

Atomic Operations

Checkpoint uses atomic operations to ensure backup integrity:

• Temporary Directory: Backups are created in a .tmp.* directory first
• Atomic Rename: Only after all operations succeed is the backup renamed to its final name
• Automatic Cleanup: Temporary directories are removed on interruption or failure
• No Partial States: You'll never see incomplete or corrupted backups

This means:

• Interrupted backups leave no trace
• Concurrent operations are safe (with locking enabled)
• Backup directories appear instantaneously when complete
• Failed operations are automatically cleaned up

Performance Characteristics

• Backup Speed: Limited by rsync performance and storage I/O
• Comparison Speed: Byte-level file comparison (cmp) with optional pattern filtering
• Scalability: Handles projects from small configs to large codebases
• Memory Usage: Minimal footprint, primarily shell variables

Development

Testing

# Lint code (must pass without errors)
shellcheck checkpoint

# BCS compliance check
bcscheck checkpoint

# Run all test suites with summary
./run_all_tests.sh

# Run all test suites
bats tests/*.bats

# Run individual test suites (74 tests total)
bats tests/test_checkpoint.bats        # Core functionality (28 tests)
bats tests/test_locking.bats           # Concurrency protection (10 tests)
bats tests/test_nonroot.bats           # Non-root user operations (9 tests)
bats tests/test_high_findings.bats     # High-severity review regressions (7 tests)
bats tests/test_low_findings.bats      # Low-severity review regressions (8 tests)
bats tests/test_medium_findings.bats   # Medium-severity review regressions (6 tests)
bats tests/test_atomic.bats            # Atomic operations (6 tests)

# Run specific test by name
bats tests/test_checkpoint.bats -f "backup creation"

# Verbose testing
bats -v tests/test_checkpoint.bats

Contributing

1. Fork the repository
2. Create a feature branch
3. Make changes following existing code style:
   • 2-space indentation (never tabs)
   • set -euo pipefail error handling
   • Use [[ for conditionals, (( )) for arithmetic
   • UPPER_CASE for global variables, lowercase for local variables
   • Comprehensive function documentation headers
   • BATS tests for new functionality
   • End all scripts with #fin marker
4. Run shellcheck checkpoint (must pass without errors)
5. Run bats tests/*.bats (all tests must pass)
6. Submit pull request

Troubleshooting

Common Issues

Permission Denied: Back up to a writable directory (e.g. under $HOME), or re-run with sudo for a root-owned backup location. --no-sudo makes an unwritable target a hard error instead of advising sudo.

Insufficient Disk Space: Check available space in backup directory before large operations.

Command Not Found: Ensure all required dependencies (rsync, find, stat) are installed.

Failed to Acquire Lock: Another checkpoint process may be running. Use --force-unlock to remove stale locks from crashed processes, or wait for the other operation to complete.

Comparing with Checkpoints

# Compare with source to check differences
checkpoint --restore --diff

Security

• Input Validation: Strict pattern matching prevents injection attacks
• Path Protection: Prevents directory traversal attacks
• Privilege Management: Never self-escalates; advises sudo only when the backup directory needs it

License

This project is licensed under the GPL-3.0 License - see the LICENSE file for details.

Version

Current version: 1.7.8

Recent Features

v1.7.8 - Messaging Refactor (BCS0703)

• _msg redesigned: the core messaging function now takes the icon as its first argument and owns its stderr redirect, replacing FUNCNAME introspection and per-caller >&2. Output is byte-identical; this clears the remaining bcscheck recommended-tier finding on the messaging layer

v1.7.7 - BCS Refactor and Documentation Accuracy

• _detect_diff_tool hardened (BCS0202): the diff-tool detector returns its result through explicit local -n namerefs instead of dynamic scoping into caller locals, clearing the lone bcscheck core-tier finding
• Documentation synced to behavior: corrected the privilege-handling docs (checkpoint never self-escalates — it detects writability and advises sudo), removed a fabricated prompt timeout, refreshed the test inventory (7 suites / 74 tests), documented -k, and expanded the manpage EXIT STATUS to the full set of codes

v1.7.6 - Critical Review Low-Severity Fixes

• Hardlinking matches its documented default: when the hardlink tool is installed it is auto-enabled (the docs always said "default if available"); --no-hardlink still opts out
• Honest restore file counts: a restore reports the number of files rsync actually transferred (parsed from --stats), not a count of everything already in the target; a selective restore that matches nothing now warns instead of silently "succeeding"
• Editor-backup files no longer mis-reported by diff: _should_skip now matches *~/~* as filename globs against the basename, so --diff/--compare-with stop flagging excluded editor-backup files as "only in source"
• --list survives an unreadable checkpoint: the per-checkpoint du is guarded, so one permission-denied directory no longer aborts the whole listing under set -e
• Disk-space check hardened: check_disk_space uses single-line POSIX df -P (a long, wrapped device name no longer mis-parses the Available column) and guards both du and df
• Diff/compare documented as viewers: --help and the manpage now state that --diff/--compare-with always exit 0, whether or not differences are found

v1.7.5 - Critical Review Medium-Severity Fixes

• Stale-lock recovery hardened: A lock with an empty or non-positive PID is now reclaimed instead of being treated as alive (kill -0 0 targets the caller's own process group and would otherwise block backups for up to 24h)
• Prune and restore now lock: --prune-only and --restore acquire the backup-directory lock, so they no longer race a concurrent backup (which could delete a checkpoint mid-restore or mid-hardlink)
• Dotfile-only restore targets confirmed: The overwrite confirmation now detects hidden-file-only targets (.git, .env, …) instead of treating them as empty and silently overwriting
• Manpage corrected: Fixed the false .gudang default-storage claim in DESCRIPTION and synced the manpage version with the script

v1.7.4 - Critical Review Security and Data-Safety Fixes

• Lock arithmetic injection fixed: Lock PID/timestamp files are now read as strings and integer-validated before use, instead of being assigned to declare -i variables (which arithmetic-evaluate — and thus execute — embedded command substitutions)
• Quiet mode no longer skips restore confirmation: The overwrite-confirmation safety gate now runs regardless of -q; only its informational output is suppressed
• Checkpoint-ID path traversal blocked: --from/--compare-with reject ids containing path separators or .., and resolution is constrained to real TIMESTAMP_PATTERN checkpoints (never .tmp.* or the lock dir)
• Selective restore descends into subdirectories: --files 'docs/*.md' now restores matching files in nested directories (added --include=*/)
• Timestamp collisions no longer corrupt backups: A same-second/reused-suffix name now gets a unique -N suffix and mv -T is used, instead of silently nesting the new backup inside the existing checkpoint and reporting stale data as success
• Lock read no longer crashes on a missing timestamp file: The timestamp read is guarded so a partial lock no longer aborts the script under errexit
• Restore/compare no longer abort via SIGPIPE: Checkpoint resolution uses mapfile instead of find | sort -r | head, which could make sort die on a closed pipe with large checkpoint counts

v1.7.1 - BCS Compliance and Bug Fixes

• UPPER_CASE globals: All 23 global configuration variables renamed to UPPER_CASE convention for clear distinction from local variables
• Stream separation: Status messages now routed to stderr via messaging functions; only data output (backup paths, list output) goes to stdout
• New success() function: Completes the messaging system (info, success, warn, error)
• Bug fix: Fixed backup_timestamp unreachable guard in prune_backups() under inherit_errexit
• Performance: Replaced 6 external basename calls with ${var##*/} parameter expansion
• Arithmetic fix: Replaced -gt in [[ ]] with (( )) for BCS0501 compliance

v1.7.0 - Simplification

• Removed unused features: Removed --debug, --verify, --metadata, and --remote options
• Reduced codebase: Cut ~970 lines of unused code for easier maintenance
• Simplified CLI: Fewer options, clearer purpose

v1.6.1 - Documentation and Tooling

• Comprehensive manpage: Full Unix manpage with all 35+ options documented
• Enhanced bash completion: Dynamic checkpoint ID completion for restore/compare operations
• Script documentation: All scripts updated with headers, usage docs, and #fin markers
• Installer improvements: Now installs manpage and bash completion automatically
• Fixed test scripts: Corrected corrupted shebangs and BCS compliance issues

v1.6.0 - Code Quality and Standards Compliance

• Full BASH-CODING-STANDARD.md compliance: Refactored entire codebase to meet strict coding standards
• Enhanced messaging system: New standardized output functions with visual indicators (✓ for success, ✗ for errors)
• Improved variable handling: Proper type declarations for all variables (integers, arrays, strings)
• Better error handling: Consistent error codes and messaging throughout
• Verification improvements: Fixed file exclusion handling during backup verification
• Code modernization: Updated arithmetic operations, fixed shellcheck warnings, improved quoting

v1.5.0 - Atomic Operations

• Implemented atomic backup operations using temporary directories
• Added automatic cleanup of interrupted operations
• Ensured backup integrity with atomic rename after completion
• Applied atomic pattern to both local and remote operations

v1.4.0 - Concurrency Protection

• Added lockfile mechanism to prevent concurrent operations
• Implemented PID-based lock ownership verification
• Added stale lock detection and automatic cleanup
• Introduced --no-lock, --lock-timeout, and --force-unlock options

For detailed version history, see the commit log or check checkpoint --version.