marcoallegretti/karapace
1
0
Fork 0
mirror of https://github.com/marcoallegretti/karapace.git synced 2026-03-26 13:33:09 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs: rewrite all documentation from source code

Browse source 
Delete 14 old docs files (AI-generated, riddled with Phase/M1/1.0
jargon, references to non-existent commands, stale CI snippets).

New documentation (6 files), written from repository source analysis:
- docs/architecture.md — crate graph, engine lifecycle, identity
  computation, runtime backends, store design, WAL, GC, unsafe blocks
- docs/cli-reference.md — all 23 commands with syntax, args, flags,
  exit codes, env vars, verified against crates/karapace-cli/src/main.rs
- docs/storage-format.md — directory layout, objects, layers, metadata,
  manifest format, lock file, WAL, atomic write contract
- docs/security-model.md — mount/device/env var policies with exact
  defaults from security.rs, trust assumptions, what is NOT protected
- docs/build-and-reproducibility.md — CI env vars, RUSTFLAGS, cargo
  profile, reproducibility verification, toolchain pinning
- docs/contributing.md — setup, verification, project layout, code
  standards, testing, CI workflows

README.md rewritten: concise, no marketing language, prerequisites
first, usage example, command table, limitations section.

CONTRIBUTING.md now points to docs/contributing.md.
CHANGELOG.md cleaned: removed M1-M8 labels, Phase refs, stale counts.
This commit is contained in:
Marco Allegretti 2026-02-23 01:06:42 +01:00
parent 864d5c45f6
commit e6e0f3dd6d
21 changed files with 1047 additions and 1898 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

26
CHANGELOG.md
View file

@ -4,7 +4,7 @@ All notable changes to Karapace will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
## [Unreleased] — 2.0 Hardening
## [Unreleased]
### Breaking Changes
@ -14,19 +14,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
- **`Engine::gc()` requires `&StoreLock`** — compile-time enforcement that callers hold the store lock before garbage collection.
- **MetadataStore checksum** — `EnvMetadata` now includes an optional `checksum` field (blake3). Written on every `put()`, verified on every `get()`. Backward-compatible via `serde(default)`.
### Added — 2.0 Hardening (M1–M8)
- **M1: WAL Crash Safety** — Fixed race windows in `build()` and `restore()` (rollback registered before side-effects). Added WAL protection to `destroy()`, `commit()` (layer manifest rollback), and `gc()` (WAL marker). 8 new WAL crash-safety tests.
- **M2: Integrity Hardening** — `LayerStore::get()` verifies blake3 hash on every read. `MetadataStore` embeds and verifies blake3 checksum. `verify_store_integrity()` expanded to check objects, layers, and metadata. 4 new integrity tests.
- **M3: GC Safety** — `Engine::gc()` now requires `&StoreLock` parameter (type-enforced). Snapshot layers whose parent is a live base layer are preserved during GC.
- **M4: Remote Protocol** — `X-Karapace-Protocol: 1` header sent on all HTTP backend requests (PUT, GET, HEAD). `PROTOCOL_VERSION` constant exported from `karapace-remote`. 4 new header/auth verification tests via header-capturing mock server.
- **M5: unwrap() Audit** — 0 `unwrap()` in production `src/` code. 4 `Mutex::lock().unwrap()` calls in `MockBackend` replaced with proper `RuntimeError` propagation.
- **M6: Failure Mode Testing** — 11 new tests: WAL write failure on read-only dir, build failure when WAL dir is read-only (disk-full simulation), stop() SIGTERM with real process (ESRCH path), stop() with non-existent PID, permission denied on object read, read-only metadata dir, concurrent GC lock contention, layer corruption detection, metadata corruption detection, destroy nonexistent env, invalid manifest.
- **M7: Coverage Expansion** — `verify_store_integrity()` now checks objects + layers + metadata (was objects-only). `IntegrityReport` expanded with `layers_checked/passed` and `metadata_checked/passed` fields. New tests: freeze/archive state transitions, rename environment, verify-store after fresh build, rebuild preserves new and cleans old, HTTP list_blobs, large (1MB) blob roundtrip.
- **Total: 417 tests** (24 ignored, require privileged operations). Clippy `-D warnings` clean. `cargo fmt` clean. Release build OK.
### Added — 1.0 Preparation
### Added
- **WAL crash safety** — Fixed race windows in `build()` and `restore()` (rollback registered before side-effects). Added WAL protection to `destroy()`, `commit()` (layer manifest rollback), and `gc()` (WAL marker).
- **Integrity hardening** — `LayerStore::get()` verifies blake3 hash on every read. `MetadataStore` embeds and verifies blake3 checksum. `verify_store_integrity()` expanded to check objects, layers, and metadata.
- **GC safety** — `Engine::gc()` now requires `&StoreLock` parameter (type-enforced). Snapshot layers whose parent is a live base layer are preserved during GC.
- **Remote protocol headers** — `X-Karapace-Protocol: 1` header sent on all HTTP backend requests (PUT, GET, HEAD). `PROTOCOL_VERSION` constant exported from `karapace-remote`.
- **unwrap() audit** — 0 `unwrap()` in production code. `Mutex::lock().unwrap()` calls in `MockBackend` replaced with proper `RuntimeError` propagation.
- **Failure mode tests** — WAL write failure, build on read-only WAL dir, stop() with SIGTERM/non-existent PID, permission denied on object read, read-only metadata dir, concurrent GC lock contention, layer/metadata corruption detection.
- **Coverage expansion** — `verify_store_integrity()` now checks objects + layers + metadata. `IntegrityReport` expanded with `layers_checked/passed` and `metadata_checked/passed` fields.
- **Real tar layers** — `pack_layer()`/`unpack_layer()` in karapace-store: deterministic tar creation (sorted entries, zero timestamps, owner 0:0) for regular files, directories, and symlinks. Content-addressed via blake3.
- **Snapshot system** — `Engine::commit()` captures overlay upper as a tar snapshot; `Engine::restore()` atomically unpacks a snapshot via staging directory swap; `Engine::list_snapshots()` lists snapshots for an environment.
- **CLI: `snapshots` and `restore`** — new commands for snapshot management.
@ -34,9 +30,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
- **Newtype wrappers threaded through all structs** — `EnvId`, `ShortId`, `ObjectHash`, `LayerHash` now used in `EnvMetadata` across all 8 crates. Transparent serde for backward compatibility.
- **Engine::push/pull** — transfer logic moved from `karapace-remote` to `Engine` methods. `karapace-remote` is now pure I/O.
- **CoreError::Remote** — new error variant for remote operation failures.
- **CLI stability contract** — `docs/cli-stability.md` defines stable command signatures for 1.x.
- **CLI stability contract** — `docs/cli-stability.md` defines CLI stability expectations.
- **Remote protocol spec** — `docs/protocol-v1.md` (v1-draft) documents blob store routes, push/pull protocol, registry format.
- **Layer limitations doc** — `docs/layer-limitations-v1.md` documents Phase 1 limits (no xattrs, device nodes, hardlinks).
- **Layer limitations doc** — `docs/layer-limitations.md` documents current limits (no xattrs, device nodes, hardlinks).
### Changed

98
CONTRIBUTING.md
View file

@ -1,97 +1,5 @@
# Contributing to Karapace
# Contributing
## Development Setup
See [docs/contributing.md](docs/contributing.md) for setup, code standards, testing, and CI details.
```bash
# Clone and build
git clone https://github.com/marcoallegretti/karapace.git
cd karapace
cargo build
# Run tests
cargo test --workspace
# Full verification (must pass before submitting)
cargo fmt --all --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
cargo build --release --workspace
```
## Project Structure
```
crates/
karapace-schema/ # Manifest parsing, normalization, lock file, identity hashing
karapace-store/ # Content-addressable store, metadata, layers, GC, integrity
karapace-runtime/ # Container runtime: images, sandbox, host integration, security
karapace-core/ # Build engine, lifecycle state machine, drift control, concurrency
karapace-remote/ # Remote store client, push/pull, registry
karapace-server/ # Reference remote server (tiny_http)
karapace-tui/ # Terminal UI (ratatui)
karapace-cli/ # CLI interface (23 commands)
karapace-dbus/ # D-Bus desktop integration (optional)
docs/ # Public documentation and specifications
examples/ # Ready-to-use manifest examples
data/ # systemd and D-Bus service files
```
## Architecture Principles
Before implementing any feature, verify it aligns with these principles:
1. **Determinism first.** Same manifest + lock = identical environment, always.
2. **No hidden mutable state.** All state changes are explicit and tracked.
3. **No silent drift.** Overlay changes are visible via `diff` and must be committed explicitly.
4. **No privilege escalation.** Everything runs as the unprivileged user.
5. **Convenience must not break reproducibility.** If there's a conflict, determinism wins.
See [Architecture Overview](docs/architecture.md) for the full design.
## Code Standards
- **Zero warnings**: `cargo clippy --workspace --all-targets -- -D warnings` must pass.
- **Formatted**: `cargo fmt --all --check` must pass.
- **No `unwrap()` in production code** (test code is fine).
- **No `TODO`/`FIXME`/`HACK`** in committed code.
- **All values interpolated into shell scripts must use `shell_quote()`.**
- **All mutating operations must hold a `StoreLock`.**
- **All file writes must be atomic** (use `NamedTempFile` + `persist()`).
## Testing
- All new features must include tests.
- Run the full test suite: `cargo test --workspace`
- Integration tests go in `crates/karapace-core/tests/`.
- Unit tests go in the relevant module as `#[cfg(test)] mod tests`.
## Submitting Changes
1. Fork the repository.
2. Create a feature branch from `main`.
3. Make your changes, ensuring all verification checks pass.
4. Submit a pull request with a clear description.
## Building the D-Bus Service
The D-Bus service is not compiled by default:
```bash
# Core CLI only (default)
cargo build --release
# Include D-Bus service
cargo build --release --workspace
```
## Generating Shell Completions
```bash
karapace completions bash > /etc/bash_completion.d/karapace
karapace completions zsh > /usr/share/zsh/site-functions/_karapace
karapace completions fish > ~/.config/fish/completions/karapace.fish
```
## License
By contributing, you agree that your contributions will be licensed under the European Union Public Licence v1.2 (EUPL-1.2).
By contributing, you agree that your contributions will be licensed under EUPL-1.2.

290
README.md
View file

@ -3,165 +3,20 @@
[![CI](https://github.com/marcoallegretti/karapace/actions/workflows/ci.yml/badge.svg)](https://github.com/marcoallegretti/karapace/actions/workflows/ci.yml)
[![License: EUPL-1.2](https://img.shields.io/badge/License-EUPL_1.2-blue.svg)](LICENSE)
A deterministic container environment engine for immutable Linux systems.
Deterministic, content-addressed container environments for Linux. No root. No daemon.
Karapace creates isolated, reproducible development environments using Linux namespaces and overlay filesystems — no root, no daemon, no Docker. Environments are content-addressed artifacts derived from declarative TOML manifests, with full host integration for GPU, audio, Wayland, and desktop applications.
Karapace builds isolated development environments from declarative TOML manifests using Linux user namespaces and overlay filesystems. Each environment gets a unique identity (blake3 hash) derived from its fully resolved dependencies.
## What Karapace Is (and Isn't)
## Prerequisites
Karapace is **the identity and certainty layer** for reproducible environments. It is not a general-purpose container runtime.
- Linux with user namespaces (`CONFIG_USER_NS=y`)
- `fuse-overlayfs`
- `curl`
- Optional: `crun`/`runc`/`youki` (OCI backend)
| Need | Tool |
|---|---|
| Full system container lifecycle, advanced networking, snapshots | Incus, LXD, Podman |
| Deterministic, content-addressed, reproducible environments | **Karapace** |
| Quick disposable containers with zero config | Distrobox |
| Determinism + simplicity for casual use | **Karapace** `quick` command |
Run `karapace doctor` to check.
Karapace is **complementary** — it can sit on top of or alongside any container runtime. Users needing full container features use a runtime; users needing reproducibility and traceability use Karapace. The `quick` command bridges the gap for users who want both simplicity and determinism.
## Features
- **Real container isolation** — Linux user namespaces (`unshare`), `fuse-overlayfs`, `chroot`
- **No root required** — runs entirely as an unprivileged user
- **No daemon** — direct CLI, no background service needed
- **Multi-distro images** — openSUSE, Ubuntu, Debian, Fedora, Arch from LXC image servers
- **Package installation** — `zypper`, `apt`, `dnf`, `pacman` inside the container
- **Host integration** — home directory, Wayland, PipeWire, D-Bus, GPU (`/dev/dri`), audio (`/dev/snd`), SSH agent, fonts, themes
- **Desktop app export** — export GUI apps as `.desktop` files on the host
- **OCI runtime support** — optional `crun`/`runc`/`youki` backend
- **Content-addressable store** — deterministic hashing, deduplication, integrity verification
- **Overlay drift control** — diff, freeze, commit, export writable layer changes
- **OSC 777 terminal markers** — container-aware terminal integration (Konsole, etc.)
## Crate Layout
| Crate | Purpose |
|---|---|
| `karapace-schema` | Manifest v1 parsing, normalization, identity hashing, lock file |
| `karapace-store` | Content-addressable store, layers, metadata, GC, integrity |
| `karapace-runtime` | Container runtime: image download, sandbox, host integration, app export |
| `karapace-core` | Build engine, lifecycle state machine, drift control, concurrency |
| `karapace-cli` | Full CLI interface (23 commands) |
| `karapace-dbus` | Socket-activated D-Bus desktop integration (**optional**, feature-gated) |
| `karapace-remote` | Remote content-addressable store, push/pull, registry |
| `karapace-server` | Reference remote server implementing protocol v1 (tiny_http) |
| `karapace-tui` | Terminal UI for environment management (ratatui) |
## CLI Commands (23)
```
karapace build [manifest] # Build environment from manifest
karapace rebuild [manifest] # Destroy + rebuild
karapace enter <env_id> [-- cmd...] # Enter environment (or run a command)
karapace exec <env_id> -- <cmd...> # Execute command inside environment
karapace destroy <env_id> # Destroy environment
karapace stop <env_id> # Stop a running environment
karapace freeze <env_id> # Freeze environment
karapace archive <env_id> # Archive environment (preserve, prevent entry)
karapace list # List all environments
karapace inspect <env_id> # Show environment metadata
karapace diff <env_id> # Show overlay drift
karapace snapshots <env_id> # List snapshots for an environment
karapace commit <env_id> # Commit overlay drift as snapshot
karapace restore <env_id> <snapshot> # Restore overlay from snapshot
karapace gc [--dry-run] # Garbage collect store
karapace verify-store # Check store integrity
karapace push <env_id> [--tag name@tag] # Push environment to remote store
karapace pull <reference> [--remote url] # Pull environment from remote store
karapace rename <env_id> <name> # Rename environment
karapace doctor # Run diagnostic checks on system and store
karapace migrate # Check store version and migration guidance
karapace completions <shell> # Generate shell completions
karapace man-pages [dir] # Generate man pages
```
All commands support `--json` for structured output, `--store <path>` for custom store location, `--verbose` / `-v` for debug logging, and `--trace` for trace-level output.
Set `KARAPACE_LOG=debug` (or `info`, `warn`, `error`, `trace`) for fine-grained log control.
## Quick Start
```bash
# Build (core CLI only — D-Bus service is opt-in)
cargo build --release
# Build with D-Bus desktop integration
cargo build --release --workspace
```
```bash
# Write a manifest
cat > karapace.toml << 'EOF'
manifest_version = 1
[base]
image = "rolling" # openSUSE Tumbleweed (or "ubuntu/24.04", "fedora/41", "arch", etc.)
[system]
packages = ["git", "curl", "vim"]
[hardware]
gpu = true
audio = true
[runtime]
backend = "namespace"
EOF
karapace build
karapace enter <env_id>
# Run a command inside without interactive shell
karapace exec <env_id> -- git --version
# Snapshot and restore
karapace commit <env_id>
karapace snapshots <env_id>
karapace restore <env_id> <snapshot_hash>
# Push/pull to remote
karapace push <env_id> --tag my-env@latest
karapace pull my-env@latest
# List environments
karapace list
```
## Example Manifests
Ready-to-use manifests in `examples/`:
| File | Description |
|---|---|
| `examples/minimal.toml` | Bare openSUSE system, no extras |
| `examples/dev.toml` | Developer tools (git, vim, tmux, gcc, clang) |
| `examples/gui-dev.toml` | GUI development with GPU + audio passthrough |
| `examples/ubuntu-dev.toml` | Ubuntu-based with Node.js, Python, build-essential |
| `examples/rust-dev.toml` | Rust development environment |
## Shell Completions
```bash
# Bash
karapace completions bash > /etc/bash_completion.d/karapace
# Zsh
karapace completions zsh > /usr/share/zsh/site-functions/_karapace
# Fish
karapace completions fish > ~/.config/fish/completions/karapace.fish
```
## Man Pages
```bash
karapace man-pages /usr/share/man/man1
```
## Installation
### From Source (recommended)
## Install
```bash
git clone https://github.com/marcoallegretti/karapace.git
@ -170,53 +25,106 @@ cargo build --release
sudo install -Dm755 target/release/karapace /usr/local/bin/karapace
```
### With D-Bus Service
```bash
cargo build --release --workspace
sudo install -Dm755 target/release/karapace /usr/local/bin/karapace
sudo install -Dm755 target/release/karapace-dbus /usr/local/bin/karapace-dbus
sudo install -Dm644 data/dbus/org.karapace.Manager1.service /usr/share/dbus-1/services/
sudo install -Dm644 data/systemd/karapace-dbus.service /usr/lib/systemd/user/
```
### Via Cargo
Or via cargo:
```bash
cargo install --git https://github.com/marcoallegretti/karapace.git karapace-cli
```
## Prerequisites
## Usage
- Linux with user namespace support (`CONFIG_USER_NS=y`)
- `fuse-overlayfs` (for overlay filesystem)
- `curl` (for image downloads)
- Optional: `crun`/`runc`/`youki` (for OCI backend)
```bash
# Create a manifest
cat > karapace.toml << 'EOF'
manifest_version = 1
Karapace checks for missing prerequisites at startup and provides distro-specific install instructions.
[base]
image = "rolling"
[system]
packages = ["git", "curl"]
[runtime]
backend = "namespace"
EOF
# Build and enter
karapace build
karapace enter <env_id>
# Run a command
karapace exec <env_id> -- git --version
# Snapshot
karapace commit <env_id>
karapace restore <env_id> <snapshot_hash>
```
See `examples/` for more manifests: `minimal.toml`, `dev.toml`, `gui-dev.toml`, `ubuntu-dev.toml`, `rust-dev.toml`.
## Commands
23 commands. All accept `--json`, `--store <path>`, `--verbose`, `--trace`.
```
build [manifest] [--name] Build environment
rebuild [manifest] [--name] Destroy + rebuild
enter <id> [-- cmd...] Enter environment
exec <id> -- <cmd...> Run command in environment
destroy <id> Destroy environment
stop <id> Stop running environment
freeze <id> Freeze (prevent writes)
archive <id> Archive (prevent entry)
list List environments
inspect <id> Show metadata
diff <id> Show overlay changes
snapshots <id> List snapshots
commit <id> Snapshot overlay
restore <id> <hash> Restore snapshot
gc [--dry-run] Garbage collect
verify-store Check store integrity
push <id> [--tag] [--remote] Push to remote
pull <ref> [--remote] Pull from remote
rename <id> <name> Rename environment
doctor Check prerequisites
migrate Check store version
completions <shell> Shell completions
man-pages [dir] Generate man pages
```
## Workspace
9 crates:
```
karapace-schema Manifest, normalization, lock file, identity
karapace-store Objects, layers, metadata, WAL, GC, integrity
karapace-runtime Backends (namespace/oci/mock), images, security
karapace-core Engine: lifecycle orchestration
karapace-cli CLI (23 commands)
karapace-dbus D-Bus service (optional)
karapace-tui Terminal UI (optional)
karapace-remote Remote store client, push/pull
karapace-server Reference HTTP server
```
## Limitations
- Linux only.
- Layer packing drops: extended attributes, device nodes, hardlinks, SELinux labels, ACLs.
- Base images are content-hashed but not GPG-verified.
- No MAC enforcement (SELinux/AppArmor) inside containers.
- Remote protocol has no authentication yet.
## Documentation
- **[Getting Started Guide](docs/getting-started.md)** — installation, first use, common workflows
- [Architecture Overview](docs/architecture.md)
- [Manifest v1 Specification](docs/manifest-spec.md)
- [Lock File v2 Specification](docs/lock-spec.md)
- [Store Format v2 Specification](docs/store-spec.md)
- [Hash Contract](docs/hash-contract.md)
- [Architecture](docs/architecture.md)
- [CLI Reference](docs/cli-reference.md)
- [Storage Format](docs/storage-format.md)
- [Security Model](docs/security-model.md)
- [CLI Stability Contract](docs/cli-stability.md)
- [Remote Protocol v1 (Draft)](docs/protocol-v1.md)
- [Layer Limitations (Phase 1)](docs/layer-limitations-v1.md)
- [Public API Reference](docs/api-reference.md)
- [Versioning Policy](docs/versioning-policy.md)
- [Verification & Supply Chain](docs/verification.md)
- [E2E Testing](docs/e2e-testing.md)
- [Build and Reproducibility](docs/build-and-reproducibility.md)
- [Contributing](docs/contributing.md)
## Verification
## License
```bash
cargo fmt --all --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
cargo build --release --workspace
```
[EUPL-1.2](LICENSE)

213
docs/api-reference.md
View file

@ -1,213 +0,0 @@
# Karapace Public API Reference
## CLI Commands
### Environment Lifecycle
#### `karapace init [manifest]`
Initialize an environment from a manifest without building. Creates metadata and a preliminary lock file.
- **Default manifest**: `karapace.toml`
#### `karapace build [manifest]`
Build an environment. Resolves dependencies, computes canonical identity, creates store objects/layers, and writes the lock file.
- **Default manifest**: `karapace.toml`
#### `karapace enter <env_id> [-- cmd...]`
Enter a built environment interactively, or run a command if `-- cmd` is provided. Transitions state to Running, then back to Built on exit.
- Accepts full env_id or short_id prefix.
#### `karapace exec <env_id> -- <cmd...>`
Execute a command inside a built environment (non-interactive). Prints stdout/stderr.
#### `karapace rebuild [manifest]`
Atomically rebuild an environment. Builds the new environment first; the old one is only destroyed after a successful build.
- Produces the same env_id for the same resolved manifest.
- If the build fails, the existing environment is preserved.
#### `karapace stop <env_id>`
Stop a running environment by sending SIGTERM/SIGKILL to its process.
#### `karapace freeze <env_id>`
Freeze an environment, preventing further entry. Transitions to Frozen state.
#### `karapace archive <env_id>`
Archive an environment. Preserves it in the store but prevents entry. Can be rebuilt later.
#### `karapace destroy <env_id>`
Destroy an environment's overlay and decrement its reference count.
- **Cannot destroy a running environment** — stop it first.
### Drift Control
#### `karapace diff <env_id>`
Show drift in the writable overlay. Lists added, modified, and removed files.
#### `karapace commit <env_id>`
Commit overlay drift into the content store as a snapshot layer.
- Only works on Built or Frozen environments.
#### `karapace export <env_id> <dest>`
Copy the writable overlay contents to a destination directory.
### Store Management
#### `karapace gc [--dry-run]`
Run garbage collection. Removes orphaned environments, layers, and objects.
- `--dry-run`: report only, do not delete.
#### `karapace verify-store`
Verify integrity of all objects in the store (blake3 content hash check).
#### `karapace verify-lock [manifest]`
Verify lock file integrity (recomputed env_id matches) and manifest consistency (no drift between manifest and lock).
### Inspection
#### `karapace inspect <env_id>`
Show environment metadata: state, layers, ref count, timestamps.
#### `karapace list`
List all known environments with short_id, state, and env_id.
#### `karapace validate [manifest]`
Validate a manifest file and print its preliminary env_id.
### Desktop Integration
#### `karapace export-app <env_id> <name> <binary>`
Export a GUI application from an environment as a `.desktop` file on the host.
#### `karapace unexport-app <env_id> <name>`
Remove an exported application's `.desktop` file from the host.
### Image Management
#### `karapace list-images`
List cached container images with status and size.
#### `karapace remove-image <name>`
Remove a cached container image from the store.
### Quick Start
#### `karapace quick [image] [-p packages] [--gpu] [--audio] [--enter]`
One-step environment creation for casual users. Generates a manifest from CLI flags, builds, and optionally enters.
- **Default image**: `rolling` (openSUSE Tumbleweed)
- `-p` / `--packages`: Comma-separated list of packages to install.
- `--gpu`: Enable GPU passthrough.
- `--audio`: Enable audio passthrough.
- `-e` / `--enter`: Enter the environment immediately after building.
- A real manifest and lock file are still generated (determinism is preserved).
Examples:
```bash
karapace quick rolling -p git,curl --enter
karapace quick ubuntu/24.04 -p build-essential,cmake --gpu
karapace quick fedora/41 --enter
```
### Tooling
#### `karapace completions <shell>`
Generate shell completions for the specified shell. Supported: `bash`, `zsh`, `fish`, `elvish`, `powershell`.
```bash
karapace completions bash > /etc/bash_completion.d/karapace
karapace completions zsh > /usr/share/zsh/site-functions/_karapace
karapace completions fish > ~/.config/fish/completions/karapace.fish
```
#### `karapace man-pages [dir]`
Generate man pages for all commands in the specified directory.
- **Default directory**: `man`
#### `karapace push <env_id> [--tag <name@tag>] [--remote <url>]`
Push an environment (metadata + layers + objects) to a remote store. Skips blobs that already exist remotely.
- **`--tag`**: Publish under a registry key (e.g. `my-env@latest`).
- **`--remote`**: Remote store URL (overrides `~/.config/karapace/remote.json`).
#### `karapace pull <reference> [--remote <url>]`
Pull an environment from a remote store. Reference can be a registry key (e.g. `my-env@latest`) or a raw env_id.
- **`--remote`**: Remote store URL (overrides config).
#### `karapace remote-list [--remote <url>]`
List environments in the remote registry.
## Global Flags
| Flag | Description |
|---|---|
| `--store <path>` | Custom store location (default: `~/.local/share/karapace`). |
| `--json` | Structured JSON output for all applicable commands. |
| `--verbose` / `-v` | Enable debug-level logging output. |
### Environment Variables
| Variable | Description |
|---|---|
| `KARAPACE_LOG` | Log level filter: `error`, `warn` (default), `info`, `debug`, `trace`. |
| `KARAPACE_STORE` | Override default store path (used by the D-Bus service). |
## Exit Codes
| Code | Meaning |
|---|---|
| `0` | Success. |
| `1` | General error. |
| `2` | Manifest validation error. |
| `3` | Store integrity error. |
## D-Bus API (Optional)
Interface: `org.karapace.Manager1`
Path: `/org/karapace/Manager1`
The D-Bus service exits after 30 seconds of idle (socket activation). Build with `cargo build -p karapace-dbus`.
All methods return proper D-Bus errors (`org.freedesktop.DBus.Error.Failed`) on failure. Mutating methods acquire the store lock automatically. Methods accepting `id_or_name` resolve by env_id, short_id, name, or prefix.
Desktop notifications are sent on build success/failure via `org.freedesktop.Notifications`.
### Properties
| Property | Type | Description |
|---|---|---|
| `ApiVersion` | `u32` | API version (currently `1`). |
| `StoreRoot` | `String` | Path to the store directory. |
### Methods
| Method | Signature | Description |
|---|---|---|
| `ListEnvironments` | `() → String` | JSON array of `{env_id, short_id, name?, state}`. |
| `GetEnvironmentStatus` | `(id_or_name) → String` | JSON status. Resolves by name. |
| `GetEnvironmentHash` | `(id_or_name) → String` | Returns env_id hash. Resolves by name. |
| `BuildEnvironment` | `(manifest_path) → String` | Build from manifest path. Sends notification. |
| `BuildNamedEnvironment` | `(manifest_path, name) → String` | Build and assign a name. Sends notification. |
| `DestroyEnvironment` | `(id_or_name) → String` | Destroy environment. Resolves by name. |
| `RunEnvironment` | `(id_or_name) → String` | Enter environment. Resolves by name. |
| `RenameEnvironment` | `(id_or_name, new_name) → String` | Rename an environment. |
| `ListPresets` | `() → String` | JSON array of `{name, description}` for built-in presets. |
| `GarbageCollect` | `(dry_run) → String` | Run GC. Acquires store lock. |
| `VerifyStore` | `() → String` | Verify store integrity. Returns `{checked, passed, failed}`. |
## Rust Crate API
### `karapace-core::Engine`
- `Engine::new(store_root)` — Create engine instance
- `Engine::init(manifest_path)` — Initialize environment (Defined state)
- `Engine::build(manifest_path)` — Full resolve → lock → build pipeline
- `Engine::enter(env_id)` — Enter environment interactively
- `Engine::exec(env_id, command)` — Execute command in environment
- `Engine::rebuild(manifest_path)` — Destroy + rebuild
- `Engine::stop(env_id)` — Stop running environment
- `Engine::freeze(env_id)` — Freeze environment
- `Engine::archive(env_id)` — Archive environment (preserve, prevent entry)
- `Engine::commit(env_id)` — Commit overlay drift
- `Engine::destroy(env_id)` — Destroy environment
- `Engine::inspect(env_id)` — Get environment metadata
- `Engine::list()` — List all environments
- `Engine::gc(dry_run)` — Garbage collection (caller must hold store lock)
- `Engine::set_name(env_id, name)` — Set or clear environment name
- `Engine::rename(env_id, new_name)` — Rename environment

249
docs/architecture.md
View file

@ -1,90 +1,183 @@
# Karapace Architecture
# Architecture
## Overview
## Workspace
Karapace is a deterministic container environment engine organized as a Cargo workspace of 9 crates. Each crate has a single responsibility and clean dependency boundaries.
## Crate Dependency Graph
Karapace is a Cargo workspace of 9 crates.
```
karapace-cli ─────┬──▶ karapace-core ──┬──▶ karapace-schema
│ ├──▶ karapace-store
│ ├──▶ karapace-runtime
│ └──▶ karapace-remote
├──▶ karapace-remote ──▶ karapace-store
└──▶ karapace-tui ────▶ karapace-core
karapace-dbus ────────▶ karapace-core
karapace-server ──────▶ karapace-store (standalone HTTP server)
karapace-schema Manifest parsing, normalization, lock file, identity hashing
karapace-store Content-addressable object store, layers, metadata, WAL, GC
karapace-runtime Container backends, image cache, security policy, prerequisites
karapace-core Engine: orchestrates the full environment lifecycle
karapace-cli CLI binary (23 commands, clap)
karapace-dbus D-Bus service (org.karapace.Manager1, zbus)
karapace-tui Terminal UI (ratatui, crossterm)
karapace-remote Remote store client: HTTP backend, registry, push/pull
karapace-server Reference HTTP server for remote store (tiny_http)
```
## Crate Responsibilities
### `karapace-schema`
Manifest v1 parsing (TOML), normalization, canonical JSON serialization, environment identity hashing, lock file v2 (resolved packages, integrity/intent verification), and built-in presets.
### `karapace-store`
Content-addressable object store (blake3), layer manifests, environment metadata (with naming, ref-counting, state machine), garbage collection with signal cancellation, and store integrity verification. All writes are atomic via `NamedTempFile` + `persist`.
### `karapace-runtime`
Container runtime abstraction (`RuntimeBackend` trait) with three backends:
- **Namespace** — `unshare` + `fuse-overlayfs` + `chroot` (unprivileged)
- **OCI** — `crun`/`runc`/`youki`
- **Mock** — deterministic test backend
Also handles image downloading, sandbox scripting, host integration (Wayland, GPU, audio, D-Bus), security policy enforcement, and desktop app export.
### `karapace-core`
The `Engine` struct orchestrates the full lifecycle: init → resolve → lock → build → enter/exec → freeze → archive → destroy. Caches `MetadataStore`, `ObjectStore`, and `LayerStore` as fields. Handles drift detection (diff/commit/export via overlay upper_dir scanning) and garbage collection delegation.
### `karapace-cli`
23 CLI commands, each in its own file under `commands/`. Shared helpers in `commands/mod.rs` (spinners, colored output, environment resolution, JSON formatting). `main.rs` is a thin dispatcher. Exit codes: 0 (success), 1 (failure), 2 (manifest error), 3 (store error).
### `karapace-dbus`
Socket-activated D-Bus service (`org.karapace.Manager1`) with 11 methods. Typed serde response structs. Desktop notifications via `notify-rust`. 30-second idle timeout for socket activation. Hardened systemd unit file.
### `karapace-remote`
Remote content-addressable store with `RemoteBackend` trait, HTTP backend (ureq), push/pull transfer with blake3 integrity verification on pull, and a JSON registry for name@tag references.
### `karapace-tui`
Interactive terminal UI (ratatui + crossterm) with list/detail/help views, vim-style keybindings, search/filter, sort cycling, freeze/archive/rename actions, and confirmation dialogs.
## Key Design Decisions
1. **Content-addressed identity** — `env_id` is computed from the *resolved* lock file (pinned versions + base image content digest), not from unresolved manifest data.
2. **Atomic operations** — All store writes use `NamedTempFile` + `persist` for crash safety. Rebuild builds the new environment before destroying the old one.
3. **No `unwrap()` in production** — All error paths are handled with proper error types (`StoreError`, `CoreError`, `RemoteError`, `RuntimeError`).
4. **Store locking** — `StoreLock` file lock on all mutating operations (CLI + D-Bus). GC respects active/archived environments.
5. **Layered security** — Mount whitelist, device policy, env var allow/deny, resource limits. No privilege escalation.
## Data Flow
## Dependency graph
```
Manifest (TOML)
│
▼
NormalizedManifest (canonical JSON)
│
▼ resolve (RuntimeBackend)
ResolutionResult (base_image_digest + resolved_packages)
│
▼
LockFile v2 (pinned, verifiable)
│
▼ compute_identity()
EnvIdentity (env_id = blake3 of canonical lock)
│
▼ build (store objects + layers + metadata)
Built Environment (overlay filesystem)
karapace-cli ──┬──> karapace-core ──┬──> karapace-schema
│ ├──> karapace-store
│ ├──> karapace-runtime
│ └──> karapace-remote
├──> karapace-runtime
└──> karapace-store
karapace-dbus ────> karapace-core
karapace-tui ─────> karapace-core
karapace-remote ──> karapace-store
karapace-server ──> karapace-remote, karapace-store
```
### `karapace-server`
Reference remote server implementing protocol v1 over HTTP (tiny_http). Provides blob storage, registry, and list endpoints. Used for testing push/pull workflows.
## Engine lifecycle
## Test Coverage
`karapace-core::Engine` is the central orchestrator. All state transitions go through it.
417 tests across all crates. 24 ignored tests require privileged operations (real `unshare`, `fuse-overlayfs`, ENOSPC simulation, namespace access).
```
┌─────────┐
build() ───> │ Defined │
└────┬────┘
│ resolve → lock → build
v
┌─────────┐
│ Built │ <── rebuild()
└──┬──┬───┘
enter() │ │ │ freeze()
v │ v
┌─────────┐ ┌─────────┐
│ Running │ │ Frozen │
└─────────┘ └────┬────┘
│ archive()
v
┌──────────┐
│ Archived │
└──────────┘
```
State transitions are validated in `karapace-core/src/lifecycle.rs`. Invalid transitions return `CoreError`.
### Build pipeline
`Engine::build(manifest_path)` executes:
1. Parse manifest (`karapace-schema::parse_manifest_file`)
2. Normalize (`ManifestV1::normalize`) — sort packages, deduplicate, lowercase backend
3. Select runtime backend (`karapace-runtime::select_backend`)
4. Resolve — backend downloads base image, computes content digest, queries package manager for exact versions → `ResolutionResult`
5. Create lock file (`LockFile::from_resolved`) with pinned versions and content digest
6. Compute identity (`LockFile::compute_identity`) → `env_id` (blake3)
7. Store manifest as object, create layers, write metadata
8. Backend builds the environment filesystem
9. Write lock file to disk
### Identity computation
Defined in `karapace-schema/src/lock.rs::LockFile::compute_identity()`.
Input fed to blake3 in order:
- `base_digest:<content_hash>`
- `pkg:<name>@<version>` for each resolved package (sorted)
- `app:<name>` for each app (sorted)
- `hw:gpu` / `hw:audio` if enabled
- `mount:<label>:<host>:<container>` for each mount (sorted)
- `backend:<name>`
- `net:isolated` if enabled
- `cpu:<value>` / `mem:<value>` if set
Output: 64-character hex blake3 digest. First 12 characters = `short_id`.
## Container runtime
`karapace-runtime/src/backend.rs` defines `RuntimeBackend` trait:
```rust
pub trait RuntimeBackend: Send + Sync {
fn name(&self) -> &str;
fn available(&self) -> bool;
fn resolve(&self, spec: &RuntimeSpec) -> Result<ResolutionResult, RuntimeError>;
fn build(&self, spec: &RuntimeSpec) -> Result<(), RuntimeError>;
fn enter(&self, spec: &RuntimeSpec) -> Result<(), RuntimeError>;
fn exec(&self, spec: &RuntimeSpec, command: &[String]) -> Result<Output, RuntimeError>;
fn destroy(&self, spec: &RuntimeSpec) -> Result<(), RuntimeError>;
fn status(&self, env_id: &str) -> Result<RuntimeStatus, RuntimeError>;
}
```
Three backends (`karapace-runtime/src/backend.rs::select_backend`):
| Backend | Implementation | Use |
|---------|---------------|-----|
| `namespace` | `unshare` + `fuse-overlayfs` + `chroot` | Default. Unprivileged. |
| `oci` | `crun` / `runc` / `youki` | OCI-compatible runtimes. |
| `mock` | Deterministic stubs | Testing only. |
## Image cache
`karapace-runtime/src/image.rs::ImageCache` stores downloaded base images under `<store_root>/images/<cache_key>/rootfs/`.
Images are fetched from `images.linuxcontainers.org`. The content digest is a blake3 hash of the rootfs directory tree (`compute_image_digest`). Package manager is auto-detected from rootfs contents (`detect_package_manager`).
## Content-addressable store
All persistent data lives under `<store_root>/store/`. See [storage-format.md](storage-format.md) for the full layout.
- **Objects**: keyed by blake3 hash of content. Written atomically (tempfile + rename). Verified on every read.
- **Layers**: JSON manifests describing tar archives. Kinds: `Base`, `Dependency`, `Policy`, `Snapshot`.
- **Metadata**: JSON per environment. Includes state, layers, ref count, checksum.
## Snapshot mechanism
`Engine::commit(env_id)`:
1. Pack the overlay upper directory into a deterministic tar (`pack_layer`)
2. Store tar as object
3. Create a `Snapshot` layer manifest with composite hash: `blake3("snapshot:{env_id}:{base_layer}:{tar_hash}")`
`Engine::restore(env_id, snapshot_hash)`:
1. Retrieve snapshot layer and its tar object
2. Unpack to `store/staging/restore-{env_id}`
3. Atomic rename-swap with the environment's upper directory
Deterministic packing: entries sorted, timestamps zeroed, owner `0:0`, permissions preserved. Symlinks preserved. Extended attributes, device nodes, hardlinks, ACLs, SELinux labels are dropped.
## Garbage collection
`Engine::gc(store_lock, dry_run)` in `karapace-core/src/engine.rs`. Caller must hold `StoreLock`.
Protected from collection:
- Environments with state `Running` or `Archived`
- Layers referenced by any live environment
- Snapshot layers whose parent is a live base layer
- Objects referenced by any live layer or live metadata `manifest_hash`
Everything else is orphaned and removed. GC supports `SIGINT`/`SIGTERM` cancellation.
## Write-ahead log
`karapace-store/src/wal.rs`. JSON entries in `store/wal/`.
Operations tracked: `Build`, `Rebuild`, `Commit`, `Restore`, `Destroy`, `Gc`.
Each entry records rollback steps (`RemoveDir`, `RemoveFile`). On `Engine::new()`, incomplete WAL entries are replayed in reverse order, then deleted. Corrupt entries are silently removed.
## Concurrency
`karapace-core/src/concurrency.rs::StoreLock` uses `flock(2)` on `store/.lock`. All mutating CLI commands and D-Bus methods acquire this lock.
## Signal handling
`karapace-core/src/concurrency.rs::install_signal_handler()` registers `SIGINT`/`SIGTERM` via `ctrlc` crate. Sets an atomic flag checked by GC and long-running operations.
## Unsafe code
Five `unsafe` blocks in the codebase:
| Location | Call | Purpose |
|----------|------|---------|
| `karapace-core/src/engine.rs:455` | `libc::kill(SIGTERM)` | Stop running environment |
| `karapace-core/src/engine.rs:475` | `libc::kill(SIGKILL)` | Force-kill after timeout |
| `karapace-runtime/src/sandbox.rs:46` | `libc::getuid()` | Get current UID for namespace setup |
| `karapace-runtime/src/sandbox.rs:53` | `libc::getgid()` | Get current GID for namespace setup |
| `karapace-runtime/src/terminal.rs:41` | `libc::isatty()` | Detect terminal for interactive mode |

83
docs/build-and-reproducibility.md Normal file
View file

@ -0,0 +1,83 @@
# Build and Reproducibility
## Deterministic identity
The environment identity (`env_id`) is a blake3 hash computed from the fully resolved lock file — not from the unresolved manifest. Two identical lock files on any machine produce the same `env_id`.
The identity is computed in `karapace-schema/src/lock.rs::LockFile::compute_identity()`. See [architecture.md](architecture.md) for the full list of hash inputs.
## Lock file
`karapace build` writes `karapace.lock` next to the manifest. This file pins:
- Base image content digest (blake3 of rootfs)
- Exact package versions (queried from the package manager inside the image)
- All manifest-declared settings (hardware, mounts, backend, resource limits)
The lock file should be committed to version control.
## Reproducibility constraints
Given the same lock file and the same base image content, builds produce the same `env_id`. The overlay filesystem content depends on the package manager's behavior, which Karapace does not control.
Karapace guarantees identity reproducibility (same inputs → same `env_id`). It does not guarantee bit-for-bit filesystem reproducibility across different package repository states.
## CI release builds
Configured in `.github/workflows/ci.yml` and `.github/workflows/release.yml`.
### Environment variables
| Variable | Value | Purpose |
|----------|-------|---------|
| `CARGO_INCREMENTAL` | `0` | Disable incremental compilation |
| `SOURCE_DATE_EPOCH` | `0` | Deterministic timestamps in build artifacts |
### RUSTFLAGS
```
-D warnings
--remap-path-prefix /home/runner/work=src
--remap-path-prefix /home/runner/.cargo/registry/src=crate
--remap-path-prefix /home/runner/.rustup=rustup
```
Path remapping eliminates runner-specific filesystem paths from the binary.
### Cargo profile (release)
Defined in workspace `Cargo.toml`:
```toml
[profile.release]
strip = true
lto = "thin"
```
### Build procedure
1. `cargo clean` — eliminates stale intermediate artifacts
2. `cargo build --release --target <target> -p karapace-cli -p karapace-dbus`
Both glibc (`x86_64-unknown-linux-gnu`) and musl (`x86_64-unknown-linux-musl`) targets are built. Musl binaries are fully statically linked.
### Reproducibility verification in CI
CI runs two types of reproducibility checks:
- **Same-run:** two sequential builds on the same runner, output hashes compared
- **Cross-run:** builds on `ubuntu-latest` and `ubuntu-22.04`, hashes compared (warning-only — different system libraries may cause differences in glibc builds)
Musl builds are expected to be runner-independent.
**Constraint:** build invocations must use identical `-p` flags. Building `-p karapace-cli` alone may produce a different binary than `-p karapace-cli -p karapace-dbus` due to codegen unit ordering.
## Local development builds
`.cargo/config.toml` configures path remapping for the project maintainer. Other developers should update the paths or set `RUSTFLAGS` directly.
Local builds are for development. CI builds are the authoritative release artifacts.
## Pinned toolchain
CI pins Rust toolchain version via `RUST_TOOLCHAIN` environment variable (currently `1.93`). This is set in the workflow `env` block and used with `dtolnay/rust-toolchain@stable`.

286
docs/cli-reference.md Normal file
View file

@ -0,0 +1,286 @@
# CLI Reference
Binary: `karapace`. Defined in `crates/karapace-cli/src/main.rs`.
## Global flags
| Flag | Default | Description |
|------|---------|-------------|
| `--store <path>` | `~/.local/share/karapace` | Store directory path |
| `--json` | `false` | JSON output |
| `--verbose` / `-v` | `false` | Debug-level logging |
| `--trace` | `false` | Trace-level logging (implies debug) |
## Environment variables
| Variable | Used by | Description |
|----------|---------|-------------|
| `KARAPACE_LOG` | cli, dbus | Log level filter: `error`, `warn`, `info`, `debug`, `trace`. Overrides `--verbose`/`--trace`. |
| `KARAPACE_STORE` | dbus | Override default store path. |
| `KARAPACE_SKIP_PREREQS` | cli | Set to `1` to skip runtime prerequisite checks. |
## Exit codes
| Code | Constant | Condition |
|------|----------|-----------|
| 0 | `EXIT_SUCCESS` | Success |
| 1 | `EXIT_FAILURE` | General error |
| 2 | `EXIT_MANIFEST_ERROR` | Manifest parse or validation error |
| 3 | `EXIT_STORE_ERROR` | Store integrity or lock error |
Defined in `crates/karapace-cli/src/commands/mod.rs`.
---
## Commands
### `build`
Build an environment from a manifest.
```
karapace build [manifest] [--name <name>]
```
| Argument | Default | Description |
|----------|---------|-------------|
| `manifest` | `karapace.toml` | Path to manifest file |
| `--name` | — | Assign a human-readable name |
Executes: parse → normalize → resolve → lock → build. Writes `karapace.lock` next to the manifest. Requires runtime prerequisites (user namespaces, fuse-overlayfs).
### `rebuild`
Destroy the existing environment and build a new one from the manifest.
```
karapace rebuild [manifest] [--name <name>]
```
Same arguments as `build`. The old environment is destroyed only after the new one builds successfully.
### `enter`
Enter an environment interactively, or run a command.
```
karapace enter <env_id> [-- cmd...]
```
| Argument | Description |
|----------|-------------|
| `env_id` | Full env_id, short_id, or name |
| `-- cmd...` | Optional command to run instead of interactive shell |
Sets state to `Running` on entry, back to `Built` on exit.
### `exec`
Run a command inside an environment (non-interactive).
```
karapace exec <env_id> -- <cmd...>
```
| Argument | Description |
|----------|-------------|
| `env_id` | Full env_id, short_id, or name |
| `cmd...` | Required. Command and arguments. |
### `destroy`
Destroy an environment and its overlay.
```
karapace destroy <env_id>
```
Cannot destroy a `Running` environment. Stop it first.
### `stop`
Stop a running environment.
```
karapace stop <env_id>
```
Sends `SIGTERM`. If the process does not exit within the timeout, sends `SIGKILL`.
### `freeze`
Freeze an environment. Prevents further writes.
```
karapace freeze <env_id>
```
### `archive`
Archive an environment. Prevents entry but preserves store data.
```
karapace archive <env_id>
```
Archived environments are protected from garbage collection.
### `list`
List all environments.
```
karapace list
```
Output columns: `SHORT_ID`, `NAME`, `STATE`, `ENV_ID`.
### `inspect`
Show environment metadata.
```
karapace inspect <env_id>
```
### `diff`
Show changes in the writable overlay.
```
karapace diff <env_id>
```
Lists added, modified, and removed files relative to the base layer.
### `snapshots`
List snapshots for an environment.
```
karapace snapshots <env_id>
```
### `commit`
Save overlay changes as a snapshot layer.
```
karapace commit <env_id>
```
Only valid for `Built` or `Frozen` environments.
### `restore`
Restore an environment's overlay from a snapshot.
```
karapace restore <env_id> <snapshot_hash>
```
| Argument | Description |
|----------|-------------|
| `env_id` | Environment to restore |
| `snapshot_hash` | Layer hash from `snapshots` output |
### `gc`
Garbage collect orphaned store data.
```
karapace gc [--dry-run]
```
| Flag | Description |
|------|-------------|
| `--dry-run` | Report what would be removed without deleting |
### `verify-store`
Verify integrity of all objects in the store.
```
karapace verify-store
```
Re-hashes every object, layer, and metadata entry against its stored key or checksum.
### `push`
Push an environment to a remote store.
```
karapace push <env_id> [--tag <name@tag>] [--remote <url>]
```
| Flag | Description |
|------|-------------|
| `--tag` | Registry key, e.g. `my-env@latest` |
| `--remote` | Remote URL. Overrides `~/.config/karapace/remote.json`. |
Skips blobs that already exist on the remote.
### `pull`
Pull an environment from a remote store.
```
karapace pull <reference> [--remote <url>]
```
| Argument | Description |
|----------|-------------|
| `reference` | Registry key (`name@tag`) or raw `env_id` |
Downloaded objects are verified with blake3 before storage.
### `rename`
Rename an environment.
```
karapace rename <env_id> <new_name>
```
Names must match `[a-zA-Z0-9_-]`, 1–64 characters. Validated in `karapace-store/src/metadata.rs::validate_env_name`.
### `completions`
Generate shell completions.
```
karapace completions <shell>
```
Supported shells: `bash`, `zsh`, `fish`, `elvish`, `powershell`.
### `man-pages`
Generate man pages.
```
karapace man-pages [dir]
```
| Argument | Default | Description |
|----------|---------|-------------|
| `dir` | `man` | Output directory |
### `doctor`
Check system prerequisites and store health.
```
karapace doctor
```
Checks: user namespace support, `fuse-overlayfs` availability, `curl` availability. Exits non-zero if any check fails.
### `migrate`
Check store format version and show migration guidance.
```
karapace migrate
```

84
docs/cli-stability.md
View file

@ -1,84 +0,0 @@
# Karapace CLI Stability Contract
## Scope
This document defines the stability guarantee for the Karapace CLI between 1.x releases.
## Stable Commands (21)
The following commands have **stable signatures** — no breaking changes to arguments, flags, or output format between 1.x releases:
| Command | Description |
|---------|-------------|
| `build` | Build environment from manifest |
| `rebuild` | Destroy + rebuild environment |
| `enter` | Enter environment interactively |
| `exec` | Execute command inside environment |
| `destroy` | Destroy environment |
| `stop` | Stop a running environment |
| `freeze` | Freeze environment (read-only overlay) |
| `archive` | Archive environment (preserve, prevent entry) |
| `list` | List all environments |
| `inspect` | Show environment metadata |
| `diff` | Show overlay drift |
| `snapshots` | List snapshots for an environment |
| `commit` | Commit overlay drift as snapshot |
| `restore` | Restore overlay from snapshot |
| `gc` | Garbage collect orphaned store data |
| `verify-store` | Check store integrity |
| `push` | Push environment to remote store |
| `pull` | Pull environment from remote store |
| `rename` | Rename environment |
| `doctor` | Run diagnostic checks on system and store |
| `migrate` | Check store version and show migration guidance |
## Zero-Maintenance Commands (2)
These commands are auto-generated and have no hand-maintained logic:
| Command | Description |
|---------|-------------|
| `completions` | Generate shell completions (bash/zsh/fish/elvish/powershell) |
| `man-pages` | Generate man pages |
**Total: 23 commands.**
## Global Flags
All commands accept these flags (stable):
- `--store <path>` — custom store location (default: `~/.local/share/karapace`)
- `--json` — structured JSON output on all query and store commands
- `--verbose` / `-v` — enable debug logging
- `--trace` — enable trace-level logging (more detailed than `--verbose`)
## What "Stable" Means
- **No removed flags** — existing flags continue to work.
- **No changed flag semantics** — same flag produces same behavior.
- **No changed exit codes** — exit code meanings are fixed.
- **No changed JSON output keys** — new keys may be added, existing keys are never removed or renamed.
- **New flags may be added** — additive changes are allowed.
## What May Change
- Human-readable (non-JSON) output formatting.
- Spinner and progress indicator appearance.
- Error message wording (not error codes).
- Addition of new commands.
- Addition of new optional flags to existing commands.
## Removed Commands
The following commands were removed before 1.0 and will not return:
`init`, `preset`, `list-presets`, `export-app`, `unexport-app`, `quick`, `validate`, `verify-lock`, `export`, `list-images`, `remove-image`, `remote-list`, `tui`
## Exit Codes
| Code | Meaning |
|------|---------|
| 0 | Success |
| 1 | General failure |
| 2 | Manifest error (parse, validation) |
| 3 | Store error (integrity, version mismatch) |

98
docs/contributing.md Normal file
View file

@ -0,0 +1,98 @@
# Contributing
## Setup
```bash
git clone https://github.com/marcoallegretti/karapace.git
cd karapace
cargo build
cargo test --workspace
```
## Verification
All of these must pass before submitting changes:
```bash
cargo fmt --all --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
cargo build --release --workspace
```
## Project layout
```
crates/
karapace-schema/ Manifest parsing, normalization, lock file, identity
karapace-store/ Content-addressable store, metadata, layers, WAL, GC
karapace-runtime/ Container backends, images, sandbox, security policy
karapace-core/ Engine: lifecycle orchestration, drift, concurrency
karapace-cli/ CLI binary (23 commands)
karapace-dbus/ D-Bus service (optional, not in default-members)
karapace-tui/ Terminal UI (optional, not in default-members)
karapace-remote/ Remote store client, push/pull, registry
karapace-server/ Reference HTTP server for remote store
docs/ Public documentation
docu_dev/ Internal development notes (not shipped)
data/ systemd and D-Bus service files
```
`default-members` in `Cargo.toml`: schema, store, runtime, core, cli, server. The D-Bus service and TUI are opt-in.
## Code standards
- `cargo clippy -- -D warnings` — zero warnings.
- `cargo fmt` — enforced.
- No `unwrap()` in production `src/` code. Tests are fine.
- All values interpolated into shell commands must use `shell_quote()`.
- All mutating operations must hold a `StoreLock`.
- All file writes must be atomic (`NamedTempFile` + `persist()`).
## Testing
- Unit tests: `#[cfg(test)] mod tests` in the relevant module.
- Integration tests: `crates/karapace-core/tests/`.
- E2E tests: `crates/karapace-core/tests/e2e.rs` — `#[ignore]`, require user namespaces and `fuse-overlayfs`.
```bash
# Unit + integration tests
cargo test --workspace
# E2E tests (requires Linux with user namespaces)
cargo test --test e2e -- --ignored --test-threads=1
```
## CI
Three workflows in `.github/workflows/`:
| Workflow | File | What it checks |
|----------|------|----------------|
| CI | `ci.yml` | fmt, clippy, tests (Ubuntu + Fedora), E2E, ENOSPC, release builds, reproducibility, smoke tests, lockfile, cargo-deny |
| Release | `release.yml` | Builds release binaries, signs with cosign, generates SBOM and provenance |
| Supply Chain | `supply-chain-test.yml` | Tamper detection, signature verification, adversarial injection tests |
The `ci-contract` job in `ci.yml` verifies that all required jobs are present. See `CI_CONTRACT.md`.
## Building the D-Bus service
```bash
# CLI only (default)
cargo build --release
# CLI + D-Bus service
cargo build --release --workspace
```
## Shell completions
```bash
karapace completions bash > ~/.local/share/bash-completion/completions/karapace
karapace completions zsh > ~/.local/share/zsh/site-functions/_karapace
karapace completions fish > ~/.config/fish/completions/karapace.fish
```
## License
Contributions are licensed under EUPL-1.2.

77
docs/e2e-testing.md
View file

@ -1,77 +0,0 @@
# End-to-End Testing
Karapace includes end-to-end tests that exercise the real namespace backend with `unshare`, `fuse-overlayfs`, and actual container images.
## Prerequisites
- Linux with user namespace support (`CONFIG_USER_NS=y`)
- `fuse-overlayfs` installed
- `curl` installed
- Network access (images are downloaded from `images.linuxcontainers.org`)
### Install on openSUSE Tumbleweed
```bash
sudo zypper install fuse-overlayfs curl
```
### Install on Ubuntu/Debian
```bash
sudo apt-get install fuse-overlayfs curl
```
### Install on Fedora
```bash
sudo dnf install fuse-overlayfs curl
```
## Running E2E Tests
E2E tests are `#[ignore]` by default. Run them explicitly:
```bash
cargo test --test e2e -- --ignored --test-threads=1
```
The `--test-threads=1` flag is important: E2E tests mount overlays and download images, so parallel execution can cause resource conflicts.
## Test Descriptions
| Test | What it does |
|---|---|
| `e2e_build_minimal_namespace` | Build a minimal openSUSE Tumbleweed environment with no packages |
| `e2e_exec_in_namespace` | Build + exec `echo hello` inside the container |
| `e2e_destroy_cleans_up` | Build + destroy, verify env_dir is removed |
| `e2e_rebuild_determinism` | Build + rebuild, verify env_id is identical |
| `e2e_build_with_packages` | Build with `which` package, verify resolved versions in lock file |
## CI
The GitHub Actions CI workflow includes an E2E job that runs on `ubuntu-latest`:
```yaml
e2e:
name: E2E Tests
runs-on: ubuntu-latest
needs: [test]
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- name: Install prerequisites
run: |
sudo apt-get update -qq
sudo apt-get install -y -qq fuse-overlayfs curl
sudo sysctl -w kernel.unprivileged_userns_clone=1 || true
- name: Run E2E tests
run: cargo test --test e2e -- --ignored --test-threads=1
```
## Troubleshooting
- **"unshare: user namespaces not available"** — Enable with `sysctl kernel.unprivileged_userns_clone=1`
- **"fuse-overlayfs not found"** — Install the `fuse-overlayfs` package
- **"failed to download image"** — Check network connectivity and DNS
- **Stale mounts after failed test** — Run `fusermount3 -u /path/to/merged` or reboot

282
docs/getting-started.md
View file

@ -1,282 +0,0 @@
# Getting Started with Karapace
Karapace creates isolated, reproducible development environments on Linux using
namespaces and overlay filesystems. No root, no daemon, no Docker.
This guide walks you through installation, first use, and common workflows.
## Prerequisites
Karapace requires a Linux system with:
- **User namespace support** (`CONFIG_USER_NS=y` — enabled on all major distros)
- **fuse-overlayfs** (overlay filesystem in userspace)
- **curl** (for downloading base images)
Optional:
- **crun**, **runc**, or **youki** (only if using the OCI backend)
### Install prerequisites by distro
**openSUSE Tumbleweed / Leap:**
```bash
sudo zypper install fuse-overlayfs curl
```
**Ubuntu / Debian:**
```bash
sudo apt install fuse-overlayfs curl
```
**Fedora:**
```bash
sudo dnf install fuse-overlayfs curl
```
**Arch Linux:**
```bash
sudo pacman -S fuse-overlayfs curl
```
Run `karapace doctor` at any time to check that all prerequisites are met.
## Installation
### From source (recommended)
```bash
git clone https://github.com/marcoallegretti/karapace.git
cd karapace
cargo build --release
sudo install -Dm755 target/release/karapace /usr/local/bin/karapace
```
### Via cargo install
```bash
cargo install --git https://github.com/marcoallegretti/karapace.git karapace-cli
```
### Shell completions
```bash
# Bash
karapace completions bash > ~/.local/share/bash-completion/completions/karapace
# Zsh
karapace completions zsh > ~/.local/share/zsh/site-functions/_karapace
# Fish
karapace completions fish > ~/.config/fish/completions/karapace.fish
```
## Your first environment
### 1. Write a manifest
Create a file called `karapace.toml`:
```toml
manifest_version = 1
[base]
image = "rolling" # openSUSE Tumbleweed
[system]
packages = ["git", "curl"]
[runtime]
backend = "namespace"
```
Available base images: `"rolling"` (openSUSE Tumbleweed), `"ubuntu/24.04"`,
`"debian/12"`, `"fedora/41"`, `"arch"`.
### 2. Build the environment
```bash
karapace build
```
This downloads the base image, installs the requested packages, and produces
a content-addressed environment. The output shows the environment ID (`env_id`)
and a short ID for convenience.
### 3. Enter the environment
```bash
karapace enter <env_id>
```
You can use the short ID (first 8 characters) or a name instead of the full ID.
Inside the environment you have a full Linux userspace with the packages you
requested.
### 4. Run a single command
```bash
karapace exec <env_id> -- git --version
```
## Naming environments
By default, environments are identified by their content hash. You can assign
a human-readable name:
```bash
karapace build --name mydev
karapace enter mydev
```
Or rename an existing environment:
```bash
karapace rename <env_id> mydev
```
## Common workflows
### Snapshot and restore
After making changes inside an environment (installing extra packages, editing
config files), you can snapshot and later restore:
```bash
# See what changed
karapace diff mydev
# Save a snapshot
karapace commit mydev
# List snapshots
karapace snapshots mydev
# Restore a previous snapshot
karapace restore mydev <snapshot_hash>
```
### Freeze and archive
```bash
# Freeze: prevent further changes (still enterable in read-only mode)
karapace freeze mydev
# Archive: preserve metadata but prevent entry
karapace archive mydev
```
### Rebuild from scratch
If you change your manifest, rebuild destroys the old environment and builds
a new one:
```bash
karapace rebuild
```
### Push and pull (remote sharing)
```bash
# Push to a remote store
karapace push mydev --tag my-env@latest
# Pull on another machine
karapace pull my-env@latest --remote https://your-server.example.com
```
### GUI application export
Export a GUI application from inside the container to your host desktop:
```bash
karapace exec mydev -- karapace-export-app firefox
```
This creates a `.desktop` file on the host that launches the app inside the
container with GPU and audio passthrough.
## Hardware passthrough
Enable GPU and audio in your manifest:
```toml
[hardware]
gpu = true # Passes /dev/dri into the container
audio = true # Passes PipeWire/PulseAudio socket
```
Karapace also forwards Wayland, X11, D-Bus session bus, SSH agent, fonts,
and GTK/icon themes automatically when available.
## Custom bind mounts
Mount host directories into the container:
```toml
[mounts]
workspace = "~/projects:/workspace"
data = "/data/datasets:/datasets"
```
## Built-in presets
For quick setup without writing a manifest:
```bash
# List available presets
karapace list-presets
# Build from a preset
karapace preset dev-rust
```
Available presets: `dev`, `dev-rust`, `dev-python`, `gui-app`, `gaming`, `minimal`.
## Quick one-liner
The `quick` command combines build + enter in a single step:
```bash
karapace quick
```
This uses the `karapace.toml` in the current directory (or creates a minimal one).
## Diagnostics
```bash
# Check system prerequisites
karapace doctor
# Verify store integrity
karapace verify-store
# List all environments
karapace list
# Inspect an environment
karapace inspect mydev
# Garbage collect unused objects
karapace gc --dry-run
karapace gc
```
## Environment variables
| Variable | Effect |
|----------|--------|
| `KARAPACE_LOG` | Log level: `error`, `warn`, `info`, `debug`, `trace` |
| `KARAPACE_STORE` | Custom store directory (default: `~/.local/share/karapace`) |
Or use CLI flags: `--verbose` / `-v` for debug, `--trace` for trace,
`--store <path>` for a custom store, `--json` for machine-readable output.
## Next steps
- [Manifest v1 Specification](manifest-spec.md) — full manifest reference
- [Architecture Overview](architecture.md) — how Karapace works internally
- [CLI Stability Contract](cli-stability.md) — which commands are stable
- [Security Model](security-model.md) — isolation guarantees and threat model
- [Verification](verification.md) — verifying release artifact integrity

60
docs/hash-contract.md
View file

@ -1,60 +0,0 @@
# Karapace Hash Contract
## Overview
The environment identity (`env_id`) is a deterministic blake3 hash that uniquely identifies an environment's fully resolved state. Two identical lock files on any machine must produce the same `env_id`.
## Algorithm
Blake3 (256-bit output, hex-encoded, 64 characters).
## Two-Phase Identity
Karapace computes identity in two phases:
### Preliminary Identity (`compute_env_id`)
Used only during `init` (before resolution) and for internal lookup. Computed from unresolved manifest data. **Not the canonical identity.**
### Canonical Identity (`LockFile::compute_identity`)
The authoritative identity used after `build`. Computed from the fully resolved lock file state. This is what gets stored in metadata and the lock file.
## Canonical Hash Input
The canonical hash includes the following inputs, fed in order:
1. **Base image content digest**: `base_digest:<blake3_of_rootfs>` — real content hash, not a tag name hash.
2. **Resolved packages**: each as `pkg:<name>@<version>` (sorted by name).
3. **Resolved apps**: each as `app:<name>` (sorted).
4. **Hardware policy**: `hw:gpu` if GPU enabled, `hw:audio` if audio enabled.
5. **Mount policy**: each as `mount:<label>:<host_path>:<container_path>` (sorted by label).
6. **Runtime backend**: `backend:<name>` (lowercased).
7. **Network isolation**: `net:isolated` if enabled.
8. **CPU shares**: `cpu:<value>` if set.
9. **Memory limit**: `mem:<value>` if set.
## Hash MUST NOT Include
- Writable overlay state (mutable drift).
- Timestamps (creation, modification).
- Host-specific non-declared paths.
- Machine identifiers (hostname, MAC, etc.).
- Store location.
- Unresolved package names without versions.
## Properties
- **Deterministic**: same resolved inputs → same hash, always.
- **Stable**: consistent across identical systems with same resolved packages.
- **Immutable**: once built, the env_id never changes for that lock state.
- **Version-sensitive**: different package versions produce different identities.
## Short ID
The `short_id` is the first 12 hex characters of `env_id`. Used for display and prefix-matching in the CLI.
## Implementation
- **Canonical**: `karapace-schema/src/lock.rs::LockFile::compute_identity()`
- **Preliminary**: `karapace-schema/src/identity.rs::compute_env_id()`

77
docs/layer-limitations-v1.md
View file

@ -1,77 +0,0 @@
# Karapace Layer Limitations — Phase 1 (v1.0)
## Overview
Karapace 1.0 ships with Phase 1 layer support: deterministic tar-based content-addressed layers. This document describes what is supported, what is not, and what is planned for future phases.
## Supported (Phase 1)
- **Regular files** — full content preservation, deterministic packing.
- **Directories** — including empty directories.
- **Symbolic links** — target path preserved exactly.
- **Deterministic packing** — sorted entries, zero timestamps (`mtime = 0`), owner `0:0`, consistent permissions.
- **Content addressing** — blake3 hash of the tar archive.
- **Snapshot layers** — `commit` captures overlay upper directory as a tar, `restore` unpacks it atomically.
- **Composite snapshot identity** — snapshot layer hash is `blake3("snapshot:{env_id}:{base_layer}:{tar_hash}")` to prevent collision with base layers.
- **Atomic restore** — unpack to staging directory, then rename-swap with the upper directory.
## Not Supported (Phase 1)
The following filesystem features are **silently dropped** during `pack_layer`:
| Feature | Status | Planned |
|---------|--------|---------|
| Extended attributes (xattrs) | Dropped | Phase 2 (1.1) |
| Device nodes | Dropped | Phase 2 (1.1) |
| Hardlinks | Stored as regular files (deduplicated content) | Phase 2 (1.1) |
| SELinux labels | Dropped | Phase 2 (1.1) |
| ACLs | Dropped | Phase 2 (1.1) |
| Sparse files | Stored as full files | Phase 2 (1.1) |
| UID/GID remapping | Not supported | Phase 3 (2.0) |
| Per-file dedup | Not supported | Phase 3 (2.0) |
| Whiteout files (overlay deletion markers) | Included as regular files | Phase 2 (1.1) |
## Implications
### Security-Sensitive Workloads
Environments relying on SELinux labels, xattrs for capabilities (`security.capability`), or ACLs will not have those attributes preserved across `commit`/`restore` cycles. This is acceptable for development environments but not for production container images.
### Hardlinks
If two files in the upper directory are hardlinked, they will be stored as separate regular files in the tar. This means:
- Restoring from a snapshot may increase disk usage.
- File identity (inode sharing) is not preserved.
### Device Nodes
Device nodes (`/dev/*`) created inside the environment are dropped during packing. This is intentional — device nodes are host-specific and should not be stored in content-addressed layers.
### Whiteout Handling
Overlay whiteout files (`.wh.*` and `.wh..wh..opq`) are currently stored as-is in the tar. They are only meaningful when applied on top of the correct base layer. Restoring a snapshot to a different base layer may produce incorrect results.
## Determinism Guarantees
| Property | Guaranteed |
|----------|-----------|
| Same directory content → same tar bytes | Yes |
| Same tar bytes → same blake3 hash | Yes |
| Roundtrip fidelity (regular files, dirs, symlinks) | Yes |
| Timestamp preservation | No (zeroed for determinism) |
| Owner/group preservation | No (set to 0:0) |
| Permission preservation | Yes (mode bits preserved) |
## Phase 2 Roadmap (1.1)
- Extended attribute support via `tar` crate's xattr feature.
- Hardlink detection and deduplication within a single layer.
- Overlay whiteout awareness (proper deletion semantics).
- Device node opt-in for privileged builds.
- SELinux label preservation.
## Phase 3 Roadmap (2.0)
- UID/GID remapping for rootless environments.
- Per-file content deduplication across layers.
- Layer diffing and incremental snapshots.

94
docs/lock-spec.md
View file

@ -1,94 +0,0 @@
# Karapace Lock File Specification (v2)
## Overview
The lock file (`karapace.lock`) captures the fully resolved state of an environment at build time. It ensures reproducible builds by pinning all dependency versions and the base image content digest.
The environment identity (`env_id`) is computed from the **locked** state — resolved package versions and content digest — not from unresolved manifest data. This guarantees: same lockfile → same env_id → same environment.
## Format
TOML with the following fields:
| Field | Type | Description |
|---|---|---|
| `lock_version` | `u32` | Lock format version. Must be `2`. |
| `env_id` | `string` | Full 64-character blake3 hex digest computed from locked state. |
| `short_id` | `string` | First 12 characters of `env_id`. |
| `base_image` | `string` | Base image identifier from manifest (e.g. `"rolling"`). |
| `base_image_digest` | `string` | Blake3 content digest of the actual base image rootfs. |
| `resolved_packages` | `array` | Sorted list of `{ name, version }` tables with pinned versions. |
| `resolved_apps` | `string[]` | Sorted, deduplicated list of resolved GUI apps. |
| `runtime_backend` | `string` | Normalized runtime backend name (lowercased). |
| `hardware_gpu` | `bool` | Whether GPU passthrough is requested. |
| `hardware_audio` | `bool` | Whether audio passthrough is requested. |
| `network_isolation` | `bool` | Whether network is isolated. |
| `mounts` | `array` | Sorted list of `{ label, host_path, container_path }` tables. |
| `cpu_shares` | `u64?` | CPU shares limit (optional). |
| `memory_limit_mb` | `u64?` | Memory limit in MB (optional). |
## Resolved Packages
Each entry in `resolved_packages` is a table:
```toml
[[resolved_packages]]
name = "git"
version = "2.44.0-1"
[[resolved_packages]]
name = "clang"
version = "17.0.6-1"
```
Versions are queried from the actual package manager inside the base image during the resolve phase. This ensures identity is based on real installed versions, not just package names.
## Invariants
- The lock file contains **content digests**, not tags or mutable references.
- `base_image_digest` is a blake3 hash of the actual rootfs content, not a hash of the image tag name.
- `env_id` is computed from all locked fields via `LockFile::compute_identity()`.
- Verification is split into two checks:
- **Integrity**: `verify_integrity()` — recomputes `env_id` and compares to the stored value.
- **Manifest intent**: `verify_manifest_intent()` — checks that the manifest hasn't drifted from what was locked.
## Generation
- Generated automatically by `karapace init` (with preliminary unresolved versions) and `karapace build` (with fully resolved versions).
- Written atomically (tempfile + rename) to the same directory as the manifest.
- Should be committed to version control for reproducible builds.
## Manual Override
- The lock file can only be regenerated by re-running `karapace build`.
- There is no `karapace lock --update` command; rebuilding is the only path.
- `karapace verify-lock` checks both integrity and manifest consistency.
## Example
```toml
lock_version = 2
env_id = "46e1d96fdd6fd988092fbcd19b1d89f2b080f3e74d0f4984b4ba45ca5b95e594"
short_id = "46e1d96fdd6f"
base_image = "rolling"
base_image_digest = "a1b2c3d4e5f6..."
runtime_backend = "namespace"
hardware_gpu = true
hardware_audio = false
network_isolation = false
[[resolved_packages]]
name = "clang"
version = "17.0.6-1"
[[resolved_packages]]
name = "git"
version = "2.44.0-1"
resolved_apps = ["debugger", "ide"]
[[mounts]]
label = "workspace"
host_path = "./"
container_path = "/workspace"
```

113
docs/manifest-spec.md
View file

@ -1,113 +0,0 @@
# Karapace Manifest v0.1 Specification
## Overview
The Karapace manifest is a TOML file (typically `karapace.toml`) that declaratively defines an environment. It is the single source of truth for environment identity.
## Schema
### Required Fields
| Field | Type | Description |
|---|---|---|
| `manifest_version` | `u32` | Must be `1`. |
| `base.image` | `string` | Base image identifier. Must not be empty. |
### Optional Sections
#### `[system]`
| Field | Type | Default | Description |
|---|---|---|---|
| `packages` | `string[]` | `[]` | System packages to install. Duplicates are deduplicated during normalization. |
#### `[gui]`
| Field | Type | Default | Description |
|---|---|---|---|
| `apps` | `string[]` | `[]` | GUI applications to install. |
#### `[hardware]`
| Field | Type | Default | Description |
|---|---|---|---|
| `gpu` | `bool` | `false` | Request GPU passthrough (`/dev/dri`). |
| `audio` | `bool` | `false` | Request audio device passthrough (`/dev/snd`). |
#### `[mounts]`
Flat key-value pairs. Each key is a label; each value is `<host_path>:<container_path>`.
- Labels must not be empty.
- The `:` separator is required.
- Absolute host paths are validated against the mount whitelist.
- Relative host paths (e.g. `./`) are always permitted.
#### `[runtime]`
| Field | Type | Default | Description |
|---|---|---|---|
| `backend` | `string` | `"namespace"` | Runtime backend: `namespace`, `oci`, or `mock`. |
| `network_isolation` | `bool` | `false` | Isolate network from host. |
#### `[runtime.resource_limits]`
| Field | Type | Default | Description |
|---|---|---|---|
| `cpu_shares` | `u64?` | `null` | CPU shares limit. |
| `memory_limit_mb` | `u64?` | `null` | Memory limit in MB. |
## Validation Rules
1. `manifest_version` must equal `1`.
2. Unknown fields at any level cause a parse error (`deny_unknown_fields`).
3. `base.image` must not be empty or whitespace-only.
4. Mount specs must contain exactly one `:` with non-empty sides.
## Normalization
During normalization:
- All string values are trimmed.
- `system.packages` and `gui.apps` are sorted, deduplicated.
- Mounts are sorted by label.
- `runtime.backend` is lowercased.
The normalized form is serialized to canonical JSON for hashing.
## Example
```toml
manifest_version = 1
[base]
image = "rolling"
[system]
packages = ["clang", "cmake", "git"]
[gui]
apps = ["ide", "debugger"]
[hardware]
gpu = true
audio = true
[mounts]
workspace = "./:/workspace"
[runtime]
backend = "namespace"
network_isolation = false
[runtime.resource_limits]
cpu_shares = 1024
memory_limit_mb = 4096
```
## Versioning
- The manifest format is versioned via `manifest_version`.
- Only version `1` is supported in Karapace 0.1.
- Future versions will increment this field.
- Backward-incompatible changes require a version bump.

109
docs/protocol-v1.md
View file

@ -1,109 +0,0 @@
# Karapace Remote Protocol v1 (Draft)
> **Status: v1-draft** — This protocol is subject to change before 1.1.
## Overview
The Karapace remote protocol defines how environments are transferred between local stores and a remote HTTP backend. It is a content-addressable blob store with an optional registry index for named references.
## Blob Types
| Kind | Key | Content |
|------|-----|---------|
| `Object` | blake3 hex hash | Raw object data (tar layers, manifests) |
| `Layer` | layer hash | JSON `LayerManifest` |
| `Metadata` | env_id | JSON `EnvMetadata` |
## HTTP Routes
All routes are relative to the remote base URL.
### Blobs
| Method | Route | Description |
|--------|-------|-------------|
| `PUT` | `/blobs/{kind}/{key}` | Upload a blob |
| `GET` | `/blobs/{kind}/{key}` | Download a blob |
| `HEAD` | `/blobs/{kind}/{key}` | Check if blob exists |
| `GET` | `/blobs/{kind}` | List blob keys |
### Registry
| Method | Route | Description |
|--------|-------|-------------|
| `PUT` | `/registry` | Upload registry index |
| `GET` | `/registry` | Download registry index |
## Push Protocol
1. Read local `EnvMetadata` for the target env_id.
2. Collect layer hashes: `base_layer` + `dependency_layers`.
3. For each layer, collect `object_refs`.
4. Upload objects (skip if `HEAD` returns 200).
5. Upload layer manifests (skip if `HEAD` returns 200).
6. Upload metadata blob.
7. If a registry tag is provided, download current registry, merge entry, upload updated registry.
## Pull Protocol
1. Download `Metadata` blob for the target env_id.
2. Collect layer hashes from metadata.
3. Download layer manifests (skip if local store has them).
4. Collect `object_refs` from all layers.
5. Download objects (skip if local store has them).
6. **Verify integrity**: compute blake3 hash of each downloaded object; reject if hash does not match key.
7. Store metadata locally.
## Registry Format
```json
{
"entries": {
"my-env@latest": {
"env_id": "abc123...",
"short_id": "abc123def456",
"name": "my-env",
"pushed_at": "2026-01-15T12:00:00Z"
}
}
}
```
### Reference Resolution
References follow the format `name@tag` (default tag: `latest`). Resolution:
1. Parse reference into `(name, tag)`.
2. Look up `{name}@{tag}` in the registry.
3. Return the associated `env_id`.
## Integrity
- All objects are keyed by their blake3 hash.
- On pull, every downloaded object is re-hashed and compared to its key.
- Mismatches produce `RemoteError::IntegrityFailure`.
- Layer manifests and metadata are JSON — no hash verification (they are keyed by logical identifiers, not content hashes).
## Headers
| Header | Value | When |
|--------|-------|------|
| `Content-Type` | `application/octet-stream` | Blob uploads/downloads |
| `Content-Type` | `application/json` | Registry uploads/downloads |
## Version Negotiation
Not yet implemented. The protocol version will be negotiated via a `X-Karapace-Protocol` header in 1.1.
## Error Responses
| Status | Meaning |
|--------|---------|
| 200 | Success |
| 404 | Blob or registry not found |
| 500 | Server error |
## Security Considerations
- No authentication in v1-draft. Authentication headers will be added in 1.1.
- HTTPS is strongly recommended for all remote URLs.
- Object integrity is verified client-side via blake3.

146
docs/security-model.md
View file

@ -1,83 +1,105 @@
# Karapace Security Model
# Security Model
## Core Invariants
Defined in `karapace-runtime/src/security.rs::SecurityPolicy`.
1. **Host filesystem protected by default.** No writes outside the store and overlay.
2. **No implicit `/dev` access.** Device passthrough is explicit and opt-in.
3. **GPU passthrough requires `hardware.gpu = true`** in the manifest.
4. **Audio passthrough requires `hardware.audio = true`** in the manifest.
5. **Network isolation is configurable** via `runtime.network_isolation`.
6. **No permanent root daemon required.** All operations are rootless.
7. **No SUID binaries required.**
## Privilege model
## Mount Policy
Karapace runs entirely as an unprivileged user. No SUID binaries. No root daemon. Isolation is provided by Linux user namespaces (`unshare`).
- Absolute host paths are checked against a whitelist.
- Default allowed prefixes: `/home`, `/tmp`.
- Relative paths (e.g. `./`) are always allowed.
- Mounts outside the whitelist are rejected at build time.
The OCI backend delegates to rootless runtimes (`crun`, `runc`, `youki`).
## Device Policy
## Mount policy
- Default policy denies all device access.
- `hardware.gpu = true` adds `/dev/dri` to allowed devices.
- `hardware.audio = true` adds `/dev/snd` to allowed devices.
- No implicit device passthrough.
Absolute host paths in manifest `[mounts]` are validated against an allowlist.
## Environment Variable Control
**Default allowed prefixes:** `/home`, `/tmp`.
- A whitelist of safe environment variables is passed to the environment:
`TERM`, `LANG`, `HOME`, `USER`, `PATH`, `SHELL`, `XDG_RUNTIME_DIR`.
- A denylist prevents sensitive variables from leaking:
`SSH_AUTH_SOCK`, `GPG_AGENT_INFO`, `AWS_SECRET_ACCESS_KEY`, `DOCKER_HOST`.
- Only whitelisted, non-denied variables are propagated.
Relative paths (e.g. `./`) are always permitted. Mounts outside the allowlist are rejected at build time with `RuntimeError::MountDenied`.
## Resource Limits
Path traversal is prevented by `canonicalize_logical()` in `security.rs`, which resolves `..` components before checking the prefix.
- CPU shares and memory limits are declared in the manifest.
- The security policy enforces upper bounds.
- Exceeding policy limits causes a build failure, not a silent cap.
Defined in `SecurityPolicy::validate_mounts`.
## Privilege Model
## Device policy
- No privileged escalation at any point.
- User namespaces provide isolation without root.
- The OCI backend uses rootless runtimes (crun, runc, youki).
Default policy denies all device access.
## Threat Model
- `hardware.gpu = true` → allows `/dev/dri`
- `hardware.audio = true` → allows `/dev/snd`
### Privilege Boundary
No implicit device passthrough. Defined in `SecurityPolicy::validate_devices`.
- Karapace operates entirely within the user's privilege level.
- The store is owned by the user and protected by filesystem permissions.
- No daemon runs with elevated privileges.
## Environment variable control
### Attack Surface
**Allowed** (propagated into the container):
- **Manifest parsing**: strict TOML parser with `deny_unknown_fields`.
- **Store integrity**: blake3 verification on every object read.
- **Image integrity**: blake3 digest stored on download, verified on cache hit.
- **Shell injection**: all paths and values in sandbox scripts use POSIX single-quote escaping (`shell_quote`). Environment variable keys are validated against `[a-zA-Z0-9_]`.
- **Mount injection**: prevented by whitelist enforcement.
- **Environment variable leakage**: prevented by deny/allow lists.
- **Concurrent access**: file locking on all mutating CLI and D-Bus operations.
- **Process safety**: cannot destroy a running environment (must stop first).
- **Rebuild atomicity**: new environment is built before the old one is destroyed.
```
TERM, LANG, HOME, USER, PATH, SHELL, XDG_RUNTIME_DIR
```
### Isolation Assumptions
**Denied** (never propagated):
- The host kernel provides functioning user namespaces.
- The filesystem permissions on the store directory are correct.
- The OCI runtime (if used) is trusted and correctly installed.
```
SSH_AUTH_SOCK, GPG_AGENT_INFO, AWS_SECRET_ACCESS_KEY, DOCKER_HOST
```
### Security Review Checklist
Only variables present in the allowed list and absent from the denied list are passed through. Defined in `SecurityPolicy::filter_env_vars`.
- [x] No SUID binaries.
- [x] No root daemon.
- [x] Mount whitelist enforced.
- [x] Device passthrough explicit.
- [x] Environment variables controlled.
- [x] Resource limits supported.
- [x] Store integrity verified.
- [x] Concurrent access safe.
- [x] Signal handling (SIGINT/SIGTERM) clean.
## Resource limits
Declared in manifest `[runtime.resource_limits]`:
- `cpu_shares`: CPU shares limit
- `memory_limit_mb`: memory limit in MB
If the policy defines upper bounds, requesting values above them causes a build-time error (`RuntimeError::ResourceLimitExceeded`). Defined in `SecurityPolicy::validate_resource_limits`.
## Store integrity
- **Objects:** blake3 hash verified on every read. Key = hash of content.
- **Metadata:** blake3 checksum embedded in each metadata file, verified on every `get()`.
- **Layers:** file content re-hashed against filename on read.
- **Images:** content digest stored on download, re-verified on cache hits.
## Concurrency
File locking (`flock(2)`) on `store/.lock` for all mutating operations. Both CLI and D-Bus service acquire the lock. Defined in `karapace-core/src/concurrency.rs::StoreLock`.
Running environments cannot be destroyed (must be stopped first).
## Shell injection prevention
Sandbox scripts use POSIX single-quote escaping for all paths and values. Environment variable keys are validated against `[a-zA-Z0-9_]`. Defined in `karapace-runtime/src/sandbox.rs`.
## Manifest parsing
Strict TOML parser with `deny_unknown_fields` on all sections. Unknown keys cause a parse error.
## What is NOT protected
- Karapace does not protect against a compromised host kernel.
- Karapace does not verify the authenticity of upstream base images beyond content hashing.
- The OCI runtime (if used) is trusted.
- Filesystem permissions on the store directory are the user's responsibility.
- Network isolation (`runtime.network_isolation`) depends on the backend implementation.
- No MAC (SELinux/AppArmor) enforcement within the container.
## Trust assumptions
1. The host kernel provides functioning, secure user namespaces.
2. The store directory has correct ownership and permissions.
3. `fuse-overlayfs` is correctly installed and not compromised.
4. The OCI runtime (if used) is a trusted binary.
5. Base images from `images.linuxcontainers.org` are fetched over HTTPS but not GPG-verified.
## Unsafe code
Five `unsafe` blocks, all FFI calls to libc:
| File | Call | Purpose |
|------|------|---------|
| `karapace-core/src/engine.rs` | `libc::kill(SIGTERM)` | Stop running environment |
| `karapace-core/src/engine.rs` | `libc::kill(SIGKILL)` | Force-kill after timeout |
| `karapace-runtime/src/sandbox.rs` | `libc::getuid()` | Get UID for namespace mapping |
| `karapace-runtime/src/sandbox.rs` | `libc::getgid()` | Get GID for namespace mapping |
| `karapace-runtime/src/terminal.rs` | `libc::isatty()` | Detect terminal for interactive mode |

212
docs/storage-format.md Normal file
View file

@ -0,0 +1,212 @@
# Storage Format
Store format version: **2**. Defined in `karapace-store/src/layout.rs::STORE_FORMAT_VERSION`.
## Directory layout
Default root: `~/.local/share/karapace`.
```
<root>/
store/
version # { "format_version": 2 }
.lock # flock(2) exclusive lock
objects/<blake3_hex> # content-addressable blobs
layers/<blake3_hex> # layer manifests (JSON)
metadata/<env_id> # environment metadata (JSON)
staging/ # temp workspace for atomic operations
wal/<op_id>.json # write-ahead log entries
env/
<env_id>/
upper/ # overlay writable layer
overlay/ # overlay mount point
images/
<cache_key>/
rootfs/ # extracted base image filesystem
```
Paths defined in `karapace-store/src/layout.rs::StoreLayout`.
## Version file
```json
{ "format_version": 2 }
```
Checked on every store access. Mismatched versions are rejected with `StoreError::VersionMismatch`.
## Objects
Content-addressable blobs keyed by blake3 hex digest of their content.
- Write: `NamedTempFile` in objects dir → write content → `sync_all()` → `persist()` (atomic rename)
- Read: read file → recompute blake3 → compare to filename → reject on mismatch
- Idempotent: writing identical content is a no-op
Defined in `karapace-store/src/objects.rs::ObjectStore`.
## Layers
JSON files in `store/layers/`. Each describes a tar archive stored in the object store.
```json
{
"hash": "<layer_hash>",
"kind": "Base | Dependency | Policy | Snapshot",
"parent": "<parent_hash> | null",
"object_refs": ["<hash>", ...],
"read_only": true,
"tar_hash": "<blake3_of_tar>"
}
```
Defined in `karapace-store/src/layers.rs::LayerManifest`.
**Layer kinds:**
| Kind | Hash computation | Parent |
|------|-----------------|--------|
| `Base` | `tar_hash` | None |
| `Dependency` | `tar_hash` | Base layer |
| `Policy` | `tar_hash` | — |
| `Snapshot` | `blake3("snapshot:{env_id}:{base_layer}:{tar_hash}")` | Base layer |
Layer integrity is verified on read: the file content is re-hashed and compared to the filename.
### Deterministic tar packing
`karapace-store/src/layers.rs::pack_layer(source_dir)`:
- Entries sorted by path
- Timestamps set to 0
- Owner set to `0:0`
- Permissions preserved
- Symlink targets preserved
**Dropped during packing:** extended attributes, device nodes, hardlinks (stored as regular files), SELinux labels, ACLs, sparse file holes.
`unpack_layer(tar_data, target_dir)` reverses the process.
## Metadata
JSON files in `store/metadata/`, one per environment. Filename is the `env_id`.
```json
{
"env_id": "...",
"short_id": "...",
"name": null,
"state": "Built",
"manifest_hash": "<object_hash>",
"base_layer": "<layer_hash>",
"dependency_layers": [],
"policy_layer": null,
"created_at": "RFC3339",
"updated_at": "RFC3339",
"ref_count": 1,
"checksum": "<blake3_of_json>"
}
```
Defined in `karapace-store/src/metadata.rs::EnvMetadata`.
**States:** `Defined`, `Built`, `Running`, `Frozen`, `Archived`.
**Checksum:** blake3 of the JSON content (excluding the checksum field itself). Computed on every `put()`, verified on every `get()`. Absent in legacy metadata (`#[serde(default)]`).
**Names:** optional, validated by `validate_env_name`: pattern `[a-zA-Z0-9_-]`, 1–64 characters. Unique across all environments.
## Manifest format
File: `karapace.toml`. Parsed by `karapace-schema/src/manifest.rs`.
```toml
manifest_version = 1
[base]
image = "rolling"
[system]
packages = ["git", "curl"]
[gui]
apps = []
[hardware]
gpu = false
audio = false
[mounts]
workspace = "./:/workspace"
[runtime]
backend = "namespace"
network_isolation = false
[runtime.resource_limits]
cpu_shares = 1024
memory_limit_mb = 4096
```
**Required:** `manifest_version` (must be `1`), `base.image` (non-empty).
**Optional:** all other sections. Unknown fields cause a parse error (`deny_unknown_fields`).
**Normalization** (`ManifestV1::normalize`): trim strings, sort and deduplicate packages/apps, sort mounts by label, lowercase backend name. Produces `NormalizedManifest` with a `canonical_json()` method.
## Lock file
File: `karapace.lock`. Written next to the manifest. TOML format.
```toml
lock_version = 2
env_id = "46e1d96f..."
short_id = "46e1d96fdd6f"
base_image = "rolling"
base_image_digest = "a1b2c3d4..."
runtime_backend = "namespace"
hardware_gpu = false
hardware_audio = false
network_isolation = false
[[resolved_packages]]
name = "git"
version = "2.44.0-1"
```
Defined in `karapace-schema/src/lock.rs::LockFile`.
**Verification:**
- `verify_integrity()`: recomputes `env_id` from locked fields, compares to stored value
- `verify_manifest_intent()`: checks manifest hasn't drifted from what was locked
## Hashing
All hashing uses **blake3**, 256-bit output, hex-encoded (64 characters).
Used for: object keys, layer hashes, env_id computation, metadata checksums, image content digests.
## Write-ahead log
`store/wal/<op_id>.json`. Defined in `karapace-store/src/wal.rs`.
```json
{
"op_id": "20260215120000123-a1b2c3d4",
"kind": "Build",
"env_id": "...",
"timestamp": "RFC3339",
"rollback_steps": [
{ "RemoveDir": "/path" },
{ "RemoveFile": "/path" }
]
}
```
**Operations:** `Build`, `Rebuild`, `Commit`, `Restore`, `Destroy`, `Gc`.
**Recovery:** on `Engine::new()`, all WAL entries are scanned. Each entry's rollback steps execute in reverse order. The entry is then deleted. Corrupt entries are silently removed.
## Atomic write contract
All store writes follow: `NamedTempFile::new_in(dir)` → write → `flush()` → `persist()` (atomic rename). No partial files are visible. Defined throughout `karapace-store`.

150
docs/store-spec.md
View file

@ -1,150 +0,0 @@
# Karapace Store Format Specification (v2)
## Overview
The Karapace store is a content-addressable filesystem structure that holds all environment data: objects, layers, metadata, environment directories, and crash recovery state.
## Directory Layout
```
<store_root>/
store/
version # JSON: { "format_version": 2 }
.lock # flock(2) file for exclusive access
objects/<hash> # Content-addressable blobs (blake3)
layers/<hash> # Layer manifests (JSON)
metadata/<env_id> # Environment metadata (JSON)
staging/ # Temporary workspace for atomic operations
wal/ # Write-ahead log entries (JSON)
env/
<env_id>/
upper/ # Writable overlay layer (fuse-overlayfs upperdir)
lower -> ... # Symlink to base image rootfs
work/ # Overlay workdir (ephemeral)
merged/ # Overlay mount point
images/
<cache_key>/
rootfs/ # Extracted base image rootfs
```
## Format Version
- Current version: **2**
- Stored in `store/version` as JSON.
- Checked on every store access; mismatches are rejected.
- Version 1 stores are not auto-migrated; a clean rebuild is required.
## Objects
- Keyed by blake3 hex digest of their content.
- Written atomically: write to tempfile, then rename.
- Integrity verified on every read: content re-hashed and compared to filename.
- Idempotent: writing the same content twice is a no-op.
## Layers
Each layer is a JSON manifest:
```json
{
"hash": "<layer_hash>",
"kind": "Base" | "Dependency" | "Policy" | "Snapshot",
"parent": "<parent_hash>" | null,
"object_refs": ["<hash>", ...],
"read_only": true,
"tar_hash": "<blake3_hash>"
}
```
- `tar_hash` (v2): blake3 hash of the deterministic tar archive stored in the object store.
- Base layers have no parent. Their `hash` equals their `tar_hash`.
- Dependency layers reference a base parent.
- Snapshot layers are created by `commit`. Their `hash` is a composite identity: `blake3("snapshot:{env_id}:{base_layer}:{tar_hash}")` to prevent collision with base layers.
## Metadata
Each environment has a JSON metadata file:
```json
{
"env_id": "...",
"short_id": "...",
"name": "my-env",
"state": "Defined" | "Built" | "Running" | "Frozen" | "Archived",
"manifest_hash": "<object_hash>",
"base_layer": "<layer_hash>",
"dependency_layers": ["<hash>", ...],
"policy_layer": null | "<hash>",
"created_at": "RFC3339",
"updated_at": "RFC3339",
"ref_count": 1
}
```
- `name` is optional (`#[serde(default)]`). Old metadata without this field deserializes correctly.
## Atomic Write Contract
All writes follow the pattern:
1. Create `NamedTempFile` in the target directory.
2. Write full content.
3. `flush()`.
4. `persist()` (atomic rename).
This ensures no partial files are visible.
## Garbage Collection
- Environments with `ref_count == 0` and state not in {`Running`, `Archived`} are eligible for collection.
- Layers not referenced by any live environment are orphaned.
- Objects not referenced by any live layer or live metadata (`manifest_hash`) are orphaned.
- GC never deletes running or archived environments.
- GC supports graceful cancellation via signal handler (`SIGINT`/`SIGTERM`).
- `--dry-run` reports what would be removed without acting.
- The caller must hold the store lock before running GC.
## Write-Ahead Log (WAL)
The `store/wal/` directory contains JSON entries for in-flight mutating operations. Each entry tracks:
```json
{
"op_id": "20260215120000123-a1b2c3d4",
"kind": "Build" | "Rebuild" | "Commit" | "Restore" | "Destroy",
"env_id": "...",
"timestamp": "RFC3339",
"rollback_steps": [
{ "RemoveDir": "/path/to/orphaned/dir" },
{ "RemoveFile": "/path/to/orphaned/file" }
]
}
```
### Recovery Protocol
1. On `Engine::new()`, the WAL directory is scanned for incomplete entries.
2. Each entry's rollback steps are executed in **reverse order**.
3. The WAL entry is then removed.
4. Corrupt or unreadable WAL entries are silently deleted.
### Invariants
- **INV-W1**: Kill during rebuild → next startup rolls back orphaned env_dir.
- **INV-W2**: Kill during build → orphaned env_dir cleaned.
- **INV-W3**: Successful operations leave zero WAL entries.
## Staging Directory
The `store/staging/` directory is a temporary workspace used for atomic operations:
- **Restore**: snapshot tar is unpacked into `staging/restore-{env_id}`, then renamed to replace the overlay upper directory.
- **Layer packing**: temporary files during tar creation.
The staging directory is cleaned up after each operation. Leftover staging data is safe to delete.
## Backward Compatibility
- Layout changes require a format version bump.
- Karapace 1.0 requires format version 2.
- Version 1 stores are not auto-migrated; environments must be rebuilt.
- The `name` and `tar_hash` fields use `#[serde(default)]` for forward-compatible deserialization.

132
docs/verification.md
View file

@ -1,132 +0,0 @@
# Verifying Karapace Downloads
Every tagged release includes signed binaries for both glibc and musl (static) targets,
SHA256 checksums, and in-toto provenance attestations. All artifacts are signed with
cosign using GitHub Actions OIDC — no manual keys involved.
These instructions are mechanically verified in CI: the `verify-docs-executable` job
in `supply-chain-test.yml` executes these exact commands on every PR.
## Choosing a Binary
| Binary | Linking | Use Case |
|--------|---------|----------|
| `karapace-linux-x86_64-gnu` | Dynamic (glibc) | Standard Linux distributions |
| `karapace-linux-x86_64-musl` | Static (musl) | Minimal containers, Alpine, any distro |
| `karapace-dbus-linux-x86_64-gnu` | Dynamic (glibc) | D-Bus service on standard distros |
| `karapace-dbus-linux-x86_64-musl` | Static (musl) | D-Bus service in containers |
The musl binaries are fully statically linked and require no system libraries.
## 1. Verify SHA256 Checksums
```bash
# For glibc binaries:
sha256sum -c SHA256SUMS-gnu
# For musl binaries:
sha256sum -c SHA256SUMS-musl
```
Both `karapace` and `karapace-dbus` must show `OK`.
## 2. Verify Cosign Signatures
Install [cosign](https://docs.sigstore.dev/cosign/system_config/installation/):
```bash
# Verify karapace binary (use -gnu or -musl suffix as appropriate)
cosign verify-blob karapace-linux-x86_64-gnu \
--signature karapace-gnu.sig \
--certificate karapace-gnu.crt \
--certificate-identity-regexp 'https://github.com/marcoallegretti/karapace' \
--certificate-oidc-issuer https://token.actions.githubusercontent.com
# Verify karapace-dbus binary
cosign verify-blob karapace-dbus-linux-x86_64-gnu \
--signature karapace-dbus-gnu.sig \
--certificate karapace-dbus-gnu.crt \
--certificate-identity-regexp 'https://github.com/marcoallegretti/karapace' \
--certificate-oidc-issuer https://token.actions.githubusercontent.com
```
Both commands must print `Verified OK`. Replace `-gnu` with `-musl` for static binaries.
## 3. Verify Provenance Attestation
```bash
cosign verify-blob provenance-gnu.json \
--signature provenance-gnu.json.sig \
--certificate provenance-gnu.json.crt \
--certificate-identity-regexp 'https://github.com/marcoallegretti/karapace' \
--certificate-oidc-issuer https://token.actions.githubusercontent.com
```
Inspect the provenance to verify the build origin:
```bash
python3 -c "
import json
with open('provenance-gnu.json') as f:
prov = json.load(f)
src = prov['predicate']['invocation']['configSource']
print(f'Commit: {src[\"digest\"][\"sha1\"]}')
print(f'Repo: {src[\"uri\"]}')
print(f'Workflow: {src[\"entryPoint\"]}')
print(f'Builder: {prov[\"predicate\"][\"builder\"][\"id\"]}')
"
```
## 4. Inspect SBOM
The CycloneDX SBOM lists all Rust dependencies and their versions:
```bash
python3 -m json.tool karapace_bom.json | head -50
```
## Build Reproducibility
All CI release builds enforce:
- `CARGO_INCREMENTAL=0` — disables incremental compilation
- `cargo clean` before every release build — eliminates stale intermediate artifacts
- `SOURCE_DATE_EPOCH=0` — deterministic timestamps
- `--remap-path-prefix` — eliminates runner-specific filesystem paths
- `strip = true` + `lto = "thin"` — deterministic output
**Reproducibility requirement:** Build invocations must use identical `-p` flags.
Building `-p karapace-cli` alone may produce a different binary than
`-p karapace-cli -p karapace-dbus` due to codegen unit ordering.
Both glibc and musl builds are verified for same-run and cross-run reproducibility
(ubuntu-latest vs ubuntu-22.04). Musl static builds are expected to be fully
runner-independent since they have no dynamic library dependencies.
## Local Development Builds
Local dev builds use `.cargo/config.toml` to remap dependency paths via `--remap-path-prefix`.
This eliminates local filesystem paths from release binaries. The remapping is configured for
the project maintainer's paths; other developers should update the paths in `.cargo/config.toml`
or set `RUSTFLAGS` directly.
**Local builds are for development only. CI builds are the authoritative release artifacts.**
## Release Artifacts
Each GitHub release contains artifacts for both `x86_64-unknown-linux-gnu` (glibc) and
`x86_64-unknown-linux-musl` (static) targets:
| File | Description |
|------|-------------|
| `karapace-linux-x86_64-gnu` | CLI binary (glibc) |
| `karapace-linux-x86_64-musl` | CLI binary (static musl) |
| `karapace-dbus-linux-x86_64-gnu` | D-Bus service binary (glibc) |
| `karapace-dbus-linux-x86_64-musl` | D-Bus service binary (static musl) |
| `SHA256SUMS-gnu` | SHA256 checksums for glibc binaries |
| `SHA256SUMS-musl` | SHA256 checksums for musl binaries |
| `karapace-gnu.sig` / `.crt` | Cosign signature + certificate (glibc CLI) |
| `karapace-musl.sig` / `.crt` | Cosign signature + certificate (musl CLI) |
| `karapace-dbus-gnu.sig` / `.crt` | Cosign signature + certificate (glibc D-Bus) |
| `karapace-dbus-musl.sig` / `.crt` | Cosign signature + certificate (musl D-Bus) |
| `provenance-gnu.json` / `.sig` / `.crt` | Provenance attestation (glibc) |
| `provenance-musl.json` / `.sig` / `.crt` | Provenance attestation (musl) |

66
docs/versioning-policy.md
View file

@ -1,66 +0,0 @@
# Karapace Versioning Policy
## Versioned Artifacts
Karapace versions three independent artifacts:
| Artifact | Current Version | Location |
|---|---|---|
| Manifest format | `1` | `manifest_version` field in manifest |
| Store layout | `2` | `store/version` file |
| DBus API | `1` | `org.karapace.Manager1` interface |
| Remote protocol | `v1-draft` | `X-Karapace-Protocol` header |
## Freeze Policies
### Store Format Freeze
- **Store layout v2 is frozen as of Karapace 1.0.**
- No breaking changes to the store directory layout, object naming scheme, or metadata JSON schema within 1.x releases.
- New optional fields may be added to metadata JSON with `#[serde(default)]`.
- Store v1 is not supported; Karapace 1.0+ rejects v1 stores with a clear error.
- If a future major version changes the store format, `karapace migrate` will be provided.
### Remote Protocol Freeze
- **Remote protocol v1 is currently `v1-draft`** and may change before Karapace 1.1.
- The protocol will be frozen (declared stable) in Karapace 1.1.
- After freeze: blob routes, registry format, and integrity checking are stable.
- New optional endpoints may be added without a version bump.
- Breaking changes require a protocol version bump and `X-Karapace-Protocol` negotiation.
## Compatibility Rules
### Manifest Format
- The `manifest_version` field is required and checked on parse.
- Only version `1` is supported in Karapace 0.1.
- Adding optional fields is a backward-compatible change (no version bump).
- Removing or renaming fields requires a version bump.
- Changing normalization or hashing behavior requires a version bump.
### Store Layout
- The store format version is stored in `store/version`.
- Karapace refuses to operate on a store with a different version.
- Adding new file types to the store is backward-compatible.
- Changing the object naming scheme or layout requires a version bump.
### DBus API
- The API version is returned by `ApiVersion()`.
- Adding new methods is backward-compatible.
- Changing method signatures requires a version bump.
- The interface name includes the major version (`Manager1`).
## Release Policy
- **Patch releases** (0.1.x): bug fixes only, no format changes.
- **Minor releases** (0.x.0): may add backward-compatible features.
- **Major releases** (x.0.0): may break backward compatibility with version bumps.
## Migration
- Karapace 0.1 does not support migration from other tools.
- Future versions may include `karapace migrate` for store upgrades.
- Manifest version migration is the user's responsibility.