WEFT_OS/docs/building.md

132 lines
4.6 KiB
Markdown
Raw Permalink Normal View History

# Building WEFT OS
## Prerequisites
Linux (x86_64 or aarch64). WEFT OS is built and validated on Linux only; runtime components require Linux kernel interfaces.
System packages (openSUSE):
```sh
sudo zypper install -y \
libwayland-devel libxkbcommon-devel \
libglvnd-devel libgbm-devel libdrm-devel \
libinput-devel seatd-devel libudev-devel systemd-devel \
pkg-config clang cmake python3
```
Rust toolchain: pinned in `rust-toolchain.toml`. Run `rustup show` to confirm the active toolchain matches.
## Workspace crates (non-Servo)
```sh
cargo build --workspace \
--exclude weft-servo-shell \
--exclude weft-app-shell
```
These crates do not require Servo and build in under two minutes.
## weft-compositor, weft-servo-shell, weft-app-shell (Linux)
```sh
cargo build -p weft-compositor
cargo build -p weft-servo-shell
cargo build -p weft-app-shell
```
fix(shells): paint and present Servo frames and report readiness truthfully With servo-embed, weft-servo-shell and weft-app-shell did not compile against the selected Servo revision: the keyboard event shape, key and mouse action variants, frame readback, shutdown, URL type and window handle APIs had changed. The frame path never called WebView::paint; the software path rebuilt its softbuffer surface every frame and read pixels without painting, and the EGL path presented nothing. weft-app-shell printed READY right after creating its WebView, before any frame, and its feature-disabled build printed READY and parked. Servo also disables CSS Grid by default, which the system UI uses. Both hosts now build against the selected API. Each frame is painted with WebView::paint and presented: the software path reads the painted frame back, presents the context and copies it into a persistent softbuffer surface; the EGL path presents its window context. Nothing is presented before Servo's first repaint request. Shutdown drops the WebView and Servo, whose drop completes the shutdown, and exits the event loop. Both hosts enable layout_grid_enabled. weft-app-shell prints READY on the first frame presented after the document has loaded and Servo reports its rendering settled: on load completion it requests WebView::take_screenshot, which completes only once every frame has fired load, render-blocking resources, images and web fonts are done and the rendering is up to date. Feature-disabled builds of both hosts exit with an error instead of reporting readiness. weft-appd does not consume the app shell's READY yet. tests/frame/check_frame.py runs weft-compositor nested on Xvfb with the winit backend, starts a host showing tests/frame/reference.html and checks the presented pixels (four colour quadrants and text). For the app host the page must be on screen within two seconds of READY; --slow-style delays a render-blocking stylesheet that reveals the page, so premature readiness fails. Validation (Ubuntu 24.04, Mesa 25.2.8 llvmpipe, software path): - check_frame.py --host system, --host app and --host app --slow-style 3: 5 of 5 runs pass each - negative fixtures fail: page with red and green swapped; a fake host printing READY without rendering; a local build printing READY on its first presented frame (--slow-style 4, 3 of 3) - cargo clippy -D warnings with and without servo-embed; cargo fmt --check - unverified: EGL path, DRM, resize, output scale, idle wakeup, input
2026-10-09 01:02:26 +00:00
Without `--features servo-embed`, the servo-shell and app-shell binaries compile for type and lint checks but exit with an error at startup; they never report readiness. This is the default and the CI baseline.
## Servo embedding (optional, slow)
```sh
cargo build -p weft-servo-shell --features servo-embed
cargo build -p weft-app-shell --features servo-embed
```
build: select compiling Servo and Stylo fork revisions Both Servo hosts tracked the marcoallegretti/servo `servo-weft` branch, which the lockfile resolved to 8e7dc40. That revision's servo-layout does not compile (E0308 and E0716 in build_backdrop_filter). WEFT also resolved upstream Stylo dca3934, where backdrop-filter is gated behind layout.unimplemented, because the Servo fork's own [patch] for Stylo does not apply to dependents. Pin Servo by revision to f0bb1aa (8e7dc40 plus the build fix) and add a root [patch."https://github.com/servo/stylo"] selecting the marcoallegretti/stylo fork at f1ba496 for all eight Stylo crates Servo uses. Cargo.lock changes only the 66 source lines of those packages. Record both pins, their reachable branches, purpose and removal conditions in SERVO_PIN.md, and restate its limitations against executed evidence. Update the Nix outputHashes: the Stylo hash is recomputed for f1ba496 with the method that reproduces the previous dca3934 value; the Servo hash is not yet known for f0bb1aa and is left as lib.fakeHash, so Nix builds report the expected value instead of using a stale one. Validation: - cargo metadata --locked: ok; no servo/stylo sources remain in the lock - cargo check -p weft-servo-shell --features servo-embed --locked: all of Servo, including servo-layout, compiles; the remaining errors are in the shell's own embedder code - cargo fmt --check; portable workspace tests: 78 passed - Nix fetchgit could not run here (archive downloads and the sandbox's TLS to the proxy fail), so the Servo hash remains unverified
2026-10-09 00:13:08 +00:00
This fetches and compiles the Servo and Stylo forks at the revisions recorded in `Cargo.lock` (see `crates/weft-servo-shell/SERVO_PIN.md`). Expect 30–60 minutes on a clean build. Servo's dependencies include SpiderMonkey (C++), which requires `clang` and `python3`.
fix(shells): paint and present Servo frames and report readiness truthfully With servo-embed, weft-servo-shell and weft-app-shell did not compile against the selected Servo revision: the keyboard event shape, key and mouse action variants, frame readback, shutdown, URL type and window handle APIs had changed. The frame path never called WebView::paint; the software path rebuilt its softbuffer surface every frame and read pixels without painting, and the EGL path presented nothing. weft-app-shell printed READY right after creating its WebView, before any frame, and its feature-disabled build printed READY and parked. Servo also disables CSS Grid by default, which the system UI uses. Both hosts now build against the selected API. Each frame is painted with WebView::paint and presented: the software path reads the painted frame back, presents the context and copies it into a persistent softbuffer surface; the EGL path presents its window context. Nothing is presented before Servo's first repaint request. Shutdown drops the WebView and Servo, whose drop completes the shutdown, and exits the event loop. Both hosts enable layout_grid_enabled. weft-app-shell prints READY on the first frame presented after the document has loaded and Servo reports its rendering settled: on load completion it requests WebView::take_screenshot, which completes only once every frame has fired load, render-blocking resources, images and web fonts are done and the rendering is up to date. Feature-disabled builds of both hosts exit with an error instead of reporting readiness. weft-appd does not consume the app shell's READY yet. tests/frame/check_frame.py runs weft-compositor nested on Xvfb with the winit backend, starts a host showing tests/frame/reference.html and checks the presented pixels (four colour quadrants and text). For the app host the page must be on screen within two seconds of READY; --slow-style delays a render-blocking stylesheet that reveals the page, so premature readiness fails. Validation (Ubuntu 24.04, Mesa 25.2.8 llvmpipe, software path): - check_frame.py --host system, --host app and --host app --slow-style 3: 5 of 5 runs pass each - negative fixtures fail: page with red and green swapped; a fake host printing READY without rendering; a local build printing READY on its first presented frame (--slow-style 4, 3 of 3) - cargo clippy -D warnings with and without servo-embed; cargo fmt --check - unverified: EGL path, DRM, resize, output scale, idle wakeup, input
2026-10-09 01:02:26 +00:00
## Frame check
`tests/frame/check_frame.py` runs `weft-compositor` nested on Xvfb with the
winit backend, starts a Servo host showing `tests/frame/reference.html`, and
checks the presented pixels. It needs `Xvfb`, `xwd`, ImageMagick's `convert`
and a Mesa OpenGL driver (llvmpipe is sufficient).
```sh
cargo build -p weft-compositor
cargo build -p weft-servo-shell -p weft-app-shell \
--features weft-servo-shell/servo-embed,weft-app-shell/servo-embed
python3 tests/frame/check_frame.py --host system
python3 tests/frame/check_frame.py --host app
python3 tests/frame/check_frame.py --host app --slow-style 3
```
`--host app` also requires the page on screen within two seconds of
`weft-app-shell` printing `READY`; `--slow-style` delays a stylesheet that
reveals the page so premature readiness becomes visible.
Logs and the last capture are written to `target/frame-check`.
test: verify native pointer and keyboard input through weft-compositor Nothing checked that input reaches a page through the compositor; the Servo hosts' input forwarding had only been compiled. tests/frame/check_input.py runs weft-servo-shell with tests/frame/input.html on the nested desktop and injects real X input with xdotool into weft-compositor's window. The compositor's winit backend routes it through its seat to the Wayland client, which has no X display. The page must react in its presented pixels: clicks 6 px either side of the boundary between its halves change only the half under the pointer, and the a and Shift+a key presses arrive with the right key value and modifier. The harness gives the compositor's X window input focus, as a window manager would; the client's keyboard focus comes from the compositor's click-to-focus. The nested desktop setup shared with check_frame.py moves to tests/frame/session.py. Xvfb now reports the display it picked through -displayfd, so concurrent runs cannot share an X server, and a run clears only the logs and capture it writes. Validation: - check_input.py: 3 of 3 runs pass; two concurrent runs pass - negative pages fail: no script; left click handled by the right half; an element extending the left half 20 px across the boundary; a key handler ignoring Shift - check_frame.py --host system, --host app, --host app --slow-style 3 pass after the refactor - unverified: key release and repeat, pointer motion and button release events, wheel, focus loss, non-US layouts, non-ASCII text, scale changes, weft-app-shell input
2026-10-09 01:18:53 +00:00
`tests/frame/check_input.py` uses the same nested desktop to send real X
pointer and keyboard input (with `xdotool`) through `weft-compositor` to
`weft-servo-shell` showing `tests/frame/input.html`, and checks that the
page reacts to clicks on either side of a boundary and to the `a` and
`Shift+a` keys. Logs and the last capture are written to `target/input-check`.
```sh
python3 tests/frame/check_input.py
```
## Demo apps (wasm32-wasip2)
Each demo is a standalone crate in `examples/`. Pre-built `app.wasm` binaries are committed. To rebuild:
```sh
rustup target add wasm32-wasip2
cd examples/org.weft.demo.counter
cargo build --release
# output: target/wasm32-wasip2/release/app.wasm
```
## weft-runtime with Wasmtime
```sh
cargo build -p weft-runtime --features wasmtime-runtime,net-fetch
```
Without `--features wasmtime-runtime`, the runtime compiles for type, lint and unit checks (the portable CI configuration) but exits with an error when asked to run a component; it never reports readiness.
## Signing packages
```sh
weft-pack generate-key ./keys
weft-pack sign ./examples/org.weft.demo.counter --key ./keys/weft-sign.key
weft-pack verify ./examples/org.weft.demo.counter --key ./keys/weft-sign.pub
```
## NixOS VM (requires Nix with flakes)
Build the VM image:
```sh
bash infra/vm/build.sh
```
Run with QEMU:
```sh
bash infra/vm/run.sh
```
build: select compiling Servo and Stylo fork revisions Both Servo hosts tracked the marcoallegretti/servo `servo-weft` branch, which the lockfile resolved to 8e7dc40. That revision's servo-layout does not compile (E0308 and E0716 in build_backdrop_filter). WEFT also resolved upstream Stylo dca3934, where backdrop-filter is gated behind layout.unimplemented, because the Servo fork's own [patch] for Stylo does not apply to dependents. Pin Servo by revision to f0bb1aa (8e7dc40 plus the build fix) and add a root [patch."https://github.com/servo/stylo"] selecting the marcoallegretti/stylo fork at f1ba496 for all eight Stylo crates Servo uses. Cargo.lock changes only the 66 source lines of those packages. Record both pins, their reachable branches, purpose and removal conditions in SERVO_PIN.md, and restate its limitations against executed evidence. Update the Nix outputHashes: the Stylo hash is recomputed for f1ba496 with the method that reproduces the previous dca3934 value; the Servo hash is not yet known for f0bb1aa and is left as lib.fakeHash, so Nix builds report the expected value instead of using a stale one. Validation: - cargo metadata --locked: ok; no servo/stylo sources remain in the lock - cargo check -p weft-servo-shell --features servo-embed --locked: all of Servo, including servo-layout, compiles; the remaining errors are in the shell's own embedder code - cargo fmt --check; portable workspace tests: 78 passed - Nix fetchgit could not run here (archive downloads and the sandbox's TLS to the proxy fail), so the Servo hash remains unverified
2026-10-09 00:13:08 +00:00
See `infra/nixos/weft-packages.nix` for the package derivations. Its `outputHashes` hold one hash per git source in `Cargo.lock` (the Servo and Stylo forks); every package vendors them, so they must match the pinned revisions for any Nix package to build. The Nix shell packages do not yet enable `servo-embed`.
## CI
fix(runtime): close fetch and preopen gaps in grant enforcement A component with a host-specific fetch grant could reach another origin by setting its own Host header, and could inject request lines through the method string, which ureq writes unvalidated. Fetch now accepts only standard methods and refuses Host, framing and hop-by-hop headers. Fetch grants accept canonical DNS names and dotted IPv4 addresses only, so a grant can never be written in a form that silently never matches. --preopen split host and guest at the first `::`, so a host path containing `::` widened the preopened directory; it now splits from the right, and appd refuses host paths that are not UTF-8 or contain `::`. appd validated capabilities one at a time, creating the data directory before a later capability was refused, and required HOME even for packages without filesystem capabilities. It now checks every capability first, resolves host directories only when needed, validates the app ID with the rule weft-pack uses (now shared in weft_ipc_types::package), and reports 400, 404, 403 or 500 according to the cause. user-dirs.dirs is read as a shell would: the last assignment wins, $HOME must be followed by a slash, escaped values are ignored and a relative XDG_CONFIG_HOME is not used. The grants probe now also overwrites, renames, removes and creates directories in read-only data, and sends a forged Host header, an injected method and a file: URL; it fails when directory permissions of a read-only preopen are widened or the header check is removed. A new CI job builds the runtime with its engine and fetch support and runs the check. The mount test that sets WEFT_APP_STORE now holds the env lock. Package images are still located separately from directory installs; deriving grants from the verified image is part of resolving one package descriptor.
2026-10-09 08:25:40 +00:00
Jobs run on every push to `main` and on pull requests:
- `portable` — fmt, clippy, tests for the crates that need no Linux system libraries (excludes Wayland crates)
- `linux-only` — clippy and tests for `weft-compositor`, `weft-servo-shell`, `weft-app-shell`
fix(runtime): close fetch and preopen gaps in grant enforcement A component with a host-specific fetch grant could reach another origin by setting its own Host header, and could inject request lines through the method string, which ureq writes unvalidated. Fetch now accepts only standard methods and refuses Host, framing and hop-by-hop headers. Fetch grants accept canonical DNS names and dotted IPv4 addresses only, so a grant can never be written in a form that silently never matches. --preopen split host and guest at the first `::`, so a host path containing `::` widened the preopened directory; it now splits from the right, and appd refuses host paths that are not UTF-8 or contain `::`. appd validated capabilities one at a time, creating the data directory before a later capability was refused, and required HOME even for packages without filesystem capabilities. It now checks every capability first, resolves host directories only when needed, validates the app ID with the rule weft-pack uses (now shared in weft_ipc_types::package), and reports 400, 404, 403 or 500 according to the cause. user-dirs.dirs is read as a shell would: the last assignment wins, $HOME must be followed by a slash, escaped values are ignored and a relative XDG_CONFIG_HOME is not used. The grants probe now also overwrites, renames, removes and creates directories in read-only data, and sends a forged Host header, an injected method and a file: URL; it fails when directory permissions of a read-only preopen are widened or the header check is removed. A new CI job builds the runtime with its engine and fetch support and runs the check. The mount test that sets WEFT_APP_STORE now holds the env lock. Package images are still located separately from directory installs; deriving grants from the verified image is part of resolving one package descriptor.
2026-10-09 08:25:40 +00:00
- `runtime-grants` — clippy and a build of `weft-runtime` with `wasmtime-runtime,net-fetch`, then `tests/runtime/check_grants.py`, which runs a probe component under each kind of capability grant
- `servo-embed-linux` — `cargo check --features servo-embed` for servo-shell and app-shell
docs: add engineering contract, contributor toolkit and templates Add AGENTS.md as the single engineering contract: sources of truth, evidence and proof requirements, ownership, working modes, the reviewed change loop, maintenance isolation, decision authority, dependency and fork rules, the 0.0.x to 0.1.0 release progression, and branch and publication rules. Add .agents/ with scope, implement, verify, maintenance and release procedures; reviewer, architecture debt scout, simplicity and refactoring roles; skills for the compositor, Servo rendering, runtime capabilities, appd sessions, packages and data, Linux deployment, integration verification and documentation; and verify.py, which runs the existing check profiles with --locked from any directory, stops on the first failure and supports --dry-run. check_toolkit.py validates local links and heading anchors, metadata, catalog coverage and issue forms. A new contributor-toolkit CI job runs both. The structure follows the Resina Design System toolkit at 1e768064f57a1e403ab271822383e732795905a6, adapted to WEFT. Validation, against the committed lockfiles: - verify.py toolkit: checker passes, 28 runner/checker tests pass - verify.py portable, demos, packages: pass - verify.py linux, runtime, servo-embed: fail on existing defects (clippy collapsible_if in the winit backend, three clippy lints in weft-runtime with wasmtime-runtime, and an E0308 in the selected Servo revision's servo-layout); the runner reports the failing status Add CONTRIBUTING.md, issue forms for defects and changes, maintenance problems and decisions of direction, and a pull request template asking for ownership, proof and the validation actually performed. README.md links the contribution guide.
2026-10-08 23:40:08 +00:00
- `contributor-toolkit` — `python .agents/scripts/verify.py toolkit`