coredesktop/skills/coredesktop-dev-sibling.md

Skill: Building coredesktop-dev, the CoreDesktop Development Sibling

Overview

coredesktop-dev is a CoreDesktop sibling project for people who work on CoreDesktop itself. Installing it with sibling add <url> gives a machine the tools to change CoreDesktop safely: the linters and test runner, a command that runs the checks the way CI should, a way to try a working copy without touching the installed one, and a guard that keeps the installed copy updatable.

Follow this document when creating the coredesktop-dev repository from scratch. It covers what to build, why, and how to test it. It assumes CoreDesktop is installed at ~/.local/share/coredesktop. The rules for any sibling are in that checkout's skills/sibling-project-format.md: read it first. This document only adds what is specific to this project.

Why a sibling and not part of CoreDesktop

• Only contributors need it. Every package in CoreDesktop's packages.conf lands on every user's machine. shellcheck, bats and the Gitea CLI don't belong there.
• It changes on its own schedule. Development tooling can change without a CoreDesktop release, and update still keeps it current on every developer's machine.

Background: the two copies of CoreDesktop

Every problem this project solves comes from one fact. A developer's machine has two copies of CoreDesktop:

Copy	Path	What it is
Deployed	~/.local/share/coredesktop	The machine's running CoreDesktop. A git clone that update pulls with --ff-only. ~/.local/bin commands, updates.d hooks, configure items and sandbox configs all run from here.
Working	e.g. ~/Projects/coredesktop	Where development happens: branches, uncommitted changes. Nothing runs from here unless you run it yourself.

Lessons learned the hard way, which the tools below exist to enforce:

1. Never edit the deployed copy. Copying working files into it to "try them" leaves tracked files modified, and the next update fails to pull (--ff-only refuses). If it happens, git -C ~/.local/share/coredesktop checkout -- . restores it, but it's easy to forget.
2. Scripts hard-code the deployed path. Most CoreDesktop scripts set COREDESKTOP="$HOME/.local/share/coredesktop". To run working-copy code as if it were deployed, point a throwaway HOME at the working copy with a symlink. Don't swap the real deployed copy.
3. A throwaway HOME doesn't isolate the session. XDG_RUNTIME_DIR (/run/user/<uid>) is shared with the live GNOME session. A test that ran gnome-keyring-daemon --replace with only HOME changed found the real keyring daemon through /run/user/<uid>/keyring, killed it, and left the user's keyring locked, which hung Bitwarden. Any test that starts session daemons needs its own XDG_RUNTIME_DIR (mode 700), no DISPLAY or WAYLAND_DISPLAY, and a dbus-run-session. Never use --replace.
4. sudo, yay, pacman and systemctl ignore HOME. A throwaway home protects your files, not your system. Code that changes the system has to be tested with fake commands (bats stubs) or on a machine or VM you don't mind changing.
5. Editors can run in a Flatpak. VSCodium from Flathub, for one, runs its terminal and any AI agent inside the Flatpak sandbox, where pacman, systemctl, journalctl and the host's /tmp don't exist. Commands there must go through flatpak-spawn --host. The tools should notice this and say so, rather than fail in confusing ways.
6. make lint only lints tracked files. CoreDesktop's Makefile builds its list from git ls-files, so a new script isn't linted until it's committed. The checks have to include untracked scripts.
7. CI runs only shellcheck. .gitea/workflows/shellcheck.yml runs make lint, not the bats tests, so make check has to run locally before pushing.

What to build

Repository layout

coredesktop-dev/
  install.sh            run once by `sibling add`
  update.sh             run on every `update`
  bin/coredesktop-dev   the command (symlinked into ~/.local/bin)
  hooks/pre-push        git hook for the working copy (opt-in)
  lib/                  one file per subcommand, plus common.sh and vm.sh
  vm/                   provision.sh and cloud-init user-data for the smoketest VM
  tests/                bats tests for this project
  Makefile              `make check`: shellcheck + bats for this repo
  .shellcheckrc         copy CoreDesktop's
  README.md             install with: sibling add <url>

Packages

Installed by update.sh with CoreDesktop's install_packages, so packages added later reach existing machines on their next update. All are in Arch's extra repo:

Package	Why
shellcheck	make lint
bats	make test (CoreDesktop's tests use plain bats, no helper libraries)
tea	Gitea's CLI, for pull requests on gitea.federated.computer
qemu-base	The smoketest's virtual machines
libisoburn	xorriso, to make the smoketest VM's cloud-init disk

make, git and jq already come with CoreDesktop. Don't add shfmt until CoreDesktop decides on a formatting style; it has no config for it today.

Settings

~/.config/coredesktop-dev/config, a shell file install.sh writes and every other script sources:

WORKING_COPY=/home/<user>/Projects/coredesktop

Write it outside the repo: siblings must never write tracked files.

install.sh

1. Source ~/.local/share/coredesktop/utils.sh. Stop with a clear message if it's missing (CoreDesktop isn't installed).
2. Ask where the working copy is, defaulting to ~/Projects/coredesktop.
   • If that folder is a git clone of CoreDesktop, use it. Check that its origin points at a CoreDesktop repository.
   • If it doesn't exist, offer to clone it there from the deployed copy's origin URL (git -C ~/.local/share/coredesktop remote get-url origin).
   • Never use the deployed copy itself as the working copy. Refuse if the answer resolves to ~/.local/share/coredesktop.
3. Save the answer in the settings file.
4. Run update.sh, which installs the packages and the command.
5. Offer to install the pre-push hook (see hooks below). Default no.
6. Run coredesktop-dev doctor and show the result.

It must be safe to re-run: re-running sibling add is how users repair an install. If the settings file exists, show the current working copy as the default answer.

update.sh

No prompts. It prints what it does and exits non-zero only on real failure:

1. install_packages shellcheck bats tea qemu-base libisoburn
2. Make ~/.local/bin/coredesktop-dev a symlink to bin/coredesktop-dev in this repo, so each pull updates the command without copying.
3. If the pre-push hook is installed in the working copy, refresh it from hooks/pre-push (compare the files, copy only if different).

The coredesktop-dev command

Subcommands, each with --help. With no subcommand, it prints usage.

coredesktop-dev doctor

Read-only. It reports each of these as ok or a problem, with the fix:

• shellcheck, bats and tea are installed.
• The working copy from the settings file exists and is a CoreDesktop clone.
• The deployed copy has no local changes (git status --porcelain --untracked-files=no is empty). If it has some, list them and point to coredesktop-dev deployed restore.
• The deployed copy can fast-forward to its upstream, or is already there.
• Whether it is running inside a Flatpak (/.flatpak-info exists, or $FLATPAK_ID is set). If it is, say that host commands need flatpak-spawn --host. If the subcommand that was asked for needs the host (anything that runs make check with the host's tools), re-run itself on the host through flatpak-spawn --host instead of failing.
• Whether the pre-push hook is installed.

Exit 0 if everything is ok, 1 otherwise.

coredesktop-dev check [--fast]

Runs the checks on the working copy, from its root:

1. make check (shellcheck on tracked scripts, then bats).
2. shellcheck on untracked scripts too, with the same arguments the Makefile uses (--severity=warning, the repo's .shellcheckrc). The files are git ls-files --others --exclude-standard whose first line is a bash or sh shebang, plus untracked *.sh files. That closes the lint gap in lesson 6.
3. --fast runs only the shellcheck steps.

Exit non-zero if anything fails. Print a one-line summary at the end, e.g. shellcheck: ok (91 files, 6 untracked) bats: ok (85 tests).

coredesktop-dev try [command...]

Runs working-copy code as if it were the deployed CoreDesktop, without changing the deployed copy or the real home folder:

1. Make a temporary directory (mktemp -d) holding a home/ and a runtime/ folder (mode 700).
2. In the temporary home, symlink .local/share/coredesktop to the working copy and put .local/bin at the front of PATH. Link .local/bin/{update,refresh,sibling,version,configure,audit} to the working copy's scripts of the same names, as CoreDesktop's installer installs them.
3. Set HOME and the XDG_*_HOME variables to the temporary home, and XDG_RUNTIME_DIR to runtime/. Unset DISPLAY, WAYLAND_DISPLAY, DBUS_SESSION_BUS_ADDRESS and SSH_AUTH_SOCK.
4. Run the command, or an interactive bash if none was given, under dbus-run-session, so anything that needs a session bus gets a private one.
5. Before starting, print a warning: sudo, yay, pacman and systemctl still change this machine (lesson 4).
6. Remove the temporary directory afterwards, unless --keep was given; then print its path.

Example: coredesktop-dev try configure --list shows what the working copy's configure would list.

coredesktop-dev deployed [status|restore]

• status (the default): the deployed copy's branch, commit, how far behind upstream it is, and any local changes.
• restore: show git diff --stat of the local changes, ask for confirmation, then git checkout -- . in the deployed copy. Untracked files are listed but left alone. Never git clean or git reset --hard there.

coredesktop-dev hooks [install|remove]

Installs hooks/pre-push into the working copy's .git/hooks/pre-push. The .git folder isn't tracked, so this is allowed. The hook runs coredesktop-dev check and blocks the push if it fails. git push --no-verify skips it, and the hook's failure message says so. Refuse to overwrite a pre-push hook that this project didn't write: write a marker comment into the hook and check for it.

coredesktop-dev smoketest

Runs CoreDesktop's end-to-end scenarios on QEMU virtual machines. The scenarios, the fingerprint and health scripts, and the helpers they use live in CoreDesktop's smoketest/ folder, because they change with the code they test; read its README for the contract between them and this driver. This project only provides the VMs: vm_exec, vm_put, vm_reboot, VM_GIT_SSH and VM_GIT_URL, then calls smoketest_prepare and the scenario.

• list, run [--tier pr|all] [--ref REF] [--keep] [scenario...], images [base|installed [REF]], shell [base|installed], clean [--all].
• Images, the shared pacman cache, the SSH key and run logs live in ~/.cache/coredesktop-dev/smoketest, never in either copy of CoreDesktop.
• base is Arch's cloud image plus archinstall's GNOME desktop profile (vm/provision.sh), with automatic login and passwordless sudo for tester. installed is base after install.sh at the commit the branch started from. Each scenario runs on a throwaway qcow2 overlay of one of them.
• Commands in the VM run in an interactive bash with the GNOME session's environment, as they would in GNOME Terminal.

What not to do

• Don't copy hooks into CoreDesktop's updates.d/: use update.sh, as the sibling format says. Using both updates the project twice.
• Don't change CoreDesktop's files from this project, including the working copy's tracked files. Suggest such changes to CoreDesktop instead (below).
• Don't start, stop or --replace real session services (keyring, portals, GNOME Shell) in any command or test.

Changes to suggest to CoreDesktop itself

These belong in CoreDesktop's repository, not in this project. Once they're made, coredesktop-dev check still works, and step 2 above simply finds nothing extra to lint.

1. Lint untracked scripts in the Makefile. Build SCRIPTS from git ls-files --cached --others --exclude-standard.
2. Run the bats tests in CI. Add a make test job to .gitea/workflows/, installing bats alongside shellcheck.

Testing this project

Use the same habits as CoreDesktop's own tests:

• bats with stubs. Copy CoreDesktop's tests/test_helper.bash pattern: a fresh HOME and a bin/ of fake commands at the front of PATH for each test. Fake pacman, yay, sudo, flatpak-spawn and make, so no test changes the machine.
• Prove the tests catch bugs. After the tests pass, break the code on purpose once per behavior (e.g. make deployed restore skip the confirmation, or make check skip untracked files) and confirm a test fails each time.
• Test the sibling flow in a throwaway home, as described in sibling-project-format.md. update.sh calls install_packages, which uses yay, so stub it there too, or run it only where installing packages is fine.

Cases worth a test each:

• install.sh refuses the deployed copy as the working copy, and is safe to re-run.
• update.sh doesn't prompt and doesn't re-copy an unchanged hook.
• doctor reports a dirty deployed copy, and reports Flatpak use.
• check lints an untracked script and fails on a shellcheck warning in it.
• try gives the command a private HOME and XDG_RUNTIME_DIR, no DISPLAY, and the working copy at ~/.local/share/coredesktop, and cleans up afterwards.
• deployed restore asks first, restores tracked files, and leaves untracked ones alone.
• hooks install won't overwrite another hook.

Checklist

• The repo's README says to install with sibling add <url>
• install.sh asks for the working copy, refuses the deployed copy, saves ~/.config/coredesktop-dev/config, and is safe to re-run
• update.sh installs shellcheck, bats, tea, QEMU and xorriso, keeps the ~/.local/bin/coredesktop-dev symlink, and never prompts
• doctor, check, try, deployed, hooks and smoketest work and have --help
• check lints untracked scripts in the working copy
• try isolates HOME, XDG_* and XDG_RUNTIME_DIR, unsets the display and session variables, uses dbus-run-session, and warns that sudo, yay, pacman and systemctl still reach the real system
• Nothing writes to the deployed copy except deployed restore, after confirmation
• The project's own make check passes: shellcheck plus bats tests with stubs, and the tests catch deliberately broken code
• The two suggested CoreDesktop changes are raised with CoreDesktop