This commit is contained in:
@@ -0,0 +1,292 @@
|
||||
# Skill: Building coredesktop-dev, the CoreDesktop Development Sibling
|
||||
|
||||
## Overview
|
||||
|
||||
`coredesktop-dev` is a CoreDesktop sibling project for people who work on
|
||||
CoreDesktop itself. Installing it with `sibling add <url>` gives a machine
|
||||
the tools to change CoreDesktop safely: the linters and test runner, a
|
||||
command that runs the checks the way CI should, a way to try a working copy
|
||||
without touching the installed one, and a guard that keeps the installed copy
|
||||
updatable.
|
||||
|
||||
Follow this document when creating the `coredesktop-dev` repository from
|
||||
scratch. It covers what to build, why, and how to test it. It assumes
|
||||
CoreDesktop is installed at `~/.local/share/coredesktop`. The rules for any
|
||||
sibling are in that checkout's `skills/sibling-project-format.md`: read it
|
||||
first. This document only adds what is specific to this project.
|
||||
|
||||
## Why a sibling and not part of CoreDesktop
|
||||
|
||||
- **Only contributors need it.** Every package in CoreDesktop's
|
||||
`packages.conf` lands on every user's machine. shellcheck, bats and the
|
||||
Gitea CLI don't belong there.
|
||||
- **It changes on its own schedule.** Development tooling can change without
|
||||
a CoreDesktop release, and `update` still keeps it current on every
|
||||
developer's machine.
|
||||
|
||||
## Background: the two copies of CoreDesktop
|
||||
|
||||
Every problem this project solves comes from one fact. A developer's machine
|
||||
has two copies of CoreDesktop:
|
||||
|
||||
| Copy | Path | What it is |
|
||||
|------|------|------------|
|
||||
| **Deployed** | `~/.local/share/coredesktop` | The machine's running CoreDesktop. A git clone that `update` pulls with `--ff-only`. `~/.local/bin` commands, `updates.d` hooks, `configure` items and sandbox configs all run from here. |
|
||||
| **Working** | e.g. `~/Projects/coredesktop` | Where development happens: branches, uncommitted changes. Nothing runs from here unless you run it yourself. |
|
||||
|
||||
Lessons learned the hard way, which the tools below exist to enforce:
|
||||
|
||||
1. **Never edit the deployed copy.** Copying working files into it to "try
|
||||
them" leaves tracked files modified, and the next `update` fails to pull
|
||||
(`--ff-only` refuses). If it happens, `git -C ~/.local/share/coredesktop
|
||||
checkout -- .` restores it, but it's easy to forget.
|
||||
2. **Scripts hard-code the deployed path.** Most CoreDesktop scripts set
|
||||
`COREDESKTOP="$HOME/.local/share/coredesktop"`. To run working-copy code
|
||||
as if it were deployed, point a throwaway `HOME` at the working copy with
|
||||
a symlink. Don't swap the real deployed copy.
|
||||
3. **A throwaway `HOME` doesn't isolate the session.** `XDG_RUNTIME_DIR`
|
||||
(`/run/user/<uid>`) is shared with the live GNOME session. A test that ran
|
||||
`gnome-keyring-daemon --replace` with only `HOME` changed found the real
|
||||
keyring daemon through `/run/user/<uid>/keyring`, killed it, and left the
|
||||
user's keyring locked, which hung Bitwarden. Any test that starts session
|
||||
daemons needs its own `XDG_RUNTIME_DIR` (mode 700), no `DISPLAY` or
|
||||
`WAYLAND_DISPLAY`, and a `dbus-run-session`. Never use `--replace`.
|
||||
4. **`sudo`, `yay`, `pacman` and `systemctl` ignore `HOME`.** A throwaway home
|
||||
protects your files, not your system. Code that changes the system has to
|
||||
be tested with fake commands (bats stubs) or on a machine or VM you don't
|
||||
mind changing.
|
||||
5. **Editors can run in a Flatpak.** VSCodium from Flathub, for one, runs its
|
||||
terminal and any AI agent inside the Flatpak sandbox, where `pacman`,
|
||||
`systemctl`, `journalctl` and the host's `/tmp` don't exist. Commands
|
||||
there must go through `flatpak-spawn --host`. The tools should notice
|
||||
this and say so, rather than fail in confusing ways.
|
||||
6. **`make lint` only lints tracked files.** CoreDesktop's Makefile builds its
|
||||
list from `git ls-files`, so a new script isn't linted until it's
|
||||
committed. The checks have to include untracked scripts.
|
||||
7. **CI runs only shellcheck.** `.gitea/workflows/shellcheck.yml` runs
|
||||
`make lint`, not the bats tests, so `make check` has to run locally
|
||||
before pushing.
|
||||
|
||||
## What to build
|
||||
|
||||
### Repository layout
|
||||
|
||||
```
|
||||
coredesktop-dev/
|
||||
install.sh run once by `sibling add`
|
||||
update.sh run on every `update`
|
||||
bin/coredesktop-dev the command (symlinked into ~/.local/bin)
|
||||
hooks/pre-push git hook for the working copy (opt-in)
|
||||
lib/ shared functions, if bin/ grows large
|
||||
tests/ bats tests for this project
|
||||
Makefile `make check`: shellcheck + bats for this repo
|
||||
.shellcheckrc copy CoreDesktop's
|
||||
README.md install with: sibling add <url>
|
||||
```
|
||||
|
||||
### Packages
|
||||
|
||||
Installed by `update.sh` with CoreDesktop's `install_packages`, so packages
|
||||
added later reach existing machines on their next `update`. All are in Arch's
|
||||
`extra` repo:
|
||||
|
||||
| Package | Why |
|
||||
|---------|-----|
|
||||
| `shellcheck` | `make lint` |
|
||||
| `bats` | `make test` (CoreDesktop's tests use plain bats, no helper libraries) |
|
||||
| `tea` | Gitea's CLI, for pull requests on gitea.federated.computer |
|
||||
|
||||
`make`, `git` and `jq` already come with CoreDesktop. Don't add `shfmt` until
|
||||
CoreDesktop decides on a formatting style; it has no config for it today.
|
||||
|
||||
### Settings
|
||||
|
||||
`~/.config/coredesktop-dev/config`, a shell file `install.sh` writes and every
|
||||
other script sources:
|
||||
|
||||
```bash
|
||||
WORKING_COPY=/home/<user>/Projects/coredesktop
|
||||
```
|
||||
|
||||
Write it outside the repo: siblings must never write tracked files.
|
||||
|
||||
### `install.sh`
|
||||
|
||||
1. Source `~/.local/share/coredesktop/utils.sh`. Stop with a clear message if
|
||||
it's missing (CoreDesktop isn't installed).
|
||||
2. Ask where the working copy is, defaulting to `~/Projects/coredesktop`.
|
||||
- If that folder is a git clone of CoreDesktop, use it. Check that its
|
||||
`origin` points at a CoreDesktop repository.
|
||||
- If it doesn't exist, offer to clone it there from the deployed copy's
|
||||
`origin` URL (`git -C ~/.local/share/coredesktop remote get-url origin`).
|
||||
- Never use the deployed copy itself as the working copy. Refuse if the
|
||||
answer resolves to `~/.local/share/coredesktop`.
|
||||
3. Save the answer in the settings file.
|
||||
4. Run `update.sh`, which installs the packages and the command.
|
||||
5. Offer to install the pre-push hook (see `hooks` below). Default no.
|
||||
6. Run `coredesktop-dev doctor` and show the result.
|
||||
|
||||
It must be safe to re-run: re-running `sibling add` is how users repair an
|
||||
install. If the settings file exists, show the current working copy as the
|
||||
default answer.
|
||||
|
||||
### `update.sh`
|
||||
|
||||
No prompts. It prints what it does and exits non-zero only on real failure:
|
||||
|
||||
1. `install_packages shellcheck bats tea`
|
||||
2. Make `~/.local/bin/coredesktop-dev` a symlink to `bin/coredesktop-dev` in
|
||||
this repo, so each pull updates the command without copying.
|
||||
3. If the pre-push hook is installed in the working copy, refresh it from
|
||||
`hooks/pre-push` (compare the files, copy only if different).
|
||||
|
||||
### The `coredesktop-dev` command
|
||||
|
||||
Subcommands, each with `--help`. With no subcommand, it prints usage.
|
||||
|
||||
#### `coredesktop-dev doctor`
|
||||
|
||||
Read-only. It reports each of these as ok or a problem, with the fix:
|
||||
|
||||
- shellcheck, bats and tea are installed.
|
||||
- The working copy from the settings file exists and is a CoreDesktop clone.
|
||||
- **The deployed copy has no local changes** (`git status --porcelain
|
||||
--untracked-files=no` is empty). If it has some, list them and point to
|
||||
`coredesktop-dev deployed restore`.
|
||||
- The deployed copy can fast-forward to its upstream, or is already there.
|
||||
- Whether it is running inside a Flatpak (`/.flatpak-info` exists, or
|
||||
`$FLATPAK_ID` is set). If it is, say that host commands need
|
||||
`flatpak-spawn --host`. If the subcommand that was asked for needs the
|
||||
host (anything that runs `make check` with the host's tools), re-run
|
||||
itself on the host through `flatpak-spawn --host` instead of failing.
|
||||
- Whether the pre-push hook is installed.
|
||||
|
||||
Exit 0 if everything is ok, 1 otherwise.
|
||||
|
||||
#### `coredesktop-dev check [--fast]`
|
||||
|
||||
Runs the checks on the working copy, from its root:
|
||||
|
||||
1. `make check` (shellcheck on tracked scripts, then bats).
|
||||
2. **shellcheck on untracked scripts too**, with the same arguments the
|
||||
Makefile uses (`--severity=warning`, the repo's `.shellcheckrc`). The
|
||||
files are `git ls-files --others --exclude-standard` whose first line is a
|
||||
bash or sh shebang, plus untracked `*.sh` files. That closes the lint gap
|
||||
in lesson 6.
|
||||
3. `--fast` runs only the shellcheck steps.
|
||||
|
||||
Exit non-zero if anything fails. Print a one-line summary at the end, e.g.
|
||||
`shellcheck: ok (91 files, 6 untracked) bats: ok (85 tests)`.
|
||||
|
||||
#### `coredesktop-dev try [command...]`
|
||||
|
||||
Runs working-copy code as if it were the deployed CoreDesktop, without
|
||||
changing the deployed copy or the real home folder:
|
||||
|
||||
1. Make a temporary directory (`mktemp -d`) holding a `home/` and a
|
||||
`runtime/` folder (mode 700).
|
||||
2. In the temporary home, symlink `.local/share/coredesktop` to the working
|
||||
copy and put `.local/bin` at the front of `PATH`. Link
|
||||
`.local/bin/{update,refresh,sibling,version,configure,audit}` to the
|
||||
working copy's scripts of the same names, as CoreDesktop's installer
|
||||
installs them.
|
||||
3. Set `HOME` and the `XDG_*_HOME` variables to the temporary home, and
|
||||
`XDG_RUNTIME_DIR` to `runtime/`. Unset `DISPLAY`, `WAYLAND_DISPLAY`,
|
||||
`DBUS_SESSION_BUS_ADDRESS` and `SSH_AUTH_SOCK`.
|
||||
4. Run the command, or an interactive `bash` if none was given, under
|
||||
`dbus-run-session`, so anything that needs a session bus gets a private
|
||||
one.
|
||||
5. Before starting, print a warning: `sudo`, `yay`, `pacman` and `systemctl`
|
||||
still change this machine (lesson 4).
|
||||
6. Remove the temporary directory afterwards, unless `--keep` was given;
|
||||
then print its path.
|
||||
|
||||
Example: `coredesktop-dev try configure --list` shows what the working copy's
|
||||
`configure` would list.
|
||||
|
||||
#### `coredesktop-dev deployed [status|restore]`
|
||||
|
||||
- `status` (the default): the deployed copy's branch, commit, how far behind
|
||||
upstream it is, and any local changes.
|
||||
- `restore`: show `git diff --stat` of the local changes, ask for
|
||||
confirmation, then `git checkout -- .` in the deployed copy. Untracked
|
||||
files are listed but left alone. Never `git clean` or `git reset --hard`
|
||||
there.
|
||||
|
||||
#### `coredesktop-dev hooks [install|remove]`
|
||||
|
||||
Installs `hooks/pre-push` into the working copy's `.git/hooks/pre-push`. The
|
||||
`.git` folder isn't tracked, so this is allowed. The hook runs
|
||||
`coredesktop-dev check` and blocks the push if it fails. `git push
|
||||
--no-verify` skips it, and the hook's failure message says so. Refuse to
|
||||
overwrite a pre-push hook that this project didn't write: write a marker
|
||||
comment into the hook and check for it.
|
||||
|
||||
### What not to do
|
||||
|
||||
- Don't copy hooks into CoreDesktop's `updates.d/`: use `update.sh`, as the
|
||||
sibling format says. Using both updates the project twice.
|
||||
- Don't change CoreDesktop's files from this project, including the working
|
||||
copy's tracked files. Suggest such changes to CoreDesktop instead (below).
|
||||
- Don't start, stop or `--replace` real session services (keyring, portals,
|
||||
GNOME Shell) in any command or test.
|
||||
|
||||
## Changes to suggest to CoreDesktop itself
|
||||
|
||||
These belong in CoreDesktop's repository, not in this project. Once they're
|
||||
made, `coredesktop-dev check` still works, and step 2 above simply finds
|
||||
nothing extra to lint.
|
||||
|
||||
1. **Lint untracked scripts in the Makefile.** Build `SCRIPTS` from
|
||||
`git ls-files --cached --others --exclude-standard`.
|
||||
2. **Run the bats tests in CI.** Add a `make test` job to
|
||||
`.gitea/workflows/`, installing `bats` alongside `shellcheck`.
|
||||
|
||||
## Testing this project
|
||||
|
||||
Use the same habits as CoreDesktop's own tests:
|
||||
|
||||
- **bats with stubs.** Copy CoreDesktop's `tests/test_helper.bash` pattern: a
|
||||
fresh `HOME` and a `bin/` of fake commands at the front of `PATH` for each
|
||||
test. Fake `pacman`, `yay`, `sudo`, `flatpak-spawn` and `make`, so no test
|
||||
changes the machine.
|
||||
- **Prove the tests catch bugs.** After the tests pass, break the code on
|
||||
purpose once per behavior (e.g. make `deployed restore` skip the
|
||||
confirmation, or make `check` skip untracked files) and confirm a test
|
||||
fails each time.
|
||||
- **Test the sibling flow in a throwaway home**, as described in
|
||||
`sibling-project-format.md`. `update.sh` calls `install_packages`, which
|
||||
uses `yay`, so stub it there too, or run it only where installing
|
||||
packages is fine.
|
||||
|
||||
Cases worth a test each:
|
||||
|
||||
- `install.sh` refuses the deployed copy as the working copy, and is safe to
|
||||
re-run.
|
||||
- `update.sh` doesn't prompt and doesn't re-copy an unchanged hook.
|
||||
- `doctor` reports a dirty deployed copy, and reports Flatpak use.
|
||||
- `check` lints an untracked script and fails on a shellcheck warning in it.
|
||||
- `try` gives the command a private `HOME` and `XDG_RUNTIME_DIR`, no
|
||||
`DISPLAY`, and the working copy at `~/.local/share/coredesktop`, and
|
||||
cleans up afterwards.
|
||||
- `deployed restore` asks first, restores tracked files, and leaves
|
||||
untracked ones alone.
|
||||
- `hooks install` won't overwrite another hook.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] The repo's README says to install with `sibling add <url>`
|
||||
- [ ] `install.sh` asks for the working copy, refuses the deployed copy,
|
||||
saves `~/.config/coredesktop-dev/config`, and is safe to re-run
|
||||
- [ ] `update.sh` installs shellcheck, bats and tea, keeps the
|
||||
`~/.local/bin/coredesktop-dev` symlink, and never prompts
|
||||
- [ ] `doctor`, `check`, `try`, `deployed` and `hooks` work and have `--help`
|
||||
- [ ] `check` lints untracked scripts in the working copy
|
||||
- [ ] `try` isolates `HOME`, `XDG_*` and `XDG_RUNTIME_DIR`, unsets the
|
||||
display and session variables, uses `dbus-run-session`, and warns that
|
||||
sudo, yay, pacman and systemctl still reach the real system
|
||||
- [ ] Nothing writes to the deployed copy except `deployed restore`, after
|
||||
confirmation
|
||||
- [ ] The project's own `make check` passes: shellcheck plus bats tests
|
||||
with stubs, and the tests catch deliberately broken code
|
||||
- [ ] The two suggested CoreDesktop changes are raised with CoreDesktop
|
||||
Reference in New Issue
Block a user