coredesktop/smoketest

Smoketest

End-to-end tests of install.sh and update on a real Arch + GNOME virtual machine. The bats tests in tests/ check the logic with fake commands; these check what fakes can't: pacman, yay, Flatpak, systemd, ufw, Snapper on btrfs, GNOME settings, and a reboot.

This folder holds what to test: the scenarios, and the scripts that describe a CoreDesktop machine. The VM itself (QEMU, images, the package cache) comes from the coredesktop-dev sibling project, which only contributors install:

sibling add <coredesktop-dev url>
coredesktop-dev smoketest list             # scenarios and their tier
coredesktop-dev smoketest run              # the quick (pr) scenarios
coredesktop-dev smoketest run --tier all   # everything, including a fresh install
coredesktop-dev smoketest run 40-failing-migration

It tests the working copy's HEAD commit, so commit first.

How it works

Each scenario runs on a throwaway copy of one of two VM images:

• base: Arch with GNOME, as archinstall's desktop/GNOME profile leaves it, with a user tester who logs in to GNOME automatically.
• installed: base after install.sh, at an older commit of CoreDesktop. Before the scenario starts, update brings it to the commit under test.

Inside the VM, a bare repo (/srv/smoketest/coredesktop.git) stands in for Gitea. git's insteadOf sends CoreDesktop's real URLs there, so install.sh and update run unchanged. The harness pushes the commit under test to it, and scenarios push further commits to simulate a new release: a new migration, a broken hook, a change to update itself.

After a scenario, fingerprint describes the machine as text (packages, Flatpaks, services, firewall, CoreDesktop's files in /etc, GNOME settings...). 10-fresh-install saves the fingerprint of a fresh install, and 30-upgrade-from-baseline checks that installing an older commit from baselines and then running update gives the same machine. If they differ, an install.sh change is probably missing its migration.

Files

File	What it is
scenarios/NN-name.sh	One scenario each (see below)
lib/scenario.sh	Helpers and assertions the scenarios use
fingerprint	Runs in the VM; prints the machine's fingerprint
health	Runs in the VM; checks the machine works, exits 1 if not
baselines	Older commits that 30-upgrade-from-baseline installs first

Writing a scenario

A scenario is a bash file that sets a few variables and defines scenario:

DESCRIPTION="update runs a new migration once"
IMAGE=installed      # base or installed
TIER=pr              # pr (quick, uses the installed image) or nightly

scenario() {
  upstream_commit "Add a migration" 'echo "touch ~/migrated" > migrations/9999999999.sh'
  cd_update
  assert_status 0
  assert_vm "the migration ran" 'test -f ~/migrated'
}

VARIANTS=(a b c) runs the scenario once per value, each in its own VM, with $VARIANT set. scenario runs with set -e, so any failing command or assertion fails it. skip "reason" skips it.

Helpers, from lib/scenario.sh:

Helper
cd_install	Run install.sh as wget ... | bash would, from the fake upstream's main
cd_update [VAR=value...]	Run update
run_in_vm NAME SCRIPT	Run SCRIPT in the VM; sets $OUTPUT and $STATUS, logs to NAME.log
upstream_set REF	Point the fake upstream's main at REF, a commit in this repo
upstream_commit MESSAGE SCRIPT	Run SCRIPT in a clone of the upstream (in the VM), commit, push
fingerprint FILE / health	Save the fingerprint / run the health checks
assert_status N, assert_output TEXT, refute_output TEXT	Check the last run_in_vm
assert_vm DESCRIPTION SCRIPT	SCRIPT must succeed in the VM
assert_same_fingerprint EXPECTED ACTUAL	The two must match; saves the diff

The contract with the driver

coredesktop-dev boots the VM, defines these, sources lib/scenario.sh, calls smoketest_prepare, then the scenario's scenario function:

Name
vm_exec SCRIPT	Run bash SCRIPT in the VM as tester, inside the GNOME session's environment, stdin from /dev/null. Returns its exit status.
vm_put LOCAL REMOTE	Copy a file into the VM
vm_reboot	Reboot and wait until tester is logged in to GNOME again
VM_GIT_SSH, VM_GIT_URL	GIT_SSH_COMMAND and the ssh:// prefix to push to a path in the VM
SMOKETEST_REF	The commit under test
SMOKETEST_RESULTS	This scenario's folder for logs and fingerprints
SMOKETEST_SHARED	A folder shared by all scenarios in the run
IMAGE_REF	For the installed image, the commit it was installed at

Anything that boots a VM can drive these scenarios, e.g. a CI runner with KVM.

Not covered

NVIDIA drivers, firmware updates (updates.d/90-firmware.sh), and anything that needs a real Core, a Tailscale login or a Claude login (configure). Check those by hand.