Federated_Computer/coredesktop
3
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

fix backup

Browse Source
This commit is contained in:
david
2026-10-26 16:21:27 -06:00
parent 8c8c59d6d5
commit 488b085713
7 changed files with 180 additions and 115 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+12 -7
View File
@@ -213,7 +213,7 @@ configure
It covers: It covers:
- **Your Federated Computer Core.** Give it your Superpanel's address (e.g. https://superpanel.example.com), your Core email, name and password, and it adds Superpanel to your applications, joins this computer to your Core's Tailscale network, adds your Nextcloud (calendar, contacts and files) and email to GNOME's Online Accounts, and mounts your Nextcloud files at /mnt/example.com ("example.com" in the Files sidebar), and points Element at your Core's Matrix server. For Nextcloud and the files it creates a Nextcloud app password, which you can revoke in Nextcloud under Settings > Security. - **Your Federated Computer Core.** Give it your Superpanel's address (e.g. https://superpanel.example.com), your Core email, name and password, and it adds Superpanel to your applications, joins this computer to your Core's Tailscale network, adds your Nextcloud (calendar, contacts and files) and email to GNOME's Online Accounts, and mounts your Nextcloud files at /mnt/example.com ("example.com" in the Files sidebar), and points Element at your Core's Matrix server. For Nextcloud and the files it creates a Nextcloud app password, which you can revoke in Nextcloud under Settings > Security.
- **Daily backups of your home folder** to your Nextcloud files, in /mnt/example.com/.backup-{machine name}, with borg. Set up your Core first. The backup is encrypted with a passphrase that `configure` shows you once: save it somewhere other than this computer, such as Bitwarden. It keeps 7 daily, 4 weekly and 6 monthly backups, and leaves out caches and things that can be reinstalled. To leave out more, list paths in ~/.config/coredesktop/backup-exclude, one per line. See "Restoring from a backup" below. - **Daily backups of your home folder** to your Nextcloud files, in a .backup-{machine name} folder, with restic. Set up your Core first. The backup is encrypted with a passphrase that `configure` shows you once: save it somewhere other than this computer, such as Bitwarden. It keeps 7 daily, 4 weekly and 6 monthly backups, and leaves out caches and things that can be reinstalled. To leave out more, list paths in ~/.config/coredesktop/backup-exclude, one per line. restic reaches Nextcloud directly through rclone with its own Nextcloud app password, so the backup doesn't depend on the /mnt mount. See "Restoring from a backup" below.
- Signing in to Claude Code, opencode and Hermes Agent and choosing their model providers. - Signing in to Claude Code, opencode and Hermes Agent and choosing their model providers.
To set up one thing, name it, e.g. `configure hermes`; `configure --list` shows the names. To set up one thing, name it, e.g. `configure hermes`; `configure --list` shows the names.
@@ -223,14 +223,19 @@ To set up one thing, name it, e.g. `configure hermes`; `configure --list` shows
To see the backups and get files back: To see the backups and get files back:
``` ```
export BORG_REPO=/mnt/example.com/.backup-$HOSTNAME source ~/.local/share/coredesktop/system/backup-env.sh
export BORG_PASSCOMMAND="secret-tool lookup coredesktop backup repo $BORG_REPO" restic snapshots
borg list restic restore latest --target ~/restored --include ~/Documents
mkdir ~/restored && cd ~/restored
borg extract ::{backup name} home/{you}/Documents
``` ```
On a new computer, set up your Core with `configure core` first. Then point `BORG_REPO` at the old computer's .backup-{name} folder and leave out `BORG_PASSCOMMAND`: borg asks for the passphrase you saved. `systemctl --user list-timers coredesktop-backup.timer` shows when the next backup runs, and `journalctl --user -u coredesktop-backup` shows how the last ones went. On a new computer, run `configure core` and `configure backup` first. Then name the old computer's folder, and restic asks for the passphrase you saved:
```
source ~/.local/share/coredesktop/system/backup-env.sh .backup-{old machine name}
restic snapshots
```
`systemctl --user list-timers coredesktop-backup.timer` shows when the next backup runs, and `journalctl --user -u coredesktop-backup` shows how the last ones went.
## Sandboxed AI agents ## Sandboxed AI agents
configure.d/10-core.sh
+2 -25
View File
@@ -6,6 +6,7 @@ DESCRIPTION="Superpanel, Tailscale, Nextcloud, email, files and Element"
# What you entered last time, so running this again only does what's missing # What you entered last time, so running this again only does what's missing
CORE_CONF="$HOME/.config/coredesktop/core.conf" CORE_CONF="$HOME/.config/coredesktop/core.conf"
GOA_ACCOUNT="$COREDESKTOP/configure.d/lib/goa-account" GOA_ACCOUNT="$COREDESKTOP/configure.d/lib/goa-account"
source "$COREDESKTOP/configure.d/lib/nextcloud.sh"
[ -f "$CORE_CONF" ] && source "$CORE_CONF" [ -f "$CORE_CONF" ] && source "$CORE_CONF"
# CORE_DOMAIN, CORE_EMAIL and CORE_NAME come from core.conf; the rest follow # CORE_DOMAIN, CORE_EMAIL and CORE_NAME come from core.conf; the rest follow
@@ -70,30 +71,6 @@ save_conf() {
} > "$CORE_CONF" } > "$CORE_CONF"
} }
# Basic auth header for curl, read from stdin (-H @-) so the password isn't
# in the process list
auth_header() {
printf 'Authorization: Basic %s\n' "$(printf '%s:%s' "$1" "$2" | base64 -w0)"
}
# Create a Nextcloud app password for the Online Accounts account and the
# files mount, so they don't keep your Core password. It shows in Nextcloud
# under Settings > Security as "CoreDesktop on <this computer>", where it can
# be revoked. Returns 2 if Nextcloud says the password is wrong.
nextcloud_app_password() {
local response code
response="$(auth_header "$CORE_USER" "$PASSWORD" | curl -s -w '\n%{http_code}' -H @- \
-H 'OCS-APIRequest: true' -A "CoreDesktop on $HOSTNAME" \
"$NEXTCLOUD/ocs/v2.php/core/getapppassword?format=json")"
code="${response##*$'\n'}"
case "$code" in
200) APP_PASSWORD="$(jq -r '.ocs.data.apppassword // empty' <<< "${response%$'\n'*}")"
[ -n "$APP_PASSWORD" ] ;;
401) return 2 ;;
*) return 1 ;;
esac
}
desktop_entry() { desktop_entry() {
local file="$1" name="$2" comment="$3" url="$4" icon="$5" local file="$1" name="$2" comment="$3" url="$4" icon="$5"
mkdir -p "$(dirname "$file")" mkdir -p "$(dirname "$file")"
@@ -332,7 +309,7 @@ run_configure() {
for tries in 1 2 3; do for tries in 1 2 3; do
read -rsp "Your Core password: " PASSWORD read -rsp "Your Core password: " PASSWORD
echo echo
nextcloud_app_password nextcloud_app_password "CoreDesktop on $HOSTNAME"
case $? in case $? in
0) break ;; 0) break ;;
2) echo "Nextcloud says that password is wrong."; PASSWORD="" ;; 2) echo "Nextcloud says that password is wrong."; PASSWORD="" ;;
configure.d/20-backup.sh
+73 -56
View File
@@ -6,86 +6,103 @@ DESCRIPTION="Back up your home folder daily to your Nextcloud files"
CORE_CONF="$HOME/.config/coredesktop/core.conf" CORE_CONF="$HOME/.config/coredesktop/core.conf"
BACKUP_CONF="$HOME/.config/coredesktop/backup.conf" BACKUP_CONF="$HOME/.config/coredesktop/backup.conf"
UNITS="$HOME/.config/systemd/user" UNITS="$HOME/.config/systemd/user"
source "$COREDESKTOP/configure.d/lib/nextcloud.sh"
[ -f "$CORE_CONF" ] && source "$CORE_CONF" [ -f "$CORE_CONF" ] && source "$CORE_CONF"
[ -f "$BACKUP_CONF" ] && source "$BACKUP_CONF"
is_installed() { is_installed() {
command -v borg > /dev/null && command -v secret-tool > /dev/null command -v restic > /dev/null && command -v rclone > /dev/null && command -v secret-tool > /dev/null
}
# The passphrase is in the keyring under the repository's path, so the
# backup can run without asking
passphrase() {
secret-tool lookup coredesktop backup repo "$1"
} }
is_configured() { is_configured() {
[ -n "$BORG_REPO" ] && [ -n "$(passphrase "$BORG_REPO")" ] \ source "$COREDESKTOP/system/backup-env.sh" > /dev/null || return 1
&& systemctl --user is-enabled -q coredesktop-backup.timer [ -n "$RESTIC_PASSWORD_COMMAND" ] && systemctl --user is-enabled -q coredesktop-backup.timer
} }
# system/backup uses borg 1.2 to 1.4; borg 2 changed its commands save_secret() {
borg_supported() { printf '%s' "$3" | secret-tool store --label "CoreDesktop backup $1 ($2)" \
local version coredesktop backup item "$1" url "$2"
version="$(borg --version | awk '{print $2}')"
[[ "$version" =~ ^1\.([2-9]|[1-9][0-9])\. ]] && return 0
echo "This needs borg 1.2 to 1.4, and borg $version is installed."
return 1
} }
save_passphrase() { # The backup gets its own app password, so revoking it stops only backups
printf '%s' "$2" | secret-tool store --label "CoreDesktop backup ($1)" coredesktop backup repo "$1" nextcloud_login() {
local tries
[ -n "$(backup_secret nextcloud "$WEBDAV")" ] \
&& rclone lsf --max-depth 1 ":webdav:" > /dev/null 2>&1 && return 0
for tries in 1 2 3; do
read -rsp "Your Core password: " PASSWORD
echo
nextcloud_app_password "CoreDesktop backup on $HOSTNAME"
case $? in
0) break ;;
2) echo "Nextcloud says that password is wrong." ;;
*) echo "Couldn't reach Nextcloud at $NEXTCLOUD."; return 1 ;;
esac
done
unset PASSWORD
[ -n "$APP_PASSWORD" ] || return 1
save_secret nextcloud "$WEBDAV" "$APP_PASSWORD" || return 1
unset APP_PASSWORD
RCLONE_WEBDAV_PASS="$(backup_secret nextcloud "$WEBDAV" | rclone obscure -)"
}
new_passphrase() {
local pass
pass="$(head -c 24 /dev/urandom | base64)"
echo "Your backup is encrypted with this passphrase:"
echo
echo " $pass"
echo
echo "Save it somewhere other than this computer, such as Bitwarden. Without it, the"
echo "backup can't be read if this computer is lost. It's kept in your keyring so the"
echo "daily backup can run."
until confirm "Have you saved it?"; do :; done
save_secret passphrase "$WEBDAV/$BACKUP_DIR" "$pass"
} }
run_configure() { run_configure() {
local mount="/mnt/$CORE_DOMAIN" repo pass local pass
borg_supported || return 1 if [ -z "$CORE_DOMAIN" ] || [ -z "$CORE_EMAIL" ]; then
if [ -z "$CORE_DOMAIN" ] || ! mountpoint -q "$mount"; then echo "Backups go to your Nextcloud files, so set up your Core first: configure core"
echo "Backups go to your Nextcloud files, so set those up first (configure core), and check they're mounted."
return 1 return 1
fi fi
repo="$mount/.backup-$HOSTNAME" CORE_USER="${CORE_EMAIL%@*}"
NEXTCLOUD="https://nextcloud.$CORE_DOMAIN"
if [ -f "$repo/config" ]; then WEBDAV="$NEXTCLOUD/remote.php/dav/files/$CORE_USER"
# Already there, e.g. after reinstalling: it needs the passphrase from then BACKUP_DIR=".backup-$HOSTNAME"
pass="$(passphrase "$repo")"
if [ -z "$pass" ] || ! BORG_PASSPHRASE="$pass" borg info "$repo" &> /dev/null; then
echo "There's already a backup at $repo."
read -rsp "Its passphrase: " pass
echo
BORG_PASSPHRASE="$pass" borg info "$repo" > /dev/null || { echo "borg couldn't open it with that passphrase."; return 1; }
save_passphrase "$repo" "$pass" || return 1
fi
echo "Using the backup at $repo."
else
pass="$(head -c 24 /dev/urandom | base64)"
echo "Your backup is encrypted with this passphrase:"
echo
echo " $pass"
echo
echo "Save it somewhere other than this computer, such as Bitwarden. Without it, the"
echo "backup can't be read if this computer is lost. It's kept in your keyring so the"
echo "daily backup can run."
until confirm "Have you saved it?"; do :; done
save_passphrase "$repo" "$pass" || return 1
echo "Creating the backup at $repo..."
BORG_PASSPHRASE="$pass" borg init --encryption=repokey-blake2 "$repo" || return 1
# davfs2 copies each file to a local cache before uploading it, and some
# servers refuse large uploads, so keep borg's files to 50 MB, not 500
BORG_PASSPHRASE="$pass" borg config "$repo" max_segment_size 52428800 || return 1
fi
mkdir -p "$(dirname "$BACKUP_CONF")" mkdir -p "$(dirname "$BACKUP_CONF")"
{ {
printf 'BORG_REPO=%q\n' "$repo" printf 'BACKUP_WEBDAV=%q\n' "$WEBDAV"
printf 'BACKUP_MOUNT=%q\n' "$mount" printf 'BACKUP_USER=%q\n' "$CORE_USER"
printf 'BACKUP_DIR=%q\n' "$BACKUP_DIR"
} > "$BACKUP_CONF" } > "$BACKUP_CONF"
source "$COREDESKTOP/system/backup-env.sh"
nextcloud_login || { echo "Couldn't sign in to Nextcloud."; return 1; }
if rclone lsf --files-only ":webdav:$BACKUP_DIR" 2> /dev/null | grep -qx config; then
# Already there, e.g. after reinstalling: it needs the passphrase from then
if [ -z "$RESTIC_PASSWORD_COMMAND" ] || ! restic cat config &> /dev/null; then
echo "There's already a backup in $BACKUP_DIR in your Nextcloud files."
read -rsp "Its passphrase: " pass
echo
RESTIC_PASSWORD_COMMAND= RESTIC_PASSWORD="$pass" restic cat config > /dev/null \
|| { echo "restic couldn't open it with that passphrase."; return 1; }
save_secret passphrase "$WEBDAV/$BACKUP_DIR" "$pass" || return 1
fi
echo "Using the backup in $BACKUP_DIR."
else
new_passphrase || return 1
source "$COREDESKTOP/system/backup-env.sh"
echo "Creating the backup in $BACKUP_DIR in your Nextcloud files..."
restic init || return 1
fi
mkdir -p "$UNITS" mkdir -p "$UNITS"
cp "$COREDESKTOP"/system/systemd/coredesktop-backup.{service,timer} "$UNITS/" || return 1 cp "$COREDESKTOP"/system/systemd/coredesktop-backup.{service,timer} "$UNITS/" || return 1
systemctl --user daemon-reload systemctl --user daemon-reload
systemctl --user enable --now coredesktop-backup.timer || return 1 systemctl --user enable --now coredesktop-backup.timer || return 1
echo "Your home folder will be backed up daily to $repo." echo "Your home folder will be backed up daily to $BACKUP_DIR in your Nextcloud files."
if confirm_yes "Start the first backup now? It runs in the background and can take hours."; then if confirm_yes "Start the first backup now? It runs in the background and can take hours."; then
systemctl --user start --no-block coredesktop-backup.service systemctl --user start --no-block coredesktop-backup.service
configure.d/lib/nextcloud.sh
+27
View File
@@ -0,0 +1,27 @@
#!/bin/bash
#
# Purpose : Nextcloud app passwords for configure.d items. Source this file.
# Basic auth header for curl, read from stdin (-H @-) so the password isn't
# in the process list
auth_header() {
printf 'Authorization: Basic %s\n' "$(printf '%s:%s' "$1" "$2" | base64 -w0)"
}
# Create a Nextcloud app password, so what CoreDesktop sets up doesn't keep
# your Core password. It shows in Nextcloud under Settings > Security with the
# name given, where it can be revoked. Uses CORE_USER, PASSWORD and NEXTCLOUD;
# sets APP_PASSWORD. Returns 2 if Nextcloud says the password is wrong.
nextcloud_app_password() {
local name="$1" response code
response="$(auth_header "$CORE_USER" "$PASSWORD" | curl -s -w '\n%{http_code}' -H @- \
-H 'OCS-APIRequest: true' -A "$name" \
"$NEXTCLOUD/ocs/v2.php/core/getapppassword?format=json")"
code="${response##*$'\n'}"
case "$code" in
200) APP_PASSWORD="$(jq -r '.ocs.data.apppassword // empty' <<< "${response%$'\n'*}")"
[ -n "$APP_PASSWORD" ] ;;
401) return 2 ;;
*) return 1 ;;
esac
}
packages.conf
+2
View File
@@ -42,6 +42,8 @@ SYSTEM_UTILS=(
nmap nmap
nss-mdns nss-mdns
openssh openssh
rclone
restic
ripgrep ripgrep
screen screen
sniffnet sniffnet
system/backup
+20 -27
View File
@@ -1,18 +1,19 @@
#!/bin/bash #!/bin/bash
# #
# Purpose : Back up your home folder with borg to your Nextcloud files, in # Purpose : Back up your home folder with restic to your Nextcloud files, in
# /mnt/<domain>/.backup-<hostname>. Run daily by # .backup-<hostname>. Run daily by coredesktop-backup.timer;
# coredesktop-backup.timer; `configure backup` sets it up. # `configure backup` sets it up.
# #
# Keeps 7 daily, 4 weekly and 6 monthly backups. To leave out more, list # Keeps 7 daily, 4 weekly and 6 monthly backups. To leave out more, list
# paths or borg patterns in ~/.config/coredesktop/backup-exclude, one per line. # paths or patterns in ~/.config/coredesktop/backup-exclude, one per line.
# Wrapped in { ...; exit; } so bash reads the whole script up front; update # Wrapped in { ...; exit; } so bash reads the whole script up front; update
# replaces this file (git pull) while a backup may be running. # replaces this file (git pull) while a backup may be running.
{ {
BACKUP_CONF="$HOME/.config/coredesktop/backup.conf" COREDESKTOP="$HOME/.local/share/coredesktop"
USER_EXCLUDE="$HOME/.config/coredesktop/backup-exclude" USER_EXCLUDE="$HOME/.config/coredesktop/backup-exclude"
LOG_HINT="See: journalctl --user -u coredesktop-backup"
# Tell the desktop when a backup fails, as no one watches the timer # Tell the desktop when a backup fails, as no one watches the timer
fail() { fail() {
@@ -21,22 +22,17 @@ fail() {
exit 1 exit 1
} }
[ -f "$BACKUP_CONF" ] || fail "Backups aren't set up. Run: configure backup" source "$COREDESKTOP/system/backup-env.sh" || exit 1
source "$BACKUP_CONF" [ -n "$RESTIC_REPOSITORY" ] || fail "Backups aren't set up. Run: configure backup"
[ -n "$RESTIC_PASSWORD_COMMAND" ] || fail "The backup's passphrase isn't in your keyring. Run: configure backup"
# The files mount comes up at login, perhaps before the network; give it a # Run at login, perhaps before the network is up; give it a few minutes
# few minutes rather than writing into the empty folder underneath
for try in {1..10}; do for try in {1..10}; do
mountpoint -q "$BACKUP_MOUNT" && break curl -sf --max-time 10 -o /dev/null "${BACKUP_WEBDAV%%/remote.php/*}/status.php" && break
mount "$BACKUP_MOUNT" 2> /dev/null && break
sleep 30 sleep 30
done done
mountpoint -q "$BACKUP_MOUNT" || fail "Your Nextcloud files aren't mounted at $BACKUP_MOUNT."
export BORG_REPO # Rebuilt by restic, or by reinstalling, so not worth the space
export BORG_PASSCOMMAND="secret-tool lookup coredesktop backup repo $BORG_REPO"
# Rebuilt by borg, or by reinstalling, so not worth the space
excludes=( excludes=(
"$HOME/.cache" "$HOME/.cache"
"$HOME/.davfs2/cache" "$HOME/.davfs2/cache"
@@ -52,23 +48,20 @@ excludes=(
) )
exclude_args=(--exclude-caches) exclude_args=(--exclude-caches)
for path in "${excludes[@]}"; do for path in "${excludes[@]}"; do
exclude_args+=(--exclude "sh:$path") exclude_args+=(--exclude "$path")
done done
[ -f "$USER_EXCLUDE" ] && exclude_args+=(--exclude-from "$USER_EXCLUDE") [ -f "$USER_EXCLUDE" ] && exclude_args+=(--exclude-file "$USER_EXCLUDE")
echo "Backing up $HOME to $BORG_REPO..." echo "Backing up $HOME to $BACKUP_DIR in your Nextcloud files..."
# --one-file-system leaves out other mounts and btrfs subvolumes inside home # --one-file-system leaves out other mounts and btrfs subvolumes inside home
borg create --one-file-system --compression zstd --stats \ restic backup --one-file-system --host "$HOSTNAME" "${exclude_args[@]}" "$HOME"
"${exclude_args[@]}" "::{hostname}-{now:%Y-%m-%dT%H:%M}" "$HOME"
status=$? status=$?
# 1 is a warning, e.g. a file changed while it was read; the backup still exists # 3 means some files couldn't be read; the rest were backed up
(( status <= 1 )) || fail "borg couldn't back up $HOME (exit $status). See: journalctl --user -u coredesktop-backup" (( status == 0 || status == 3 )) || fail "restic couldn't back up $HOME (exit $status). $LOG_HINT"
echo "Removing old backups..." echo "Removing old backups..."
borg prune --glob-archives '{hostname}-*' \ restic forget --host "$HOSTNAME" --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prune \
--keep-daily 7 --keep-weekly 4 --keep-monthly 6 \ || fail "restic couldn't remove old backups. $LOG_HINT"
|| fail "borg couldn't remove old backups. See: journalctl --user -u coredesktop-backup"
borg compact || fail "borg couldn't free space from old backups. See: journalctl --user -u coredesktop-backup"
echo "Backup finished." echo "Backup finished."
exit 0 exit 0
system/backup-env.sh
+44
View File
@@ -0,0 +1,44 @@
#!/bin/bash
#
# Purpose : Point restic at your backup in your Nextcloud files. Sourced by
# system/backup and configure.d/20-backup.sh, and by you to look at
# or restore from a backup:
#
# source ~/.local/share/coredesktop/system/backup-env.sh
# restic snapshots
#
# To use another computer's backup, name its folder:
#
# source ~/.local/share/coredesktop/system/backup-env.sh .backup-<name>
#
# restic reaches Nextcloud over WebDAV through rclone, not the davfs2 mount,
# which can't safely replace files. The Nextcloud app password and the
# backup's passphrase are kept in your keyring.
BACKUP_CONF="$HOME/.config/coredesktop/backup.conf"
# A secret from the keyring: nextcloud (app password) or passphrase
backup_secret() {
secret-tool lookup coredesktop backup item "$1" url "$2"
}
if [ -f "$BACKUP_CONF" ]; then
source "$BACKUP_CONF"
BACKUP_DIR="${1:-$BACKUP_DIR}"
export RCLONE_WEBDAV_URL="$BACKUP_WEBDAV"
export RCLONE_WEBDAV_VENDOR=nextcloud
export RCLONE_WEBDAV_USER="$BACKUP_USER"
RCLONE_WEBDAV_PASS="$(backup_secret nextcloud "$BACKUP_WEBDAV" | rclone obscure -)"
export RCLONE_WEBDAV_PASS
export RESTIC_REPOSITORY="rclone::webdav:$BACKUP_DIR"
# Without a passphrase in the keyring, e.g. for another computer's backup,
# restic asks for it
unset RESTIC_PASSWORD_COMMAND
if [ -n "$(backup_secret passphrase "$BACKUP_WEBDAV/$BACKUP_DIR")" ]; then
export RESTIC_PASSWORD_COMMAND="secret-tool lookup coredesktop backup item passphrase url $BACKUP_WEBDAV/$BACKUP_DIR"
fi
else
echo "Backups aren't set up. Run: configure backup" >&2
fi