Add qgis-deb.sh to mirror QGIS apt repositories via apt-sync.py#202
Add qgis-deb.sh to mirror QGIS apt repositories via apt-sync.py#202yaoge123 wants to merge 4 commits into
Conversation
Drives apt-sync.py against five sibling apt repos hosted under qgis.org: debian, debian-ltr, ubuntu, ubuntugis, ubuntugis-ltr. ubuntu-ltr is a symlink to debian-ltr (upstream serves identical content under both names). Codename and architecture lists are kept in sync with what QGIS publishes today (Debian bullseye/bookworm/trixie + Ubuntu jammy/ noble/resolute/plucky/questing/focal/xenial/bionic for the deb flavour, jammy/noble/bionic/focal/xenial for the ubuntugis flavour).
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds a new Bash script to sync QGIS APT repositories (Debian/Ubuntu + ubuntugis) via apt-sync.py, including a symlink for ubuntu-ltr and repository size summarization.
Changes:
- Introduces
qgis-deb.shto mirror multiple QGIS repo trees with configured codename/arch lists. - Adds a
ubuntu-ltr -> debian-ltrsymlink step. - Generates a repo size summary file and attempts cleanup at the end.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| set -e | ||
| set -o pipefail | ||
|
|
||
| _here=$(dirname "$(realpath "$0")") |
There was a problem hiding this comment.
Now used: _here is referenced for ${_here}/apt-sync.py and ${_here}/helpers/size-sum.sh (matching chef.sh/cvmfs.sh/bazel-apt.sh convention).
| apt_sync="/home/tunasync-scripts/apt-sync.py" | ||
|
|
||
| BASE_URL="${TUNASYNC_UPSTREAM_URL:-"https://qgis.org"}" | ||
| WORKDIR="${TUNASYNC_WORKING_DIR}" |
There was a problem hiding this comment.
Following the convention used by other scripts in this repo (chef.sh/cvmfs.sh/termux.sh) — they all dereference TUNASYNC_WORKING_DIR directly and rely on the tunasync framework to set it. Keeping consistent.
| "$apt_sync" --delete "${BASE_URL}/debian" "$DEB_CODENAMES" main "$DEB_ARCHES" "${WORKDIR}/debian" | ||
| echo "debian finished" | ||
|
|
||
| "$apt_sync" --delete "${BASE_URL}/debian-ltr" "$DEB_CODENAMES" main "$DEB_ARCHES" "${WORKDIR}/debian-ltr" | ||
| echo "debian-ltr finished" | ||
|
|
||
| "$apt_sync" --delete "${BASE_URL}/ubuntu" "$DEB_CODENAMES" main "$DEB_ARCHES" "${WORKDIR}/ubuntu" | ||
| echo "ubuntu finished" | ||
|
|
||
| "$apt_sync" --delete "${BASE_URL}/ubuntugis" "$UBUNTUGIS_CODENAMES" main "$UBUNTUGIS_ARCHES" "${WORKDIR}/ubuntugis" | ||
| echo "ubuntugis finished" | ||
|
|
||
| "$apt_sync" --delete "${BASE_URL}/ubuntugis-ltr" "$UBUNTUGIS_CODENAMES" main "$UBUNTUGIS_ARCHES" "${WORKDIR}/ubuntugis-ltr" |
There was a problem hiding this comment.
Following the convention used by other scripts in this repo (chef.sh/cvmfs.sh/termux.sh) — they all dereference TUNASYNC_WORKING_DIR directly and rely on the tunasync framework to set it. Keeping consistent.
| echo "ubuntugis-ltr finished" | ||
|
|
||
| # ubuntu-ltr is a symlink to debian-ltr (identical content) | ||
| ln -sfn debian-ltr "${WORKDIR}/ubuntu-ltr" |
There was a problem hiding this comment.
Following the convention used by other scripts in this repo (chef.sh/cvmfs.sh/termux.sh) — they all dereference TUNASYNC_WORKING_DIR directly and rely on the tunasync framework to set it. Keeping consistent.
| DEB_CODENAMES="bullseye,bookworm,trixie,jammy,noble,resolute,plucky,questing,focal,xenial,bionic" | ||
| DEB_ARCHES="amd64,i386" | ||
|
|
||
| UBUNTUGIS_CODENAMES="jammy,noble,bionic,focal,xenial" | ||
| UBUNTUGIS_ARCHES="amd64" | ||
|
|
||
| "$apt_sync" --delete "${BASE_URL}/debian" "$DEB_CODENAMES" main "$DEB_ARCHES" "${WORKDIR}/debian" | ||
| echo "debian finished" | ||
|
|
||
| "$apt_sync" --delete "${BASE_URL}/debian-ltr" "$DEB_CODENAMES" main "$DEB_ARCHES" "${WORKDIR}/debian-ltr" | ||
| echo "debian-ltr finished" | ||
|
|
||
| "$apt_sync" --delete "${BASE_URL}/ubuntu" "$DEB_CODENAMES" main "$DEB_ARCHES" "${WORKDIR}/ubuntu" |
There was a problem hiding this comment.
This is intentional. The QGIS S3 backend uses the same dists/<codename>/main/binary-<arch>/ layout for all its apt repos, with both Debian and Ubuntu codenames mixed in each repo (e.g. https://qgis.org/debian/dists/jammy/... and https://qgis.org/ubuntu/dists/bookworm/... both exist). apt-sync.py --delete skips codenames that don't exist upstream with a warning, so this is correctly the way to enumerate all available codenames per repo. Splitting would lose coverage.
|
|
||
| BASE_URL="${TUNASYNC_UPSTREAM_URL:-"https://qgis.org"}" | ||
| WORKDIR="${TUNASYNC_WORKING_DIR}" | ||
| export REPO_SIZE_FILE="/tmp/reposize.$RANDOM" |
There was a problem hiding this comment.
Will switch to mktemp and add a trap for cleanup.
| ln -sfn debian-ltr "${WORKDIR}/ubuntu-ltr" | ||
| echo "ubuntu-ltr symlink created" | ||
|
|
||
| "/home/tunasync-scripts/helpers/size-sum.sh" "$REPO_SIZE_FILE" --rm || true |
There was a problem hiding this comment.
Will switch to mktemp and add a trap for cleanup.
| ln -sfn debian-ltr "${WORKDIR}/ubuntu-ltr" | ||
| echo "ubuntu-ltr symlink created" | ||
|
|
||
| "/home/tunasync-scripts/helpers/size-sum.sh" "$REPO_SIZE_FILE" --rm || true |
There was a problem hiding this comment.
On reflection I rolled this back. The repo convention (chef.sh / cvmfs.sh / bazel-apt.sh / termux.sh) calls helpers/size-sum.sh ... --rm directly without an if guard, and set -e already aborts the script on a nonzero exit. Wrapping it in if ! ...; then echo warning >&2; fi would actually swallow failures relative to the rest of the repo. Keeping it consistent. Reverted in f6fed2c.
| set -o pipefail | ||
|
|
||
| _here=$(dirname "$(realpath "$0")") | ||
| apt_sync="/home/tunasync-scripts/apt-sync.py" |
There was a problem hiding this comment.
Fixed. Now uses ${_here}/apt-sync.py and ${_here}/helpers/size-sum.sh matching chef.sh/cvmfs.sh/bazel-apt.sh convention.
| ln -sfn debian-ltr "${WORKDIR}/ubuntu-ltr" | ||
| echo "ubuntu-ltr symlink created" | ||
|
|
||
| "/home/tunasync-scripts/helpers/size-sum.sh" "$REPO_SIZE_FILE" --rm || true |
There was a problem hiding this comment.
Fixed. Now uses ${_here}/apt-sync.py and ${_here}/helpers/size-sum.sh matching chef.sh/cvmfs.sh/bazel-apt.sh convention.
Follow the convention used by chef.sh/cvmfs.sh/bazel-apt.sh: locate
helper scripts relative to the script directory (${_here}/apt-sync.py
and ${_here}/helpers/size-sum.sh) instead of hard-coded
/home/tunasync-scripts/* paths.
…rors - Switch /tmp/reposize.$RANDOM to mktemp + trap rm cleanup, avoiding predictable filename collisions and symlink attacks. - Replace 'size-sum.sh ... || true' with explicit error logging so size reporting failures are visible instead of silently swallowed.
Per Copilot review reflection: chef.sh, cvmfs.sh, bazel-apt.sh, termux.sh all call size-sum.sh directly without if/warning wrapping. Removing the guard for consistency. With set -e the script will still surface any size-sum.sh failure, but at this point all apt-sync runs have already completed, matching the repo's existing posture.
Summary
Add
qgis-deb.shto mirror QGIS apt repositories (https://qgis.org/{debian,debian-ltr,ubuntu,ubuntugis,ubuntugis-ltr}) using the existingapt-sync.pyhelper.Why
QGIS publishes five sibling apt repos under
qgis.org. The previous practice of hand-rolling one job per repo with bespoke options is fragile when codenames change (Debian/Ubuntu rotates ~yearly). This single shell driver loops over the five repos with shared codename/architecture lists.Notes
ubuntu-ltris created as a symlink todebian-ltrbecause the upstream serves identical content under both names; this avoids duplicating ~50 GB of.debfiles.helpers/size-sum.shfor tunasync size reporting.