Skip to content

Cached CI apt downloads and bounded the install scripts' network calls - #506

Merged
fdesbiens merged 1 commit into
eclipse-threadx:devfrom
fdesbiens:ci/apt-package-cache
Oct 24, 2026
Merged

fdesbiens merged 1 commit into
eclipse-threadx:devfrom
fdesbiens:ci/apt-package-cache

Conversation

@fdesbiens

Copy link
Copy Markdown
Contributor

On 23 Oct 2026 the Ubuntu mirror served package files at 50 to 250 KB/s while indexes stayed fast, and CI install steps timed out. Every job downloads the same packages again, and no apt-get, pip or wget call in the install scripts had a timeout or a retry, so a stalled fetch ran until the step cap without saying which one.

_test-suite.yml now restores apt's .deb files from the Actions cache before the install step and saves them after it, as eclipse-threadx/threadx#823 does for the ThreadX template. The key is the runner image and a hash of install.sh and the optional installer, and the apt_cache input turns it off. The new scripts/ci_common.sh gives the install scripts retried, time-bounded apt-get, pip and download helpers. A failed index update only warns, because a third-party repository on the image can fail it; the install remains the gate. CMake comes from apt only when none is on PATH, since the image's own shadows it. install_mqtt.sh now runs under set -euo pipefail.

shellcheck and bash -n clean. Two dispatched ci-dev runs on a fork: the first saved the base, MQTT and secure caches (35, 54 and 53 MB). The mirror slowed to 20 to 50 KB/s for two netxduo shards, which stopped after three bounded attempts at 7m52s; a third shard recovered on its last attempt. The second run restored an exact match in all 23 suite jobs, fetched 0 B of 36.4 MB (and 0 B of 20.0 and 19.4 MB for the MQTT and secure installers), and passed.

On 23 Oct 2026 the Ubuntu mirror served package files at 50 to 250 KB/s while indexes
stayed fast, and CI install steps timed out. Every job downloads the same packages
again, and no apt-get, pip or wget call in the install scripts had a timeout or a
retry, so a stalled fetch ran until the step cap without saying which one.

_test-suite.yml now restores apt's .deb files from the Actions cache before the install
step and saves them after it, as eclipse-threadx/threadx#823 does for the ThreadX
template. The key is the runner image and a hash of install.sh and the optional
installer, and the apt_cache input turns it off. The new scripts/ci_common.sh gives the
install scripts retried, time-bounded apt-get, pip and download helpers. A failed index
update only warns, because a third-party repository on the image can fail it; the
install remains the gate. CMake comes from apt only when none is on PATH, since the
image's own shadows it. install_mqtt.sh now runs under set -euo pipefail.

shellcheck and bash -n clean. Two dispatched ci-dev runs on a fork: the first saved the
base, MQTT and secure caches (35, 54 and 53 MB). The mirror slowed to 20 to 50 KB/s for
two netxduo shards, which stopped after three bounded attempts at 7m52s; a third shard
recovered on its last attempt. The second run restored an exact match in all 23 suite
jobs, fetched 0 B of 36.4 MB (and 0 B of 20.0 and 19.4 MB for the MQTT and secure
installers), and passed.

Assisted-by: Claude Code (Opus 5.5) <noreply@anthropic.com>
@fdesbiens
fdesbiens merged commit cff5312 into eclipse-threadx:dev Oct 24, 2026
41 of 52 checks passed
@fdesbiens
fdesbiens deleted the ci/apt-package-cache branch October 24, 2026 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant