Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/gates.yml
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ jobs:
- name: every headline number has exactly one home
run: |
fail=0
for n in 3470 358 64/64 60/60 45/45 40/40 26/26; do
for n in 3470 358 65/65 61/61 45/45 40/40 26/26; do
homes=$(git grep -lF "$n" -- '*.md' ':(exclude)docs/history/*' |
sed 's|^docs/report/.*|docs/report/|' | sort -u | tr '\n' ' ')
count=$(printf '%s' "$homes" | wc -w)
Expand Down
44 changes: 43 additions & 1 deletion docs/report/03-defects-found-by-measurement.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
## 3. Six defects found by measurement
## 3. Seven defects found by measurement

None of these were in the problem statement. Each was found by running
something, and each is fixed.
Expand Down Expand Up @@ -144,6 +144,48 @@ comment saying it makes no `dlerror()` call, which was true and not enough.
**Fix:** re-run the load after the report, which puts the real message back.
One extra failed `dlopen`, only in a trace run. **E29**.

### 3.7 The shim defined the stack canary on the architectures whose loader owns it

Found outside this repository's own suite, which is the part worth reading:
chromium built as an anylinux AppImage crashed on aarch64, only there and
only with the preload present, and its GPU process died with SIGSEGV under
`CROSS_LIBC_DLOPEN=0` too (issue #37). The switch is irrelevant because the
damage is done at relocation time, before any of the loader's own code runs.

The generated shim emitted `__stack_chk_guard` as a function stub. The name
is a musl-only symbol against the x86-64 floor, which is what the generator
consulted for its type, so it became a `SHIM(void)` stub. But on every glibc
architecture without `THREAD_SET_STACK_GUARD` the dynamic loader itself
exports that name as a process-wide data object, the stack canary, and
writes it in `security_init()`. Measured against the floor libc packages
(`libc6-<target>-cross` 2.31, and trixie's for loongarch64):

| target | loader exports `__stack_chk_guard` | consequence |
|---|---|---|
| x86_64 | no, the canary is `%fs:0x28` | unaffected, and the shim's definition is load-bearing for musl guests |
| ppc64le | no, the canary is in the TCB | the same |
| aarch64 | yes, `ld-linux-aarch64.so.1` | the interposition |
| riscv64 | yes, `ld-linux-riscv64-lp64d.so.1` | the interposition |
| loongarch64 | yes, `ld-linux-loongarch-lp64d.so.1` | the interposition |

An unversioned definition in a preload wins the dynamic lookup for a
versioned reference, so every `__stack_chk_guard` GOT slot in the process,
`libc.so.6`'s own included, bound the shim's stub in read-only `.text`
instead of the object the loader initializes. Canary reads then disagreed
across the process and chromium's GPU process died before any cross-libc
code ran.

**Fix, in three layers.** The generator excludes the name on the
architectures whose loader exports it and emits it as zeroed data of the
real size where it is needed, because the merged kind table, not the x86
target's, owns the type. `scripts/verify-artifacts.sh` re-measures every
build against the target's own `libc.so.6` and loader and refuses any
exported name shared with them, beyond the audited interpositions. **E102**
is the suite's case, and on the aarch64 runner it is the case that failed
before this fix and passes after; on the x86-64 runner it passes both ways,
because that libc exports no such name, and it still guards that row against
a collision of the same shape.

---

---
Expand Down
11 changes: 11 additions & 0 deletions docs/report/05-design-b-generated-shim.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,17 @@ The generator now takes `--musl <inventory>` and folds musl's 46 floor-absent
exports into the same enumerable gap, rather than relying on a hand-maintained
list. That is what took the corpus from 243/247 to **247/247**.

⚠ **The musl inventory is measured on one architecture, and one of its
symbols is the loader's on others.** `__stack_chk_guard` is a musl-only name
against the x86-64 floor, and musl exports it as an 8-byte object, but on
aarch64, riscv64 and loongarch64 the dynamic loader exports that very name as
the process-wide stack canary. The generator excludes it per architecture
and emits it as zeroed data of the real size where the loader does not
provide it; the merged kind table, not the x86-64 target's, decides the type.
[Section 3.7](03-defects-found-by-measurement.md) has the measurement and the
crash it caused, E102 is the case, and `scripts/verify-artifacts.sh` refuses
any build whose preload exports a name the target's own libc family has.

### 5.4 The `___environ` rename

Applied, and confirmed firing on the real `libLLVM.so.20.1`:
Expand Down
10 changes: 6 additions & 4 deletions docs/report/08-test-results.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ wrong.

### Tier 1, the evidence table

`sh scripts/run-evidence.sh` reports **64/64 predictions held on x86-64** and
**60/60 on aarch64**. The x86-64 total was measured at the change that added
E101; the aarch64 runner runs the same table, so its total is the
x86-64 total minus the four skips below, and CI re-runs both on every push.
`sh scripts/run-evidence.sh` reports **65/65 predictions held on x86-64** and
**61/61 on aarch64**. The x86-64 total was measured at the change that added
E102; the aarch64 runner runs the same table, so its total is the x86-64
total minus the four skips below, and CI re-runs both on every push.
`experiments/run.ps1` drives the same three stage scripts for a machine with
PowerShell and no POSIX shell.

Expand All @@ -44,6 +44,7 @@ naming the capability it lacks rather than the difference being unexplained:
⭐ **E23's skip is the one worth reading.** It was reporting MATCH on the ARM
runner while asserting nothing, and skipping it with E22 is what stopped that.
64 minus 4 is 60, and no case is missing for a reason nobody wrote down.
65 minus 4 is 61 under the same accounting.

E1 through E13 measure the problem. E14 through E21 are one per fix from the first pass: the
ELF self-test, the generated-shim compile and behaviour, and five selector
Expand All @@ -63,6 +64,7 @@ objects small enough that the mechanism is the only thing being measured:
| E26 | the audit: no glibc may add a trap `version-compat.c` neither forwards nor declines |
| E28 | the report names the dependency that failed to open, instead of accusing the libc |
| E29 | and the caller still gets ld.so's message, not one of the report's own `dlsym` misses |
| E102 | the built preload exports no name this host's own libc family also exports, beyond the audited interpositions: the case that caught `__stack_chk_guard` being defined where the loader owns it (3.7), and on the aarch64 runner the case that failed before that fix |
| E54 | a plugin's undeclared import cannot see its loader's closure when that closure was loaded `RTLD_LOCAL` |
| E55 | its control: `RTLD_GLOBAL`, same two files, and it resolves |
| E56 | preload constructors run in REVERSE of the `.preload` order |
Expand Down
154 changes: 83 additions & 71 deletions docs/todo/PROGRESS.md
Original file line number Diff line number Diff line change
@@ -1,74 +1,83 @@

# PROGRESS

⭐ **Read this first, every session.** It is the only file that carries a work
order. [`INDEX.md`](INDEX.md) carries the list; this carries the order and the
baseline.
⭐ **Read this first, every session.** [`INDEX.md`](INDEX.md) carries the list; this carries the order and the baseline.

⚠ **Rewritten every session. It carries no history.** That is
[`../history/`](../history/README.md)'s job.
⚠ **Rewritten every session. It carries no history.** That is [`../history/`](../history/README.md)'s job.

---

## Where the work is right now

⭐ **[`v0.2.3` is published.](https://github.com/pkgforge-dev/cross-libc-dlopen/releases/tag/v0.2.3)**
66 assets: all six architectures (x86_64, aarch64, riscv64, ppc64, ppc64le,
loongarch64), loose objects plus `.tar`, `.zip` and `.sha256` for each, both
variants. Tagged on `34482c7`, built on the glibc 2.31 floor, body generated
from the manifests by `scripts/release-notes.sh`. The `release` workflow went
green end to end on the tag: run
[34025633281](https://github.com/pkgforge-dev/cross-libc-dlopen/actions/runs/34025633281).
[v0.2.5](https://github.com/pkgforge-dev/cross-libc-dlopen/releases/tag/v0.2.5) is
published. The aarch64, riscv64 and loongarch64 artefacts of every release so
far carry the defect below; there is no release yet without it.

| workflow | latest on `main` |
|---|---|
| `gates` | ✅ run [34025411777](https://github.com/pkgforge-dev/cross-libc-dlopen/actions/runs/34025411777) |
| `secret-sweep` | ✅ on `34482c7` |
| `release` | ✅ on `v0.2.3`, published |
| `gates` | ✅ |
| `secret-sweep` | ✅ |
| `release` | ✅ on `v0.2.5` |

**This branch changes what a release ships.** One variant, and no CET request
from any build. See the work below.
**This branch fixes issue #37 and needs a release once merged.** The next
consumer (Helium, Brave, Chrome AppImages via quick-sharun) is crashing on
aarch64 CI today.

---

## ⛔ The work order

### 1. Ship one variant, and stop asking for CET

⭐ **This is the session's work, on branch `drop-portable-variant`.** Two
decisions, both the operator's:

- **The release ships the default build only.** The strict build (reads only
`CROSS_LIBC_DLOPEN_ROOT`, never `APPDIR`) stays a build-time choice:
`cd src && make portable`, measured by E87 and E88. quick-sharun sets
`CROSS_LIBC_DLOPEN_ROOT` itself, so the strict assets had no consumer; the
`APPDIR` fallback stays, because upstream's own AppImage relies on it and
[`src/cld-env.h`](../../src/cld-env.h) has that argument in full.
- **No build asks for `-fcf-protection=full`.** Measured in
[`../report/09-the-second-boundary.md`](../report/09-the-second-boundary.md)
9.13: the flag adds six `endbr64` and cannot produce the IBT property note,
so it does no protective work here. It stays askable with
`make CET_CFLAGS=-fcf-protection=full`.

**The case that proves it.** E101 in `experiments/30-run-tests.sh` builds the
shim by the default recipe and again with the flag asked for, and requires the
default to come out strictly lighter:

- FAILS before, against the Makefile that still asked for the flag:
`predictions matched: 63, mismatched: 1` with
`E101 MISMATCH predicted=OK (exit 1, wanted OK)`; both arms tied.
- PASSES after: `E101 MATCH predicted=OK fewer endbr64 than the flag arm:
default 3472, asked for: 3478`, and the table is green end to end on
x86-64. The suite total moves 63 to 64 and every one-home record moved with
it: [`../report/`](../report/README.md) 01, 08, 09 and 10, the list in
[`gates.yml`](../../.github/workflows/gates.yml), and the same list in
`scripts/verify-gates.sh`. The aarch64 total and the four-skip list are in
report 08, which is that number's home.

Measured locally besides the suite: `build.sh --arch x86_64` and
`--arch x86_64 --portable` both exit 0 with the right manifest variant, the
default build's `gl-fwd.so` carries 3472 `endbr64`, and both directories
package and generate a body.
### 1. `__stack_chk_guard` must not be defined where the loader owns it

⭐ **This is the session's work, on branch `no-loader-owned-exports`.**

The generated shim emitted `__stack_chk_guard` as a function stub, because
the generator consulted the x86-64 target inventory for its type and the name
is musl-only there. On aarch64, riscv64 and loongarch64 the dynamic loader
exports that name as the process-wide stack canary, and an unversioned
preload definition wins every lookup in the process, `libc.so.6`'s own
included. Chromium's GPU process died with SIGSEGV under the preload with the
feature switch off; x86-64 and ppc64le were never affected, because their
loaders keep the canary in thread storage and never export the name.
[../report/03](../report/03-defects-found-by-measurement.md) 3.7 has the
whole chain with the measurement table.

Three layers, all on this branch:

- the generator excludes the name on the three architectures whose loader
exports it, emits it as zeroed data of the real size everywhere else, and
decides FUNC versus OBJECT from the merged kind table rather than the x86-64
target's. `src/forward-shim.c` and the manifest are regenerated.
- `scripts/verify-artifacts.sh` refuses any build whose preload exports a
name the target's own libc family also exports, beyond `dlopen` and
`version-compat.c`'s forwarders. `CLD_SYSROOT` names the target sysroot for
a verification outside a build; that is how its refusal was proven on this
x86-64 machine against the real bullseye arm64 cross libc.
- **E102** is the suite's case, run on both rows of the evidence table. Its
logic was proven against the real floor cross libc on this machine: the
released v0.2.5 aarch64 object makes it report `__stack_chk_guard` and the
fixed one reports nothing. The row on the ARM runner itself is the PR's CI
run, which is where E101's aarch64 number was measured too.

The suite total moved by one on both rows with E102. Report 08 owns the new
numbers, and every one-home record moved with it: `gates.yml`,
`scripts/verify-gates.sh` and that script's probe string.

Measured locally besides the suite: the fixed x86-64 build exports the same
140 names as before, with seven musl-object symbols re-typed from FUNC stubs
to data of their real sizes (`___environ`, `__optpos`, `__optreset`,
`__stack_chk_guard`, `_ns_flagdata`, `h_errno`, `optreset`); the planted
defect, the released v0.2.5 aarch64 object, makes both the gate and E102's
logic refuse naming the symbol; the gate's `defined_names` was caught reading
only the first member of the target list, which measured a whole pass against
`libc.so.6` alone and reported the loader's canary absent.

And measured on real silicon, in a consumer's CI rather than this
repository's: an unmodified Helium AppImage build whose only change is
compiling this branch and dropping the object into `AppDir/lib/sharun-preload`
passes its packaged `--test` on aarch64 and x86-64, with zero GPU-process
SIGSEGVs in both the `CROSS_LIBC_DLOPEN=0` control arm and the default arm,
where the released object had six in each. The runs are linked from the pull
request, because a fork's logs are not files this repository can cite.

### 2. What is still open

Expand All @@ -79,20 +88,23 @@ work order lives nowhere else.

## ⚠ What a new session should distrust

- **The aarch64 total on this branch is expected, not yet measured here.**
This machine has no ARM silicon: it is the x86-64 total minus four named
skips (E22, E23, E58 and now E101). The PR's CI run is the measurement.
- **`verify-gates.sh`'s one-home list claimed to be identical to
`gates.yml`'s and was not** (it still gated the previous suite totals after
the totals had moved on) until this branch aligned them. A comment that
asserts sameness is a claim; diff it.
- **`skip E76` and `skip E76b` at the foot of `experiments/30-run-tests.sh`
name a function that does not exist in that file.** On an x86-64 machine
with neither qemu nor an aarch64 cross compiler those lines would fail with
`skip: command not found` and leave E76 and E76b unscored rather than
SKIPPED by name. CI never reaches that path (it installs both), so it has
never been seen to fire. Named, not fixed: a change there belongs to its
own decision.
- **A guard that has never been seen to refuse is a guard nobody knows
works.** Three were found decorative or unarmed in earlier sessions and
every one of them looked fine.
- **The aarch64 and riscv64 and loongarch64 rows of `build` in CI are the
only builds that exercise the new gate against a real target sysroot.** A
green row means the artefact exports no name that target's libc family has;
a machine without the cross libc installed prints `name collisions
unverified` and that is a SKIP by name, not a pass.
- **`forward-shim.c` is one file compiled for every architecture, and its
arch-conditioned emission is the only per-architecture behaviour in it.**
The `#if` guard spells the three excluded architectures with the same
macros `src/cross-libc-dlopen.c` uses for its triplets. A new architecture
added to the build emits the symbol by default; the gate then refuses the
build if that architecture's loader exports the name, which is the intended
failure rather than a shipped crash.
- **The wide appimage suite has failed at its extraction step since late
August**, on both rows, for reasons unrelated to this branch. Its aarch64
row is the only real-driver run this repository has, and it is not running.
Fixing that is not this branch's work.
- **`__stack_chk_guard` is load-bearing on x86-64 and ppc64le**, where musl
guests bind it by name. Removing it entirely rather than per architecture
would turn a working musl load into an unresolved strong symbol at `dlopen`
time. E49 covers that path on x86-64 and still passes.
45 changes: 45 additions & 0 deletions experiments/30-run-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -516,6 +516,51 @@ else
echo " exists; without it the unstripped answer is already 0."
fi

# E102: the preload exports no name this host's own libc family also
# exports, beyond the audited interpositions. This is the case the
# aarch64 artefact shipped without: the generated shim defined
# __stack_chk_guard as a function stub, this architecture's loader
# exports the name as the process-wide stack canary, a preload
# definition wins every lookup in the process including the
# loader's own, and chromium's GPU process died with SIGSEGV under
# the preload with the feature switch off (issue #37). On the x86-64
# runner this libc exports no such name, because the canary lives
# at %fs:0x28 there, so the case passes both ways on that row and
# its teeth are the architectures whose loader owns the canary. It
# still guards every row against a collision of the same shape.
#
# ⚠ readelf over "$@", not "$1": the target is a LIST, libc plus its
# loader, and the first shape of this check read only the first member
# and found nothing. A case that cannot see its own defect is the
# failure mode this repository calls a silent pass.
#
# ⚠ The exemptions are read out of the source that defines them, so the
# assertion cannot drift from the forwarder set: dlopen is the project's
# own interposition, and version-compat.c's forwarders each forward to
# the default definition they displaced.
host_libc="/lib/$TRIPLET/libc.so.6"
[ -f "$host_libc" ] || host_libc="/usr/lib/$TRIPLET/libc.so.6"
defined_names() {
# The [<localentry>: 8] column PPC64 ELFv2 readelf prints between
# visibility and index is stripped for the same reason as in
# scripts/verify-artifacts.sh: unstripped, every name parses as 8].
readelf --dyn-syms -W "$@" 2>/dev/null |
sed 's/\[<localentry>:[^]]*\]//' |
awk '$7 != "UND" && ($5 == "GLOBAL" || $5 == "WEAK") &&
$6 == "DEFAULT" { n = $8; sub(/@.*/, "", n); print n }' |
sort -u
}
{ printf '%s\n' dlopen cross_libc_dlopen_init_now
sed -n 's/^VC_VISIBLE .*[ *]\([A-Za-z_][A-Za-z0-9_]*\)(.*/\1/p' \
/repo/src/version-compat.c
} | sort -u > /work/e102-exempt
{ defined_names /work/cross-libc-dlopen.so
defined_names "$host_libc" "$LIBDIR2/$LDSO"
} | sort | uniq -d | grep -vxF -f /work/e102-exempt \
> /work/e102-collisions || true
run E102 OK "no libc-family name reexported" sh -c \
'[ ! -s /work/e102-collisions ] && echo no libc-family name reexported'

# ---- the deprecated ANYLINUX_* spellings, and that they are gone ------
#
# Every control here used to have a second spelling, read as a deprecated
Expand Down
Loading