RTL8821AU + RTL8814AU 5GHz TX/RX: phydm IQK calibration port required #59

Problem

Devourer's 5GHz code path silently fails on multiple chip + direction combos. The 2026-05-29 + 2026-05-30 investigation traced the root cause to missing phydm IQK / DPK calibration. At 2.4GHz, the chip's reset-state calibration is tolerant enough for TX/RX; at 5GHz, the PA / LO mismatch tolerance is tighter and IQK is effectively mandatory.

Affected matrix cells (5GHz, ch100 / ch36)

The same cells at 2.4GHz (ch6) all pass with hundreds-to-thousands of hits. Cross-reference: PR #58 added the band asymmetry documentation; PR bodies #34 / #42 / #49 record the 5GHz status as "still broken" because their matrix runs were captured at --channel 100.

The persistent RTL8814AU TX gate is a separate, deeper issue (broken on both bands) tracked elsewhere and NOT addressed by this port. See kaeru ref RTL8814AU libusb-userspace bulk-OUT does not produce on-air TX.

Root cause analysis

Verified 2026-05-30 against upstream aircrack-ng/rtl8812au-5.6.4.2_35491.20191025 (DKMS source on the devourer-testrig VM at /usr/src/8812au-5.6.4.2_35491.20191025/).

1. Devourer's phy_SwChnl8812 matches upstream's production-chip code byte-for-byte. No missing channel-set helpers for RTL8821U normal-chip. Verified by line-by-line comparison of src/RadioManagementModule.cpp:834-887 vs hal/rtl8812a/rtl8812a_phycfg.c:1773-1864.

2. The kernel-side RF[0x42] writes I captured during channel-set (ch6: data=0x37cf8; ch100: data=0x378f0; 4 writes each) come from inside the IQK code, NOT from phy_SwChnl8812. Specifically hal/phydm/halrf/rtl8812a/halrf_8812a_win.c:1989, 2678 (entry: do_iqk_8812a).

3. Smoke-tested a hardcoded phy_set_rf_reg(RF_PATH_A, 0x42, 0xfffff, 0x378f0) at end of phy_SwChnl8812 — devourer 8821AU TX @ ch100 still produced 0 hits on-air. So the gate isn't one register; it's the full IQK sequence.

4. Kernel's BB+RF write footprint per channel-set at ch100 (203 writes total): 0x0994 ×56, 0x08fc/0x08f8/0x09a4/0x0a2c/0x0b58/0x08a4 ×18 each, 0x0830/0x0834/0x08b0 ×9 each. This is the classic phydm IQK loop signature.

5. Devourer has no IQK code at all. grep -rn IQK src/ returns only the _needIQK flag, no implementation.

Pickup checklist for the next person

1. Read the kaeru evidence first

The investigation captured detailed evidence and intermediate hypothesis tests in two episodes (search via mcp__kaeru with initiative: devourer):

• 8821au-5ghz-tx-gate-investigation-started-2026-05-30 — initial wire-diff, smoke tests, dead-ends ruled out
• 8821au-5ghz-tx-gate-iqk-port-required-2026-05-30 — final conclusion + port scope

Plus the related cite devourer 5GHz vs 2.4GHz cell asymmetry — matrix --channel 100 default hides working 2.4G state.

2. Reproduce the gate

sudo python3 tests/regress.py \
    --tx-pid 0120 --rx-pid 8812 \
    --channel 100 --duration 12 \
    --vm-name devourer-testrig \
    --vm-ssh <user>@<VM-IP> \
    --keep-logs --no-baseline-abort
# Expect: dk cell = 0 ✗ (devourer 8821 TX → kernel 8812 RX, broken at 5G)
# Same test at --channel 6 = 5544 ✓ (works at 2.4G)

You need a devourer-testrig libvirt VM with aircrack-ng/88XXau loaded (tests/setup_vm.sh provisions it).

3. Pull the port-target source

In the VM, the relevant upstream source is at:

• /usr/src/8812au-5.6.4.2_35491.20191025/hal/phydm/halrf/rtl8821a/halrf_iqk_8821a_ce.c — 773 LOC, 8821a-specific IQK
• /usr/src/8812au-5.6.4.2_35491.20191025/hal/phydm/halrf/rtl8812a/halrf_8812a_ce.c — 1975 LOC, shared 8812a/8821a radio config + IQK entry point do_iqk_8812a
• Plus phydm framework deps it imports: dm_struct, odm_set_rf_reg, BB table reads, phydm_precomp.h

Same files are pullable from https://github.com/aircrack-ng/rtl8812au at the matching upstream tag.

4. Port strategy

Suggested approach (smallest-blast-radius first):

1. Port the minimum IQK entry point first — just enough to call do_iqk_8812a after channel-set on 8821AU. Stub everything in dm_struct that isn't strictly required. Run regress.py against ch100; if frame counts come back, the port path is right and you can iterate on completeness.
2. The CE (Customer Edition) sources are the right port-target (vs *_win.c / *_ap.c which are Windows / AP-mode variants).
3. Reuse devourer's existing PhyTableLoader (src/PhyTableLoader.cpp) for any phydm-table reads the IQK code does.
4. Reuse phy_set_rf_reg / phy_query_rf_reg (already in RadioManagementModule) instead of porting odm_set_rf_reg directly — just rename calls.

Watch out for:

• phydm framework calls like odm_get_bb_reg that may require porting the phy_query_bb_reg path with the right bit-mask semantics.
• Time-dependent IQK steps that use kernel msleep — devourer is single-threaded; use std::this_thread::sleep_for.
• The IQK code may write to chip-cut-specific phydm tables (devourer ports those in hal/phydm/).

5. Validation

End-to-end smoke test per port iteration:

# Full matrix at both bands — should pass on all non-8814-TX cells:
sudo python3 tests/regress.py --full-matrix --channel 6   --vm-name devourer-testrig --vm-ssh <user>@<VM-IP>
sudo python3 tests/regress.py --full-matrix --channel 100 --vm-name devourer-testrig --vm-ssh <user>@<VM-IP>

Wire-shape verification (optional, useful for confirming IQK writes hit the chip in the right order):

sudo modprobe usbmon
sudo tshark -i usbmonN -s 0 -w /tmp/dev-8821-ch100.pcapng    # while WiFiDriverTxDemo runs
python3 tools/usbmon_pcap_diff.py /tmp/kernel-8821-ch100.pcap /tmp/dev-8821-ch100.pcapng \
    --devnum-a 2 --devnum-b 3
# Expect: 0x0994 / 0x08xx write counts to converge with kernel after IQK port

The kernel reference captures from 2026-05-30 may still be on the test rig at /tmp/kernel-8821-ch{6,100}.pcap — if not, re-capture via sudo tshark -i usbmon1 while the kernel driver is loaded and a channel-set is performed.

6. Pre-existing tooling that helps

• tools/usbmon_pcap_diff.py (RTL8814AU: pcapng-fidelity USB diff + replay tooling #55) — binary-fidelity USB wire diff, surfaces per-register count + value divergences.
• tools/pcapng_to_urbscript.py + tools/usbmon_replay.c (RTL8814AU: pcapng-fidelity USB diff + replay tooling #55) — verbatim URB replay; not directly relevant for IQK porting but useful for proving "these N writes match the kernel byte-for-byte".

7. Scope estimate

• Minimum viable port: 1-2 weeks for someone familiar with Realtek phydm
• Full faithful port with all chip-cut branches: longer
• Same IQK port may also resolve 8814AU 5GHz RX and 8821AU 5GHz RX (both stuck on the same band-specific calibration issue)

Out of scope

• 8814AU TX gate: separate issue, broken on both bands, deeper root cause not yet identified. Don't conflate.
• Tooling improvements: PR tests: default matrix --channel to 6 (2.4GHz); surface band asymmetry #58 already changed the matrix default to ch6 to surface the 2.4G working state; further test infra work isn't blocked on this port.

References

• PR RTL8814AU: post-fwdl init block was mis-gated on CHIP_8821 #57 — RTL8814AU post-fwdl init correctness (related: 8814 path, not 5G specifically)
• PR tests: default matrix --channel to 6 (2.4GHz); surface band asymmetry #58 — tests/regress.py default to ch6 (documents the band asymmetry this issue investigates)
• PR RTL8814AU: pcapng-fidelity USB diff + replay tooling #55 — diff/replay tooling (the analysis harness used here)
• PR tests: --full-matrix runs N-adapter cross-driver interop tables #34 — original full-matrix infrastructure (baseline that recorded "still broken" for these cells)
• PR RTL8821AU: 8821-specific init flow (replaces trace-derived parity) #42 — 8821AU init flow proper (the post-fwdl correctness fix; doesn't address 5G calibration)
• kaeru episode 8821au-5ghz-tx-gate-iqk-port-required-2026-05-30 — final verdict
• kaeru episode 8821au-5ghz-tx-gate-investigation-started-2026-05-30 — initial wire-diff
• kaeru cite devourer 5GHz vs 2.4GHz cell asymmetry — matrix --channel 100 default hides working 2.4G state — the umbrella observation

🤖 Generated with Claude Code