Support R 4.x, modernise the build, and bring the console up to current libraries

[SamLovick]

Summary

Makes BERT work with current R (tested with 4.5.2) without dropping older R, moves the native build onto a current toolchain, and brings the console up to current versions of Electron, TypeScript, Monaco, xterm, rxjs and protobuf. Addresses #194, #207 and #124.

R controller

• The controller refused any R other than 3.5. It now checks the version at startup: a floor of 3.5, a warning above the tested major series, no ceiling. One ControlR.exe, built against R 4.5.2, starts under R 3.5.0, 4.2.2 and 4.5.2.
• String encoding is explicit on every R version: strings from Excel and the console are marked UTF-8 when handed to R, strings taken from R are translated to UTF-8, and console text is converted to and from the Windows code page only when R's native encoding is not UTF-8.
• ControlR.exe carries the UTF-8 code-page manifest that R's own executables use. Without it, R 4.2+ embedded in another process runs on the system code page, and strings outside it cannot exist in the session.
• Console messages are built from the length R supplies rather than a terminating NUL; the old code could send stale bytes after the text.
• R_LEGACY_RCOMPLEX is defined because MSVC cannot compile the _Complex member R 4.3 added to Rcomplex.

Build

• v145 toolset, Windows SDK from the installed VS, ARM64EC configurations alongside x64.
• protobuf 5.29 via vcpkg in manifest mode; generated code regenerated, .proto unchanged so the wire format is unaffected.
• The R installation is an overridable MSBuild property (BertRHome); import libraries regenerated from R 4.5.2 with their .def files checked in, and R's exported variables marked DATA so the ARM64X import libraries link.
• controlr.cc is stored as UTF-8 (it was UTF-16, which git shows as binary).

Console

• Electron 1.8 → 44, TypeScript 2.7 → 7, Monaco 0.10 → 0.56, xterm 3 → 6, rxjs 5 → 7, google-protobuf 3 → 4, and the smaller libraries. The architecture is unchanged (Node in the renderer as an explicit opt-in, @electron/remote for the few main-process calls; navigator.clipboard replaces the removed renderer clipboard module).
• The xterm port no longer patches xterm's prototype; the two remaining reaches into its internals (cell size, scrollback trim event) are isolated in one file.
• The release update check is skipped for tagged development builds, and the add-in's version string carries a branch tag and build time so a local build is distinguishable in the console.
• npm with a committed lockfile instead of yarn.

Docs

docs/BUILDING.md describes the build, the R version policy and what has been exercised; docs/MODERNISATION.md records what changed in R and the toolchain since 2018 and the remaining work (installer still bundles R 3.5, Julia 0.x controllers, strict TypeScript, Monaco as ES modules, file paths from the add-in in the code page).

Verification

• Full solution builds Release x64 (ControlR.exe, BERT64.xll, BERTRibbon2x64.dll); ControlR also builds Release ARM64EC.
• Used with Excel and R 4.5.2: functions from cells, the console shell and editor, copy/paste, and non-ASCII strings ("café ñ 日本" round-trips through both a cell and the shell; nchar("café") is 4).
• R 4.2.2 and 3.5.0 have had the controller's startup check only.

Generated with the assistance of Claude Code

[SamLovick]

Since opening this I found a bug in the add-in that predates everything in this branch. Flagging it separately because it is independent of the R 4.x work and affects anyone calling the Excel API from a spreadsheet function — it is worth taking on its own even if the rest of the PR is not of interest.

Any callback into Excel from a cell function deadlocks

Excel shows "Calculating" and stops responding, the controller sits idle, and neither recovers. That takes out R graphics completely, since BERT.graphics.device(cell=TRUE) calls xlfCaller and Device::UpdateSize asks Excel for the shape size — so the worked example on bert-toolkit.com/r-graphics-in-excel hangs as soon as it calculates.

Smallest reproduction I could get to, in the functions directory:

callback.test <- function() {
  ref <- BERT$.Excel(89)   # xlfCaller
  "ok"
}

=R.callback.test() in a cell hangs Excel. A cell function that does not call back is fine, which is why ordinary R functions never showed the problem.

Cause

LanguageService::Call sends the call, then does Sleep(1000) before entering its wait loop, and only resets callback_info_.default_signaled_event_ inside that loop. BERT::HandleCallback uses that same event to choose a path: signaled means "shell function, switch context through Application.Run("BERT.ContextSwitch")", unsignaled means "spreadsheet function, the main thread is blocked in the call, hand it over".

R answers in about a millisecond. So for the whole of that second the flag still reads "shell", and the callback is pushed through COM into an Excel that is mid-calculation and can never service it. Both processes then wait forever.

Trace from an instrumented build, with the timestamps that give it away:

09:26:21.040  CB: message from R (36 bytes), wait=1
09:26:21.040  HandleCallback: entry (R)
09:26:21.040  HandleCallback: shell function branch (COM context switch)   <- wrong path
09:26:22.043  CALL: waiting for a reply from R                            <- a second later

Fix

Commit 4fdb756: record Excel's main thread in BERT::Init (new CallbackInfo::OnMainThread()); when a call is made on that thread, reset the event before the call goes out rather than after; drop the sleep; and make the in-loop reset conditional so console calls keep the COM path that exists for them. Which branch a callback takes then follows from the thread that made the call rather than from timing.

After the fix the same trace reads:

CB: message from R (36 bytes), wait=1
HandleCallback: spreadsheet branch; waking the main thread
CALL: servicing callback on the main thread
CB: response written to R (result 1, 18 bytes)

Verified with the documented graphics example plus cell-linked and named devices, rotated text, transparency, rasters and UTF-8 labels; console callbacks still round-trip through the COM path.

Two others worth picking up separately

• b62bfac — startup.R, .resolveObject compares a whole vector: inside while (length(n) > 1), n == "" is length 2. R warns from 4.2 and refuses from 4.3, so asking for a function's arguments prints 'length = 2' in coercion to 'logical(1)' into the shell. Comparing n[1] gives identical results on 3.5.0, 4.2.2 and 4.5.2.
• DebugOut now compiles into a release build when BERT_TRACE is defined, with named trace points across the callback handshake. That is how the above was found, and it turns a field hang into a readable log.

Binaries for anyone who wants to check without building: https://github.com/SamLovick/Basic-Excel-R-Toolkit/releases/tag/v2.4.3-r7

Generated with the assistance of Claude Code.