CI: Run every SDK's JavaScript builder tests, not only .NET's - #29
Conversation
The templates are consumed as a submodule, so a change here is only ever exercised by the SDKs that embed it. Four do, being .NET, Java, Node and Python, and until now the consumer job built one of them. That is how a change that was correct for .NET reached a package registry and broke the session storage cache for Python. Every builder is expected to render the same script from the same evidence, and one of them did not, which nothing here could see because nothing here built it. Each SDK now gets a job that checks it out, swaps in the revision under review and runs its JavaScript builder tests, with the same guard the .NET job already had that says so when the copy changed nothing and the run would otherwise report a pass about the revision the SDK pins rather than about this change. Java runs on 11 rather than 8 on purpose. The module drops JavaScriptBuilderTests at test compilation on 8, because Selenium's classes are built for 11, so a run on 8 reports a pass without having built the tests that drive the template at all. What this does not do. It runs each SDK's own tests, so it is only as good as those tests, and only .NET drives the template in a real browser today. It would not by itself have caught the change that prompted it, because Python has no test that loads a second page view. What it does is make a template change visible to every builder that embeds it, which is the precondition for any of them catching anything. The check that would settle it is rendering one fixed set of evidence through all four and comparing the output, since all four are meant to produce the same script. That needs a way to render from fixed evidence in each builder, which none of them expose today.
Six SDKs render these templates, not four. PHP and Rust carry a copy of the file rather than a submodule, so a change here never reaches them on its own and the copies drift. Today the PHP copy matches the create last merge without the fix to the inputs record, and the Rust copy is the revision before create last, so Rust serves a different script from the other five. Both now get a job that swaps the revision under review into the copy and runs the SDK's tests. Because a copy that differs is itself a difference in what that SDK serves, the job reports it as a warning rather than a notice.
The three red consumer jobs, and what each one wasAll seven checks now pass. None of the three failures was in this branch's rust JavaScript builder
pipeline-java JavaScript builder
pipeline-node JavaScript builderThe same check, "the script parameters leave out the session id and the Worth noting for whoever reviews thisThe java job's swap step reported the template it copies in as 763 lines EvidenceRun 35186467794 on 4378f81, all seven checks green, being pipeline-dotnet, |
Why
Every builder that embeds these templates is expected to render the same
script from the same evidence. Until now the consumer job built one of them,
.NET, so a change that suited .NET and not another builder could not be seen
here at all.
That is what happened. A change here was correct for .NET, reached a package
registry through another SDK, and broke the session storage cache for pages
using it: the record a page keeps of its request inputs carried a value that
differs on every page view, so it could never match, the cached response was
thrown away and a request went on every page view for the life of the tab.
Four SDKs embed the templates. All four now get a job.
FiftyOne.Pipeline.Elements/FiftyOne.Pipeline.JavaScriptBuilderElement/Templatespipeline.javascriptbuilder/src/main/resources/fiftyone/pipeline/javascriptbuilder/templatesfiftyone.pipeline.core/javascript-templatesfiftyone_pipeline_core/src/fiftyone_pipeline_core/js_templatesEach keeps the guard the .NET job already had, which reports when the copy
changed nothing so a green run cannot be mistaken for a check of this change
when it actually exercised the revision that SDK pins.
Java runs on 11 rather than 8 deliberately. The module drops
JavaScriptBuilderTestsat test compilation on 8, because Selenium's classesare built for 11:
So a run on 8 reports a pass without ever having built the tests that drive
the template. That is worth knowing separately from this change.
What this does not do
It runs each SDK's own tests, so it is only as good as those tests, and only
.NET drives the template in a real browser today. It would not by itself
have caught the change that prompted it, because the SDK that broke has no
test that loads a second page view. What it does is make a template change
visible to every builder that embeds it, which is the precondition for any of
them catching anything.
The check that would settle it is rendering one fixed set of evidence through
all four and comparing the output, since all four are meant to produce the
same script. That needs a way to render from fixed evidence in each builder,
which none of them exposes today, and it is a larger piece of work in four
repositories rather than one file here.
Verified
The workflow parses and declares the four jobs:
The test commands are the ones I ran by hand in each SDK today whilst fixing
the builders, so they are known to work rather than inferred from the
repositories' own CI.
The three new jobs have not run yet. This pull request is their first run,
which is the point at which the paths and commands are proved rather than
asserted.
Merge order
The three ready pull requests in this repository take this order:
29, 32, 31
This one goes first, so nothing has to merge before it.
This one changes only
.github/workflows/consumer-tests.yml, which neither of the others touches.No branch had to be changed for this order. Every pair was measured with
git merge-tree, the three were then merged intomainin that order as real merges with no conflict, andnpm testintestson the result reports207 checks, 0 failures. Smallest and most self-contained first. Checked on 17 September 2026.