Store snippet results only under names the snippet writes - #31
Merged
Merged
Conversation
A snippet that builds its result name as it runs, such as the location snippet's "51D_Pos_" + key + "=", left an empty value in session storage under the text of the join, fod_data_51D_Pos_" + key + ", and every request from the tab then sent 51D_Pos_%22+%2B+key+%2B+%22 to the server. The rewrite of the store itself was already right, because the join is copied inside the session storage key, but the empty result written before the snippet runs took the name group literally. A fixed name joined from two strings, as in "51D_Bandwidth" + "=", was stored the same wrong way. The empty result now removes joins between quoted strings, and stores nothing for a name that is joined to a variable, because that name is only known as the snippet runs. A value an earlier script stored under a name holding a quote is no longer sent. The pattern now reads a quoted name as a fixed start joined only to quoted text and plain variable names, and leaves a space before the equals sign out of the name, so any other form is left to write its cookie rather than being stored under a name the pattern did not understand. The template literal form used by the high entropy values snippet was already read correctly and is now covered by tests, as are the forms that are left alone.
The description of this change said a check confirmed the README pattern and the template's were the same, and no such check existed. The template holds the pattern as a single quoted string with its backslashes doubled, so the check halves them and looks for the result in the README. Changing one character of the README pattern turns the check red, which is how it was proved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Pages running the cloud client script end up with this session storage key.
Every request from that tab then sends
51D_Pos_%22+%2B+key+%2B+%22=to the server. The location snippet writes its result with a name it builds as it runs.Cause
In
processJsProperties(JavaScriptResource.mustache, thevalueSetPrefixpattern andstoreEmptyValues), group 3 of the pattern matched the text between the outer quotes,51D_Pos_" + key + ".window.sessionStorage["fod_data_51D_Pos_" + key + ""]=pos.coords[key], so the snippet builds the right key as it runs.storeEmptyValuesruns before each snippet in both cookie modes, and it used group 3 as a literal name. It therefore stored""under the text of the join, andgetFodSavedValuessent that key."51D_Bandwidth" + "=" + ...(the bandwidth snippet in the device data), was stored under51D_Bandwidth" + "."51D_A =" + vwas stored under51D_Awith a trailing space.The template literal form used by the high entropy values snippet was already handled correctly.
Group 6 is
51D_GetHighEntropyValuesand the rewrite giveswindow.sessionStorage["fod_data_51D_GetHighEntropyValues"]=btoa(JSON.stringify(t)). This PR adds tests for that form.Change
+only to further quoted text and to plain variable names. The pattern leaves a space before=out of the name. Group numbers are unchanged. Anything else is not matched, for example a name joined toitem.key, a template literal with a built name, or a template literal with text after the value. Those stores are left to write their cookie, and nothing is stored for them under a name the pattern did not understand.storeEmptyValuesremoves joins between quoted strings, so51D_Bandwidth" + "becomes51D_Bandwidth. It stores nothing for a name joined to a variable, because that name is only known as the snippet runs. The location snippet still gets its empty51D_Pos_Error, which is a fixed name.getFodSavedValuesskips a stored name that holds a double quote, so a tab that an earlier script left with the bad key stops sending it.I ran the old and new patterns over every store form in the device data file (
51D_ProfileIds,51D_ScreenPixelsWidth,51D_ScreenPixelsHeight,51D_PixelRatio,51D_ThirdPartyCookiesEnabled,51D_Bandwidth,51D_GetHighEntropyValues) and over both paths of the location snippet. Both patterns rewrite every one of them to the same text. The only change for these snippets is the name used for the empty result.The rendered script gains no
document.cookietext, so the ports' occurrence counts are unchanged.Tests
A new section in
tests/template-tests.js, "Result names the snippets build or join", adds 30 checks. The location and high entropy values snippets are copied as the service serves them.fod_data_51D_Pos_latitudeandfod_data_51D_Pos_longitude,51D_Pos_Errorgets its empty result, no key holds a quote, and the body carries no%22.51D_Pos_Erroris stored and no bad key is left.51D_GetHighEntropyValues. Wherenavigator.userAgentDatais missing, the empty result is sent."51D_Bandwidth" + "="is stored and sent as51D_Bandwidth."51D_Spaced =" + vis stored and sent as51D_Spaced.Without the template change (template from
main, new tests):One of the failing request bodies, which matches what production pages send:
With the change:
For consumers
The server side of the 51Degrees cloud service keeps a copy of this pattern so that it can find the result names a snippet writes, and a test there compares the copy with the template the service serves. That copy will need the new pattern when a builder package carrying this template is taken. This PR does not change the snippet text in the device data or the location engine, because every store those snippets use is read correctly now.
Not changed
+ "; path=/"to its value. When cookies are off, the rewrite therefore stores the value with; path=/on the end. This was already the case before this PR and is unchanged here.Added after the run recorded above
Item 4 above said a check confirmed the README pattern and the template's were the same, and no such check existed. One was added in
6d92f23, namedthe README prints the pattern the template uses. It takes the pattern out of the template, halves the doubled backslashes of the single quoted string that holds it, and looks for the result in the README. Changing one character of the README pattern turns it red, which is how it was proved. The suite now reports188 checks, 0 failures.Related, and what has to follow
This and #32, which makes the missing preference platform warning wait for the page to load, both change
JavaScriptResource.mustache, in different parts of the file. Merging the two branches together locally in both orders needed no conflict resolution, andnode tests/template-tests.json the result reports207 checks, 0 failureseither way, so the merge order does not matter.The Rust SDK keeps its own copy of the template at
javascript-builder/assets/JavaScriptResource.mustacheand checks it against this repository. Its pull request and push builds compare against a pinned commit, but the Monday 04:00 schedule and a manual run compare againstmain, so that weekly check goes red once this is merged and stays red until 51Degrees/rust#52 moves the bundled copy and the pinned reference together in one pull request. The .NET, Java, Node and Python pipelines take this repository as a submodule and the PHP pipeline readsJavaScriptResource.mustachefrom ajavascript-templatesdirectory beside its source, so each of them picks the change up only when it next moves that pointer and publishes a package.Merge order
The three ready pull requests in this repository take this order:
29, 32, 31
Merges before this one: 29, 32.
This and 32 both change
JavaScriptResource.mustacheandtests/template-tests.js, in different parts of each file, andgit merge-treefinds no conflict between them in either direction.No branch had to be changed for this order. Every pair was measured with
git merge-tree, the three were then merged intomainin that order as real merges with no conflict, andnpm testintestson the result reports207 checks, 0 failures. Smallest and most self-contained first. Checked on 17 September 2026.