Skip to content

Store snippet results only under names the snippet writes - #31

Merged
jwrosewell merged 2 commits into
mainfrom
fix/cookie-rewrite-names
Sep 18, 2026
Merged

jwrosewell merged 2 commits into
mainfrom
fix/cookie-rewrite-names

Conversation

@jwrosewell

@jwrosewell jwrosewell commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Problem

Pages running the cloud client script end up with this session storage key.

fod_data_51D_Pos_" + key + "

Every request from that tab then sends 51D_Pos_%22+%2B+key+%2B+%22= to the server. The location snippet writes its result with a name it builds as it runs.

document.cookie = "51D_Pos_" + key + "=" + pos.coords[key];

Cause

In processJsProperties (JavaScriptResource.mustache, the valueSetPrefix pattern and storeEmptyValues), group 3 of the pattern matched the text between the outer quotes, 51D_Pos_" + key + ".

  • The rewrite was already right. It copies that text inside the session storage key, which gives window.sessionStorage["fod_data_51D_Pos_" + key + ""]=pos.coords[key], so the snippet builds the right key as it runs.
  • The empty result was wrong. storeEmptyValues runs before each snippet in both cookie modes, and it used group 3 as a literal name. It therefore stored "" under the text of the join, and getFodSavedValues sent that key.
  • A second form has the same fault. A fixed name made by joining two strings, "51D_Bandwidth" + "=" + ... (the bandwidth snippet in the device data), was stored under 51D_Bandwidth" + ".
  • A space before the equals sign was kept in the name. The old pattern's name class allowed spaces, so "51D_A =" + v was stored under 51D_A with a trailing space.

The template literal form used by the high entropy values snippet was already handled correctly.

document.cookie=`51D_GetHighEntropyValues=${btoa(JSON.stringify(t))}`

Group 6 is 51D_GetHighEntropyValues and the rewrite gives window.sessionStorage["fod_data_51D_GetHighEntropyValues"]=btoa(JSON.stringify(t)). This PR adds tests for that form.

Change

  1. The pattern. A quoted name is now a fixed start joined with + only to further quoted text and to plain variable names. The pattern leaves a space before = out of the name. Group numbers are unchanged. Anything else is not matched, for example a name joined to item.key, a template literal with a built name, or a template literal with text after the value. Those stores are left to write their cookie, and nothing is stored for them under a name the pattern did not understand.
  2. The empty result. storeEmptyValues removes joins between quoted strings, so 51D_Bandwidth" + " becomes 51D_Bandwidth. It stores nothing for a name joined to a variable, because that name is only known as the snippet runs. The location snippet still gets its empty 51D_Pos_Error, which is a fixed name.
  3. Old keys. getFodSavedValues skips a stored name that holds a double quote, so a tab that an earlier script left with the bad key stops sending it.
  4. README. The README describes the joined name forms and the updated pattern. A check confirms the README pattern is the same as the template's.

I ran the old and new patterns over every store form in the device data file (51D_ProfileIds, 51D_ScreenPixelsWidth, 51D_ScreenPixelsHeight, 51D_PixelRatio, 51D_ThirdPartyCookiesEnabled, 51D_Bandwidth, 51D_GetHighEntropyValues) and over both paths of the location snippet. Both patterns rewrite every one of them to the same text. The only change for these snippets is the name used for the empty result.

The rendered script gains no document.cookie text, so the ports' occurrence counts are unchanged.

Tests

A new section in tests/template-tests.js, "Result names the snippets build or join", adds 30 checks. The location and high entropy values snippets are copied as the service serves them.

  • Location, cookies off, position allowed. The results are stored as fod_data_51D_Pos_latitude and fod_data_51D_Pos_longitude, 51D_Pos_Error gets its empty result, no key holds a quote, and the body carries no %22.
  • Location, cookies off, position refused. 51D_Pos_Error is stored and no bad key is left.
  • Location, cookies on. The cookies are written and no bad key is stored or sent.
  • High entropy values, cookies off. The result is stored and sent under 51D_GetHighEntropyValues. Where navigator.userAgentData is missing, the empty result is sent.
  • "51D_Bandwidth" + "=" is stored and sent as 51D_Bandwidth.
  • "51D_Spaced =" + v is stored and sent as 51D_Spaced.
  • A member join, a built template literal name, and a template literal with a trailing attribute each still write their cookie, store nothing, and let the round finish.
  • A bad key left by an earlier script is not sent.

Without the template change (template from main, new tests):

  FAIL  a built name leaves nothing stored under a wrong key
  FAIL  the request carries no name with a quote in it
  FAIL  the error path leaves nothing stored under a wrong key
  FAIL  the error path request carries no name with a quote in it
  FAIL  a cookie page stores nothing under a wrong key
  FAIL  a cookie page sends no name with a quote in it
  FAIL  a name joined from two strings leaves no wrong key
  FAIL  a space before the equals sign is left out of the name
  FAIL  a space before the equals sign is left out of the sent name
  FAIL  a value stored under a wrong key earlier is not sent
187 checks, 10 failures

One of the failing request bodies, which matches what production pages send:

mark=second&id.usage=personalized&51D_Pos_%22+%2B+key+%2B+%22=&51D_Pos_Error=&51D_Pos_latitude=51&51D_Pos_longitude=-1&session-id=abc-123&sequence=1

With the change:

187 checks, 0 failures

For consumers

The server side of the 51Degrees cloud service keeps a copy of this pattern so that it can find the result names a snippet writes, and a test there compares the copy with the template the service serves. That copy will need the new pattern when a builder package carrying this template is taken. This PR does not change the snippet text in the device data or the location engine, because every store those snippets use is read correctly now.

Not changed

  • The bandwidth snippet appends + "; path=/" to its value. When cookies are off, the rewrite therefore stores the value with ; path=/ on the end. This was already the case before this PR and is unchanged here.

Added after the run recorded above

Item 4 above said a check confirmed the README pattern and the template's were the same, and no such check existed. One was added in 6d92f23, named the README prints the pattern the template uses. It takes the pattern out of the template, halves the doubled backslashes of the single quoted string that holds it, and looks for the result in the README. Changing one character of the README pattern turns it red, which is how it was proved. The suite now reports 188 checks, 0 failures.

Related, and what has to follow

This and #32, which makes the missing preference platform warning wait for the page to load, both change JavaScriptResource.mustache, in different parts of the file. Merging the two branches together locally in both orders needed no conflict resolution, and node tests/template-tests.js on the result reports 207 checks, 0 failures either way, so the merge order does not matter.

The Rust SDK keeps its own copy of the template at javascript-builder/assets/JavaScriptResource.mustache and checks it against this repository. Its pull request and push builds compare against a pinned commit, but the Monday 04:00 schedule and a manual run compare against main, so that weekly check goes red once this is merged and stays red until 51Degrees/rust#52 moves the bundled copy and the pinned reference together in one pull request. The .NET, Java, Node and Python pipelines take this repository as a submodule and the PHP pipeline reads JavaScriptResource.mustache from a javascript-templates directory beside its source, so each of them picks the change up only when it next moves that pointer and publishes a package.

Merge order

The three ready pull requests in this repository take this order:

29, 32, 31

Merges before this one: 29, 32.

This and 32 both change JavaScriptResource.mustache and tests/template-tests.js, in different parts of each file, and git merge-tree finds no conflict between them in either direction.

No branch had to be changed for this order. Every pair was measured with git merge-tree, the three were then merged into main in that order as real merges with no conflict, and npm test in tests on the result reports 207 checks, 0 failures. Smallest and most self-contained first. Checked on 17 September 2026.

A snippet that builds its result name as it runs, such as the location
snippet's "51D_Pos_" + key + "=", left an empty value in session storage
under the text of the join, fod_data_51D_Pos_" + key + ", and every
request from the tab then sent 51D_Pos_%22+%2B+key+%2B+%22 to the
server. The rewrite of the store itself was already right, because the
join is copied inside the session storage key, but the empty result
written before the snippet runs took the name group literally. A fixed
name joined from two strings, as in "51D_Bandwidth" + "=", was stored
the same wrong way.

The empty result now removes joins between quoted strings, and stores
nothing for a name that is joined to a variable, because that name is
only known as the snippet runs. A value an earlier script stored under a
name holding a quote is no longer sent. The pattern now reads a quoted
name as a fixed start joined only to quoted text and plain variable
names, and leaves a space before the equals sign out of the name, so any
other form is left to write its cookie rather than being stored under a
name the pattern did not understand. The template literal form used by
the high entropy values snippet was already read correctly and is now
covered by tests, as are the forms that are left alone.
The description of this change said a check confirmed the README pattern
and the template's were the same, and no such check existed. The template
holds the pattern as a single quoted string with its backslashes doubled,
so the check halves them and looks for the result in the README. Changing
one character of the README pattern turns the check red, which is how it
was proved.
@jwrosewell
jwrosewell marked this pull request as ready for review September 17, 2026 20:01
@jwrosewell
jwrosewell merged commit 6cc744a into main Sep 18, 2026
2 checks passed
@jwrosewell
jwrosewell deleted the fix/cookie-rewrite-names branch September 18, 2026 07:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant