Skip to content

fix(backend): Add current date to etag that's used in get-released-data - #7015

Open
maverbiest wants to merge 1 commit into
mainfrom
etag-with-date
Open

fix(backend): Add current date to etag that's used in get-released-data#7015
maverbiest wants to merge 1 commit into
mainfrom
etag-with-date

Conversation

@maverbiest

@maverbiest maverbiest commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

resolves #7013

The ETag get-released-data returns is only updated when something is written to released data tables. However, a sequence's DUT can change from RESTRICTED to OPEN on a new day if the restricted until date is reached. This does not result in a database write so the ETag does not change, but get-released-data will start returning DUT OPEN for the sequence as it is computed from the current date and the restricted until date each time the endpoint is called.

This can cause a mismatch between the DUT reported by the endpoint and what is show in SILO/LAPIS, since SILO only imports new data when the ETag changes. This was noticed as a mismatch between data returned by the endpoint and what was visible on the website (website reflects LAPIS state): https://loculus.slack.com/archives/C0757PTR607/p1785492200559359.

Fix

This PR adds a getLastDatabaseWriteETagWithDate function to be used in the get-released-data endpoint controller. The only difference is that it adds the current date to the ETag. I'm still keeping the old ETag implementation around as it's also used by extract-unprocessed-data, where a change in DUT doesn't matter as far as I can tell? (let me know if this is not the case, happy to update)

This does mean that all organisms will have a SILO refresh when the UTC date changes. This is some extra load but hopefully shouldn't be too bad.

PR Checklist

  • All necessary documentation has been adapted.
  • The implemented feature is covered by appropriate, automated tests.
    - [ ] Any manual testing that has been done is documented (i.e. what exactly was tested?)

🚀 Preview: Add preview label to enable

@claude claude Bot added the backend related to the loculus backend component label Jul 31, 2026
@anna-parker
anna-parker requested a review from clonder August 4, 2026 20:07
@anna-parker

Copy link
Copy Markdown
Contributor

side note: I realized the /extract-unprocessed-data etag is not scoped by organism, we should ideally add organism scope and a test there - can be a follow up PR :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend related to the loculus backend component

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Include current date in get-released-data ETag so newly OPEN sequences don't wrogly show as RESTRICTED in SILO

2 participants