fix(backend): Add current date to etag that's used in get-released-data - #7015
Open
maverbiest wants to merge 1 commit into
Open
fix(backend): Add current date to etag that's used in get-released-data#7015maverbiest wants to merge 1 commit into
maverbiest wants to merge 1 commit into
Conversation
Contributor
|
side note: I realized the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
resolves #7013
The ETag
get-released-datareturns is only updated when something is written to released data tables. However, a sequence's DUT can change from RESTRICTED to OPEN on a new day if the restricted until date is reached. This does not result in a database write so the ETag does not change, butget-released-datawill start returning DUT OPEN for the sequence as it is computed from the current date and the restricted until date each time the endpoint is called.This can cause a mismatch between the DUT reported by the endpoint and what is show in SILO/LAPIS, since SILO only imports new data when the ETag changes. This was noticed as a mismatch between data returned by the endpoint and what was visible on the website (website reflects LAPIS state): https://loculus.slack.com/archives/C0757PTR607/p1785492200559359.
Fix
This PR adds a
getLastDatabaseWriteETagWithDatefunction to be used in theget-released-dataendpoint controller. The only difference is that it adds the current date to the ETag. I'm still keeping the old ETag implementation around as it's also used byextract-unprocessed-data, where a change in DUT doesn't matter as far as I can tell? (let me know if this is not the case, happy to update)This does mean that all organisms will have a SILO refresh when the UTC date changes. This is some extra load but hopefully shouldn't be too bad.
PR Checklist
- [ ] Any manual testing that has been done is documented (i.e. what exactly was tested?)🚀 Preview: Add
previewlabel to enable