Repository navigation
Count soft-deleted learners when picking a free picture password - #15664
namanjain24-sudo wants to merge 10 commits into
Conversation
A soft-deleted learner keeps its picture_password until it is hard-deleted 30 days later, so the row still occupies the (dataset, picture_password) unique constraint. get_assigned_sequences read through the default manager, which hides soft-deleted rows, so for those 30 days the sequence was reported as free and handed straight back out, raising IntegrityError. PicturePasswordCollisionOperation had the same blind spot: when the local holder of an incoming record's sequence was soft-deleted it found no user to reassign, returned COMPLETED without clearing the store error, and the incoming record failed to deserialize on every later sync. Read through all_objects in both places, so the set treated as taken matches the set the constraint enforces. Fixes learningequality#15662
|
👋 Hi @namanjain24-sudo, thanks for contributing! For the review process to begin, please verify that the following is satisfied:
📢✨ Before we assign a reviewer:
Also check that issue requirements are satisfied & you ran Pull requests that don't follow the guidelines will be closed. Reviewer assignment can take up to 2 weeks. |
|
Confirming both checklist items:
|
Build Artifacts
Smoke test screenshot |
…pool runs out Soft-deleted learners now keep their sequence, so a facility that fills the pool has no way to assign another. Fall back to taking one back from the learner deleted longest ago, who is least likely to want theirs again, and only raise NoAvailableSequences when there is nothing left to reclaim. Requested on learningequality#15662.
|
Pushed the reclaim half that @rtibbles asked for on #15662, so this PR now covers the whole of the described outcome:
The one assumption, flagged so it is easy to overturn: I read "least recently deleted" as LRU-style — reclaim from the learner deleted longest ago first — on the grounds their sequence is least likely to be wanted back. That is a single
Still outstanding, deliberately not in this push: |
|
CI shows two reds on the reclaim commit. Both trace to one infrastructure hang rather than the change, and I checked rather than assumed, because the same two jobs passed on this PR's previous commit:
The change in this commit is confined to Locally: I can't re-run jobs on this repo from a fork, and I'd rather not force-push a no-op commit just to clear a red that isn't telling you anything. Happy to push a rebase if you'd like a clean run, or it will sort itself out on the next push. |
|
A dependency between this PR and #15660 that I found while testing the new reclaim path, and that is worth knowing before either lands.
Measured both ways on a tree with
To be precise about what this is and isn't: the retry path is already broken on The practical upshot is only about ordering: if #15660 lands first, or they land together, the reclaim works as intended. If this one lands alone, reclaim-on-exhaustion will fail for any caller that hits a collision first — which is the narrow case of a facility whose pool is full and that races on the same sequence. Both branches merge together cleanly; I ran the combined tree at 886 passed, 16 skipped. |
|
📢✨ Before we assign a reviewer, we'll turn on |
🟡 Waiting for changesOutcome: reviewed Last updated: 2026-10-29 11:19 UTC |
rtibblesbot
left a comment
There was a problem hiding this comment.
PR #15664: the reclaim sits in get_available_sequence, not in assign_picture_password as #15662 asked. Counting soft-deleted learners via all_objects fixes #15662 as filed.
CI: only "Build Assets" failed, and it was cancelled. Test jobs pass. Tests not run locally (no venv).
- blocking: move the reclaim into
assign_picture_password, per the maintainer (inline). - suggestion: restored learners can come back without a picture password (inline).
- suggestion: the sync collision path falls back to
Noneinstead of reclaiming (inline). - suggestion: no test covers the read-only contract (inline).
- suggestion:
are_picture_passwords_exhaustedundercounts, becauseLearnerCounterCache.countstill usesFacilityUser.objects. The #15662 thread planned to fix this here. Is that fix still coming? - nitpick:
date_deletedties need a tie-break (inline). - nitpick: comments explain the change rather than the code (inline).
Next step: Needs a full code review by a human engineer.
- The author is outside the team: confirm this should be built at all.
- Check that the reclaim sits in
assign_picture_passwordand runs in one transaction, as requested on #15662. - Decide whether restoring a learner whose sequence was reclaimed should assign a new one, and whether sync collisions should reclaim too.
Written by rtibblesbot, an LLM-based coding agent.
Requested on learningequality#15662: get_available_sequence is a read it should not write through. It now raises NoAvailableSequences again, and assign_picture_password catches that and takes the sequence from the oldest soft-deleted holder. Clearing the holder and saving the user happen in one transaction.atomic(), so an IntegrityError on the user's save puts the sequence back rather than leaving it assigned to nobody. Order by id as well as date_deleted: a bulk soft-delete writes one timestamp across the whole selection, so date_deleted alone leaves the choice arbitrary.
The "30 days" one also duplicated the retention window in tasks.py, where it would have gone stale.
|
Thanks — pushed the blocking change and the two nitpicks. Taking the points in order. blocking — reclaim moved into suggestion — no test covers the read-only contract. Added nitpick — nitpick — comments that explain the change. Deleted four: the suggestion — restored learners come back without a sequence. Agreed, and it is already open as #15690 against suggestion — the sync collision path falls back to suggestion —
|
rtibblesbot
left a comment
There was a problem hiding this comment.
PR #15664: 4 of 6 prior findings resolved; 2 suggestions still open on their existing threads.
CI pending.
- suggestion: a restored learner whose sequence was reclaimed has no picture password, so can't use picture login. Please handle this on restore, or reply on the existing thread with why it is acceptable.
- suggestion:
sync_operations.py:396: an active conflicting local learner getsNoneonce soft-deleted holders fill the pool. Please handle the exhausted pool there, or reply on the existing thread with whyNoneis acceptable. - nitpick: new docstring lines restate the code (inline).
Prior-finding status
RESOLVED — kolibri/core/auth/utils/picture_passwords.py — get_available_sequence writes another user's row
UNADDRESSED — kolibri/core/auth/utils/picture_passwords.py — restored learner whose sequence was reclaimed can't use picture login
UNADDRESSED — kolibri/core/auth/sync_operations.py:396 — active local learner gets None once soft-deleted holders fill the pool
RESOLVED — kolibri/core/auth/test/test_picture_passwords.py:180 — no test that get_available_sequence stays read-only
RESOLVED — kolibri/core/auth/utils/picture_passwords.py — .first() tie on date_deleted
RESOLVED — kolibri/core/auth/sync_operations.py — comments explaining the change
Next step: Needs a full code review by a human engineer.
- The author is outside the team: confirm this should be built at all.
- Decide whether taking a picture password from the learner deleted longest ago is acceptable, since restoring that learner leaves them without picture login.
- Check that sync collision handling, which now also matches soft-deleted local learners, behaves correctly once the pool is exhausted.
Written by rtibblesbot, an LLM-based coding agent.
Superseded by my review of bb21baf: no blocking findings remain.
|
One red check on this run, and it is not the change — leaving the evidence here so nobody has to dig.
That assertion is Three things say it is environmental rather than this branch:
The change here is Python logic in |
Moving the reclaim out of get_available_sequence left the retry calling a function that now raises instead of reclaiming, so a collision that coincided with the pool emptying raised NoAvailableSequences where the first attempt would have taken a sequence back. That contradicted this function's own docstring. Both paths reclaim again.
|
One more push, from re-reading the restructure rather than from a review finding. Moving the reclaim out of Both paths reclaim again now.
Also, thank you for the four withdrawals — the sync-path and docstring reasoning landing rather than being argued down was useful. |
rtibblesbot
left a comment
There was a problem hiding this comment.
PR #15664: no new findings. All 8 prior findings are resolved or acknowledged. CI is pending.
Prior-finding status
RESOLVED — kolibri/core/auth/utils/picture_passwords.py — get_available_sequence writes another user's row
RESOLVED — kolibri/core/auth/test/test_picture_passwords.py:180 — no test covers get_available_sequence staying read-only
RESOLVED — kolibri/core/auth/utils/picture_passwords.py — .first() picks an arbitrary row on date_deleted ties
RESOLVED — kolibri/core/auth/sync_operations.py — comments explain the change, not the code
ACKNOWLEDGED — kolibri/core/auth/utils/picture_passwords.py — restored learner whose sequence was reclaimed (moved to #15690)
RESOLVED — kolibri/core/auth/sync_operations.py:396 — sync collision path doesn't reclaim (withdrawn)
RESOLVED — kolibri/core/auth/utils/picture_passwords.py:135 — docstring restates body / Raises sentence
RESOLVED — kolibri/core/auth/utils/picture_passwords.py:182 — docstring paragraph restates the except branch (withdrawn)
Next step: Needs a full code review by a human engineer.
- The author is outside the team: confirm this should be built at all.
- Confirm reclaiming a picture password from the longest-deleted learner (writing another user's soft-deleted row) is the intended product behaviour.
- Check the reclaimed row's change syncs cleanly when that learner's facility data is held on other devices.
Written by rtibblesbot, an LLM-based coding agent.
|
Taking the third item in your next-step list — whether the reclaimed row's change syncs cleanly when that learner's facility data is held on other devices — since I could check it rather than leave it open. The narrow save does mark the row for sync. if update_dirty_bit_to is not None and kwargs.get("update_fields") is not None:
kwargs["update_fields"] = set(kwargs["update_fields"]) | {"_morango_dirty_bit"}Clearing to NULL cannot collide. The constraint is The interesting case is ordering, and this PR already covers it. The reclaim produces two records: the holder losing its sequence and the new user gaining it. If the gaining record deserializes on another device before the clearing one, it hits the unique constraint against a holder that is still soft-deleted there. That is exactly the case the first commit on this branch fixes — before it, So the reclaim and the sync-side fix in this PR are two halves of the same thing, which I had not spelled out. Worth a human eye on the claim, but that is what the code does. |
|
Checked and agreed — that closes the sync question from my review:
Nothing further from me on this item. |
|
A merge-order note, since this PR and #15660 now overlap and that is a consequence of the restructure rather than something either PR set out to do. Moving the reclaim into I checked rather than just flagging it, and it is mild:
So whichever lands second needs a trivial rebase and no behavioural decision. Happy to rebase this one onto #15660 now if you would rather carry the conflict here than discover it at merge time — I have left both branches as they are for now rather than guessing at the order you want. |
The atomic() added for the reclaim was the point of the blocking change, but nothing exercised what it is there for. Two tests: - a failed save after the holder was cleared must roll the clear back, or the sequence is left assigned to nobody. This is red without the atomic(). - the reclaim run inside a caller's own atomic block, which is how both real callers invoke it, must leave that transaction usable.
|
Closing a loop on my own claim rather than leaving it asserted. When I moved the reclaim I said the
|
rtibblesbot
left a comment
There was a problem hiding this comment.
PR #15664: all 8 prior findings are resolved. One new nitpick. CI is pending.
Prior-finding status
RESOLVED — kolibri/core/auth/utils/picture_passwords.py — get_available_sequence writes another user's row
RESOLVED — kolibri/core/auth/utils/picture_passwords.py — restored learner can't use picture login (moved to #15690)
RESOLVED — kolibri/core/auth/sync_operations.py:396 — active conflicting learner gets None (withdrawn)
RESOLVED — kolibri/core/auth/test/test_picture_passwords.py:181 — no test that get_available_sequence stays read-only
RESOLVED — kolibri/core/auth/utils/picture_passwords.py — .first() arbitrary on date_deleted ties
RESOLVED — kolibri/core/auth/sync_operations.py — comments explain the change
RESOLVED — kolibri/core/auth/utils/picture_passwords.py:135 — docstring restates body
RESOLVED — kolibri/core/auth/utils/picture_passwords.py:182 — docstring paragraph (withdrawn)
Next step: Needs a full code review by a human engineer.
- The author is outside the team: confirm this should be built at all.
Written by rtibblesbot, an LLM-based coding agent.
rtibblesbot
left a comment
There was a problem hiding this comment.
PR #15664: all 9 prior findings resolved. CI still pending.
Prior-finding status
RESOLVED — kolibri/core/auth/utils/picture_passwords.py — get_available_sequence writes another user's row
RESOLVED — kolibri/core/auth/utils/picture_passwords.py — Restored learner with a reclaimed sequence can't use picture login (moved to #15690)
RESOLVED — kolibri/core/auth/sync_operations.py:396 — Conflicting active learner gets None once the pool is full (withdrawn)
RESOLVED — kolibri/core/auth/test/test_picture_passwords.py:181 — No test covers get_available_sequence staying read-only
RESOLVED — kolibri/core/auth/utils/picture_passwords.py — .first() picks an arbitrary row when date_deleted ties
RESOLVED — kolibri/core/auth/sync_operations.py — Comments explain the change, not the code
RESOLVED — kolibri/core/auth/utils/picture_passwords.py:135 — Docstring restates the body / Raises line
RESOLVED — kolibri/core/auth/utils/picture_passwords.py:182 — Docstring paragraph restates the except branch (withdrawn)
RESOLVED — kolibri/core/auth/test/test_picture_passwords.py — Test docstring restates the name and the assertion
Next step: Needs a full code review by a human engineer.
- The author is outside the team: confirm this should be built at all.
- Confirm that reclaiming a soft-deleted user's picture sequence (writing that user's row) is acceptable, given it syncs to other devices.
Written by rtibblesbot, an LLM-based coding agent.
rtibbles
left a comment
There was a problem hiding this comment.
Please trim this to the all_objects fix plus a single reclaim path.
- Keep the
all_objectschanges inget_assigned_sequencesandPicturePasswordCollisionOperation. - Reclaim only when the first
get_available_sequencecall raisesNoAvailableSequences: a helper clears thepicture_passwordof the learner deleted longest ago and returns that sequence, then the existing save and retry run unchanged. - Drop the reclaim from the retry branch and the helper's own
transaction.atomic(). - Tests: keep
test_includes_soft_deleted_learners,test_exhausted_pool_reclaims_from_longest_deleted_learnerand the sync operation test. Drop the other four. - Leave
LearnerCounterCacheas is. Counting soft-deleted learners would refuse new learners at the limit before reclaim could run. - Cut the docstring additions.
|
Deferring to @rtibbles's review: withdrawing my earlier asks for |
Per review: the reclaim helper clears the longest-deleted learner's sequence and returns it, and assign_picture_password uses it only when the first get_available_sequence call finds nothing. The save and retry run as they did before -- no reclaim in the retry branch, and no transaction.atomic() of its own. Drops the four tests that covered the paths that no longer exist, and the docstring additions.
|
Trimmed to that shape — the PR is now 4 files, +111/-3.
On
|
|
Correcting my own merge-order note above. It no longer applies. I flagged that this PR and #15660 had started to conflict, because moving the reclaim into Re-checked rather than assumed: So there is no merge-order constraint between the two any more, and nothing needs rebasing whichever lands first. Sorry for the noise — the note was accurate when I wrote it and the trim is what undid it. |
rtibblesbot
left a comment
There was a problem hiding this comment.
PR #15664: the trim removed two docstrings that still carry needed facts (2 inline suggestions). All 8 prior findings are resolved or acknowledged.
- CI failure: the Mac job hit a network timeout, unrelated to this change.
Prior-finding status
RESOLVED — kolibri/core/auth/utils/picture_passwords.py — get_available_sequence writes another user's row
ACKNOWLEDGED — kolibri/core/auth/utils/picture_passwords.py:145 — restored learner can't use picture login (moved to #15690)
RESOLVED — kolibri/core/auth/sync_operations.py:396 — reclaim on the sync path (withdrawn)
RESOLVED — kolibri/core/auth/test/test_picture_passwords.py:179 — no test that get_available_sequence stays read-only (moot: it has no write path now)
RESOLVED — kolibri/core/auth/utils/picture_passwords.py — .first() arbitrary on date_deleted ties
RESOLVED — kolibri/core/auth/sync_operations.py — comments explain the change
RESOLVED — kolibri/core/auth/utils/picture_passwords.py — docstring line / Raises restate the body
RESOLVED — kolibri/core/auth/utils/picture_passwords.py — docstring paragraph restates the except branch (withdrawn)
Next step: Needs a full code review by a human engineer.
- The author is outside the team: confirm this should be built at all.
- Confirm that taking a soft-deleted learner's picture password, so that a restore leaves them without one (handled in #15690), is an acceptable product call.
- Check that the reclaim writing the holder's row outside
transaction.atomicis safe: if the user's save fails, the holder has already lost the sequence.
Written by rtibblesbot, an LLM-based coding agent.
|
Three red checks on this run, none of them the change. Evidence rather than assertion, since the trim is exactly the commit they landed on.
That one is worth more than a shrug, because the workflow has 24 successes and 1 failure in its last 25 runs repo-wide and the failure is mine. What rules it out as this branch's doing is the branch's own history: this job has passed eight times here, including on I cannot re-run jobs from a fork. Happy to push a no-op to retrigger if you would rather see it green than take my word for it. |
|
Small correction to the note above. I wrote "a commit that only deletes code". The conclusion is unchanged and does not rest on that phrasing: those eight lines are sequence-selection logic in |
The all_objects paragraph is what stops get_assigned_sequences reading like a slip and being "fixed" back to objects, which is learningequality#15662. And the Raises line had gone stale: with a full pool and a soft-deleted holder the function now reclaims instead of raising, so NoAvailableSequences escapes only when there is no holder, or when the retry finds the pool full. Also records that the reclaim writes another learner's row, which nothing else in the signature shows.
|
Both restored — you were right that the trim took facts with it, and one of them had gone stale rather than merely missing.
@rtibbles — flagging a tension rather than deciding it: your review said to cut the docstring additions, and this puts two of them back on @rtibblesbot's suggestion. If you would rather they stay out, say so and they come out again; the
|
rtibblesbot
left a comment
There was a problem hiding this comment.
PR #15664: all 11 prior findings are resolved. No new findings. CI is still pending.
Prior-finding status
RESOLVED — kolibri/core/auth/utils/picture_passwords.py:157 — get_available_sequence writes another user's row
RESOLVED — kolibri/core/auth/utils/picture_passwords.py:148 — restored learner can't use picture login (moved to #15690)
RESOLVED — kolibri/core/auth/sync_operations.py:396 — reclaim for conflicting active local learner (withdrawn)
RESOLVED — kolibri/core/auth/test/test_picture_passwords.py:179 — no test that get_available_sequence stays read-only
RESOLVED — kolibri/core/auth/utils/picture_passwords.py:139 — .first() tie ordering on date_deleted
RESOLVED — kolibri/core/auth/sync_operations.py — comments restating the change
RESOLVED — kolibri/core/auth/utils/picture_passwords.py:154 — docstring restating body / Raises (summary kept, withdrawn)
RESOLVED — kolibri/core/auth/utils/picture_passwords.py:171 — docstring paragraph restating except (withdrawn)
RESOLVED — kolibri/core/auth/test/test_picture_passwords.py — test docstring restating test name
RESOLVED — kolibri/core/auth/utils/picture_passwords.py:119 — trim dropped the soft-deleted-holder paragraph
RESOLVED — kolibri/core/auth/utils/picture_passwords.py:177 — Raises line wrong once reclaim exists
Next step: Needs a full code review by a human engineer.
- The author is outside the team: confirm this should be built at all.
- Confirm that silently taking a soft-deleted learner's picture password once the pool is full is the intended product behaviour.
- Check the reclaim writing a deleted learner's row is safe for sync, given that row may later be restored or synced from another device.
Written by rtibblesbot, an LLM-based coding agent.
Summary
FacilityUserenforcesunique_together = (("dataset", "picture_password")), and soft-deleting a learner setsdate_deletedwithout clearingpicture_password— so the row keeps holding its sequence against that constraint untilcleanup_expired_deleted_usershard-deletes it 30 days later.The pool of free sequences was read through the default manager, which hides exactly those rows:
So for 30 days after a learner is removed, their sequence was occupied in the database but reported as free, and
get_available_sequencehanded it straight back out — producingIntegrityError: UNIQUE constraint failed: kolibriauth_facilityuser.dataset_id, kolibriauth_facilityuser.picture_passwordon a sequence that had just been reported as available.PicturePasswordCollisionOperationhad the same blind spot, with a worse outcome. It repairs an incoming record that hit this constraint by reassigning the local holder, but looked that holder up through the same manager. When the holder was soft-deleted it found nothing, hitif not local_users: return COMPLETED, and returned without clearing the store error — so the incomingFacilityUserstayed un-deserialized and failed again on every later sync until the soft-deleted row was hard-deleted.Both now read through
all_objects, so the set treated as taken matches the set the constraint actually enforces.References
Fixes #15662
Reviewer guidance
developand green here:test_includes_soft_deleted_learners— fails withAssertionError: '1.2.3' not found in set()test_soft_deleted_local_user_reassigned— fails withAssertionError: '1.2.3' == '1.2.3', i.e. the soft-deleted holder was left in place and the collision never resolvedpytest kolibri/core/auth/test/— 824 passed, 16 skipped, no regressions.prek run --files …clean on all four files.databases = "__all__"added, because soft-deleting throughsave()triggersSession.delete_all_sessions, which touches thesessionsdatabase.roles__isnull=Truefilter rather than widening it further. Role holders should never carry a picture password —Role.save()clears it when a role is granted — so that filter is not implicated in this bug, and dropping it would be a separate change with its own reasoning.None. Freeing the slot outright may well be better, since a user due for hard deletion cannot log in anyway, but that is a behaviour decision rather than part of this fix — happy to change it if you'd prefer.assign_picture_passwordin the same file but a different function; the two do not overlap textually and can land in either order.AI usage
Claude Code (on my instruction to audit the newer code for a significant bug, then fix what it found) traced the mismatch between the
unique_togetherconstraint and the manager used to compute the free pool, and I filed it as #15662 before this fix. It worked test-first here: both regression tests went in before the change and were confirmed red, then green. Notably the sync-path test turned a claim I had only reasoned about in the issue into something demonstrated — reverting the fix leaves the soft-deleted holder untouched and the collision unresolved. It also checked morango's deserialization path (skip_erroringdefaults toFalseand Kolibri never passesTrue) to confirm the stuck record really is retried on later syncs rather than dropped, so the scope of the sync impact is stated accurately. I reviewed the constraint-versus-manager reasoning, the decision to leave theroles__isnullfilter and the reassignment policy alone, and every test run above before opening this PR.