Repository navigation
0.3.1: fix state-safety, EWC, merge, loading, multi-label, server and strategic-mode bugs; Node.js port; Docker check - #69
Merged
Conversation
scripts/adaptive_router was an incomplete, pre-0.2.0 save (no examples.json, tensors named tensors.safetensors, no tokenizer) that AdaptiveClassifier.load could not read.
…inference Work in progress: neither has been run yet. The Docker build cannot run in the authoring sandbox (TLS proxy blocks the PyTorch wheel download), so the new CI job is the first real check.
The port matches AdaptiveClassifier.predict for both pooling modes, legacy and current scoring, fixed and ramped new-class weights, and over-length inputs. Deliberately breaking the ramp or the truncation is caught.
Moved from the main test workflow to its own workflow with path filters (plus a weekly schedule), so ordinary PRs do not build the image.
…locking Found by probing MultiLabelAdaptiveClassifier with bad input and lifecycle calls: - forget() and remove_examples() crashed (the head had no remove_classes) - load(use_onnx=...) raised TypeError; thresholds and settings were not saved - a bare-string label was split into one class per character - max_labels=0 meant no limit, and min_predictions could exceed max_labels - the head was trained for ~10 steps and later-added labels went through a softmax loss that does not apply to sigmoid outputs; it now trains with BCE for head_steps steps with a cosine schedule, and is retrained when labels are added - public methods were not covered by the instance lock 28 new tests (27 fail on the old code).
…l and strategic mode Strategic mode: validate cost_coefficients (the README example silently disabled it), sample best-response candidates across all dimensions with a fixed seed, make predict/predict_strategic/predict_robust scores independent of k, and fix the robustness evaluation (division by zero, unknown labels, repeatability, lock). Pin datasets/huggingface-hub in the benchmark and scripts extras.
…multi-label modes The 0.3.0 instance lock made deepcopy and pickle fail. predict_batch skipped the strategic blend and softmaxed multi-label sigmoid scores, so it disagreed with predict.
The prototype index was not rebuilt after a merge, so the other classifier's classes were never found (0% accuracy on them). Also: empty target, training counts, stale calibration, shared example objects.
…bugs found in review Core: add_examples is all-or-nothing; EWC penalty now acts on the live head; clear_memory really clears; reloaded prototypes fold in new examples by weight; re-saving an ONNX-loaded classifier keeps the encoder name; merge_classifiers cannot deadlock and keeps OOD radii; embeddings are encoded in batches. Multi-label: explicit threshold honoured, base-class load returns the right type, calibration refused, abstain options, thresholds follow merge/remove. Server: updates/saves/info off the event loop, multi-label refused. Strategic: input validation, gaming levels. sklearn: declared classes, label clashes. 18 + 40 new tests; the full suite run was still in progress when this was committed.
basic_usage.py no longer calls .eval() on an ONNX model; docs match real signatures and drop the nonexistent --workers flag; the release workflow gates publishing on the full test workflow, scopes PyPI credentials to the publish job and checks tag == version; docker.yml triggers aligned.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Version 0.3.1. Follow-up to #68, from a systematic hunt for bugs in features that had little or no test coverage: lifecycle operations, multi-label, strategic mode, the HTTP server, the scikit-learn wrapper,
merge_classifiers, and the CI/release plumbing. Three independent reviewers (core library, feature modules, CI/packaging/docs) read the code cold and tried to break it; every finding below was reproduced with a script before being fixed.CHANGELOG.mdhas the full list.Core library
add_examplesis now all-or-nothing. An error or Ctrl-C while training used to leave the label map ahead of the neural head, so every laterpredictraised "selected index k out of range" and every retry failed; or left the search index stale, so predictions silently returned the wrong class. Texts are encoded before any state changes, and on failure everything is restored.merge_classifierswas badly broken: the other classifier's classes were never searchable afterwards (the prototype index was not rebuilt; 0% accuracy on them), merging into an empty classifier raised, training counts were not combined, stale calibration was kept. Also fixed a deadlock when two threads merge each other (it never holds two classifiers' locks at once) and the other classifier's OOD radii are kept.clear_memory()with no labels cleared only the prototypes; the label map, counts, calibration and neural head stayed, sopredictkept answering from the head. It now empties the classifier.add_examples(moved by 0.32 versus 0.03 live for a 30-example class). Prototypes now fold in new examples with the weight of the examples they were saved from.model_name;add_examplesignoredbatch_size;copy.deepcopy/picklefailed because of the instance lock;predict_batchdisagreed withpredictin strategic and multi-label modes; blank training texts/labels were silently learned (behaviour change: they now raiseValueError);remove_examples("str")removed nothing;PrototypeMemory.get_strategic_prototypesraisedNameError.Multi-label, strategic mode, server, scikit-learn
forget/remove_examples/load(use_onnx=...)crashed; an explicitthresholdwas ignored; a saved multi-label model loaded throughAdaptiveClassifier.load(or the server) came back as single-label; the head was barely trained;calibrate/predict_setreturned meaningless numbers (nowNotImplementedError); bare-string labels were split into one class per character.k; the robustness evaluation divided by zero and crashed unless the levels included 0.0 and 1.0./infono longer block the event loop (a save with ONNX export froze/healthfor seconds); multi-label models are refused with a clear message instead of failing every request with 500.partial_fit(classes=...)now lists declared classes; labels that collapse to the same class name (1and"1"),NoneandNaNare rejected.New
examples/javascript/portable_inference.mjs: a Node.js port of the portable inference, tested againstpredict()for both pooling modes, every scoring branch and over-length inputs (tests/test_js_example.py, run in CI).scripts/docker_smoke.pyand.github/workflows/docker.yml: build the serving image and query it (path-filtered plus weekly, so ordinary PRs do not build it).docs/API.mdnow covers the real signatures,MultiLabelAdaptiveClassifier, strategic mode and what a saved classifier keeps; README documents the new-class recall trade-off (new_class_example_threshold) with measurements.Testing
Known gaps
docker build), theonnxruntime-nodepath of the JavaScript example (onlyonnxruntime-webcould be installed locally), and the reworked release workflow.docs/API.md); only the prototypes were fixed.🤖 Generated with Claude Code