Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .claude/skills/mblt-vision/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,10 @@ description: >-
and prepared query/gallery loaders. Preserve ops stable distance ties, camera exclusions
and dataset-specific junk handling; see the ReID API section in `mblt_vision/README.md`.

- Face verification uses prepared 112x112 crops without realignment or implicit flip fusion.
Keep pair/fold order, source threshold search, Results embeddings and readiness connected;
require exact metric and preprocessing parity (face verification API section).

## Runtime and Packaging

- Route NPU runtime access through mblt-npu-python; do not copy backend classes into Vision.
Expand Down
4 changes: 4 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,10 @@ The current ownership boundary is deliberate:
stable distance ties, camera exclusions, junk rules, and preprocessing numeric parity tested.
Named Normalize styles retain legacy arithmetic; explicit mean/std uses source float32 arithmetic.

- Face verification utilities preserve prepared alignment and official pair/fold order.
Use dataset-bound `FaceRecognitionPost`, `Results.embeddings`, and face readiness checks.
Protocol details and source-parity requirements are in the face verification API section.

## Python-First Architecture

- Keep the public layer independent from a particular backend. Define small internal interfaces
Expand Down
2 changes: 1 addition & 1 deletion benchmark/benchmark_vision_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
tuple[CoreMode, ...], core_modes_for_target_device("aries-rb")
)
# Utility-only embedding tasks have no benchmark engine/CLI route yet.
TASK_CHOICES = tuple(task for task in VISION_TASKS if task != "person_reid")
TASK_CHOICES = tuple(task for task in VISION_TASKS if task not in {"person_reid", "face_recognition"})
SUPPORTED_TARGET_DEVICES = frozenset({"aries-rb", "regulus-ra", "regulus-rb"})


Expand Down
43 changes: 43 additions & 0 deletions mblt_vision/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1022,3 +1022,46 @@ Offline tests vendor the inspected ops functions at
distance/CMC/mAP equality; synthetic RGB preprocessing requires byte equality
(`atol=0`) against the ops adapter for explicit source mean/std profiles. These
are protocol tests, not measured model accuracy claims.

## Face verification utilities

`mblt_vision.face_recognition` exports `FaceRecognitionPost(EmbeddingPost)`,
`LFWDataset`, `AgeDB30Dataset`, `CFPFPDataset`, and their `LFWEvaluator`,
`AgeDB30Evaluator`, `CFPEvaluator` subclasses of `VerificationEvaluator`.
The processing-only `bases/face_recognition.yaml` profile is 112x112 PIL bilinear,
explicit float32 0.5 mean/std, 512 dimensions, row-wise L2 normalization. Engine
postprocessing stores dense vectors in `Results.embeddings` and `output`. Model
wrappers/artifact YAMLs are a separate stage; CLI/benchmark routing is unchanged.

Pair loaders read prepared `pairs.csv` columns `pair_id,fold_id,is_same,image_a,image_b`,
retain pair order and official one-based fold labels, and enumerate unique image
basenames in sorted order. LFW uses `aligned/`; AgeDB-30 and CFP-FP use `images/`.
They require existing 112x112 crops and never detect or realign faces. Source pair
counts are checked by default; caller-owned subsets explicitly opt out with
`validate_counts=False`. Default cache roots end in `<dataset>/eval`.
`dataset_ready(root, "face_recognition", dataset)` additionally requires all ten
folds and decodable crops; its default dataset is LFW.

```python
from mblt_vision.face_recognition import LFWDataset, LFWEvaluator

data = LFWDataset("/path/to/lfw/eval")
# embeddings: one row per data.samples, from your inference callback.
# result = LFWEvaluator().evaluate(embeddings, data.samples, data.pairs)
# print(result.as_dict())
```

Ops at `ec537764e36ca10da963fcb7afab157ea60ca022` owns the protocol: one
normalized embedding per crop, **no flip fusion**, float64 cosine scores, threshold
grid `linspace(-1,1,4001)`, inclusive `score >= threshold`, strictest threshold on
training ties, leave-one-fold-out threshold fitting, mean accuracy and population
standard deviation over folds. `extract_embeddings` shares the ordered bounded
inference helper with retrieval. There is no implicit fusion in scoring.

Offline tests compare exact cosine scores, fold accuracies, thresholds and aggregate
metrics against a verbatim source snapshot over five synthetic seeds. Three RGB
fixtures (including no-op resize) require preprocessing byte equality (`atol=0`).
Dataset YAMLs record exact source paths/revision. CFP-FP specifically refers to
the prepared cat-claws mirror: ops warns it was not checked against the original
`cfp_fp.bin` and records a training-log discrepancy. This package preserves that
provenance and makes no new accuracy claim.
2 changes: 2 additions & 0 deletions mblt_vision/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@

from . import depth_estimation as depth_estimation
from . import face_detection as face_detection
from . import face_recognition as face_recognition
from . import image_classification as image_classification
from . import instance_segmentation as instance_segmentation
from . import obb as obb
Expand Down Expand Up @@ -54,6 +55,7 @@
"obb",
"pose_estimation",
"person_reid",
"face_recognition",
"semantic_segmentation",
] + sorted(_LEGACY_MODEL_EXPORTS)
# Keep legacy compatibility exports synchronized with their task packages.
Expand Down
1 change: 1 addition & 0 deletions mblt_vision/_tasks.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
"pose_estimation",
"face_detection",
"person_reid",
"face_recognition",
)


Expand Down
34 changes: 34 additions & 0 deletions mblt_vision/datasets/agedb-30.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: agedb-30
path: ~/.mblt_model_zoo/datasets/agedb-30
tasks:
- face_recognition
val: eval
image_directory: images
pairs: 6000
folds: 10
aligned_size:
- 112
- 112
source:
repository: https://git.mobilint.com/algorithm-team/testing/mblt-model-ops
revision: ec537764e36ca10da963fcb7afab157ea60ca022
config_path: datasets/agedb_30/agedb_30.yaml
name: agedb-30
dataloader_path: datasets/agedb_30/_private/dataloader.py
eval_protocol:
id: agedb-30-ten-fold-verification.v1
evaluation:
split: eval/pairs.csv
protocol: agedb-30-ten-fold-verification.v1
primary_metric: agedb_30_accuracy
secondary_metrics:
- agedb_30_accuracy_std
embedding_fusion: none; one normalized embedding per crop
implementation_reference:
path: datasets/agedb_30/_private/evaluator.py
shared_paths:
- datasets/_shared/face_pairs.py
- datasets/_shared/face_verification.py
- models/face_recognition/_shared/evaluate.py
- models/_shared/embeddings.py
revision: ec537764e36ca10da963fcb7afab157ea60ca022
36 changes: 36 additions & 0 deletions mblt_vision/datasets/cfp-fp.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: cfp-fp
path: ~/.mblt_model_zoo/datasets/cfp-fp
tasks:
- face_recognition
val: eval
image_directory: images
pairs: 7000
folds: 10
aligned_size:
- 112
- 112
source:
repository: https://git.mobilint.com/algorithm-team/testing/mblt-model-ops
revision: ec537764e36ca10da963fcb7afab157ea60ca022
config_path: datasets/cfp_fp/cfp_fp.yaml
name: cfp-fp
dataloader_path: datasets/cfp_fp/_private/dataloader.py
caveat: Prepared cat-claws mirror is not checked against original cfp_fp.bin; ops
records lower CFP-FP accuracy than training logs.
eval_protocol:
id: cfp-fp-ten-fold-verification.v1
evaluation:
split: eval/pairs.csv
protocol: cfp-fp-ten-fold-verification.v1
primary_metric: cfp_fp_accuracy
secondary_metrics:
- cfp_fp_accuracy_std
embedding_fusion: none; one normalized embedding per crop
implementation_reference:
path: datasets/cfp_fp/_private/evaluator.py
shared_paths:
- datasets/_shared/face_pairs.py
- datasets/_shared/face_verification.py
- models/face_recognition/_shared/evaluate.py
- models/_shared/embeddings.py
revision: ec537764e36ca10da963fcb7afab157ea60ca022
34 changes: 34 additions & 0 deletions mblt_vision/datasets/lfw.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: lfw
path: ~/.mblt_model_zoo/datasets/lfw
tasks:
- face_recognition
val: eval
image_directory: aligned
pairs: 6000
folds: 10
aligned_size:
- 112
- 112
source:
repository: https://git.mobilint.com/algorithm-team/testing/mblt-model-ops
revision: ec537764e36ca10da963fcb7afab157ea60ca022
config_path: datasets/lfw/lfw.yaml
name: lfw
dataloader_path: datasets/lfw/_private/dataloader.py
eval_protocol:
id: lfw-ten-fold-verification.v1
evaluation:
split: eval/pairs.csv
protocol: lfw-ten-fold-verification.v1
primary_metric: lfw_accuracy
secondary_metrics:
- lfw_accuracy_std
embedding_fusion: none; one normalized embedding per crop
implementation_reference:
path: datasets/lfw/_private/evaluator.py
shared_paths:
- datasets/_shared/face_pairs.py
- datasets/_shared/face_verification.py
- models/face_recognition/_shared/evaluate.py
- models/_shared/embeddings.py
revision: ec537764e36ca10da963fcb7afab157ea60ca022
40 changes: 40 additions & 0 deletions mblt_vision/face_recognition/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
"""Aligned face pair verification utilities; no implicit face detection/alignment."""

from ..utils.datasets.face_pairs import (
AgeDB30Dataset,
CFPFPDataset,
FacePair,
FacePairDataset,
LFWDataset,
PairImage,
)
from ..utils.evaluation.verification import (
AgeDB30Evaluator,
CFPEvaluator,
LFWEvaluator,
VerificationEvaluator,
VerificationResult,
cosine_scores,
extract_embeddings,
fold_accuracies,
)
from ..utils.postprocess.embedding_post import EmbeddingPost, FaceRecognitionPost

__all__ = [
"AgeDB30Dataset",
"AgeDB30Evaluator",
"CFPEvaluator",
"CFPFPDataset",
"EmbeddingPost",
"FacePair",
"FacePairDataset",
"FaceRecognitionPost",
"LFWDataset",
"LFWEvaluator",
"PairImage",
"VerificationEvaluator",
"VerificationResult",
"cosine_scores",
"extract_embeddings",
"fold_accuracies",
]
18 changes: 18 additions & 0 deletions mblt_vision/models/bases/face_recognition.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Processing only; concrete approved HF artifacts are added separately.
# ops ec537764e36ca10da963fcb7afab157ea60ca022
# models/face_recognition/ArcFace_IResNet_100/distribution.yaml
DEFAULT:
pre_cfg:
Reader:
style: pil
Resize:
size: [112, 112]
interpolation: bilinear
Normalize:
mean: [0.5, 0.5, 0.5]
std: [0.5, 0.5, 0.5]
post_cfg:
task: face_recognition
dataset: lfw
embedding_dim: 512
normalize: true
Loading
Loading