Teaching a classifier to say “I don’t know.”
A research study prepared for the Research Methodologies course (Wrocław University of Science and Technology / Politechnika Wrocławska). It grew out of two seminar presentations and is reworked here into one clean, self-contained written report.
A standard classifier is trained on a fixed list of K categories and, by construction, must
map every input onto one of them — even inputs belonging to a class it has never seen.
Open-set recognition (OSR) removes that assumption: the model is allowed to recognise the
known classes and to reject an input as unknown. This report explains the problem, maps
the main families of methods, and works through the generative approach in depth, ending with
OpenGAN, which reaches state-of-the-art results by learning the boundary in feature space.
- The blind spot in ordinary classifiers
- What open-set recognition is
- Why it matters
- A map of the methods
- The generative route: GANs → OpenGAN
- Results
- Key takeaways
- Repository layout
- Authors and credits
- References
Two deep-dive chapters carry the full detail: Open-Set Recognition — foundations & methods · GANs and OpenGAN.
Ordinary image classification is a closed-set problem: the categories in the training set are exactly the categories in the test set, so the model never meets a class it wasn’t taught.
Train a model on cats, dogs, horses. Show it a frog. Because “frog” was never an option, the network cannot answer “frog” — it quietly returns whichever known class looks most similar, usually with high confidence. The mistake is silent, and that is the danger.
A softmax layer makes this unavoidable: its outputs are forced to sum to 1 across the known classes, so there is nowhere for probability mass to go except onto a wrong label.
flowchart LR
A["Input"] --> M["Closed-set classifier<br/>(K known classes)"]
M --> S["softmax over K classes"]
S --> P["Always outputs one<br/>of the K labels"]
P -. frog in horse out .-> W["Silent, confident error"]
Closed-set recognition is simple and lets the model focus on clean classification. Its weakness is equally simple: it cannot represent the possibility of an unknown class.
Open-set recognition keeps the closed-set skill and adds one new ability — rejection. Faced with an input, an OSR system may either assign a known label or declare the input unknown.
| Ability | Closed-set | Open-set |
|---|---|---|
| Classify a known class | ✓ | ✓ |
| Recognise “this is none of the known classes” | ✗ | ✓ |
| Cope with a changing, open world | ✗ | ✓ |
flowchart LR
A["Input"] --> C["Classifier + rejection rule"]
C --> Q{"Confident it belongs<br/>to a known class?"}
Q -->|yes| K["Predict the known class"]
Q -->|no| U["Reject as UNKNOWN"]
Both outcomes are correct answers: a well-behaved OSR model is right when it names a known class and right when it refuses to guess on something genuinely new. The real world is open — new objects, new categories, and edge cases arrive constantly — so this is the setting most deployed systems actually face.
- Safety and robustness. A closed-set model will confidently attach a familiar label to an unfamiliar object. In a face-recognition system that means a stranger can be matched to a friend or family member; in a perception stack a stroller can be read as a motorcycle. Rejecting the unknown is often safer than guessing.
- Lower annotation cost. Supervised learning needs every class labelled by hand, which is expensive and can never be exhaustive. OSR lets a model behave sensibly on categories nobody labelled, instead of demanding a label for everything that could ever appear.
- A better fit to the real world. Deployment is open-ended by nature. A model that can flag “I have not seen this before” generalises more honestly than one forced to choose.
Approaches to OSR fall into three broad families, from simple score thresholds to modern deep models. The foundations chapter covers each in detail; the summary:
| Family | Method | Core idea |
|---|---|---|
| Traditional | Threshold on confidence / distance | Reject any sample whose score falls below a chosen threshold. Simple, effective baseline; the hard part is choosing the threshold. |
| Distance-based rejection | Reject samples that lie too far from every known-class prototype in feature space. | |
| Statistical | Extreme Value Theory (EVT) | Model the tail of the score distribution to estimate how likely a sample is to be an outlier. |
| OpenMax | Replace softmax with a layer that redistributes probability onto an explicit “unknown” class, calibrated with a Weibull (EVT) fit. | |
| Deep learning | Metric learning | Learn compact, well-separated features; classify by proximity to class prototypes. |
| Generative models | Use GANs to synthesise “unknown-looking” samples, or reconstruction error to spot outliers. | |
| Self-supervised / contrastive | Learn robust features without labels; a stronger closed-set representation is itself a stronger open-set detector. |
flowchart TD
OSR["Open-set recognition"] --> T["Traditional"]
OSR --> S["Statistical"]
OSR --> D["Deep learning"]
T --> T1["Confidence / distance threshold"]
S --> S1["EVT"]
S --> S2["OpenMax"]
D --> D1["Metric learning"]
D --> D2["Generative (GAN / reconstruction)"]
D --> D3["Self-supervised · contrastive · transformers"]
OpenMax in one line. Softmax spreads all probability across the known classes and leaves no
room for doubt; OpenMax rescales the network’s activations and hands some of that mass to an
unknown class:
| Class 1 | Class 2 | Class 3 | Unknown | |
|---|---|---|---|---|
| Softmax | 0.30 | 0.50 | 0.20 | 0.00 ✗ |
| OpenMax | 0.25 | 0.40 | 0.15 | 0.20 ✓ |
A recurring modern finding ties the families together: a good closed-set classifier is already a good open-set detector — improving plain accuracy tends to improve the confidence scores that OSR relies on (Vaze et al., ICLR 2022).
The most interesting family generates the unknown instead of only thresholding it. The full derivation is in the GANs & OpenGAN chapter; the essentials:
A Generative Adversarial Network is a two-player game. A generator turns random noise into fake samples; a discriminator learns to tell real data from those fakes. Training the two against each other pushes the generator to produce ever more realistic output.
flowchart LR
z["Noise z"] --> G["Generator"]
G --> fake["Fake sample"]
x["Real data"] --> Dsc["Discriminator"]
fake --> Dsc
Dsc --> out{"Real or fake?"}
out -. train G to fool D .-> G
out -. train D to catch fakes .-> Dsc
Formally the two networks optimise a minimax objective:
where D(x) is the discriminator’s estimate that a real sample is real, and G(z) is the
sample the generator builds from noise z. D maximises this score; G minimises it.
If closed-set models fail on unknowns, why not just detect the unknowns directly?
- Fix 1 — train on outlier data. Collect some “not-a-known-class” examples and learn to flag them. But the outliers you can collect are never all possible unknowns, so the detector overfits to the particular outliers it saw.
- Fix 2 — reuse a GAN’s discriminator as the detector. A GAN discriminator already separates real from fake, so treat “fake” as “unknown.” But GAN training is unstable; the raw discriminator is unreliable as an off-the-shelf detector.
OpenGAN (Kong & Ramanan, ICCV 2021) fixes both problems with two moves:
- Discriminate in feature space, not pixel space. Instead of judging raw images, OpenGAN works on the feature vectors produced by an existing, frozen classifier. This is faster and more stable, and the open-set head is a lightweight add-on to any backbone.
- Select the model on a small held-out open set. GAN training is unstable, so OpenGAN keeps a tiny validation set of real open examples and picks the discriminator checkpoint that best separates known from unknown. Without this step performance is essentially random; with it, the method is stable and reaches state of the art.
flowchart LR
img["Image"] --> F["Frozen feature extractor<br/>(existing K-class network)"]
F --> feat["Real feature vector"]
z["Noise z"] --> Gg["Generator"]
Gg --> ff["Fake feature vector"]
feat --> Dd["Open discriminator<br/>(operates in FEATURE space)"]
ff --> Dd
open["Held-out open features<br/>(model selection)"] -.-> Dd
Dd --> score{"Known vs. unknown"}
The generator manufactures endless “fake feature” outliers so the discriminator never depends on a fixed, finite outlier set — which is exactly what defeated Fix 1 and Fix 2.
Redrawn from the OpenGAN paper (Kong & Ramanan, ICCV 2021, Table 1). The metric is AUROC for separating known from unknown inputs — higher is better. OpenGAN pulls clearly ahead of the classic Softmax (MSP) and OpenMax baselines, and the gap widens on the harder datasets.
| Dataset | Softmax (MSP) | OpenMax | OpenGAN |
|---|---|---|---|
| MNIST | 97.7 | 98.1 | 99.9 |
| SVHN | 88.6 | 89.4 | 98.8 |
| CIFAR-10 | 75.7 | 81.1 | 97.3 |
| TinyImageNet | 57.7 | 57.6 | 90.7 |
On the hardest benchmark, TinyImageNet, the baselines sit near coin-flip AUROC (~57%) while OpenGAN reaches ~91%. The same feature-space discriminator also transfers to open-set semantic segmentation, flagging pixels that belong to no known class.
These are literature results reported by the OpenGAN authors, reproduced here to explain the method — not experiments run for this coursework.
- Closed-set classifiers cannot signal “unknown”; softmax forces a wrong, confident label.
- Open-set recognition adds a rejection option — being right about known classes and right to refuse on genuinely new ones.
- Methods run from simple thresholds, through EVT / OpenMax, to deep metric and generative models; a stronger closed-set classifier is usually a stronger open-set detector.
- OpenGAN wins by moving the fight into feature space and selecting the model on a small held-out open set — turning an unstable idea into a state-of-the-art detector.
.
├── README.md this report — the guided tour
├── docs/
│ ├── 01-open-set-recognition.md foundations, taxonomy, every method in detail
│ └── 02-gans-and-opengan.md GAN fundamentals, minimax, training, OpenGAN in full
├── figures/
│ └── opengan-auroc.png AUROC comparison chart (redrawn from the OpenGAN paper)
├── slides/
│ └── open-set-recognition.pptx the original seminar deck
├── NOTICE third-party attributions (papers and figures)
└── LICENSE MIT
Researched and presented by Wuxi Bao and Alimuzzaman Bhuiyan (student IDs 294496 / 254736) for the Research Methodologies course, Electronic Media in Business and Commerce, Wrocław University of Science and Technology, 2026.
The report synthesises the team’s two seminar presentations into one document. All papers,
datasets, and figures that informed it are credited in the references and in
NOTICE.
- W. J. Scheirer, A. Rocha, A. Sapkota, T. E. Boult. Toward Open Set Recognition. IEEE TPAMI, 2013.
- W. J. Scheirer, L. P. Jain, T. E. Boult. Probability Models for Open Set Recognition. IEEE TPAMI, 36(11):2317–2324, 2014.
- A. Bendale, T. E. Boult. Towards Open Set Deep Networks (OpenMax). CVPR, 2016, pp. 1563–1572.
- C. Geng, S. Huang, S. Chen. Recent Advances in Open Set Recognition: A Survey. IEEE TPAMI, 43(10):3614–3631, 2020.
- S. Vaze, K. Han, A. Vedaldi, A. Zisserman. Open-Set Recognition: A Good Closed-Set Classifier is All You Need. ICLR, 2022. arXiv:2110.06207
- I. Goodfellow et al. Generative Adversarial Nets. NeurIPS, 2014. arXiv:1406.2661
- M. Arjovsky, S. Chintala, L. Bottou. Wasserstein GAN. ICML, 2017. arXiv:1701.07875
- S. Kong, D. Ramanan. OpenGAN: Open-Set Recognition via Open Data Generation. ICCV, 2021 (Best Paper Honorable Mention). arXiv:2104.02939 · project page
- Gao Fei, Yang Liu, Li Hui. A Survey on Open Set Recognition. Journal of Nanjing University, 58(1):115–134, 2022.
Released under the MIT License — open source and free to use. The papers, datasets,
and figures discussed remain the property of their respective authors and are cited above and in
NOTICE; the reported benchmark numbers are the work of the OpenGAN authors.