A Low-Shot Object Counting Network With Iterative Prototype Adaptation
IEEE/CVF International Conference on Computer Vision (ICCV) 2023
Nikola Djukic, Alan Lukezic, Vitjan Zavrtanik, Matej Kristan
We consider low-shot counting of arbitrary semantic categories in the image using only few annotated exemplars (few-shot) or no exemplars (no-shot). The standard few-shot pipeline follows extraction of appearance queries from exemplars and matching them with image features to infer the object counts. Existing methods extract queries by feature pooling which neglects the shape information (e.g., size and aspect) and leads to a reduced object localization accuracy and count estimates. We propose a Low-shot Object Counting network with iterative prototype Adaptation (LOCA). Our main contribution is the new object prototype extraction module, which iteratively fuses the exemplar shape and appearance information with image features. The module is easily adapted to zero-shot scenarios, enabling LOCA to cover the entire spectrum of low-shot counting problems. LOCA outperforms all recent state-of-the-art methods on FSC147 benchmark by 20-30% in RMSE on one-shot and few-shot and achieves state-of-the-art on zero-shot scenarios, while demonstrating better generalization capabilities.
Paper: ICCV2023
Source code: GitHub
BibTex citation:
@InProceedings{Dukic_2023_ICCV,
author = {{\DJ}uki'c, Nikola and Luke\v{z}i\v{c}, Alan and Zavrtanik, Vitjan and Kristan, Matej},
title = {A Low-Shot Object Counting Network With Iterative Prototype Adaptation},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2023},
pages = {18872-18881}
}
DAVE – A Detect-and-Verify Paradigm for Low-Shot Counting
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024
Jer Pelhan, Alan Lukežič, Vitjan Zavrtanik, Matej Kristan
Low-shot counters estimate the number of objects corresponding to a selected category, based on only few or no exemplars annotated in the image. The current state-of-the-art estimates the total counts as the sum over the object location density map, but does not provide individual object locations and sizes, which are crucial for many applications. This is addressed by detection-based counters, which, however fall behind in the total count accuracy. Furthermore, both approaches tend to overestimate the counts in the presence of other object classes due to many false positives. We propose DAVE, a low-shot counter based on a detect-and-verify paradigm, that avoids the aforementioned issues by first generating a high-recall detection set and then verifying the detections to identify and remove the outliers. This jointly increases the recall and precision, leading to accurate counts. DAVE outperforms the top density-based counters by $\sim$20% in the total count MAE, it outperforms the most recent detection-based counter by $\sim$20% in detection quality and sets a new state-of-the-art in zero-shot as well as text-prompt-based counting.
Paper: CVPR2024
Source code: GitHub
BibTex citation:
@InProceedings{Pelhan_2024_CVPR,
author = {Jer Pelhan and Alan Lukežič and Vitjan Zavrtanik and Matej Kristan},
title = {DAVE – A Detect-and-Verify Paradigm for Low-Shot Counting},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2024}
}
A Novel Unified Architecture for Low-Shot Counting by Detection and Segmentation
Advances in Neural Information Processing Systems (NeurIPS) 2024
Jer Pelhan, Alan Lukežič, Vitjan Zavrtanik, Matej Kristan
Low-shot object counters estimate the number of objects in an image using few or no annotated exemplars. Objects are localized by matching them to prototypes, which are constructed by unsupervised image-wide object appearance aggregation.
Due to potentially diverse object appearances,
the existing approaches often lead to overgeneralization and false positive detections.
Furthermore, the best-performing methods train object localization by a surrogate loss, that predicts a unit Gaussian at each object center. This loss is sensitive to annotation error, hyperparameters and does not directly optimize the detection task, leading to suboptimal counts.
We introduce GeCo, a novel low-shot counter that achieves accurate object detection, segmentation, and count estimation in a unified architecture.
GeCo robustly generalizes the prototypes across objects appearances through a novel dense object query formulation.
In addition, a novel counting loss is proposed, that directly optimizes the detection task and avoids the issues of the standard surrogate loss.
GeCo surpasses the leading few-shot detection-based counters by 25% in the total count MAE, achieves superior detection accuracy and
sets a new solid state-of-the-art result across all low-shot counting setups.
Paper: NeurIPS 2024
Source code: GitHub
BibTex citation:
@InProceedings{Pelhan_2024_NeurIPS,
author = {Jer Pelhan and Alan Lukežič and Vitjan Zavrtanik and Matej Kristan},
title = {A Novel Unified Architecture for Low-Shot Counting by Detection and Segmentation},
booktitle = {Advances in Neural Information Processing Systems},
volume={37},
year={2024},
publisher={Curran Associates, Inc.}
}