Semantic Correspondence:
Unified Benchmarking and a Strong Baseline

TPAMI 2025

Kaiyan Zhang, Xinghui Li, Jingyi Lu, Kai Han
Visual AI Lab, The University of Hong Kong

Paper Icon Paper arXiv Icon arXiv Code Icon Code Code Icon Paper List Bibtex Icon BibTeX
Qualitative Result GIF 1 Qualitative Result GIF 2 Qualitative Result GIF 3 Qualitative Result GIF 4 Qualitative Result GIF 5 Qualitative Result GIF 6

This paper presents the first extensive survey of semantic correspondence methods. We first propose a taxonomy to classify existing methods based on the type of their method designs. These methods are then categorized accordingly, and we provide a detailed analysis of each approach. Furthermore, we aggregate and summarize the results of methods in literature across various benchmarks into a unified comparative table, with detailed configurations to highlight performance variations. Additionally, we thoroughly conduct controlled experiments to analyse the effectiveness of the components of different methods. Finally, we propose a simple yet effective baseline that achieves sota performance on multiple benchmarks, providing a solid foundation for future research in this field. We hope this survey serves as a comprehensive reference and consolidated baseline for future development.

Taxonomy

Taxonomy

This taxonomy provides a comprehensive overview of semantic correspondence methods to enhance feature quality, matching performance, or training strategies.

A Strong Baseline

We propose a simple yet effective baseline that achieves state-of-the-art performance on multiple benchmarks, offering a strong and solid foundation for future research in the field.

Methods Backbone Reso. SPair-71k PF-PASCAL PF-WILLOW
PCK @ αbbox PCK @ αimg PCK @ αbbox-kp
0.05 0.1 0.15 0.05 0.1 0.15 0.05 0.1 0.15
SimSC-iBOT iBOT 256 43.0 63.5 - 88.4 95.6 97.3 44.9 71.4 84.5
DHF SD+DINOv2 Ori - 64.6 - - 86.7 - - - -
LPMFlow ViT-B/16 256 46.7 65.6 - 82.4 94.3 97.2 - 81.0 -
SD-DINO SD+DINOv2 960, 840 - 74.6 - 80.9 93.6 96.9 - - -
SD4Match SD+DINOv2 768 59.5 75.5 - 84.4 95.2 97.5 52.1 80.4 91.2
GeoAware-SC SD+DINOv2 960, 840 72.6 82.9 - 85.5 95.1 97.4 - - -
Ours(DINOv2) DINOv2 840 73.3 85.1 89.3 87.6 95.8 98.2 48.8 73.7 85.8
Ours(DINOv2+ResNet) DINOv2 840 72.7 85.2 89.3 80.3 90.8 94.9 46.6 74.1 87.1
Ours(DINOv2+NC) DINOv2 840 70.2 85.2 89.6 80.6 91.5 95.0 39.8 62.3 74.9
Ours(DINOv3) DINOv3 960 81.3 90.6 93.5 89.7 96.8 98.5 51.5 78.1 89.1

Qualitative Results

cat cat cat cat cat cat cat cat cat

BibTeX

    @article{zhang2025semantic,
      title={Semantic Correspondence: Unified Benchmarking and a Strong Baseline}, 
      author={Kaiyan Zhang and Xinghui Li and Jingyi Lu and Kai Han},
      journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
      year={2025},
      doi={10.1109/TPAMI.2025.3640429}
      }