Indoor VPR with DINOv2

Visual place recognition using self-supervised learning

Evaluation of DINOv2 foundation model for indoor visual place recognition, comparing zero-shot retrieval performance against VGGT geometric re-ranking on the NYC Indoor VPR dataset.

Overview

This project explored using DINOv2, a self-supervised vision transformer, for indoor place recognition tasks. The goal was to evaluate whether modern foundation models can perform zero-shot visual place recognition without task-specific fine-tuning.

Tech Stack

  • Python - Implementation language
  • DINOv2 - Vision foundation model (Meta AI)
  • PyTorch - Deep learning framework
  • NYC Indoor VPR Dataset - Benchmark dataset

Methodology

Zero-Shot Retrieval

  • Extracted DINOv2 embeddings from indoor images
  • Computed cosine similarity for place matching
  • Evaluated top-k retrieval accuracy

Geometric Re-ranking (VGGT)

  • Applied VGGT spatial verification
  • Analyzed improvement over pure embedding similarity
  • Diagnosed limited improvement due to top_k_rerank=10 constraint

Attempted Fine-tuning

  • Explored training optimization strategies
  • Addressed slow training performance issues

Results

  • DINOv2 demonstrated strong zero-shot retrieval capabilities
  • VGGT re-ranking showed limited improvement due to parameter constraints
  • Results archived and documented for reproducibility

Collaborator

Eric Kim, Henry Chen

What I Learned

  • Working with large vision foundation models
  • Visual place recognition evaluation metrics
  • Geometric verification techniques
  • Optimization strategies for transformer fine-tuning
  • Research experimentation and documentation
<!--
  See https://www.debugbear.com/blog/responsive-images#w-descriptors-and-the-sizes-attribute and
  https://developer.mozilla.org/en-US/docs/Learn/HTML/Multimedia_and_embedding/Responsive_images for info on defining 'sizes' for responsive images
-->

  
    <source
      class="responsive-img-srcset"
      
        srcset="/assets/img/vpr-results-480.webp 480w,/assets/img/vpr-results-800.webp 800w,/assets/img/vpr-results-1400.webp 1400w,"
        type="image/webp"
      
      
        sizes="95vw"
      
    >
  

<img
  src="/assets/img/vpr-results.png"
  
    class="img-fluid rounded z-depth-1"
  
  
    width="100%"
  
  
    height="auto"
  
  
  
  
    title="Retrieval results"
  
  
  
    loading="lazy"
  
  onerror="this.onerror=null; $('.responsive-img-srcset').remove();"
>

</picture>

</figure>

</div>

</div> –>