Multi-camera BEV perception: Lift-Splat with DINOv2
I built a Lift-Splat bird's-eye-view network with a DINOv2 foundation-model backbone, module by module, and verified its six-camera geometry end to end on real nuScenes calibration.
- Role
- Architecture and geometry implementation
- Setting
- Architecture and camera geometry verified on real calibration; no trained-model metrics reported
- Stack
- PyTorch
- DINOv2 ViT-S/14
- Lift-Splat-Shoot
- nuScenes devkit

Key results
- frustum points per sample
125,952
frustum points per sample
6 × 41 depth bins × 16 × 32
- BEV cell
0.5 m
BEV cell
200 × 200 grid, 100 m × 100 m
01
Problem & constraints
Camera-only BEV perception is attractive because cameras are cheap, but lifting 2D features into 3D requires per-pixel depth reasoning and an exact camera rig model.
02
My contribution
Architecture and geometry implementation
- Implemented the Lift, Geometry and Splat modules with shape tests.
- Replaced the image encoder with DINOv2 ViT-S/14 patch tokens.
- Verified the geometry with the real nuScenes calibration of all six cameras.
03
System architecture
01 · Input
data
6 cameras
3×224×448
data
Intrinsics + extrinsics
02 · Backbone
model
DINOv2 ViT-S/14
384×16×32 per camera
03 · Lift
process
41 depth bins, 4–45 m
softmax ⊗ 64-d context
04 · Splat
process
scatter_add into 200×200
0.5 m cells
05 · Heads
output
Occupancy (3 classes)
output
Detection heatmap
04
Results

05
Failures, limits & next steps
Next steps · what would prove it
- Extension: rasterise drivable area, lanes and 3D box centres into BEV targets.
- Extension: train with DINOv2 frozen, then unfreeze the last blocks.
- What would prove it: BEV IoU and nuScenes mAP / NDS on v1.0-trainval.
06
Code & references
- Philion & Fidler, Lift, Splat, Shoot, ECCV 2020.
- Oquab et al., DINOv2, 2023.
- Caesar et al., nuScenes, CVPR 2020 (CC BY-NC-SA 4.0).
Related projects
Next: Infrared trackingSensor Fusion · Autonomous Systems
Camera–LiDAR fusion on raw autonomous-vehicle data
I built LiDAR-to-camera projection for a 120° f-θ lens from raw NVIDIA AV files and gave every segmented object a metric range, over two 605-frame drives.
2 × 605frames fused end to end
Deep Learning · Autonomous Systems
Pedestrian crossing intent with a vision-language model
I fine-tuned Qwen2-VL-2B with QLoRA to 100% valid JSON, then showed on held-out clips that its intent decision stayed at the majority baseline, traced why, and measured what a VLM does to a real-time tracker.
100%valid JSON after fine-tuning
Computer Vision · Autonomous Systems
Aerial multi-object tracking on VisDrone
I wrote a SORT tracker from scratch, ran it against BoT-SORT with camera-motion compensation on all 17 VisDrone test sequences, and caught an ID-leak bug through the output analysis.
201,931boxes tracked
Contact
Want to go deeper on this project? I'm happy to walk through it.
Open to full-time engineering roles. If the problem is hard and the data is messy, I'm interested.