Aerial multi-object tracking on VisDrone
I wrote a SORT tracker from scratch, ran it against BoT-SORT with camera-motion compensation on all 17 VisDrone test sequences, and caught an ID-leak bug through the output analysis.
- Role
- Tracker implementation, batch pipeline and output analysis
- Setting
- Offline tracking on recorded drone video
- Stack
- RT-DETR-L
- Kalman filter
- Hungarian assignment
- BoT-SORT
- motmetrics
- Ultralytics
Key results
- test-dev sequences
17
test-dev sequences
6,635 frames
- objects per frame, peak sequence mean
67
objects per frame, peak sequence mean
crowded intersection
- leaked IDs found and fixed
32
leaked IDs found and fixed
tracker state reused across sequences
01
Problem & constraints
From a drone, people and vehicles are a few pixels tall, crowded, and the camera itself moves. A tracker has to keep identities through detector flicker and ego-motion.
02
My contribution
Tracker implementation, batch pipeline and output analysis
- Implemented SORT from scratch: 7-state constant-velocity Kalman filter, IoU cost, Hungarian assignment, hit/age track management.
- Ran RT-DETR-L + BoT-SORT over every sequence and wrote MOT-format output.
- Built a motmetrics MOTA / IDF1 / ID-switch harness.
- Analysed track statistics and found 32 identities leaking between sequences; fixed the per-sequence reset.
03
System architecture
01 · Input
data
VisDrone frame
02 · Detect
model
RT-DETR-L
COCO weights, conf 0.3, 5 classes
03 · Track
process
A · SORT (mine)
Kalman · IoU · Hungarian
process
B · BoT-SORT
camera-motion compensation
04 · Output
output
MOT txt + video
process
motmetrics harness
04
Evaluation
- Dataset
- VisDrone2019-MOT test-dev, 17 sequences
- Metrics
- MOTA / IDF1 not yet reported: ground-truth files were not kept with the outputs; the harness is ready
| uav0000161_00000 | 66.8 |
|---|---|
| uav0000297_02761 | 49.8 |
| uav0000297_00000 | 41.0 |
| uav0000188_00000 | 37.9 |
| uav0000009_03358 | 36.5 |
| uav0000120_04775 | 36.2 |
| uav0000306_00230 | 36.0 |
| uav0000355_00001 | 33.6 |
| uav0000119_02301 | 30.2 |
| uav0000077_00720 | 25.9 |
| uav0000073_04464 | 24.8 |
| uav0000201_00000 | 21.8 |
| uav0000249_02688 | 20.0 |
| uav0000073_00600 | 18.3 |
| uav0000249_00001 | 17.9 |
| uav0000088_00290 | 17.2 |
| uav0000370_00001 | 6.2 |
Median track lifetime is 7–37 frames, and in most sequences 40–55% of IDs last under 10 frames: the COCO-pretrained detector flickers on tiny aerial objects.
05
Results
SORT (mine)BoT-SORT06
Failures, limits & next steps
Fragmented identities
Each detector gap longer than the track buffer starts a new ID; tiny-object flicker is the main source of churn.
A bug only the statistics revealed
One model instance with persist=True was reused for all sequences without a reset, so 32 track IDs leaked into the next sequence's first frames. The batch script now resets tracker state per sequence.
Next steps · what would prove it
- MOTA / IDF1 / HOTA on class-filtered ground truth, for both trackers on identical detections.
- Fine-tune the detector on VisDrone-DET for small objects.
- Tune track_buffer and new_track_thresh against IDF1.
07
Code & references
- Zhu et al., Detection and Tracking Meet Drones Challenge, IEEE TPAMI 2021.
- Zhao et al., DETRs Beat YOLOs on Real-time Object Detection, CVPR 2024.
DataVisDrone is for academic use; frames shown are tracker outputs for illustration.
Related projects
Next: Pedestrian intent VLMComputer Vision · Edge Deployment
Infrared small-target detection & tracking
I built a detector and tracker for 3–7 pixel aerial targets in infrared video, then ported it from Python to C++/CUDA/TensorRT: 1.7 ms per frame, 9.3× faster, with tracks identical to the prototype.
1.7 msper 1280×1024 frame, enhance → detect → track
Sensor Fusion · Autonomous Systems
Camera–LiDAR fusion on raw autonomous-vehicle data
I built LiDAR-to-camera projection for a 120° f-θ lens from raw NVIDIA AV files and gave every segmented object a metric range, over two 605-frame drives.
2 × 605frames fused end to end
Deep Learning · Autonomous Systems
Pedestrian crossing intent with a vision-language model
I fine-tuned Qwen2-VL-2B with QLoRA to 100% valid JSON, then showed on held-out clips that its intent decision stayed at the majority baseline, traced why, and measured what a VLM does to a real-time tracker.
100%valid JSON after fine-tuning
Contact
Want to go deeper on this project? I'm happy to walk through it.
Open to full-time engineering roles. If the problem is hard and the data is messy, I'm interested.
