Skip to content

Aerial multi-object tracking on VisDrone

I wrote a SORT tracker from scratch, ran it against BoT-SORT with camera-motion compensation on all 17 VisDrone test sequences, and caught an ID-leak bug through the output analysis.

Role
Tracker implementation, batch pipeline and output analysis
Setting
Offline tracking on recorded drone video
Stack
  • RT-DETR-L
  • Kalman filter
  • Hungarian assignment
  • BoT-SORT
  • motmetrics
  • Ultralytics
BoT-SORT with camera-motion compensation on a dense-traffic sequence.

Key results

test-dev sequences

17

test-dev sequences

6,635 frames

objects per frame, peak sequence mean

67

objects per frame, peak sequence mean

crowded intersection

leaked IDs found and fixed

32

leaked IDs found and fixed

tracker state reused across sequences

01

Problem & constraints

From a drone, people and vehicles are a few pixels tall, crowded, and the camera itself moves. A tracker has to keep identities through detector flicker and ego-motion.

02

My contribution

Tracker implementation, batch pipeline and output analysis

  • Implemented SORT from scratch: 7-state constant-velocity Kalman filter, IoU cost, Hungarian assignment, hit/age track management.
  • Ran RT-DETR-L + BoT-SORT over every sequence and wrote MOT-format output.
  • Built a motmetrics MOTA / IDF1 / ID-switch harness.
  • Analysed track statistics and found 32 identities leaking between sequences; fixed the per-sequence reset.

03

System architecture

  1. 01 · Input

    • data

      VisDrone frame

  2. 02 · Detect

    • model

      RT-DETR-L

      COCO weights, conf 0.3, 5 classes

  3. 03 · Track

    • process

      A · SORT (mine)

      Kalman · IoU · Hungarian

    • process

      B · BoT-SORT

      camera-motion compensation

  4. 04 · Output

    • output

      MOT txt + video

    • process

      motmetrics harness

04

Evaluation

Dataset
VisDrone2019-MOT test-dev, 17 sequences
Metrics
MOTA / IDF1 not yet reported: ground-truth files were not kept with the outputs; the harness is ready
Objects tracked per frame (mean, per sequence)
Objects tracked per frame (mean, per sequence)
uav0000161_0000066.8
uav0000297_0276149.8
uav0000297_0000041.0
uav0000188_0000037.9
uav0000009_0335836.5
uav0000120_0477536.2
uav0000306_0023036.0
uav0000355_0000133.6
uav0000119_0230130.2
uav0000077_0072025.9
uav0000073_0446424.8
uav0000201_0000021.8
uav0000249_0268820.0
uav0000073_0060018.3
uav0000249_0000117.9
uav0000088_0029017.2
uav0000370_000016.2

Median track lifetime is 7–37 frames, and in most sequences 40–55% of IDs last under 10 frames: the COCO-pretrained detector flickers on tiny aerial objects.

05

Results

BoT-SORT: Same frame and detections tracked by the from-scratch SORT and by BoT-SORTSORT (mine)BoT-SORT
Same frame, same detections: the from-scratch SORT against BoT-SORT.
Night sequence, BoT-SORT.

06

Failures, limits & next steps

Fragmented identities

Each detector gap longer than the track buffer starts a new ID; tiny-object flicker is the main source of churn.

A bug only the statistics revealed

One model instance with persist=True was reused for all sequences without a reset, so 32 track IDs leaked into the next sequence's first frames. The batch script now resets tracker state per sequence.

Next steps · what would prove it

  • MOTA / IDF1 / HOTA on class-filtered ground truth, for both trackers on identical detections.
  • Fine-tune the detector on VisDrone-DET for small objects.
  • Tune track_buffer and new_track_thresh against IDF1.

07

Code & references

  • Zhu et al., Detection and Tracking Meet Drones Challenge, IEEE TPAMI 2021.
  • Zhao et al., DETRs Beat YOLOs on Real-time Object Detection, CVPR 2024.

DataVisDrone is for academic use; frames shown are tracker outputs for illustration.

Contact

Want to go deeper on this project? I'm happy to walk through it.

Open to full-time engineering roles. If the problem is hard and the data is messy, I'm interested.