Real-time surgical tool detection, from dataset to device link
I validated a surgical-tool detector at 0.963 mAP50 over 5 folds, deployed it as a 0.87 ms TensorRT engine behind a Qt panel and a C++ serial relay, and measured an honest cross-dataset drop to 0.17.
- Role
- Training, validation, deployment and system integration
- Setting
- Research prototype · simulated microcontroller over a virtual serial link (no physical device)
- Stack
- YOLO11n
- TensorRT
- ONNX
- PySide6
- C++17
- POSIX termios
- socat

Key results
- mAP50 on held-out split
0.937
mAP50 on held-out split
YOLO11n, 60 epochs, 640 px
- 5-fold mean mAP50
0.963
5-fold mean mAP50
± 0.034
- TensorRT mean latency
0.87 ms
TensorRT mean latency
p99 1.03 ms, 3,451 runs, 640×640
- cross-dataset mAP50
0.167
cross-dataset mAP50
laparoscopic m2cai16, no retraining
01
Problem & constraints
A tool detector in an endoscopy suite has to run in real time and pass its state to other equipment. Its in-domain accuracy means little if it collapses on a different procedure.
How accurate and how fast is a small detector on endoscopic video, and does that accuracy transfer to a new surgical domain?
02
My contribution
Training, validation, deployment and system integration
- Converted Kvasir-Instrument to YOLO format and ran 5-fold cross-validation.
- Compared YOLO11n against a YOLOv8n baseline.
- Ran an unchanged cross-dataset test on 2,811 laparoscopic frames.
- Exported to ONNX/TensorRT and profiled latency over 3,451 runs.
- Built the PySide6 panel and a C++17 termios relay to a simulated microcontroller; wrote requirements and a test plan.
03
System architecture
01 · Data
data
Kvasir-Instrument
590 frames → 502 / 88
02 · Model
model
YOLO11n
60 epochs, 640 px
process
5-fold CV
03 · Deploy
process
ONNX → TensorRT
output
PySide6 live panel
04 · Device link
system
C++ status relay
termios 115200 8N1
05 · Device
system
Simulated MCU
socat virtual UART
04
Evaluation
- In-domain
- Kvasir-Instrument, 88-image held-out split + 5-fold CV
- Cross-domain
- m2cai16-tool-locations: 2,811 frames, 3,929 boxes, 7 tool classes merged into one
- Latency
- trtexec, FP32 engine (TensorRT 11 strongly typed), 640×640, RTX 5080
| Fold 1 | 0.993 |
|---|---|
| Fold 2 | 0.903 |
| Fold 3 | 0.979 |
| Fold 4 | 0.989 |
| Fold 5 | 0.948 |
| m2cai16 (laparoscopic) | 0.167 |
The in-domain score does not transfer to a new procedure; mixed-domain training is the next experiment.
| Model | Precision | Recall | mAP50 | mAP50-95 |
|---|---|---|---|---|
| YOLO11n (final) | 0.974 | 0.879 | 0.937 | 0.804 |
| YOLOv8n (baseline) | 0.912 | 0.923 | 0.929 | 0.818 |
| YOLO11n, 5-fold mean ± std | 0.966 ± 0.029 | 0.932 ± 0.051 | 0.963 ± 0.034 | 0.856 ± 0.039 |
05
Results



06
Failures, limits & next steps
Missed guidewire
A thin guidewire in an over-exposed view gets no detection at all.
Extra box on the lumen
The tool is found at 0.92, but a second box at 0.11 lands on the dark lumen.
Limitations
- Performance is in-domain only (cross-dataset mAP50 0.17).
- FP32 engine; the GUI still serves the PyTorch model.
- Camera-to-UART latency and the event log requirement are open in the test plan.
Next steps · what would prove it
- Train on Kvasir + m2cai16, test on a third procedure.
- FP16/INT8 engine served from the GUI.
- Measure camera → UART byte latency.
07
Code & references
- Jha et al., Kvasir-Instrument, MMM 2021.
- Jin et al., m2cai16-tool-locations, WACV 2018.
Related projects
Next: Aerial trackingComputer Vision · Deep Learning
Industrial defect segmentation with a C++ TensorRT app
I trained YOLO11-seg on MVTec AD to 0.82 mask mAP50, wrote the C++ TensorRT app that serves it, and found that rebalancing VisA's data lifted mask mAP50 from 0.25 to 0.41 where a bigger model did nothing.
Ground truthC++ output11 / 15held-out defects located by the C++ app
Computer Vision · Edge Deployment
Infrared small-target detection & tracking
I built a detector and tracker for 3–7 pixel aerial targets in infrared video, then ported it from Python to C++/CUDA/TensorRT: 1.7 ms per frame, 9.3× faster, with tracks identical to the prototype.
1.7 msper 1280×1024 frame, enhance → detect → track
Computer Vision · Deep Learning
Zero-shot wildfire smoke segmentation, scored against ground truth
I scored a Grounding DINO + SAM zero-shot pipeline against 744 annotated plumes, raised localisation from 36% to 51% through prompt design, and put the MobileSAM encoder on TensorRT at 4.3 ms.

Contact
Want to go deeper on this project? I'm happy to walk through it.
Open to full-time engineering roles. If the problem is hard and the data is messy, I'm interested.


