Skip to content

Industrial defect segmentation with a C++ TensorRT app

I trained YOLO11-seg on MVTec AD to 0.82 mask mAP50, wrote the C++ TensorRT app that serves it, and found that rebalancing VisA's data lifted mask mAP50 from 0.25 to 0.41 where a bigger model did nothing.

Role
Dataset engineering, training experiments, evaluation and the C++ inference application
Setting
Offline evaluation on public benchmark images · C++ application run on a workstation GPU
Stack
  • Ultralytics YOLO11-seg
  • TensorRT
  • C++17
  • CUDA
  • OpenCV
  • ONNX
C++ output: Bent cable wire: ground-truth outline versus the C++ application's maskGround truthC++ output
Held-out validation image through the C++/TensorRT app: mask IoU 0.79 against the ground truth.

Key results

mask mAP50 · MVTec AD

0.82

mask mAP50 · MVTec AD

15 categories, YOLO11l-seg 640 px

VisA mask mAP50

0.25 → 0.41

VisA mask mAP50

training-set rebalancing only

TensorRT YOLO11x-seg 1024²

6.7 ms

TensorRT YOLO11x-seg 1024²

mean, p99 7.2 ms (trtexec)

false alarms on good parts

2.1%

false alarms on good parts

467 unseen defect-free parts, conf 0.5

01

Problem & constraints

Visual inspection needs a mask, not just a box: the defect's extent decides whether a part is scrap. The model must work on parts it has never seen and must not reject good ones.

Does a pixel-level detector trained on a few hundred defect images hold up on held-out parts, and what actually moves its accuracy?

02

My contribution

Dataset engineering, training experiments, evaluation and the C++ inference application

  • Converted MVTec AD masks to YOLO polygon labels with an 80/20 split per defect type.
  • Ran the VisA rebalancing experiment and the 2.2×-larger-model comparison.
  • Measured false alarms on 467 defect-free parts the model never saw in training.
  • Wrote the C++ app: pinned CHW buffers, async H2D/enqueueV3/D2H on one stream, NMS and mask reconstruction from prototypes.
  • Scored the app's masks case by case against ground truth.

03

System architecture

  1. 01 · Data

    • data

      MVTec AD

      masks → polygons, 977 / 281

    • data

      VisA

      background = 10% of defects

  2. 02 · Train

    • model

      YOLO11l-seg 640

    • model

      YOLO11x-seg 1024

      evolved hyper-params

  3. 03 · Export

    • process

      ONNX → TensorRT

      engine built with trtexec

  4. 04 · C++ app

    • system

      Pinned buffers, CUDA stream

      enqueueV3

    • process

      NMS + mask prototypes

      32×256×256

  5. 05 · Output

    • output

      Box + mask + category

04

Method & key decisions

Data balance over model size

The two VisA runs have identical settings; only the share of background images changed (kept at 10% of the defect count). That removed a 20-epoch stall at zero and raised mask mAP50 by 16 points. A 2.2× larger model at 4× the pixels gave no gain on MVTec.

Mask decoding in C++

The app decodes 21,504 proposals × (4 box + 15 class + 32 mask coefficients). Each instance mask is sigmoid(coefficients · prototypes), upsampled and cropped to its box.

Fail loudly on stale engines

A TensorRT engine only loads with the version that built it. After an upgrade the app crashed on the old engine, so it now checks the load and says why.

05

Evaluation

Dataset
MVTec AD (15 categories), VisA
Validation
281 MVTec images; the same split selected best.pt, so MVTec numbers are slightly optimistic
C++ demo
1 random validation image per category (seed 0) + 2 random defect-free parts
Hardware
NVIDIA RTX 5080 (16 GB), CUDA 13, TensorRT 11
Mask AP50 per MVTec category (YOLO11l-seg, 640 px)
Mask AP50 per MVTec category (YOLO11l-seg, 640 px)
Tile0.99
Zipper0.97
Bottle0.97
Wood0.88
Metal nut0.86
Pill0.84
Grid0.82
Capsule0.81
Hazelnut0.80
Carpet0.78
Toothbrush0.75
Leather0.73
Screw0.72
Transistor0.69
Cable0.67

Objects are harder than textures. Each category has only 6–31 validation images, so single-category differences are noisy.

Training runs
RunBox mAP50Mask mAP50Mask mAP50-95
MVTec · YOLO11l-seg · 640 px0.8300.8190.482
MVTec · YOLO11x-seg · 1024 px · evolved0.8310.8230.473
VisA · YOLO11l-seg · unbalanced0.2810.2500.118
VisA · YOLO11l-seg · balanced0.4570.4100.214

06

Try it

17 held-out cases · 11 of 15 defects located at IoU ≥ 0.5

C++ output: Cable, bent wireGround truthC++ output
Part
Cable
Defect
bent wire
Detection
cable 0.91
Mask IoU
0.79

Verdict: located

Real
Outputs saved from the C++/TensorRT application on 17 held-out MVTec AD images, with mask IoU scored against ground truth.
Selection
One random validation image per category (seed 0) plus two random defect-free parts. Failures are included.
Runs where
Pre-recorded. The engine needs an NVIDIA GPU, so nothing is inferred in your browser.

07

Failures, limits & next steps

Right class, wrong extent

On the transistor's bent lead the mask spreads over the whole component (IoU 0.10). Shape and position defects are harder to delineate than surface marks.

C++ output: Transistor with a bent lead: the predicted mask covers the whole transistorGround truthC++ output

Missed: a 0.1% defect

The damaged screw thread covers about 1,200 pixels, 0.1% of the image, and nothing clears the app's 0.5 confidence threshold.

C++ output: Screw with a damaged thread: no detectionGround truthC++ output

False alarm on a good part

A small surface speck on a defect-free hazelnut is flagged; the other defect-free part, a screw, stays clean.

Limitations

  • No defect-free images in training; image-level AUROC and pixel-level PRO (the MVTec benchmark metrics) are not reported yet.
  • Hyperparameter evolution finished 2 of 20 iterations on a 16 GB GPU.
  • The engine is built from an FP32 graph; C++ end-to-end timing including pre/post-processing is not yet measured.

Next steps · what would prove it

  • Train with defect-free backgrounds and report AUROC / PRO.
  • FP16 export and end-to-end C++ timing.
  • Smaller proxy model for hyperparameter search.

08

Code & references

  • Bergmann et al., MVTec AD, CVPR 2019 (CC BY-NC-SA 4.0).
  • Zou et al., VisA, ECCV 2022 (CC BY 4.0).

DataImages shown are model outputs on MVTec AD images, for non-commercial illustration with attribution.

Contact

Want to go deeper on this project? I'm happy to walk through it.

Open to full-time engineering roles. If the problem is hard and the data is messy, I'm interested.