NMS-free detection: what NMS is, and why fotonet has none

Why detectors produce duplicates

A detector looks at thousands of positions in the image. Several neighbouring positions usually see the same object, and each of them predicts a box for it. The raw output for one person can be ten boxes, all slightly different.

What NMS does

Non-Maximum Suppression sorts the boxes by confidence, keeps the best one, and deletes every other box that overlaps it by more than a threshold, usually measured as IoU (intersection over union). Then it repeats with the next box that is left.

It works, but it has costs:

• An extra step after the network, on the CPU or as a custom GPU kernel. Its time grows with the number of boxes.
• A setting to tune. Set the IoU threshold too low and two people standing close together become one. Set it too high and duplicates survive.
• Harder deployment. Exported models need the NMS step rebuilt in every runtime.

How fotonet avoids it

During training, fotonet matches each real object to exactly one prediction, and teaches every other prediction for that object to stay quiet. This is called one-to-one assignment. A second helper head learns from many matches per object, which gives the network more examples to learn from; it is removed before export, so it costs nothing when you run the model.

The result: what comes out of the network is the answer. After it there is only a sort and a confidence cut.

results = model.predict("street.jpg", conf=0.25)   # no iou= setting: there is no NMS

What it means for you

• One setting instead of two: you choose a confidence and nothing else. See choosing a confidence threshold.
• The same output in every runtime: PyTorch, ONNX, TensorRT and the browser all return the same boxes, with nothing to reimplement.
• Predictable speed: no post-processing step that slows down in busy scenes.