mAP50-95 explained: how detection accuracy is measured

IoU: does the box fit?

Intersection over union divides the area two boxes share by the area they cover together. 1.0 is a perfect fit; 0 means they do not touch. A prediction counts as correct only if its IoU with a real object is above a threshold.

Precision and recall

• Precision: of the boxes the model drew, how many were right?
• Recall: of the objects really there, how many did the model find?

Lowering the confidence threshold finds more objects (recall goes up) but lets more mistakes through (precision goes down). One threshold gives one point on that trade-off.

AP: every threshold at once

Average precision sweeps the confidence from high to low and measures the area under the precision-recall curve. It scores the model at every threshold, so it does not depend on the one you happen to pick. mAP is the mean of AP over all classes.

mAP50 and mAP50-95

• mAP50 counts a box as correct at IoU 0.5 or more. Loose boxes still pass.
• mAP50-95 repeats the measurement at ten IoU thresholds, 0.50 to 0.95 in steps of 0.05, and averages them. A model only scores high if its boxes are tight.

When people say "AP" for COCO, they mean mAP50-95. It is the number in fotonet's tables.

How fotonet measures

We score every model with pycocotools, the official COCO evaluation code, on all 5,000 images of COCO val2017, at a confidence of 0.01 so the whole precision-recall curve is counted. fotonetn scores 32.1 AP. You can reproduce it:

fotonet val model=fotonetn data=coco.yaml

Reading the number

• Compare models only on the same dataset and protocol.
• COCO mAP tells you how a model does on COCO's 80 classes. On your own data, measure again: that is the number that matters for your project.
• COCO also reports AP for small, medium and large objects, which shows where a model struggles.