What is object detection?

Boxes, classes and scores

Give a detector a photo and it gives back a list. Each entry is one object: a box (four numbers for its corners), a class such as person or car, and a confidence between 0 and 1 that says how sure the model is.

That list is what makes detection useful. You can count objects, follow them from frame to frame, crop them out, or trigger something when one appears in a part of the picture.

How it differs from classification and segmentation

• Classification gives one label for the whole image: this is a photo of a dog.
• Detection finds every dog, and every other object it knows, each with its own box.
• Segmentation goes further and marks the exact pixels of each object.

Detection is the middle ground most products need: precise enough to locate things, cheap enough to run on every frame of a video.

How a detector works

A detector is a neural network. A backbone turns the image into features, a neck mixes features from different scales so small and large objects are both visible, and a head predicts boxes, classes and scores at many places in the image at once.

Older detectors predict many overlapping boxes for each object and clean them up afterwards with a step called NMS. fotonet is trained to predict one box per object, so there is nothing to clean up. Read why that matters.

Where it is used

• Counting people, vehicles or products on a shelf.
• Quality control on a production line.
• Wildlife monitoring from camera traps.
• Robotics and automation, where a machine needs to know where things are.
• Searching and organising large photo collections.

Try it

The fastest way to see detection is the live demo: it runs fotonetn in your browser on your own photos. To run it in Python:

pip install fotonet

from fotonet import Fotonet
model = Fotonet("fotonetn")
for box in model.predict("photo.jpg")[0].boxes:
    print(box.cls, box.conf, box.xyxy)