Exporting a detector: ONNX, TensorRT, TorchScript and CoreML
Training happens in PyTorch. Shipping usually happens somewhere else. Export turns your model into a file that runs there.
One call
from fotonet import Fotonet
model = Fotonet("fotonetn")
model.export(format="onnx") # or "tensorrt", "torchscript", "coreml"Which format
| Format | Runs on | Use it for |
|---|---|---|
| ONNX | ONNX Runtime on CPU, GPU and the browser, plus many other engines | The default: portable and well supported |
| TensorRT | NVIDIA GPUs and Jetson boards | The lowest latency on NVIDIA hardware (built with trtexec on the target) |
| TorchScript | PyTorch without Python source, including C++ | Staying in the PyTorch world |
| CoreML | iPhone, iPad and Mac | Apple apps (experimental: check accuracy on the device) |
Speed
On an RTX 4060, fotonetn's TensorRT FP16 engine runs a forward pass in 1.22 ms. FP16 (half precision) roughly halves memory and is much faster on modern GPUs, usually with no visible change in boxes. Measure on your own device: speed depends heavily on the hardware.
No NMS to rebuild
Detectors with NMS need that step reimplemented, or bolted on as a plugin, in every runtime. fotonet's network output is already final, so the exported model returns the same boxes as Python with nothing added. Why there is no NMS.
Check the export
fotonet compares an ONNX export with PyTorch on the same inputs before you ship it. For TensorRT, which is built on your target device, check speed and accuracy there with your own images.