Choosing the Rescue Detector

The problem

• Licence. The rover's detector (yolo26n) and all our YOLO models are AGPL-3.0, which would force us to publish our own source if we ship them. We needed a permissive licence for both code and weights.
• It barely finds casualties. The stock yolo26n finds a lying person in only 13% of casualty frames at the rover's own setting, because COCO-trained detectors rarely see people lying on the floor.
• It runs on the CPU. The rover runs it through OpenCV on the CPU at 140 ms per frame.
• The NVIDIA option fell short. NVIDIA's TAO RT-DETR, fine-tuned on our data, stopped at 26% recall however long we trained it.

How we measured

Accuracy: recall on 22,060 held-out hospital frames from Isaac Sim (6,724 with a casualty). We set each model's threshold so it fires on no more than 0.5% of frames that show a standing person, then count how many casualties it still finds. The same rule applies to every model, so the numbers compare directly.

Speed: time per frame on the Jetson Orin Nano (15 W), TensorRT FP16, 95th percentile, measured idle and while Cosmos3-Edge is generating.

How we chose

1. Pushed the NVIDIA stack to its limit. 12 and 16 epochs of TAO RT-DETR gave no gain: 26% recall.
2. Picked permissive candidates. RF-DETR (sizes N to L, Apache-2.0) and D-FINE (Apache-2.0). We ruled out YOLOv8 (also AGPL) and RF-DETR XL and 2XL (non-permissive licence).
3. Trained both on the same data for 12 epochs, in parallel on the cluster's 8 GPUs. RF-DETR-M at 1280 px reached 42%, between our two AGPL YOLO models.
4. Timed them on the Jetson. RF-DETR-M at 1280 px took 135 ms, too slow. D-FINE-M took 49 ms at 37%.
5. Tried INT8. Under 1 point of recall lost, but 2 to 7% slower on the Orin. Dropped.
6. Trained at a smaller input instead. RF-DETR's sizes share one backbone, so input size, not model size, sets the speed. RF-DETR-S trained at 640 px gave 37% at 28 ms, the best balance.
7. Compared against the rover's current detector on the same frames: about 5 times more casualties found, with 4 times fewer alarms on standing people.
8. Added more classes (person, backpack, handbag, suitcase, dog, car, truck, box, and a catch-all object class for Cosmos3-Edge to name). Casualty recall stayed the same, 37%.

All results

The armA and armB rows were scored by the bench and timed on 15 September; the rest were scored with a replica of the bench harness.

What we still don't know

• Real footage. Every number comes from Isaac renders or COCO photos. Rover recordings with known casualties are the test that matters, and they don't exist yet.
• The shape rule. It was designed for detectors that also fire on standing people. For ours it mostly removes real casualties: dropping it raises RF-DETR-S from 37% to 44%. The bench owns this decision and must re-score the YOLO models the same way.
• Two numbers to reconcile. The bench scored stock yolo26n at 2%; our rover-style decode gives 12%. The ranking holds either way.
• Weak classes. Boxes, backpacks and handbags are the weakest of the new classes. Labelled warehouse renders have been requested for boxes.
• Rover integration. The rover's detector code only reads YOLO output today and needs a small change to decode RF-DETR.