PicoSAM: Distilled Segmentation Models Running In-Sensor on the Sony IMX500

The Challenge

Modern segmentation models like Meta's SAM are powerful but far too large for embedded deployment. Running them on edge devices -let alone directly on a camera sensor -requires radical model compression without sacrificing accuracy.

Our Approach

PicoSAM2 and PicoSAM3 are ultra-lightweight segmentation models distilled from Meta's SAM framework, optimized for real-time inference on the Sony IMX500 image sensor. Through knowledge distillation, we compress the model down to ~1.2 MB while preserving region-of-interest segmentation quality.

Key Features

• In-sensor inference -Runs directly on the IMX500 sensor hardware, not the CPU
• Real-time on Raspberry Pi 5 -Live demo with interactive bounding box drawing
• Knowledge distillation pipeline -Train, quantize, convert, and deploy to sensor
• Pretrained checkpoints -Available on HuggingFace and Zenodo

Recognition

Presented at IEEE Sensors 2025 in Vancouver, where it received the Outstanding Lecture Award.