Towards Low-Latency Event-based Obstacle Avoidance on a FPGA-Drone

The Challenge

Autonomous drones need to detect and dodge obstacles in milliseconds, not the tens-to-hundreds of milliseconds typical of conventional RGB camera pipelines. Traditional frame-based vision introduces latency through exposure time, readout, and heavyweight neural-network inference — any of which can be fatal at high speed.

Event Cameras Meet FPGAs

We combine two technologies tailor-made for this problem: event-based vision sensors (EVS), which asynchronously fire per-pixel brightness changes at microsecond resolution, and an FPGA accelerator that can run inference on aggregated events with sub-millisecond latency.

EVS vs RGB — Head to Head

We benchmarked both approaches on the same collision-avoidance task:

• Frame rate — The EVS model achieves an effective 1 kHz, far exceeding standard cameras.
• Prediction accuracy — 20 ms lower temporal error and 20 mm lower spatial error versus the RGB baseline.
• Robustness — On out-of-distribution data, the EVS model maintains 78% precision for motion detection compared to just 19% for RGB, which is highly prone to overfitting.

End-to-End Latency

The full pipeline — from photon to evasion decision — completes in approximately 2.14 ms. Event aggregation accounts for about 1 ms, while inference on the FPGA processing unit takes just 0.94 ms. These numbers demonstrate that event-based vision on dedicated hardware is a viable path toward truly real-time obstacle avoidance on resource-constrained drones.