Summary: Researchers at Delft University of Technology have built an autonomous drone that uses neuromorphic vision and spiking neural network control inspired by animal brains. The design dramatically speeds up onboard perception while cutting energy use compared with conventional GPU-based approaches, opening the way for much smaller, more agile flying robots.
Using neuromorphic processing, the drone handles visual input and flight control far more efficiently: the team reports up to 64× faster data processing and roughly three times lower energy consumption than when running equivalent networks on an embedded GPU. These gains make tiny, insect-like autonomous drones more practical for real-world tasks.
Key Facts:
- Efficient Processing: Neuromorphic spiking neural networks process event-based visual data orders of magnitude faster than the same models run on GPUs.
- Lower Power: The neuromorphic chip used in the study consumes only milliwatts to execute the network, significantly reducing total system energy.
- Practical Applications: Compact, energy-efficient drones could be used for greenhouse crop monitoring, warehouse inventory management, swarm exploration, and other tasks that benefit from small, safe platforms.
Source: Delft University of Technology
Overview
Traditional deep learning for vision and control typically runs on graphics processing units (GPUs) and requires substantial power and computing resources. In contrast, biological brains perform perception and action using sparse, asynchronous electrical pulses called spikes. Neuromorphic hardware and spiking neural networks (SNNs) mimic those properties to achieve low-latency, energy-efficient processing, making them especially suitable for small aerial robots with tight payload and power limits.
The Delft research team demonstrates a full vision-to-control neuromorphic pipeline running onboard a drone, published in Science Robotics. They combined an event-based neuromorphic camera with a spiking neural network deployed on Intel’s Loihi research chip, enabling real-time perception and low-level control while maintaining very low energy use.
Spiking neural networks and event-based vision
Spiking neural networks represent information with sparse, timed spikes instead of continuous-valued activations. On digital neuromorphic chips, spiking neurons perform simpler integer additions rather than the floating-point multiplications typical of standard deep networks. This reduces computational complexity and energy consumption.
Event-based or neuromorphic cameras complement SNNs by producing asynchronous output: each pixel emits an event only when its brightness changes. That enables much faster motion detection, lower data rates, and robust operation across lighting conditions, including dim or rapidly flickering light. Importantly, the event stream can be fed directly to a spiking network without costly frame conversions.
First fully neuromorphic vision-and-control flight
Delft’s team created a two-part spiking network to control the drone. The perception module—trained with self-supervised learning on real event data—maps raw events to estimates of the drone’s ego-motion. The control module—trained via an evolutionary algorithm in simulation—maps those motion estimates to low-level thrust and attitude commands. After training, the combined network was ported to the Loihi chip and flown onboard the drone, demonstrating successful sim-to-real transfer.
The drone can hover, land, and maneuver laterally while yawing, and it operates across a wide range of illumination conditions, including flickering light that generates many non-motion events. The neuromorphic stack maintained reliable control and perception despite these challenges.
Performance and energy measurements
Measured execution rates for the neuromorphic network ranged from roughly 274 to 1600 updates per second, depending on conditions. On an embedded GPU the same network ran about 25 times per second—meaning the neuromorphic implementation was approximately 10–64 times faster in practice. Power measurements showed that the Loihi chip had about 1.007 W idle power with only 7–12 mW extra when running the network, while an embedded GPU used about 3 W total, with roughly 2 W consumed to execute the network. These results highlight the potential for highly responsive, energy-frugal autonomy on tiny platforms.
Implications and future directions
Neuromorphic sensing and processing can enable a new class of insect-scale or bird-scale autonomous robots that are safe, inexpensive, and capable of navigating tight spaces. Delft’s research group envisions applications such as greenhouse crop monitoring, warehouse inventory inspection, swarm exploration, and source localization tasks. Realizing these use cases will require continued miniaturization of neuromorphic hardware and extending capabilities to higher-level navigation and multi-sensor integration.
About this AI and robotics research news
Author: Marc Kool
Source: Delft University of Technology
Contact: Marc Kool – Delft University of Technology
Image: The image is credited to Neuroscience News
Original Research: Closed access. “Fully neuromorphic vision and control for autonomous drone flight” by Guido de Croon et al., Science Robotics
Abstract
Fully neuromorphic vision and control for autonomous drone flight
Biological sensing and processing are asynchronous and sparse, enabling low-latency, energy-efficient perception and action. Neuromorphic hardware for event-based vision and spiking neural networks promises similar properties for robotics, but until now implementations were limited by embedded hardware size and the challenges of training SNNs.
This work presents a complete neuromorphic vision-to-control pipeline: a spiking neural network that accepts raw event-based camera data and outputs low-level control commands for autonomous flight. The perception network, with five layers and 28,800 neurons, was trained with self-supervised learning on real event data to estimate ego-motion. The control stage was learned with an evolutionary algorithm in simulation. Experiments demonstrate successful sim-to-real transfer: the drone performs hovering, landing, and sideways maneuvers while yawing.
The pipeline runs onboard Intel’s Loihi neuromorphic processor at about 200 Hz, with approximately 0.94 W idle power and an additional 7–12 mW while executing the network. These results illustrate the potential of neuromorphic sensing and processing to enable insect-sized intelligent robots for practical tasks.