A group of researchers has developed neural network algorithms that can work on nano-UAVs weighing as little as 10 g (writes Nick Flaherty).

The nano-UAVs are increasingly useful for monitoring applications, but struggle with complex autonomous operations with multiple objectives.

Researchers from the University of Bologna, Italy, KU Leuven in Belgium, the Dalle Molle Institute for Artificial Intelligence, USI-SUPSI, in Switzerland and the Integrated Systems Laboratory at ETH Zurich, Switzerland, successfully used time-of-flight distance sensor data with a vision-based convolutional neural network (CNN) for object detection in tests of the autonomous system with multiple objectives.

The test UAV is a Bitcraze Crazyflie 2.1 quadrotor, a COTS UAV that weighs 27 g and has a diameter of 10 cm. It is equipped with two microcontroller units (MCUs), a single-core ARM Cortex-M4 (STM32) for safe navigation and exploration policies, and a parallel ultra-low power eight-core GAP8 that uses the RISC-V instruction set. The GAP8 has been optimised for CNN inference, with a power envelope of only 134 mW, including the image sensors and external memories.

The challenge of providing autonomous operation for the UAV is addressed by mapping multiple tasks on the two MCUs. The STM32F405 MCU, with a peak performance of 100 million MAC operations per second, is used for the lightweight workloads such as sensor interfaces. The GAP8 MCU is used for the machine learning and image detection algorithms.

The UAV has three additional PCBs: the Flow deck, the Multi-ranger deck and the AI deck. The Flow deck provides optical flow and height measurements to increase state estimation reliability.

The Multi-ranger deck features five single-beam VL53L1x time-of-flight distance sensors mounted on the UAV’s top and sides, providing line-of-sight distance measurements within 40 cm.

The AI deck is a visual engine running on the GAP8 chip at 175 MHz, with a low-power QVGA-resolution grayscale camera and additional off-chip memories, including 8 Mbytes of HyperRAM and a 64 Mbytes of HyperFlash memory.

The UAV was tested with various algorithms, and flew  around a room with multiple objectives. The best results achieved a final detection rate of 90%, exploiting a pseudo-random policy for exploration with the largest object detection model and a mean flight speed of 0.5 m/s. The higher detection rate can be reached by trading off the CNN’s detection capabilities with the flight speed.