Teledyne FLIR has developed a video processor for AI algorithms to use in its thermal cameras, writes Nick Flaherty.

The FLIR AVP video processor weighs 5 g and is based around the QCS8550 mobile processor from Qualcomm, which has been optimised for UAV and robot sensors.

It runs the Teledyne FLIR Prism AI and image signal pipeline (ISP) software libraries, and interfaces with the company’s Boson and Neutrino thermal infrared imaging camera modules, as well as a wide range of popular visible cameras.

The FLIR Prism AI model has been trained on the world’s largest thermal image data lake of more than five million annotations to detect, classify and track targets or objects for automotive autonomy, automotive automatic emergency braking, airborne camera payloads and counter-UAV systems.

Prism ISP is a set of image-processing algorithms that include super resolution, image fusion, atmospheric turbulence removal, electronic stabilisation, local-contrast enhancement and noise reduction for the cameras.

The QCS8550 processor is built on a leading-edge, 4 nm process technology and measures 15.6 x 14.0 mm. It is mounted on the AVP System-on-Module (SoM) board, measuring 40.27 x 33.41 mm.

The processor has eight 64-bit CPU cores and the latest AI accelerator cores. The main ARM Cortex-X3 core runs at up to 3.36 GHz, and it is supported by four performance cores running up to 2.8 GHz and three efficiency cores at up to 2 GHz.

AI models can take a lot of memory, and the chip supports up to 16 Gbytes of low-power LP DDR5 memory and 256 GB of storage using the UFS3.1 standard (see data storage feature, page 42). An Adreno 740 GPU and Hexagon Tensor Processor (HTP) with Hexagon Vector eXtensions (HVX) and Hexagon Matrix eXtensions (HMX) are used to process the 8 bit and 16 bit AI data.

The chip includes concurrent GPS, Glonass, BeiDou, Galileo, QZSS, NavIC and sensor-assisted Positioning, and it supports up to six cameras. Several tools support the AVP board to simplify and streamline development, including a Qualcomm RB5 robotic development kit.

Software and board-support packages are also available.