Chip-module system for reality capture

(Image: Lyte)
Lyte in the US has developed a chip and module that combines imaging and positional data into a single capture that can feed into a digital twin model of an autonomous vehicle, writes Nick Flaherty.
LyteVision combines RGB, 4D imaging and an IMU into a single multimodal sensor. All modalities are synchronised, calibrated and fused at the silicon level on the Lyte chip. The output is a structured, high-fidelity capture (at the edge) that operates from bright sunlight to complete darkness, and stays robust under optical interference, dust and weather.
Using custom silicon photonics and electronics integrated at the chip level, Lyte has developed a coherent 4D sensor fused with RGB and motion awareness in a single architecture, meaning that environmental robustness and interference immunity are designed together, not bolted on afterward.
The system was developed by the team behind the Kinect camera sensor and Apple’s Face ID technology.
However, the module is only part of the solution. What needs to follow is the connection of sensors to silicon to software to AI compute, with data flowing from the edge to the cloud and back, and models processing what the machine perceives, making decisions and issuing commands. The full loop needs to close in milliseconds.
Lyte points out that this is not one integration; it is integrations on top of integrations – sensors to processors, processors to networks, networks to data centres, data centres to models. Every connection is a potential failure point, and every handoff hides latency and incoherence.
The key to the Lyte approach is that the path from perception to intelligence is unified: sensors fused to silicon designed to talk to software and software designed to talk to AI compute, providing the full stack.
Lyte’s perception modules are in active deployment with leading robotics developers with a perception-to-twin (scan-to-twin) layer that turns every scan into an indexed, simulation-ready asset and every redeployment into a refresh of the library. This is built on the same stack as the perception layer.

Reality capture is the bottleneck holding back digital twins of autonomous vehicles. The hard part is not stitching geometry in a sim engine. The hard part is acquiring a scene with enough fidelity, geometric precision and semantic consistency that the downstream pipeline has something real to work with, which is where most capture stacks struggle. Sensors disagree on time, geometry and what they are looking at, meaning that the compute layer inherits a noisy, fragmented input that it tries to ‘clean’ after the fact.
Robot OEMs need a recurring digital twin for locomotion. However, environments can change daily and a one-shot scan can ‘age out’ in weeks. The same sensor the robot uses to perceive at run-time is the sensor that keeps the twin current.
LyteVision acquires geometry as a dense 3D point cloud with per-point Doppler velocity and RGB colour. The sensor hardware unifies every input stream before software ever touches it, which means the reconstruction layer always starts from a consistent, latency-aligned multimodal capture.
Lyte’s proprietary reconstruction engine produces candidate objects, runs either on-device at run time or on a server for post-processing and outputs each candidate as an asset with geometry.
UPCOMING EVENTS