The head of Censys Technologies talks to Peter Donaldson about how she sees UAVs and AI being deployed for fast, high-resolution inspections

Computer visionary

While she wasn’t born among robots and high technology, Noel Heiks was introduced to them at an early age. That set her on a career path as an engineer, serial entrepreneur and investor that has led to positions on the boards of companies including long-range BVLOS UAV systems developer Censys Technologies, and made her an evangelist for the integration of UAVs with advanced sensors and machine learning (ML).

“My dad is the culprit”, she says. “He ran a business in Greenville, South Carolina, called Advanced Automation,” she says. “I started working with him when I was about 14, first as a janitor, then a drill press operator, then a blueprinter, and when I was 18 he said, ‘OK, now you can program the robots.’”

Advanced Automation’s main customer was Bosch, for whom it made fuel injectors for cars, using robots served by long lines of conveyor belts to assemble them. Programming the robots involved writing code to send data over a local area network and put it into a Lotus 123 spreadsheet. “I had to figure out how to write the code, so my dad sort of threw a C++ book at me and said ‘Learn this!’”

Growing up in her father’s business also made starting and running technology companies seem the natural thing to do. To prepare for that, however, she needed to supplement her early hands-on experience with further education in science and technology, which she pursued through a bachelor’s degree in physics and then a master’s in electrical engineering, both from Virginia Tech.

After earning her bachelor’s, she received a practical introduction to sensor technologies during 3 years as an employee of a company that built automated processing systems for manufacturing and testing photomultiplier tubes. A return to Virginia Tech followed, where she worked as a research associate building data acquisition systems for fibre optic sensors.

Then came her first start-up, Haleos, which sold micro-fabrication and optoelectronics products. As vice-president, Heiks raised capital and grew the business for 6 years before selling it. During that time she was also studying for her master’s.

Next came two more start-ups including Nuvotronics, which built phased-array radars and millimetre-wave antenna systems, usually for aerospace applications. Growing Nuvotronics, she raised capital and built alliances with large commercial and defence companies before selling it to Cubic Corporation.

Machine vision

A stint as president and chief operating officer at Duos Technologies brought experience of very high-speed, high-resolution imaging combined with AI/ML for automated safety inspection of freight railcars. Here, Heiks’ master’s in computer vision came in handy.

“They build portals through which a train

2 miles long passes through at 70 mph, taking visible and infrared images as it is speeding by. A few minutes later you can tell whether a nut is out of place or if a wheel bearing is too hot,” she says. “We could detect all kinds of mechanical defects.”

Turning this concept around by putting the sensors or cameras on moving vehicles creates opportunities for UAVs in high-speed, high-resolution inspection services aided by computer vision and AI/ML technologies. This is where Censys Technologies comes in, with its long-range Sentaero UAVs.

Invited by Censys to meet an electric utility company for which Censys was demonstrating one of its UAVs, Heiks was impressed by the system in which truck/trailer-based mobile command centres support multiple UAVs. Some missions might cover hundreds or thousands of miles using multiple trailers and fleets of the aircraft.

Adapting AI

“I watched this in action and I thought about the implications for both government and commercial use,” she says. “I thought these guys were going to go far, but at the time they did not have the AI/ML capability to build on their data sets.

“So I imagined that very soon Censys would be adapting its payloads and taking on all kinds of data – spectrographic, hyperspectral, thermal and so on – and discerning from that certain patterns such as, ‘That is too hot, that plant is a different species, that is a car, that guy is carrying a gun and he shouldn’t be.’ These are the sorts of things you can do when you have smarts on your aircraft.”

These capabilities are now integrated into Censys’ CensWise AI computer vision system.

Heiks then hired technologists from Duos, including AI expert David Ponevac as CTO, where they set up platforms such as RailSens software. This takes data from UAV flights over miles of railway looking for vegetation encroachment, anomalies in the rails themselves, anything that shouldn’t be there.

She expects this kind of service with UAVs to replace manned aircraft and more subjective monitoring processes. “The issue is that you get invasive species that grow at exponential rates that power companies are not used to dealing with, then it gets on a powerline and you have a fire,” she says. “Doing that on a regular basis using UAVs though means you don’t have the safety issues associated with manned aircraft, and

you can cover long distances – 80 to

100 miles at a time.”

In addition to the AI software, one of the challenges with implementing this capability is the close integration required between the payloads and the UAVs. Heiks emphasises the importance of standardising the footprint of the receptacle in which the payloads are to go, on which Censys has worked closely with payload vendors to achieve. “So you can put a variety of different gimbals, Lidar systems and so on into the standard platform with standard connectivity.”

Monitoring methane

Environmental monitoring is a growing market for UAV operations, and laser-based sensors that can detect greenhouse gases, particularly methane, look to provide a useful new capability.

Heiks explains that some US state governments have been tasked with tracking down ‘orphaned’ oil wells that need to be capped, because about 20% of all 2.5 million of them are leaking methane.

“That was one big issue we could see coming up about 18 months ago, so we started test flights over areas known to be leaking methane, a lot of them over landfills,” she says.

The detection technique used is the well-established tuneable differential laser absorption spectroscopy. The challenge lies in developing a reliable, accurate, compact and lightweight system for UAV deployment.

In the initial trial implementation, the system aboard the UAV aimed two laser beams at the ground, one of which was tuned to a wavelength that is absorbed by methane and the other which was not, to act as a control.

“If methane is there, light from the tuned laser is absorbed and the amplitude of the light reflected back decreases. But if the second laser beam is also attenuated then there is no methane,” she says. “If there is a difference in absorption between the two wavelengths though then you definitely have methane.”

Up close and personal

“Usually, spectroscopy is done with small systems in which there is a chemical under test, a laser and a receiver all in one small unit. Now imagine the problems you have when your test sample is a long way from the sensor.

“You have a lot of attenuation from the reflecting surface. There is a lot of scatter from grass or the ground, so you have to pick out minute signals coming back. You must be up close and personal to do this sensing – hundreds rather than thousands of feet above the sensing surface.”

Heiks adds that the accuracy of this experimental system in mapping methane emissions in the target area was not as good as Censys wanted for an operational system, largely because of its narrow field of view as well as the scattering.

“As the drone flies, the laser shoots directly downward and captures only what is immediately under it. You can fly in a pattern, but you have to do it multiple times,” she says. “You get a noisy, averaged look at the scene, although you get some good information.”

These limitations led Heiks and her team to put together a new system called RetraSpectra. “We were thinking of inventing a system with a mobile retroreflector either in the air or on the ground, while the laser could move around with respect to that reflector as it flew around a target area to collect clear information,” she recalls. “But we found that The Aerospace Corporation, which is a federally funded lab, had patented the exact thing we wanted, so we licensed their technology to commercialise it.”

In support of that, Censys has won a Small Business Innovation Research grant from the US Environmental Protection Agency (EPA).

The system Censys is developing consists of a ground-based laser on a gimbal that is free to turn through n x 360o, and a UAV carrying a retroreflector. The UAV flies around the survey area and ‘targets’ the laser beam with the retroreflector to provide a strong return signal.

“A retroreflector is very similar to a mirror, but it is tuned to the wavelength and is very tolerant of errors in targeting,” Heiks explains. “It has many features that make it similar to reflective tape, which you can always see when driving at night, even if your lights are not shining directly at it, but it is a little more complex at the wavelengths used.”

She notes that the laser does not generate a single narrow beam but uses a lens system to create a much broader beam, characterising it as a big, wide-open eye.

Censys expects the system to tolerate errors amounting to several metres of lateral displacement and of the order of 10-20o of angular displacement. “We are conducting tests of this right now for the EPA. Initially we are doing static tests and we’ll go flying in phase two of our programme.”

The system has also demonstrated good range and sensitivity, detecting methane at concentrations as low as 10 parts per million.

“We are now concentrating on testing and training the targeting system. Laser systems have been developed before to do this kind of work, but targeting of the retroreflector is one of the difficult parts.”

Heiks adds that the CensWise system is being applied to detecting and tracking objects. More broadly, she considers that there is a great deal of as yet untapped potential for remote sensing from UAVs using sensors more often associated with military than commercial and environmental applications.

Untapped potential

“We are developing sensors such as magnetometers for detecting metal,” she says. “There are also millimetre-wave sensors that can look through tree canopies and, potentially, see metal and other objects.

“Sensors were my background before I got into UAVs, and I think there is a huge amount of capability in the millimetre-wave spectrum as well as the microwave spectrum, including being able to see underground, for example. These capabilities haven’t really been exploited, because they are usually the purview of military systems and, generally, they are bulky and heavy.”

However, what she considers most important is that advanced sensors in UAVs feed the mobile command centres with their AI/ML algorithms so that the system can discern significant patterns, particularly in methane and CO2 emissions. “I hope we can combine the data we get from the RetraSpectra system with data on humidity, wind direction and so on with modelling, and connecting that to a whole-Earth model to really understand the effect we are having on our atmosphere.”

Noel Heiks

Noel Heiks grew up around robotics and industry, spending much of her childhood and teens in her father’s automation business. She pursued her further education at Virginia Tech, earning a bachelor’s degree in physics in 1992 and a master’s in electrical engineering in 1997.

Co-founding optoelectronics company Haleos, her first start-up, in 1996, she served as vice-president before joining the company to which she sold the business, Rohm and Haas Electronic Materials, in 2002. She founded OptiSales in 2007 and defence electronics company Nuvotronics in 2008, serving in senior roles until April 2019.

She joined rail inspection and security systems company Duos Technologies as president and COO in March 2018, leaving to join the board of broadband and military navigation electronics specialist Emcore in September 2019, where she still serves.

Heiks became a board member at Censys Technologies in January 2020, and managing partner at venture capital company Kirenaga Partners in February 2021. She is now chairman of the board at Censys, and has recently launched Azultera, a new venture firm.