Open waters

The head of Greensea Systems explains the technologies behind his company’s open offshore robotics platform. Peter Donaldson reports

Ben Kinnaman, founder and CEO of Greensea Systems and originator of the Open Software and Equipment Architecture (OpenSea) robotics platform, came up through the hands-on, practical side of the marine industry.

Now in his late 40s, he left school as soon as he could and went to work offshore as a salvage diver before moving into ROVs as a technician and then a supervisor, returning to education with a focus on robotics and control systems along the way. While working with ROVs, it became obvious to him that the industry had evolved in a way that made the integration of new systems and technologies slow and difficult.

“Our industry came up very pragmatically. We had technicians developing and building machines to work deeper, safer and accomplish more, but in doing so we ended up in a lot of silos: everyone was doing things a bit differently. Other robotics industries grew out of academia and government-funded initiatives, and they brought standards forward early.

“I saw an opportunity to create a platform that would allow our industry to advance at the rate of other industries,” he says.

Breaking down the silos

The purpose of the resulting open architecture platform was to break down those silos and make it easy for the industry to transition to new technologies, share ideas and to integrate new sensors, payloads and packages. These days, OpenSea is deployed on about 2500 underwater vehicles and related systems, both crewed and uncrewed.

“From a technical perspective, the challenge is really around how to build a platform that is flexible and scalable, that people can use to advance robotics but that stops short of over-specifying the end result. So it is really about architecture: how do we build a bucket of Legos without predefining what they are supposed to create?”

Kinnaman founded Greensea in 2006, and recalls that building a business around the concept of an open architecture platform before the industry was willing to adopt one was difficult. “Open architecture is now quite a buzzword; everyone says it with their breakfast in the morning, but 16 years ago, nobody was talking about open architectures.”

Building blocks

In explaining the platform’s openness, he returns to the Lego analogy. “Lego is a little building block with a standard interface, and if you have an interface that matches that block, you can plug other blocks into it,” he says. “OpenSea is made up of hundreds of little ‘blocks’ with a public interface that people can build upon.”

Greensea provides application programming interfaces (APIs) and interface control documents (ICDs) to enable customers and partners to develop software that works with OpenSea, and generally shares its knowledge of how to build robust underwater robotic systems.

In the software stack on a vehicle’s main computer, OpenSea usually sits directly on top of the operating system. In most systems on which it is deployed, that OS is a Linux distribution, a choice that emerged from user preference, as many operators and manufacturers like it and its technical performance.

“These days, with edge processors running OpenSea in autonomous robots or sensor sets, most of the OS backbone originates out of an Ubuntu Linux distribution. However, OpenSea is completely cross-platform. We have built packages for OpenSea that support systems from Windows to Red Hat and everything in between.”

Greensea supports current and legacy systems, running a continuous process in-house in which the software engineering team is constantly porting OpenSea onto new platforms and testing it. “Every night we are testing and validating OpenSea across six or seven different platforms,” Kinnaman says.

OpenSea provides common wrappers around OS processes. The wrappers serve as an abstraction layer between the OS and the particular implementation of OpenSea so that common calls and loggers can run over any platform.

Pragmatism and architecture

On top of that layer sits the OpenSea Library, an application suite that Kinnaman describes as a mixture of pragmatism and intentional architecture.

“From a pragmatic perspective, the applications represent our 16 years of writing software for ocean robotics, and include everything from device drivers to utility functions to translation applications to the tools and processes that developers and users need.”

On the architecture side, OpenSea addresses some critical technology, particularly control, navigation and autonomy, through what Kinnaman calls its big five applications.

The first of these is OpenINS, an inertial navigation system engine, which processes and fuses sensor data to produce a cohesive state estimate (navigation solution) for a robot platform.

The second is OpenCMD, which is a vehicle platform and control package that provides full, six-degrees-of-freedom, multi-state, open- and closed-loop control for a vehicle.

The third is Open Manager, known as OpenMNGR, which Kinnaman describes as an autonomy executive, an application that provides a link between autonomous decision-making software and OpenSea.

OpenFLS is the fourth, serving as a sonar processing application that supports almost all the forward-looking imaging sonar systems currently on the market, he says. In addition to data processing, OpenFLS also provides technologies including feature detection, SLAM routines for stabilising a navigation system, target tracking and closed-loop control, plus closed-loop feature-relative positioning of sonar data.

Finally, OpenBT is a behaviour tree-derived engine for autonomous systems.

Interfaces galore

Naturally, OpenSea is also designed to run third-party applications, and to do so it provides a wide range of interfaces for them. “We have the ‘down and in’ interfaces that OpenSea needs to sit on top of computing platforms, then we have the ‘up and out’ interfaces that other applications use to work with OpenSea,” Kinnaman explains.

“With the lower-level interfaces, OpenSea supports almost every industry interface, from CAN and CAN Open to Protobus and Modbus to serial TCP/IP and more. We also have the common industry standard interfaces. Natively, OpenSea uses LCM and will soon use DDS as the inner process interfaces.

“We publish all those interfaces, and provide build and software utilities to help people connect to them.”

The company also supports other open architecture platforms, such as the Robot Operating System (ROS), providing ROS bridges and translation layers. “It is not uncommon at all, especially in the academic community, to have a pretty big ROS stack interfacing with an OpenSea platform, although it is less common in defence and commercial applications,” he says.

Greensea also provides support for several proprietary third-party control and autonomy packages, as well as a handful of other open architecture systems.

For customers and partners who want to develop their own applications to run on OpenSea, the company provides a software developer kit (SDK), which Kinnaman says contains “all the keys to the kingdom”.

In the SDK are the APIs and ICDs, the latter also including all the interfaces and examples for interfacing with the OpenSea suite of applications. Greensea also makes bespoke kits for many of its robot manufacturer partners.

The ‘easy’ button

One option aimed at customers who want a ready-made computing environment on their vehicles into which they can integrate applications is the OpenSea Hub. This is an embedded processor with a high-density I/O module associated with it, usually implemented through an FPGA chip.

Kinnaman describes the Hub as “the easy button” – a stable, proven hardware platform that comes with OpenSea installed, along with essentially everything needed to plug it in and start developing a robot.

“It shortcuts the need for porting OpenSea onto a new hardware platform, or a new OS, or creating a new build environment,” he says. “It is a platform that is known and understood, and that helps manage risk and cost.”

Work with partners on their projects can start at almost any stage in the process, he emphasises. “Sometimes we start very early and are literally there at the table when they start drawing the concepts on the whiteboard. At that point it is easy to provide the Hub as the ‘brain’.

“Sometimes the conversation begins much later, maybe even after years of operating a mature product that they now want to advance with new technologies but whose current architecture inhibits that. In that case, we might replace the processor on the vehicle with the Hub, or port OpenSea onto their existing hardware.”

Kinnaman also notes that Greensea is releasing a new product in late 2022 called OpenSea Edge. This is a vehicle-agnostic autonomy system for ROVs supporting seafloor to over-the-horizon comms, video and sonar perception systems. It also supports third-party AI/ML libraries. OpenSea Edge is a strapdown solution for untethered autonomy on subsea robotics. Greensea began deploying it in several defence applications in early 2022.

Phased development

Greensea characterises the development it undertakes with partners as a process divided into four phases, numbered 0 to 3. Phase 0 concentrates on fundamental requirements, and involves discussions about deciding how OpenSea can be of most value. It culminates in a technical roadmap.

Phase 1 is about initial development, where the team works to establish a basic, robust operational system or what is termed a minimum viable product. This phase takes the vehicle, for example, up to technology readiness level 6 (TRL 6).

Phase 2 takes the development further, by adding features and applications, culminating in a system at TRL 8. Continuous testing and development accompany any changes to the feature set, helping to refine the system.

Phase 3 takes the system to TRL 9, yielding a commercial product ready for distribution, complete with full documentation and training materials developed for operators and depot-level support.

The OpenSea platform is being evolved to make it more robust and useful, primarily with the aid of feedback from the deployed systems.

“We are hearing from manufacturing partners and end-users every day, in commercial, government and science applications. This feedback could be bug reports and issues that need to be addressed, or they could be feature suggestions. Our team is constantly rolling the feedback into OpenSea through our release schedule and into the development roadmap.”

Future developments

Beyond this routine process are some key developments that Greensea plans to bring out over the next 2 years.

The first is native support for the US Department of Defense’s Unmanned Maritime Autonomy Architecture (UMAA). Support for the UMAA, Kinnaman stresses, will allow the defence community to benefit from operational experience of thousands of OpenSea users as well as the interfaces and building blocks from which to create defence applications.

The second development is a reworking of OpenSea’s autonomy engine, which will include what Kinnaman refers to as a behaviour platform intended to enable the development of objective-based autonomous behaviours for working ROVs.

“Now and over the past few years, so much of the work on autonomy has been focused on executing a defined set of tasks without a human in the loop. To elevate that relationship, and to let OpenSea evolve into our next-generation platform, we need to augment the supervision by a human operator. Instead of executing a list of tasks, we need to focus on accomplishing an objective.

A vital part of this kind of autonomy is the computer’s ability to interpret the information provided by its sensors, so Greensea is working to integrate perception systems into OpenSea. This involves both perception software and perception platform support, as well as AI and machine learning architectures to interface well with higher-level autonomy applications, Kinnaman explains.

“This type of autonomy will enable our vehicles to work without us in some pretty complex operations,” he says.

Ben Kinnaman

Born in 1976 in North Carolina, Kinnaman grew up ‘on the beach’ in the southern area of the state and the islands of the Outer Banks. With no interest in study, he started work young, finding employment locally on tourist vessels and fishing boats.

His route back into education began with pressure from his mother, who persuaded him to apply to Davidson College, the local liberal arts institution. He was accepted and won a scholarship, starting as an arts major until an encounter with a professor (Dr John Swallow) in a required mathematics class precipitated a change of direction.

“He kept me after class one day and spent about an hour showing me what math can do, using numbers to define and model physical systems. I was absolutely blown away,” he recalls.

He left Davidson with a Bachelor’s degree in physics and independent studies in computational physics in 1998.

Work as a self-employed salvage diver followed, before he joined marine services contractor Phoenix International, hired by Michael Kutzleb, one of Phoenix’s founders, whom he regards as a mentor. “He gave me the opportunity to move into engineering, and Phoenix supported me going back to school.”

Kinnaman subsequently earned a Master’s degree in Mechanical Engineering and Control Systems from Johns Hopkins University in 2006.

While at Phoenix, he was involved with the water-based search for the wreckage of the Space Shuttle Columbia, which crashed in February 2003.

“There was no software framework for organising the data from the waterborne searches, so I sat at a table in a volunteer fire department and started writing,” he says. “That was my transition point from looking for things underwater to writing software to help look for things underwater.”

Kinnaman founded Greensea in 2006. For the first decade, the company was small, and Kinnaman was “chief architect, chief software developer and primary floor sweeper”. The company now employs more than 70 people, six of whom are department heads who report directly to Kinnaman, who describes his current role as that of a traditional CEO.