The watchkeeper

Peter Donaldson reports on how this uncrewed system keeps construction sites safe and secure

If uncrewed systems are to make the most of the opportunities offered by the need to secure large and often complex sites, they will have to exploit automation as far as possible, ensure safety and cost-effectiveness, and stay on the right side of the regulators. Centrally operated UAV-in-a-box systems run by specialist contractors are emerging as an attractive option, and Protegimus Protection, headed by founder and CEO Stephen O’Callaghan, is one such provider.

The company works in partnership with Idiployer, which makes the Nexus UAV-in-a-box system, and FlytNow, which provides the operator interface. It has been providing automated security since early 2022.

Protegimus’ Residential and Commercial Aerial Protection Services (RCAPS) for construction sites, for example, is claimed to reduce security costs by about 50% while protecting vehicles, equipment and personnel on site. It offers other benefits as well, such as site development updates, inspections, site safety and compliance checks, fire monitoring and incident response and management.

Protegimus manages flight operations directly from its control centre, but there is a regulatory requirement to place an observer on site as a safety measure to ensure that the UAV is doing what the company’s operators are telling it to do via the flight software, O’Callaghan explains.

On-site set-up

The set-up of each customer’s system is strongly influenced by the topology of the site and the customer’s priorities. “No two sites are the same,” O’Callaghan points out. “Initial consultation with the client allows us to identify key vulnerable areas that need an extra layer of safety and security on top of those already implemented or as a standalone system.”

Site visits are usually conducted during phase one of a construction project, which is usually after its location has been confirmed.

“We visit the site and conduct the site security assessment, basically to detail how the automated services and systems will help the customer solve the problems they face, whether they involve intruders, theft or criminal damage for example, or if the customer wants inspections and surveys,” O’Callaghan says. “We detail the boxes they need, how many, and obviously how we intend to run the operation.

“If an in-person visit is not feasible then we conduct a virtual visit to gain a general understanding of the site and the factors we need to consider when planning operations there.”

He adds that site surveys are conducted at ground level as well as from the air. The ground survey involves a walk-around by a specialist and is followed by a UAV mission to provide 360° coverage of the site. Information obtained from the survey enables the Protegimus team to design a bespoke set-up, which includes the number and location of UAV-in-a-box units.

The number required depends on factors including the size and environment of the site, the required frequency of patrols, inspections, mapping flights and so on, and the coverage required, which could be 24/7, on-call, daily or weekly, for example.

The safest and most efficient positions for the box (or boxes) might be well above ground level, O’Callaghan notes, in which case the company will build custom elevated platforms on which to mount them.

A system typically requires at least two people to install, followed by operational planning for the site by one of Protegimus’ UAV specialists.

“The most common challenge we’ve faced with sites is the support infrastructure in terms of power and Ethernet connectivity,” he says. “If Ethernet is not available then we can run operations over a 4G network, as the docking station has an integrated router to enable docks to be installed almost anywhere.”

Idiployer’s Nexus box hosts a DJI Mavic 2 Enterprise Advanced quadcopter, a combination that has been through thousands of hours of testing and subsequent operations, O’Callaghan says. This is the principal UAV in use by Protegimus, but Idiployer is developing a new version of the box known as Nexus Plus that will also support the Mavic 3 Enterprise and Mavic 3 Thermal.

The system can also be integrated into existing security infrastructures such as video management systems, CCTV, passive infrared sensors and alarms. “These can all act as triggers to alert the UAV to go to a specific location to investigate further. This is achieved through FlytNow’s flight software and application programming interfaces,” O’Callaghan explains.

In the Protegimus remote operations centre, the number of people dedicated to each customer depends on factors such as the level of risk associated with the site, the planned frequency of flight missions, the length of the contract and the particular service package adopted. Just as no two sites are the same, so all clients are different, and the company tailors its service package to each one individually, O’Callaghan says.

“The operational flight software is user-friendly and intuitive. Users have a clutter-free interface where they can manage the current operation and existing fleet of UAVs and/or docking stations,” he says.

“The initial trigger to alert the system that an intruder is on site will come from perimeter sensors or the CCTV,” he explains. “The UAV will then deploy automatically to where the initial trigger was received and search the area. When the intruder is detected by the UAV’s onboard thermal camera and confirmed, we have the option to take manual control of the UAV from the mission dashboard and track them, directing ground personnel to their location to intercept and deal with the situation.”

The system also enhances safety on construction sites, because the information it generates can be used to prevent accidents and fires. “During routine missions the UAV can highlight and detect anything out of the ordinary, and can alert the control room of a potential fire through hotspots detected by the onboard thermal sensors,” he says. “We can also ensure health and safety compliance by observing whether the correct PPE is being worn by on-site personnel.”

In addition to their cameras, the UAVs can be equipped with effectors such as strobe lights, spotlights and speakers to warn people that the police have been called if criminal activity has been observed.

Operator’s view

The human-machine interface used in Protegimus’ operations centre is the software dashboard developed by FlytNow. O’Callaghan explains that the box comes packaged with the UAV and a one-year subscription to FlytNow’s software, which is the heart of the system’s automation, mission management and situational awareness.

The dashboard is laid out over a Google Maps view of the site. Overlaid on this are a number of windows; one shows a live camera view of the box from which the UAV is about to be deployed. The camera and others track the UAV throughout its flight.

There is also a status window informing the operator what the system is doing at any given moment. When preparing for flight, for example, the window displays a label saying as much, and shows an indicator for the UAV and one for the docking station, either in green for go or red for no-go.

If all is well, the status window indicates that the pre-take-off routine has been successful and the operator can then click ‘Next’ in the window. This brings up a checklist that includes a link to documented permissions from authorities to fly the UAV(s), carry out their calibration, a ‘GPS lock’ that shows the number of satellites from which the UAV’s receiver is getting data, battery status in terms of charge, a figure for comms latency in milliseconds, and the readiness of the emergency landing routine shown by whether the emergency landing button is set.

When the status of all these items is acceptable and confirmed, the operator clicks the ‘Check All’ button and then the ‘Next’ button. This brings up the rest of the checklist that shows predefined actions including the low battery and critical battery fail-safes, in which the UAV automatically returns to its launch site when the battery reaches 20% and lands immediately when it reaches 10%.

The other items on the list refer to actions on the loss of any of the three comms links. The UAV returns to its launch site if the LTE link drops for 30 seconds, immediately if the RC link drops, and after 30 seconds from loss of the FlytNow link. The final item is the ‘Mission Finish’ action, which is a normal return to the docking station.

With all those items checked, the operator is free to click the ‘Execute’ button, after which the UAV lifts off and the status window confirms that the mission has started.

Throughout the mission, the dashboard has an ‘Abort’ button that stops the current action and puts the UAV into hover mode.

As the UAV takes off, its camera feed appears in a dedicated window, and another window shows the progress of its post-take-off routine. The status window then shows whether that routine has been successful.

The map also continuously shows the UAV between numbered waypoints.

A further window on the left of the dashboard shows all the missions in progress, and the operator can switch between them.

As the UAV prepares to land, the status window indicates with green labels that it has set its precision landing (PL) configuration and is descending, and shows the seconds remaining until it is due to trigger the PL process. This is followed by confirmation that the preliminary pre-landing routine has been completed successfully.

A blue indicator shows that the UAV is searching for a PL tag on the Nexus box, then a green label indicates that the PL routine has been triggered and the UAV is landing.

Potential problems that might warrant operator action are flagged up with orange caution labels, indicating for example that the UAV is unable to detect the PL tag and is increasing its altitude to 10 m, then that it is searching for the tag again.

When the UAV has detected the largest PL tag and is heading to it for landing, a blue label in the status window informs the operator. As it approaches, the UAV’s camera feed shows its view of the PL tags, which is displayed on the operator’s screen next to the image of the UAV landing provided by a camera on the ground watching the box.

A blue status label then reports that the UAV is landing on the tag, then a green one indicates that the post-landing routine has been initiated.

As the landing pad with the UAV withdraws into the box, an orange label says the UAV is ‘avoiding detected obstacles’, an indication that its sensors are still working. A green label then shows that the post-landing routine has been successful.

The operator is then given the option to stop and save a recording of the mission in a pop-up window that also offers other options such as instant replay, live broadcast, screenshot and photo modes, which can be accessed as needed during the mission.

ArUco markers

The PL tags on the Nexus box are key to automated operation, and are in the form of ArUco (Augmented Reality University of Cordoba) markers, which resemble QR codes.

These machine vision markers are widely used in the augmented reality field for camera pose estimation. If that camera is aboard a UAV, its position in three dimensions with respect to the marker can be calculated precisely.

Each marker consists of a square divided into a 7 x 7 grid, with black-and-white squares representing the binary values of 0 and 1 respectively. The squares around the outside are all black, forming a border around the inner 5 x 5 grid that encodes the marker’s identity and creates the shape that allows 3D pose estimation.

Displayed on the Nexus box landing pad, five ArUco markers enable the UAV to locate the box, confirms that it is the right one to land on and enables a precision landing independently of GNSS. The box has one large ArUco code on top, while the landing pad, which slides out on runners, shows one large marker and two medium-sized ones with a single small one between the medium pair. Each marker has its own black-and-white pattern.

O’Callaghan says the UAV can read the code on the largest marker from 15 m. With the box’s identity and the UAV’s position with respect to this marker established, it can begin its descent, picking up the smaller markers during its approach.

At 1m above the landing pad, the UAV can read the smallest marker and perform a precision final approach and touchdown. “The guided landing mode is triggered in light winds, which enables the UAV to perform a ‘down to zero’ vision-guided landing,” O’Callaghan says. “Further algorithms are in place to ensure it can find the tag in low light or poor weather.”

While the box has few moving parts, other than the retractable landing pad with its integral front door, it features a built-in weather station that incorporates a local wind sensor in the form of an anemometer, plus audio and visual beacons to provide warnings of when it is in the process of opening and launching the UAV.

Attached external lighting makes it easier for the UAV to locate the markers from the air at night or in poor visibility.

Idiployer says the system is hardware-agnostic in terms of UAVs and docking stations, and is also scalable in that users can add “unlimited” UAVs and boxes, and monitor them remotely over 4G/5G from anywhere in the world.

Safety features centre on fail-safes that are triggered automatically if battery capacity is low, comms between the UAV and the box is lost, the GPS signal drops out or the box’s connection to the internet is disrupted. Its enterprise-grade security uses token-based authentication and end-to-end encryption.

Protegimus shares responsibility for maintenance and repairs with Idiployer, O’Callaghan says. The system’s health is gauged by sensors integrated into the box and monitored remotely via the internet, and is also displayed to the operator on the FlytNow interface.

This provides an overall go/no-go indication and flags up any issues with the battery, the Nexus box or the UAV itself. “That will be fed back to the maintenance team at Idiployer,” O’Callaghan notes.

While the internet connection allows software-related problems to be tackled remotely through restarts, software patches and bug fixes, the box also has facilities intended to simplify hands-on maintenance, repair and component replacement. This is in the form of a diagnostics panel on the back that consists of 18 LED warning lights, each labelled to identify the component it represents.

“It just saves time going in there to see what’s going on,” O’Callaghan says. “If any part needs to be replaced or repaired, we will get that information remotely and then send out the parts to our guys on site who are managing the system. They will be able to open the back, change the part and get the system running again.”

Near-term upgrades centre on the launch of the new Mavic 3 Dock this February, with Protegimus extending its RCAPS service to airports and other transport and logistics sites, the retail, agriculture, energy and maritime sectors and more.