Rescue bots

AI robotics expert Robin Murphy talks to Peter Donaldson about search and rescue in extreme conditions

Professor Robin Murphy is an expert in using robots to save lives and rebuild communities following a disaster. She wrote a book on the subject, Disaster Robotics, which has become a key reference work. She is also the Raytheon professor of computer science and engineering at Texas A&M University, and vice-president of the Center for Robot-Assisted Search and Rescue.

Two major disasters pivotal to the use of robots for search and rescue took place in the US and Japan in 1995, which shaped Murphy’s career: the Oklahoma City bombing and the Kobe earthquake. The respective responses of Murphy and professor Satoshi Tadokoro of Kobe University led to their being credited with founding the field of disaster robotics.

At the same time, NASA was making final preparations for the Mars Pathfinder mission, during which a 10.6 kg, wheeled robotic rover, Sojourner, would deploy from the Carl Sagan lander.

“Everybody was looking at small robots for extreme environments and realising this technology was needed immediately on Earth. And, in 1996, I got the first National Science Foundation grant that ever had the words ‘rescue robotics’ in it,” Murphy recalls.

It might seem that the technology developed to overcome the long communication delays involved in deep space missions could be transferred naturally to difficult operations on Earth, but experience taught a different lesson.

Complementarity

“Our initial assumption was that our robots would all be autonomous, but one of our biggest research findings was that operations have to be co-operative. We call it human-robot complementarity,” Murphy says.

This, she explains, is because disasters are so different from one another and the environment so complex. “You can’t even simulate the range of things that we continue to encounter, so you want the robot to be smart enough not to drive the operator up the wall with the minutiae of teleoperation, but at the same time you want the operator to be able to stop, assess and figure out what’s going on.

“What’s really important is the people who are trapped more than 20 ft from the surface, because that’s where all of our tools fall apart unless you’re using a robot that can crawl in there. You’re talking small void spaces and people buried in rubble, so it’s not an environment we humans can walk around or even crawl in.

“That is mechanically very, very hard. Perceptually, it’s very hard as well, because you’ve got all this dust suspended in the air, and as you move around you’re kicking up more, and there’s water everywhere. With the concrete dust it makes a nice, acidic slurry that eats away at your camera systems and mobility effectors.”

In addition to the operator, construction engineers need access to the robot’s sensor feeds, as they need to work out what they are going to do if a survivor is found, as the average time to extricate someone is 10 hours, Murphy explains. This puts survivors at risk of crush syndrome, during which the sudden removal of pressure triggers potentially fatal shock and kidney failure unless the right drugs are administered at the right time. “Everything has to be super well-prepared, so it takes time.”

Keep ‘em small

Naturally, 9/11 was a major milestone in Murphy’s career. One of the key lessons was that military robots developed to enter buildings in search of enemy combatants (of which a number were made available) were too big.

“You need small, shoebox-sized or snake-like robots,” she says. “It was only these that got used in the rubble pile.”

Another lesson was that operators must be able to carry the robot and everything else they need in a backpack, and get to the scene on foot. And this lesson needs constant reinforcement, Murphy says. “We still, to this day, get people showing up to our exercises expecting to drive right up to the scene in their van, and electricity and wireless to be available.”

Murphy has to juggle fieldwork with teaching responsibilities, although they are complementary to a large extent. Fieldwork provides ample case studies through which she can discuss AI and robotics with her students, exploring what works, what doesn’t and what they think will.

“And, of course, everybody likes to be helpful, so they are happy to see the home team working with the responders and, hopefully, making a positive difference,” she adds.

The operation that stands out the most for Murphy in this respect is the Crandall Canyon Mine disaster, which took place in Utah in August 2007. This is not a story with a happy ending, because six miners and, later, three rescue workers were killed underground, but, for Murphy, it was important to be there in the local church with the families to explain what was being done to help their loved ones.

“It made a difference to those families, or it seemed to, and I still think about them,” she says.

In the search effort, boreholes were drilled down from the surface to reach areas where any survivors were expected to be. Air-sampling sensors, microphones and a small, tracked and tethered robot with a low-light camera were lowered inside the boreholes to check for signs of life.

Operating conditions for the robot were difficult as it had to pass down a 22 cm borehole, slide down a pile of tailings on the floor underneath (where the drill broke through) and move through a shallow layer of groundwater mixed with drilling fluid.

The robot was built for the job by Inuktun Systems and operated by PipeEye International.

Don’t make things worse

“Normally, I consider it unethical to bring an untried robot to the field, because you must not make things worse,” Murphy says. “At Crandall Canyon, I did so, because all of the technology existed and the company just came up with a different combination for us.”

Even now, nearly 20 years later, there is still a whiff of the ad hoc about the use of ground robots for disaster response, usually involving machines donated to first-responder organisations by the military, and commitment to the technology is low.

Part of the problem, Murphy observes, is that disasters are rare and specific kinds, such as those in mines, are even rarer, so there is a risk that a robot bought now will sit unused for years and nobody will be trained on it when it is needed. This happened during a couple of nuclear disasters. What’s more, the technology changes at a similar pace to that of smartphones, and operators need to keep up. “So that makes investment decisions hard.”

However, with ground robots generally, institutional adoption has been glacially slow, she says. “Even after 9/11, the attitude was ‘oh well, we should have ground robots, but first we must make standards for them’, and we are still working on those standards. At some point, you’ve just got to start using them.”

Disaster operations differ in many ways from regular ones. “Usually, your environment is deconstructed and unlike anything you have seen before, even if you are operating aerial or underwater vehicles,” she says.

Another difference is the high operating tempo. The robot has to be up and running in about 20 minutes or you don’t get invited to the next disaster, she cautions. “It has to work, so there is a lot of pressure on the operator.” Combined with travel, long days and little sleep, this causes high levels of cognitive fatigue.

Murphy is now working on the second edition of her book. She takes pride in the original’s acceptance by both academics and rescue workers, and is particularly heartened by the number of practitioners who have read and refer to it. The book includes checklists to run through before deploying them in actual situations, judgement calls on what will and won’t work, and both how to and how not to learn from operational experience.

One approach that is often observed in academia, but which she cautions against, is building a mock-up of the last disaster out of cheap materials, such as plywood, which are not good facsimiles of rubble and slurries. Instead, she focuses on drawing out the fundamental characteristics of environments in which rescue robots typically operate and quantifying them.

Verticality and tortuosity

“You need to know about verticality and tortuosity – how much twistiness there is. You need to know what your relative size is, if you are expecting a large robot to get into a tight space or a small robot to work in a large space. For example, if you’ve got a snake robot that moves by pushing against walls, you don’t want it to be in a big spot, because it’s got nothing to push against.”

The main technology gap is in small ground robots, but Murphy is excited about the major advancements being made in “soft” robotics; the snake-like

or vine-like machines that can go into small, irregular-shaped spaces. These have been developed mainly for underground infrastructure inspection or medical applications such as endoscopy, where they face the problem of keeping cameras free of “goop”.

AI contributions

Murphy’s PhD is in AI for robotics, and there are two areas where she believes such smarts will make a big contribution to disaster robotics in future. One of these is the nexus where computer vision meets machine learning, and the other is human-robot complementarity.

In the first case, her graduate students created a “huge” database of UAV imagery from disasters, and they are training AI to recognise and categorise building damage using imagery from the aftermath of recent hurricanes. This can amount to 60 GB per day. “One person can’t do that, five people can’t, so you definitely need the AI.”

She says human-robot co-operation gets away from the paradigm in which the machine was either fully automated or fully teleoperated when the relationship should be more like a horse and its rider.

“Finally, we’re seeing the robotics community go back to this idea of complementarity – human and robot working together in what is also called a joint cognitive system.”

In such a system, a snake bot could be trusted to find its way into a debris pile, and its human operator might ask it to stop and back up, or provide a better view. However, Murphy cautions against the common hand-waving assumption that AI will “just learn” what it needs for rescue work.

“One of the concerns I have is that my colleagues tend not to be in the field. By not working with domain experts in real conditions it’s easy to assume your way out of a problem or make an AI system that doesn’t really work.”

A solution may emerge from AI-related work at the National Science Foundation AI Institute on societal decision-making.

“That is part and parcel of helping to educate and encourage our community to get more into fieldwork to understand what it takes to make these decisions,” Murphy adds.

Professor Robin Murphy

Robin Murphy was born in Mobile, Alabama, in the year of Sputnik – 1957. Growing up and going to school in the area immersed her in the technological culture of the space race: “Everybody wanted their kids to know science and math.”

The area is also subject to hurricanes. In September 1965, Hurricane Betsy wiped out Round Island in Mobile Bay, a favourite fishing spot for Murphy and her father.

“I got an impression of just how devastating hurricanes can be,” she recalls.

The family later moved to Georgia, where Murphy attended Georgia Tech and earned a degree in mechanical engineering, followed by a masters and, in 1992, a doctorate in computer science.

Murphy’s introduction to robotics came while pursuing her PhD and, under the terms of a fellowship, she had to work on computer-integrated manufacturing systems. She took the chance to work for a new professor on AI for robotics.

At first, she says, this “sounded like a bunch of hooey”, but she was undeterred. “I got on with him, and just fell in love with AI and robotics.”

Now approaching retirement in the spring, Murphy plans to teach a three-hour class on emergency robotics, summarising the field since 1995. After that, she’s looking forward to continuing her work with CRASAR, but also spending more time hiking, camping and kayaking. “I’m a very outdoors person,” she adds.