By William J. Clancey - Florida Institute for Human and Machine Cognition
February 25, 2017
Pilot displays showing orientation, depth, and distance from bottom of D2 and Seirios; sonar from each; and their relative relation to the ship (orange chart). Large monitors above show main cameras from D2 and context view from Seirios. Image courtesy of William J. Clancey. Download larger version (jpg, 4.1 MB).
Computer science is a relatively young discipline, with the first university departments formed only in the 1960s. As a design methodology, computer science has matured from developing stand-alone programs and interfaces for individuals to developing networked, “work systems” that relate people and technology of all types including robotic systems, vehicles, displays, sensors, databases, etc. Rather than focusing just on automation, computer scientists often work with psychologists and social scientists to augment or amplify how people think and promote their ability to collaborate and learn from each other.
Having studied the Exploration Command Centers associated with NOAA Ship Okeanos Explorer expeditions previously, I have been most interested to see the other end of the process—what’s happening onboard in the control room? How are the scientists interacting with the remotely operated vehicle (ROV) engineers? How is the ROV controlled? How are samples curated? And what about the ship’s operation, which is almost completely invisible to remote scientists?
The videography work station to the right of the ROV engineers in the Okeanos Explorer Control Room. Image courtesy of William J. Clancey. Download image (jpg, 84 KB).
The different roles and responsibilities, robotic equipment, software tools, and operating procedures (including ROV/Seirios launch and recovery) constitute an “exploration system” that all comes together during a dive to satisfy the interests of the scientists. What’s often invisible off camera—and thus my interest for being onboard the Okeanos— is how we use ROV Deep Discoverer (D2) and Seirios to the best of their capabilities without risking loss or damage, how the ship is maneuvered to keep Seirios (which serves as a buffer to ship motions) near D2, and how we plan and carry out a sequence of dives on a daily 8 am – 5 pm schedule in changing weather and sea conditions, while keeping those onboard safe.
So in short, as our distributed science team is studying the geology and biology of this region, I am studying the exploration system itself. What accounts for the quality of the scientific work? Why is the exploration system succeeding and how might we make it even better?
By participating in the daily dives, observing, documenting, and asking questions of people onboard, it becomes possible in a few weeks to learn about the different responsibilities of the team—scientists, ROV engineers, videographers, and the ship’s crew—and most importantly, how their attention and activities are continuously synchronized to make exploration and telepresence possible. How do the computer tools mediate our interactions, providing common information and references of location and orientation in a mostly unknown sea bottom?
Thus, studying a work system, such as the Okeanos Explorer, is itself a scientific effort, bringing together analytic perspectives from computer science, cognitive science, and the social sciences. The study itself is called an “ethnography,” referring particularly to a written report about this “community of practice” that lives and works at sea while collaborating with scientists around the world during an expedition. A proper ethnography of the Okeanos exploration system could merit a book, so I will only describe one aspect--the coordination observable in the control room during a dive.
Even a computer scientist will be impressed by the collection of displays and computer tools onboard the Okeanos, most of it used simultaneously during the daily dive by people with different concerns at different places onboard. In the Control Room alone we have 38 screens, plus laptops and video monitors. The Dry Lab, just outside the Control Room, home for the outreach team, mapping/survey lead, and expedition coordinator, has another dozen displays and video monitors.
How do we get a grasp on this complicated system of people, technology, and facilities? One way is to map the communications: Who talks to whom? How do they communicate? Who is guiding, making requests? Who is providing support, making certain activities possible behind the scenes?
It becomes clear that the operations are neatly compartmentalized, so most people focusing on one aspect need not be concerned about what is happening elsewhere. Yet, through clear lines of authority and protocols for communication, these domains of operation—science, D2/Seirios, and the ship—are synchronized quickly and efficiently. The graph below is one way of visualizing the compartmentalization of concerns and the flow of information and requests.
This graph is one way of visualizing the compartmentalization of concerns and the flow of information and requests. Image courtesy of William J. Clancey. Download image (jpg, 36 KB).
In the Control Room, three ROV engineers are monitoring and controlling D2 and Seirios thrusters and cameras, collaborating with the videographer who zooms, focuses, and adjusts brightness, contrast, etc. Seated in the row behind, the two onboard scientists (called “watch leads”) are speaking on the telecon with the remote scientists, while reading and contributing periodically to their interactions in the chat room and logged biology and geology identifications in a web browser tool.
The onboard scientists (“watch leads”) sitting in the back row of the Control Room, narrating the dive. Image courtesy of William J. Clancey. Download larger version (jpg, 4.1 MB).
During most of the dive, the scientists are conversing about what they are seeing, where they want to explore further, and what to sample. These interests are communicated by an internal intercom (the “voice loop”) between the onboard scientists and the ROV pilot. The pilot is using D2’s thrusters, moving and pointing the craft as well as its main camera at items of potential interest. He signals to the video operator when to “come in” or do a “partial” (zoom). This operation is seamless, with the engineers and videographer often knowing by experience what to focus on and for how long, attending to what scientists are saying on the telecon.
But this is just the core of the exploration system. As D2 moves along the sea bottom, we must move the ship as well, dragging Seirios along and keeping D2, Seirios, and the ship aligned. The tether between D2 and Seirios is 30 meters long; we want to keep it relatively loose, with Seirios behind D2 and about 10 to 20 meters above (depending on topography, current, visibility, ship location, etc.), so its camera can provide context for the scientific observation and movement of D2.
The Operations Officer on the bridge interacts with the ROV navigator, positioning the ship as required. Image courtesy of William J. Clancey. Download larger version (jpg, 3.6 MB).
For someone who has only participated in expeditions from shoreside, the frequent interactions among the ROV navigator and the ship’s bridge are unexpected. While the ROV pilot focuses on moving and pointing D2 and its main camera, the ROV co-pilot and navigator are particularly responsible for monitoring the distance of the ship from Seirios and how it is pointing in relation to D2. They frequently monitor a topographic chart showing the relative positions, a graphic display of the orientation and depths of D2 and Seirios, and sonar revealing possible obstacles. The navigator will accordingly, roughly every 10–20 minutes, contact the bridge and request a ship move, such as, “Range 30 meters, bearing 065, speed 0.2 knots.” The officer at the helm reads this back and reports when it is underway. The navigator communicates this information to the pilots, and depending on depth, with as much as eight minutes’ delay, Seirios will start to move as the ship pulls its cable. If the scientists meanwhile decide to take a sample, this move may need to be halted.
During most of the dive, the ROV co-pilot is pointing the Seirios camera down and forward to D2. During sampling, the co-pilot can take over the pilot’s controls, because the pilot has both hands full manipulating the complicated multi-jointed joystick to pick up a sample. The co-pilot also moves the sample box “tray” out and over, in position for the arm. During sampling, the pilot keeps one or both of his “toes” (D2’s lower front corners) on the surface for stability and checks his different “eyes” (camera views) for coordinating his "claw" spatially.
The ROV pilot controls D2's grasping arm, while the co-pilot points the main camera. Image courtesy of William J. Clancey. Download larger version (jpg, 564 KB).
While all this is occurring, the outreach team is using other computer tools to extract, organize, describe, and post highlight imagery, video, and mission logs for use on the web. Describing their work would surely merit another mission log.
In studying the Okeanos exploration system in this way, describing how people do their work (their “practices”), we can understand what is working well and why. This helps us formulate design principles for applying the lessons learned here in the future, perhaps for more advanced systems using new forms of automation and new ways to explore and collaborate with telepresence.
