Machine Learning for Automated Detection of Shipwreck Sites from Large Area Robotic Surveys

Field expeditions were focused in Thunder Bay National Marine Sanctuary (TBNMS), which is unique in its abundance of known shipwreck sites of varying type, size, and wreck conditions. The team conducted eight shore-based day operations throughout the sanctuary on R/V Storm between May 23-June 3, 2022. Each mission consisted of traveling to the initial starting point, deploying robotic systems, and conducting surveys in a local area. When the mission was complete, we moved to the next planned site.

To collect side-scan sonar data, we deployed Michigan Technological University’s IVER-3 autonomous underwater vehicle (AUV). Surveys covered a total of 8.04 square kilometers (3.1 square miles) over the course of field trials. Figure A shows an example of the high-resolution side-scan sonar imagery we collected with the AUV over the Grecian site.

Working with maritime archaeologists at the sanctuary, we created high-resolution labels of shipwreck sites, distinguishing between background, ships, debris, and moorings. Figure B shows a sample label. Labeled data allows us to train and evaluate machine learning methods for detecting shipwreck sites in side-scan sonar imagery.

We also conducted preliminary deployments of a remotely operated vehicle (ROV) for close-range imaging surveys of shipwreck sites. Over the course of the expedition, we had a total of 13 ROV deployments with three different ROV platforms. With data collected by the ROV, we are able to create large-scale 3D reconstructions of shipwreck sites. Figure C shows initial 3D reconstructions obtained from data collected by the ROV.

The data collected during the Year 1 field expeditions will lay the foundation for developing and training machine learning methods in Year 2 of the project.

By Katie Skinner, Assistant Professor, University of Michigan

Field work took place in the Thunder Bay National Marine Sanctuary. Figure 1 shows an overview of the planned survey regions, though modifications were made to these survey regions to accommodate weather and vehicle operating status each day.

Operations consisted of shore-based day expeditions between June 5-9, June 13-16, and June 20-21, 2023, during which we visited 15 sites. Each mission consisted of traveling to the initial waypoint, deploying robotic systems, and conducting surveys in a local area. When the mission was complete, we moved to the next planned waypoint. For remotely operated vehicle operations, we used moorings to ensure safe operation.

As a collaboration between the University of Michigan, Louisiana State University, and Michigan Technological University, the full detection pipeline involved an initial survey from an IVER3 autonomous underwater vehicle (AUV) equipped with a side-scan sonar followed by our novel shipwreck detection model which produced GPS coordinates of potential wreck sites (see Figure 2).

These coordinates were passed along to an autonomous planning and navigation algorithm on a BlueROV, which was used to collect up-close camera and depth data for 3D reconstruction. Figure 3 shows the survey path that was automatically generated by algorithms developed by the Louisiana State University team.

Once the survey was planned, the modified BlueROV, Dory, was deployed to conduct an autonomous survey of the site. Figure 4 shows deployment of the Dory AUV from the R/V Storm research vessel, and Figure 5 shows imagery collected during an autonomous survey of the Barge No. 1 shipwreck site.

Overall, this field expedition successfully demonstrated the full pipeline developed by the multi-institute team to deploy robotic platforms to automatically detect shipwreck sites from sidescan sonar imagery, and then perform planning and navigation to carry out autonomous surveys of detected shipwreck sites. The collected data will be used to compute 3D models of surveyed sites.

By Anja Sheppard (Ph.D. Student, University of Michigan) and Katie Skinner (Assistant Professor, University of Michigan)