Sea Trials Begin for Artificial Intelligence "Captain"

Published Mar 7, 2020 7:55 PM by The Maritime Executive

IBM and marine research organization Promare have announced that a new artificial intelligence captain (AI Captain) will enable the Mayflower autonomous ship to self-navigate across the Atlantic later this year.

With the three hulls of the trimaran Mayflower currently reaching the final phase of construction in Gdansk, Poland, a prototype of the AI Captain will first take to the water on a manned vessel - the Plymouth Quest - a research ship owned and operated by the Plymouth Marine Laboratory in the U.K.

The trials starting this month will evaluate how the AI Captain uses cameras, artificial intelligence and edge computing systems to safely navigate around ships, buoys and other ocean hazards that the Mayflower is expected to meet during her transatlantic voyage in September 2020. She will sail from Plymouth, to Plymouth, Massachusetts, with no onboard crew, tracing the route of the original 1620 Mayflower to commemorate the 400th anniversary of the famous voyage.

"While the autonomous shipping market is set to grow from $90 billion today to over $130 billion by 2030, many of today's autonomous ships are really just automated - robots which do not dynamically adapt to new situations and rely heavily on operator override," said Don Scott, CTO of the Mayflower Autonomous Ship. "Using an integrated set of IBM's AI, cloud and edge technologies, we are aiming to give the Mayflower the ability to operate independently in some of the most challenging circumstances on the planet."

Over the past two years, the Mayflower team has been training the ship's AI models using over a million nautical images collected from cameras in the Plymouth Sound in the U.K. as well as open source databases. Using IBM's computer vision technology, the Mayflower's AI Captain should be able to independently detect and classify ships, buoys and other hazards such as land, breakwaters and debris.

To meet the processing demands of machine learning, the team used an IBM Power AC922 using by IBM Power9 CPUs and NVIDIA V100 Tensor Core GPUs, the same technologies behind the world's smartest AI supercomputers. As the Mayflower will not have access to high-bandwidth connectivity throughout its transatlantic voyage, she will use a fully autonomous IBM edge computing system powered by several onboard NVIDIA Jetson AGX Xavier devices. She will process data locally on NVIDIA Jetson, increasing the speed of decision making and reducing the amount of data flow and storage on the ship.

As well as following the overall mission objectives to reach Plymouth, Massachusetts in the shortest amount of time, the AI Captain will draw on IBM's rule management system (Operational Decision Manager - ODM) to follow the COLREGs as well as recommendations from SOLAS. Used widely across the financial services industry, ODM is particularly suited to the Mayflower project as it provides a completely transparent record of its decision-making process, avoiding “black box” scenarios, says IBM.

The weather is expected to be one of the most significant factors impacting the success of the voyage, and the AI Captain will use forecast data from The Weather Company to help make navigation decisions. A Safety Manager function (running on RHEL) will review all of the AI Captain's decisions to ensure they are safe for the Mayflower and for other vessels in its vicinity.

For example, it will cater for the situation where the Mayflower is in the open ocean, approaching Cape Cod, with no satellite connectivity. In her path ahead is a cargo ship which has had a collision with a fishing vessel that has lost cargo overboard. In this hypothetical scenario, the Mayflower's AI Captain will use the following technologies and processes to independently assess the situation, and decide what action to take:

Senses

• Radar detects multiple hazards in Mayflower's path, 2.5 nautical miles ahead
• Onboard cameras provide visual input to IBM computer vision system which identifies hazards as: a cargo ship, a fishing vessel and three partially submerged shipping containers floating in the water
• Automatic Identification System (AIS) provides specific information about the cargo ship's class, weight, speed, cargo, etc.
• GPS Navigation System – provides Mayflower's current location, heading, speed and course
• Mayflower's nautical chart server provides geospatial information about its chosen route
• Weather data provided by The Weather Company
• Attitude Sensors – assess local sea state
• Fathometer – provides water depth measurements
• Vehicle Management System – provides operational data such as Mayflower's battery charge level, power consumption, communications, science payloads etc.

Thinks

• IBM Operational Decision Manager (ODM) evaluates COLREGs with respect to the other vessels in the vicinity and generates a risk map indicating an "unsafe" situation ahead
• Mayflower's AI Captain ingests the ODM recommendation, computer vision input, current and forecast weather and assesses several options to avoid hazard

Acts

• AI Captain determines the best action for Mayflower is to steer to starboard to avoid the unexpected navigation hazard
• Mayflower's Safety Manager verifies the decision as safe
• AI Captain instructs Mayflower's Vehicle Management system to change course and speed.
• As the ocean is an ever-changing dynamic environment, the AI Captain will constantly re-evaluate the situation and update the course of the Mayflower as situations evolve.

The March sea trials will take place for approximately two months on the Plymouth Quest with the ship's human captain and crew at the helm. In the first stage of testing, the Mayflower AI Captain's inference engine will receive input from the Quest's radar, AIS, GPS and navigation system, as well as data about visibility. Cameras, computer vision, edge and autonomy capabilities will be added in the next phase of testing from April.