Vision AI Goes Underwater

AI computer vision technology has been automating imagery analysis in shoreside applications for years, identifying, categorizing and counting everything from cows to car parts. It is starting to make inroads in maritime for tasks like automatically spotting vessels and whales. Now computer vision is also standing watch underwater, thanks to a partnership between AI company Plainsight and tech startup MarineSitu. 

MarineSitu is a spinoff of the University of Washington's Applied Physics Laboratory (APL-UW), a Navy-supported institute with decades of history in oceanography. The startup builds integrated sensor / video / sonar packages for monitoring marine life, particularly for renewable energy installations. To scale up and analyze a huge volume of underwater video footage, MarineSitu has partnered with Plainsight to add vision AI to its products. The Maritime Executive recently spoke with Logan Spears, Plainsight’s Co-Founder & CTO, and James Joslin, MarineSitu’s Founder & President. to learn more about recent advances in computer vision technology and potential applications under the sea. 

TME: Can you tell us about MarineSitu?

James: Sure. MarineSitu came out of the University of Washington as a spinoff company, and we are trying to commercialize some of the research technologies that have been developed over the last decade at the university. In particular, we work with technology for installing instruments underwater for continuous long term monitoring.

When you have cameras or sonars or other instruments underwater for long periods of time, the large amount of data you accrue can get in the way of doing good science, unless you can implement real-time data processing. And so that is where using these AI machine learning tools comes in, providing the ability to do continuous data processing so you're not building up what we call a "data mortgage." This reduces storage requirements within the unit and communications requirements if you are in an area that has limited bandwidth.

TME: What makes Plainsight's AI well-suited for this task? 

Logan: One of the things we help provide is to accelerate the AI journey by automating some of that data life cycle. That includes the data collection, data processing and data labeling. Labeling in machine learning is basically telling the AI model what to care about and what specific things to look for, and then automatically training machine learning models. Once that's done, those models are available at the edge where they can be used in the offline scenarios that James was referring to.

James: We also wanted the ability to have models that are continually learning and continually being retrained, because marine environments change very quickly. At each site where you deploy a camera or a system, the machine learning model is going to need to be retrained specifically for that site and then have continual learning so that it stays optimized. With this system, the edge device pushes a subset of its data up to the cloud, where it can be used to train and update the model.

TME: What applications do you have in mind for this technology?

James: It's dependent on the interests of the customer, but our background is in monitoring marine life near renewable energy devices, like tidal turbines and wave energy converters. However, there are applications across the blue economy for the same type of technology, and the partnership with Plainsight will help us to grow and scale.  

With Plainsight AI, MarineSitu can identify and catalogue fish in its video imagery (Illustration courtey Plainsight)

TME: Can you tell us about what your systems can do?

James: As a company, we're looking to provide a turnkey solution for long-term monitoring underwater. That includes hardware components like cameras, lights and integration hubs, plus the software for data acquisition, data processing and data management. We are trying to take the best technology that's available and make it as cost-effective as possible. Right now the majority of our work is in using optical cameras, but we also are working with imaging sonar, which can give ranges out to a hundred meters or more.

With AI, this technology is ideal for monitoring for a long period of time. Machine learning is really a game changer for the space, because now aquatic environments can be quantified with actual measurements - and in a way that doesn't require a human in the loop. 

TME: Are there other promising applications out there for vision AI in the maritime world? 

Logan: There's an ocean of opportunity out there. For example, let's say you're trying to monitor marine activity in every port in the entire world. You need AI to automate it to even make it possible. We're really excited about all kinds of use cases and we love partnering with companies like MarineSitu to accelerate their journey.