Book Review: How Can We Keep Humans in Control of Autonomous Systems?
How can we ensure that humans remain in control as artificial intelligence, autonomy and remote operations become increasingly important in critical maritime operations? This is one of the key questions explored in the new book "Safety by Design: Human-Centered Approaches to AI, Automation, and Remote Operations," launched in June.
The publication brings together contributions from leading researchers and experts in safety, human-machine interaction and autonomous systems.
The growing adoption of autonomous vessels, remote operations and AI-enabled decision support systems offers significant opportunities, while also creating new requirements for safety, operator support and system design.
Safety must be built in from the start
The book argues that safety cannot be treated as an add-on once technology has been developed. Instead, human factors, usability and the role of operators must be integrated into systems from the outset. Drawing on experiences from aviation, maritime operations, energy and transportation, the authors demonstrate how serious incidents often occur when technology, organisations and human work processes are not sufficiently aligned.
A central concept in the book is “Meaningful Human Control”, a framework designed to ensure that people retain situational awareness, decision-making authority and the ability to intervene when automated systems operate in complex and safety-critical environments. This is becoming increasingly important as new AI regulations are introduced across Europe.
"As automation and AI move into safety-critical domains, safety can no longer be treated as something that is verified only at the end of development. It must be designed into the relationship between people, organizations and technology from the very beginning," said co-editor Ole Andreas Alsos, Professor of Interaction Design, NTNU. "This book is important because it brings human factors, interaction design and safety science together in a practical way: it shows how we can create systems where humans retain meaningful control, operators are supported under pressure, and complex technologies become safer, more transparent and more resilient."
"Autonomous and remotely operated maritime systems will not become safe simply by removing people from the bridge or the control loop. They become safer when we understand the work operators actually have to perform, design the technology around that work, and ensure that people can build situation awareness and intervene when conditions change," added co-editor Stig Ole Johnsen of NTNU.
Relevance for autonomous maritime operations
For organizations developing the next generation of autonomous vessels, subsea systems and remotely operated maritime services, the book offers an important perspective: technology does not become safer by removing humans from the loop—it becomes safer when the interaction between people and technology is carefully designed.
The book is edited by Frøy Birte Bjørneseth, Vidar Hepsø, Stig Ole Johnsen, Ole Andreas Alsos and Gunhild Birgitte Sætren. Several of the editors and contributors have extensive experience in research and development related to maritime technology, autonomy and safety-critical operations.
MIDAS – Humans in Future Maritime Operations
The book is initiated, funded and written as part of the project MIDAS – Humans in Future Maritime Operations.
MIDAS is a national competence project that investigates the role of humans in future maritime operations, where autonomy, artificial intelligence and automation are becoming increasingly important. The ambition is to ensure that new technology is developed in ways that safeguard safety, usability and meaningful human control. The project brings together researchers, technologists, designers and industry stakeholders across the maritime value chain, with contributions from NTNU, SINTEF Digital, DNV, Digital Norway, Ocean Autonomy Cluster and Blue Maritime Cluster. Through MIDAS, the aim is to strengthen Norwegian industry’s innovation capacity and contribute to the development and export of future autonomous maritime solutions.
that matters most
Get the latest maritime news delivered to your inbox daily.
Published by CRC Press, Safety by Design is available as an Open Access book under the Creative Commons CC BY 4.0 licence. This means that the book can be downloaded free of charge, shared, copied and reused in research, education, training, industrial development and policy work, provided that the original source is properly credited.
Lars Bugge Aarset is communications advisor for Norwegian cluster development company Fremtidens Industri.
The opinions expressed herein are the author's and not necessarily those of The Maritime Executive.