How Do You Measure Training Effectiveness?

Scenario #1: You are a vessel Master and just went through a 5 day Leadership and Management course to meet the new STCW requirements with an instructor that was fantastic; in fact, the best instructor you ever had. Not once did you nod off. She kept the participants engaged and the information fresh. When it was time to evaluate the course, you gave the instructor, facility, and overall experience the highest marks. Would you say this 40 hour training experience was effective? Are you a better leader because of it? Are you a better manager?

Scenario #2: You are the Risk Manager for your company and noticed there have been multiple incidents of dental issues onboard the vessels. Digging deeper into these incidents, you uncover significant costs, especially the fees associated with airlifting a couple of crewmembers off at different times, one because of a needed emergency root canal and the other due to an abscessed tooth. As a result, you decide to discuss preventive dental care and the company’s robust dental plan at the upcoming Masters and Mates Shoreside Seminar. Your training is about awareness: what these incidents are costing the company; reminding everyone how important preventive dental care is to their health; and, reinforcing the fact that the costs of preventive dental care is 100% covered by the company’s insurance plan, which will not cost the crew a penny. The presentation lasts a total of 30 minutes, including 10 minutes for questions and answers. The training is well received, but was it effective enough to help reduce these incidents?

Scenario #3: You are the company’s Crew Development Director, responsible for ensuring qualified and competent crew members. You want to make sure that the licensed officers are competent and skilled in basic navigation. After a considerable amount of research, you decide the best and most effective way to assess all the Masters and Mates in your company is to measure their performance in the simulator, allowing an opportunity for the participants to de-brief their experience at the end of the session to help them identify “opportunities for improvement”. You spend a year coordinating and overseeing small groups of deck officers through these sessions. Did the training make a difference in helping to improve competency in basic navigation? Was it worth the time and money spent?

All 3 training scenarios are very different in the way they might be measured for effectiveness.

The first scenario is a proactive situation, where the training is required and will impact the company negatively if not received by each deck officer. So whether or not the mariner actually gained skills to be a better leader and/or manager might not be relevant. Receiving the certificate and checking the box to meet licensing requirements was probably the item that was tracked and measured. Enjoying the class was an added benefit.

The second scenario was a reactive response to events that occurred in the past and started to create a trend, which impacted the bottom line; however, this situation could result in predictive measurement for effectiveness by monitoring future dental issues, or hopefully lack of, to see if the training made a difference and for how long did the mariners retain the information. If dental issues went away for 3 years, but came back during the 4^th year since the training, then you might determine that this type of awareness training has a shelf life of 3 years (predictive).

The third scenario is also proactive in nature; however, the measurement used to gauge effectiveness will likely be much different. The Crew Development Director will probably be measuring future performance either in the simulator or onboard the vessel now that there is a baseline to see if assessment scores have improved. The same could happen with the first scenario if the required STCW elements are assessed prior to and after the training.

Any ideas you would like to share on how best to measure training effectiveness?

This Training Blog is an open forum for us to “Learn Better Together”, your thoughts and insights are always welcome. ~ Dione