8. Support system for preventing a decline in alertness level of drivers

As part of the efforts of railway operators to ensure safe and stable transport services, preventing a decline in the alertness level of train drivers is an important challenge. While methods for estimating the alertness level of drivers have been implemented in automobiles, in the case of trains, there is an issue with ensuring accuracy due to the wide variety of driver behaviors.
This study attempted to address this problem by developing a system that estimates a driver’s alertness level based solely on information from images of the driver’s face while driving and issues an alarm sound when the alertness level drops by a specified amount (Fig. 1, left).

The system captures facial images using a fixed camera in the train, without any sensors contacting the driver’s body. Deep learning is used to evaluate alertness on a scale of six levels. In an experiment with 112 general test subjects, we confirmed that the system is capable of accurately gaging the alertness with an estimation accuracy of 80% or higher, even when masks are worn (Fig. 1, right).

For the alarm that is issued when a drop in alertness level is detected, we developed a sound that is highly rated for its functionality (strong arousing effect), perceptibility (easy to hear), and its distinguishability (not mistaken for other sounds).
When test subjects in a laboratory experiment suffered a drop in alertness, their reaction time in a simple reaction task recovered when exposed to this alarm sound (Fig. 2), with approximately 90% of subjects responding “I felt awakened” in a questionnaire.

After the system was trialed on a commercial train, the results confirmed that the system accurately captures facial images, even under diverse lighting conditions, and that it is capable of estimating the alertness level of drivers in real time. Three drivers (including a conductor) rated the arousal effect of the alarm sound, giving it a score of 5 or more out of 6 points (Fig.
3).