7. Forward Obstacle Detection System for Trains in Preparation for Future Driverless Operation

Obstacle detection ahead of the train using on-board cameras and sensors is considered one of the most promising technologies for ensuring railway track safety when introducing future driverless operations on general railway lines, such as those with level crossings. To achieve this, a system that integrates multiple sensors is required, including capabilities to detect obstacles at night and respond to incidents such as wayside fires.

To address this, we developed a forward obstacle detection system for trains using sensor fusion. This system integrates a visible light camera, a LiDAR sensor, and a far-infrared camera. It uses AI to extract the railway track area from visible light camera images, detects objects that have entered the railway track area, measures distance using LiDAR, and detects temperature using the far-infrared camera. On the other hand, integrating multiple sensors requires calibration (correction and adjustments) of each individual sensor, a process that can be complex and time-consuming. To address this, we developed a method for collectively calibrating various sensors using railway-specific rail information (Figure 1).

In verification tests conducted on actual straight railway tracks, the system was able to detect wildlife (deer) at up to 376 meters, wayside fire flames at up to 502 meters, people at up to 556 meters, and automobiles at up to 614 meters (Figure 2).

The insights gained from developing this system can be used to determine specifications of forward obstacle detection system for trains tailored to railway operators’ line conditions. Additionally, they will be applied as foundational technology for the currently in-development “automatic train operation system utilizing existing railway system.” Furthermore, the calibration method can be utilized as a core technology for various forward obstacle detection systems.