7. Method of applying image processing-based object detection and obstacle determination to safety applications

Since image processing units that use cameras and general-purpose processors cannot diagnose failures by themselves, they cannot ensure sufficient safety in the event of a failure.
Therefore, they cannot be used in safety-related equipment.
One way to address this issue of image processing unit failures might be to perform image processing using a current signalling computer which is specially designed for safety-related applications (fail-safe components). Due to slow processing speed, however, this is not a practicable solution. And since image processing uses machine learning and random numbers, conventional diagnostic methods that utilize multiple image processing units to check for bit-perfect matches of processed data cannot be used either.

To solve this problem, we developed a method that works in combination with a fail-safe component to perform high-speed self-diagnosis of failures in image processing units and cameras (Fig. 1). The method diagnoses image processing errors in image processing units by focusing on the similarity of processing results.
Specifically, the processing contents of the image processing unit are compressed in the diagnostic image and the similarities are then determined in the fail-safe component as “differences of the diagnostic image = similarities.”
In addition, by adding a function to the camera that embeds diagnostic data in the image data and outputs a test pattern, we proposed a method to diagnose image freezing and other issues using a fail-safe component (Table 1).

We trial manufactured a detection equipment that implements the proposed method, to serve as an abnormality detection equipment at level crossings. By means of an artificial fault test, we verified that the method works and is capable of fault diagnosing the operational status of the equipment at 0.1-second intervals.

The proposed method can detect large differences that influence determinations, while still allowing differences in processing results in multiple image processing units arising from machine learning (e.g., background images updated every cycle).
It can therefore be applied as a fail-safe configuration in AI and general-purpose processors that handle non-image data.
This approach to camera diagnosis can also be applied to other general-purpose sensors for LiDAR, millimeter wave radar, etc.