JP7186539B2 - RUST DETECTION PROGRAM, RUST DETECTION SYSTEM AND RUST DETECTION METHOD - Google Patents

Description

The present invention relates to a rust detection program, a rust detection system, and a rust detection method for detecting the presence of rust.

Conventionally, deterioration of infrastructure facilities such as bridges and steel towers has been regularly monitored.

For example, Patent Literature 1 describes a photograph DB that stores photographed images of social infrastructure, an inspection information DB that stores inspection information including social infrastructure position information, damage status of social infrastructure, and social infrastructure name, and a tablet terminal. A function that overlaps the already-taken image as a watermarked composition image on the camera app screen, and compares the taken image with the already-taken image and adjusts the taken image to the composition of the already-taken image when there is a difference. Disclosed is a damage situation determination system equipped with a management server having a function of performing matching correction and a function of comparing the photographed image after the matching correction with an already photographed image to determine the degree of damage due to damage and rust color density change. there is

JP 2016-133320 A

However, there is a demand for more efficient determination of the state of such infrastructure equipment, particularly detection of rust.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems of the prior art, and an object of the present invention is to provide a rust detection program, a rust detection system, and a rust detection method that can improve the efficiency of rust detection. .

In order to solve the above-described problems, the present invention inputs an image of an object to a first learning model, which is a deep learning model with high reproducibility and high expressiveness for a point-like distribution. a first rust candidate region detection procedure for detecting a first rust candidate region in the object based on information output from one learning model; inputting the image to a second learning model, which is a precision deep learning model, and detecting a second rust candidate region in the object based on information output from the second learning model; The rust candidate area detection procedure, the first rust candidate area detected by the first rust candidate area detection procedure, and the second rust candidate area detected by the second rust candidate area detection procedure and a determination procedure for determining a rust generation area in the object based on the rust candidate area .

In addition, the present invention inputs an image of an object to a first learning model, which is a deep learning model with high reproducibility that has high expressive power for a point distribution, and outputs from the first learning model. a first rust candidate region detection unit for detecting a first rust candidate region in the object based on the obtained information; and a high precision deep learning model having high expressive power for linear connectivity. A second rust candidate region detection unit for inputting the image to a second learning model and detecting a second rust candidate region in the object based on information output from the second learning model and the first rust candidate area detected by the first rust candidate area detection unit and the second rust candidate area detected by the second rust candidate area detection unit. and a determination unit that determines a rust generation area in the object based on the above.

In addition, the present invention inputs an image of an object to a first learning model, which is a deep learning model with high reproducibility that has high expressive power for a point distribution, and outputs from the first learning model. a first rust candidate region detection step of detecting a first rust candidate region in the object based on the obtained information; and a high precision deep learning model having high expressive power for linear connectivity. a second rust candidate region detection step of inputting the image to a second learning model and detecting a second rust candidate region in the object based on information output from the second learning model and the first rust candidate area detected by the first rust candidate area detecting step and the second rust candidate area detected by the second rust candidate area detecting step. and a determination step of determining a rust generation area in the object based on the above.

According to the present invention, rust detection can be made more efficient.

FIG. 1 is an explanatory diagram of a rust detection program according to this embodiment. FIG. 2 is an explanatory diagram of a specific example of rust detection. FIG. 3 is a system configuration diagram showing the system configuration of the rust detection system. FIG. 4 is an explanatory diagram of a display example of the rust detection device. FIG. 5 is a flow chart showing the processing procedure of the rust detection device. FIG. 6 is an explanatory diagram of a specific example of a high recall model. FIG. 7 is an explanatory diagram of transform kernels in convolution of a high precision model. FIG. 8 is a diagram showing an example of a hardware configuration.

EMBODIMENT OF THE INVENTION Hereinafter, with reference to an accompanying drawing, embodiment of the rust detection program, the rust detection system, and the rust detection method which concern on embodiment of this invention is described.

In the present embodiment, an image of an object is captured, the image is input to two learning models, first and second, the presence of rust is detected by each learning model, and the two detection results are used to A region where rust is generated on an object is determined.

FIG. 1 is an explanatory diagram of a rust detection program according to this embodiment. When the rust detection program captures an image of an object, it pre-processes the captured image and inputs it to two learning models.

The two learning models are models already trained by deep learning, one of which is a high recall model and the other is a high precision model. Upon receiving an image input, the high recall model and the high precision model determine the possibility of each pixel being a rust region and output it. Whether a learning model has a high recall rate or a high precision rate is relatively determined by comparing the characteristics of the learning models to be used. That is, if a learning model has a higher recall rate than another learning model used in the present invention, the learning model can be used as the first learning model, and if the precision rate is high, the learning model can be used as the second learning model. It can be used as a learning model. The recall of the first learning model is preferably, for example, about 95%, and the difference in recall from the second learning model is preferably 10% or less. The precision of the second learning model is preferably, for example, about 90%, and the difference in precision from the first learning model is preferably 10% or less.

Here, the high-recall model has high expressive power for the point-like distribution and may erroneously detect non-rust regions, but it is a model with high detection performance that leaves few rust regions undetected. On the other hand, a high precision model is a model that has high expressive power for linear connectivity and high generalization performance with few false positives.

Therefore, even if the same preprocessed image is input to the high recall model and the high relevance model, each pixel is determined by the high recall model and the high relevance model, respectively. “Gender” is not necessarily the same, and usually has different values.

The rust detection program compares the "possibility of being a rust area" for each pixel output by each model with a threshold value, and detects pixels whose "possibility of being a rust area" exceeds the threshold as pixels of candidate rust areas. do.

The threshold used for detecting the rust candidate region can be determined as appropriate, and can be made different, for example, between the high recall model and the high relevance model. That is, the output of the high-recall model may be compared to the threshold for the high-precision model, and the output of the high-precision model may be compared to the threshold for the high-precision model.

The rust detection program synthesizes the two detection results when the rust candidate area detection result by the first learning model and the rust candidate area detection result by the second learning model are obtained for each input pixel. , are the determination results of the rust region. In this synthesis, pixels detected as rust candidate regions in both the first and second learning models are determined to be rust region pixels. Further, as will be described later, in the determination of the rust area, it is possible to obtain the certainty of the determination for each pixel, that is, the degree of certainty of estimation as to whether or not it is a rust area.

The degree of certainty of each pixel may be obtained, for example, by weighting and summing the outputs of the first learning model and the second learning model. Also, the degree of certainty of estimation as to whether it is a rust region can also be used as a numerical value indicating the degree of aging of the object.

In this way, the rust detection program inputs an image of an object to two different learning models, detects the presence of rust in each learning model, synthesizes the detection results of the two learning models, Since the region of rust in is determined, rust detection can be made more efficient. In addition, the determination result of the rust area by the rust detection program can be linked to various business systems such as an aging diagnosis system.

FIG. 2 is an explanatory diagram of a specific example of rust detection. In FIG. 2, an image of a bridge as an object is captured. When the captured image is subjected to rust detection using the high-reproducibility model, a first rust candidate area map indicating the positions of rust candidate areas in the image is output as a detection result. In the detection results shown in FIG. 2, the pixels with the possibility of rust exceeding the threshold are colored white, and the pixels with the possibility of rust being equal to or less than the threshold are colored black.

Similarly, when a captured image is subjected to rust detection using a high precision model, a second rust candidate area map indicating the positions of rust candidate areas in the image is output as a detection result. In the detection results shown in FIG. 2, the pixels with the possibility of rust exceeding the threshold are colored white, and the pixels with the possibility of rust being equal to or less than the threshold are colored black.

Comparing the first rust candidate region map based on the high recall model and the second rust candidate region map based on the high precision model, there is a difference in pixels detected as rust candidate regions. This is due to the difference in model characteristics that the high precision model has few false detections and the high recall model has few omissions.

The rust detection program combines two types of rust candidate area detection results output by the first learning model with a high recall rate and the second learning model with a high precision rate to appropriately determine the rust area. can be done.

The two types of rust candidate area detection results are synthesized, for example, by setting the value of each pixel determined to be a rust candidate area to "1" and the value of each pixel determined not to be a rust candidate area to "0". This can be done by taking the logical product of the values.

Next, the system configuration of the rust detection system will be described. FIG. 3 is a system configuration diagram showing the system configuration of the rust detection system. In FIG. 3, a bridge, which is an object, is imaged by an imaging unit provided in a camera 22 or a small unmanned aerial vehicle 21, and the image of the bridge is transmitted to the rust detection device 10 via the network.

The rust detection device 10 has a communication section 11 , an input section 12 , a display section 13 , a storage section 14 and a control section 15 . The communication unit 11 is a communication interface that receives images of objects from the camera 22 and the small unmanned aerial vehicle 21 . The input unit 12 is an input device such as a keyboard and mouse. The display unit 13 is a display device such as a liquid crystal display device.

The storage unit 14 is a storage device such as a hard disk drive or a nonvolatile memory, and stores a second learning model (hereinafter also referred to as high precision model data 14a), a first learning model (hereinafter also referred to as high recall model data 14b). ), learned pattern data 14c, rust region data 14d, and deterioration evaluation data 14e.

The high-relevance model data 14a is data specifying the structure of a multi-layer neural network and interlayer parameters in the high-relevance model. Similarly, the high recall model data 14b is data specifying the structure of the multi-layer neural network and the interlayer parameters in the high recall model.

The learning pattern data 14c is sample data in which learning input data and correct output data are set. The learning pattern data 14c is used for learning the high precision model data 14a and the high recall model data 14b, that is, for optimizing the interlayer parameters. Note that the learning pattern data 14c may be deleted after the learning is completed. Alternatively, if the already learned high precision model data 14a and high recall model data 14b are used, the learning pattern data 14c is unnecessary.

The rust area data 14d is data indicating the determination result of the rust area for the input image. The aging evaluation data 14e is data indicating the evaluation result of the degree of aging of the object.

The control unit 15 is a control unit that controls the entire rust detection device 10, and includes an image acquisition unit 15a, a pre-processing unit 15b, a first rust detection unit 15c, a second rust detection unit 15d, a rust area determination unit 15e, and a rust area determination unit 15e. It has an aging degree calculator 15f. In practice, programs corresponding to these functional units are stored in a ROM or non-volatile memory (not shown), and these programs are loaded into a CPU (Central Processing Unit) and executed so that the image capture unit 15a , the pre-processing unit 15b, the first rust detection unit 15c, the second rust detection unit 15d, the rust area determination unit 15e, and the aging degree calculation unit 15f, respectively, execute corresponding processes.

The image capturing unit 15a is a processing unit that acquires an image including an image of a target object. Specifically, the image capturing unit 15a captures the image received by the communication unit 11 from the camera 22 or the small unmanned aerial vehicle 21 as an image including the image of the object.

The pre-processing unit 15b is a processing unit that pre-processes the image captured by the image capturing unit 15a. Arbitrary image processing such as color tone correction and edge enhancement can be used as the pre-processing.

The first rust detection unit 15c inputs the image pre-processed by the pre-processing unit 15b into the first learning model, and detects rust on the object based on the information output from the first learning model. It is a processing unit that detects presence.

The second rust detection unit 15d inputs the image preprocessed by the preprocessing unit 15b into a second learning model having a structure different from that of the first learning model, and outputs the image from the second learning model. It is a processing unit that detects the presence of rust on the object based on the information obtained.

Specifically, the first rust detection unit 15c reads the high-reproduction model data 14b, inputs an image, compares the probability of each pixel output from the high-reproduction model with the first threshold, Pixels exceeding the first threshold are detected as pixels of the first rust candidate region.

Then, the second rust detection unit 15d reads out the high relevance model data 14a, inputs an image, compares the probability of each pixel output from the high relevance model with the second threshold, and determines the second rust detection unit 15d. Pixels exceeding the threshold are detected as pixels of the second rust candidate region.

The rust area determination unit 15e is a processing unit that determines a rust generation area on the object based on the detection result of the first rust detection unit 15c and the detection result of the second rust detection unit 15d. Specifically, the rust area determination unit 15e determines that the pixels detected as the rust candidate area by the first rust detection unit 15c and detected as the rust candidate area by the second rust detection unit 15d are pixels forming rust. Determine that there is.

The rust area determination unit 15e stores the determination result in the storage unit 14 as rust area data 14d. Further, the rust area determination unit 15e can control display of the rust area data 14d, that is, the state of rust generation in the object on the display unit 13. FIG.

The deterioration degree calculation unit 15f is a processing unit that calculates a numerical value indicating the degree of deterioration of the object based on the determination result of the rust region determination unit 15e. For example, by weighting and summing the outputs of the high precision model and the high recall model, the degree of confidence in the estimation that it is a rusted area can be obtained, and this degree of confidence can be used as a numerical value indicating the degree of aging. good.

The aging degree calculation unit 15f stores the calculated aging degree of each pixel in the storage unit 14 as the aging evaluation data 14e. Further, the aging degree calculation unit 15f can control display of the aging evaluation data 14e on the display unit 13. FIG.

FIG. 4 is an explanatory diagram of a display example of the rust detection device. As shown in FIG. 4(a), when an image including an image of an object is captured and a rust area is determined, rust area data shown in FIG. 4(b) can be obtained. This rust area data is image data in which the pixels determined to be the rust area are replaced with a specific color.

FIG. 4C is a display example of deterioration evaluation data. FIG. 4C shows a heat map in which the degree of aging is associated with colors. In this way, by displaying the degree of aging in different display colors of the pixels, it is possible to visually express the degree of aging in an easy-to-understand manner. In the example of FIG. 4C, the lower the degree of aging, the darker the pixel color, and the higher the degree of aging, the brighter the pixel color of image data is displayed. Specifically, the degree of certainty of estimation of the rust area is calculated as a percentage, and the percentage and the color of the pixel are displayed in association with each other. Specifically, when the certainty that the area is a rust area is 100%, the area is displayed in a color close to R = 255, G = 255, B = 255, and when it is 0%, Colors close to R=0, G=0, and B=0 can be displayed, and if the reliability is intermediate between these, it can be displayed in an intermediate color selected according to the degree of reliability. In addition, it is also possible to clarify and display areas that have exceeded a certain degree of aging by adding highlights or the like. This makes it easier to visually grasp the portion where aging has progressed.

Next, a processing procedure of the rust detection device 10 will be described. FIG. 5 is a flow chart showing the processing procedure of the rust detection device 10. As shown in FIG. First, the image capturing unit 15a acquires an image including an image of a target object, which is received by the communication unit 11 from the camera 22 or the small unmanned aerial vehicle 21 (step S101). Preprocessing is performed on the image captured by 15a (step S102).

The first rust detection unit 15c inputs the image preprocessed by the preprocessing unit 15b into the first learning model (step S103), compares the output of each pixel with the first threshold, is detected as a first rust candidate area (step S104).

The second rust detection unit 15d inputs the image pre-processed by the pre-processing unit 15b into the second learning model (step S105), compares the output of each pixel with the second threshold, is detected as a second rust candidate area (step S106).

The rust area determination unit 15e determines a rust area from the first rust candidate area and the second rust candidate area (step S107). Further, the aging degree calculation unit 15f calculates the aging degree indicating the degree of aging of the rust area (step S108). Then, the determined rust area and the degree of deterioration are output to the display unit 13 or the like as appropriate (step S109), and the process ends.

Learning models with different detection results can be used as the first and second learning models. In particular, it is preferable to use models with mutually different structures. By combining rust candidate regions detected by learning models having different structures, the rust region can be determined more appropriately. In particular, as described in detail below, it is preferable to use a high recall model as the first learning model and a high precision model as the second learning model. A deep learning model is preferably used as the first and second learning models. As a result, the image can be analyzed with high accuracy, and the rust region can be determined efficiently.

Next, specific examples of the high recall model as the first learning model and the high precision model as the second learning model will be described. FIG. 6 is an explanatory diagram of a specific example of the high recall model. The high recall model shown in FIG. 6 is a multilayer neural network that performs encoding and decoding using pooling layers.

The encoding used for high-recall models is a process of gradually decreasing spatial dimensionality. Decoding then gradually restores the details and spatial dimensions of the object. Also, in order to assist restoration in decoding, a copy (shortcut) is performed from the layer before encoding to the layer of the corresponding dimension.

On the other hand, as a high precision model, a multilayer neural network that does not have a pooling layer and aggregates by convolution can be used. Pooling is effective in increasing the receptive field in the classification network, but the loss of resolution can be detrimental to classification.

Therefore, in the high precision model, a dilated convolutional layer as shown in FIG. 7 can be used. Dilated convolutional layers are also called Atrous convolutions. By using dilated convolutional layers, the field of view can be increased exponentially without reducing the spatial dimension.

FIG. 7 is an explanatory diagram of transform kernels in convolution of a high precision model. As shown in Fig. 7, setting the rate of the 3x3 transform kernel to 1 will obtain the features of the object in a 3x3 field of view, and setting the rate of the 3x3 transform kernel to 6 will result in , 13×13 field of view to acquire the features of the object. Then, setting the rate of the 3×3 transform kernel to 24 will acquire the features of the object in a 47×47 field of view. In this way, by changing the rate of the transform kernel to obtain features and synthesizing them, it is possible to aggregate features while maintaining the dimensionality of the object.

Next, the correspondence relationship between the rust detection device 10 according to the embodiment and the main hardware configuration of the computer will be described. FIG. 8 is a diagram showing an example of a hardware configuration.

A general computer has a configuration in which a CPU 31 , a ROM 32 , a RAM 33 , a nonvolatile memory 34 and the like are connected by a bus 35 . A hard disk device may be provided instead of the nonvolatile memory 34 . For convenience of explanation, only the basic hardware configuration is shown.

Here, the ROM 32 or the nonvolatile memory 34 stores programs and the like necessary for booting an operating system (hereinafter simply referred to as "OS"). to read and execute the OS program.

On the other hand, various application programs executed on the OS are stored in the nonvolatile memory 34, and the CPU 31 executes the application programs while using the RAM 33 as the main memory, thereby executing processes corresponding to the applications. be.

The rust detection program of the rust detection device 10 according to the embodiment is also stored in the non-volatile memory 34 or the like like other application programs, and the CPU 31 loads and executes the rust detection program. . In the case of the rust detection device 10 according to the embodiment, the image acquisition unit 15a, the pre-processing unit 15b, the first rust detection unit 15c, the second rust detection unit 15d, the rust region determination unit 15e and the rust area determination unit 15e shown in FIG. A rust detection program including a routine corresponding to the aging degree calculator 15f is stored in the nonvolatile memory 34 or the like. Then, the rust detection program is loaded and executed by the CPU 31, so that the image capture unit 15a, the pre-processing unit 15b, the first rust detection unit 15c, the second rust detection unit 15d, the rust area determination unit 15e, and the deterioration degree calculation are performed. A process corresponding to part 15f is generated.

As described above, the rust detection program according to the present embodiment inputs the captured image of the object to the first learning model, and based on the information output from the first learning model, inputting an image to a first rust detection procedure for detecting the presence of rust in the first learning model and a second learning model having a different structure from the first learning model, and based on the information output from the second learning model , a rust generation area on the object based on a second rust detection procedure for detecting the presence of rust on the object, a detection result by the first rust detection procedure, and a detection result by the second rust detection procedure; By causing a computer to execute the judgment procedure for judging the rust, it is possible to improve the efficiency of rust detection.

Further, the rust detection program according to the present embodiment uses a deep learning model with high recall that has high expressive power for point-like distribution as the first learning model, and a linear connection model as the second learning model. A high precision deep learning model with high expressive power for gender can be used.

Further, the rust detection program according to the present embodiment can use a multi-layer neural network that performs encoding and decoding as the first learning model, and a multi-layer neural network that performs aggregation by convolution as the second learning model.

Further, in the rust detection program according to the present embodiment, the first learning model and the second learning model output the probability that each pixel of the image is a pixel forming rust, and the first rust detection procedure is , detecting pixels for which the probability for each pixel output from the first learning model exceeds a first threshold, and a second rust detection procedure detects pixels for which the probability for each pixel output from the second learning model exceeds the first It is desirable to arrange to detect pixels exceeding a second threshold different from the threshold of .

Further, the rust detection program according to the present embodiment determines that pixels detected in the first rust detection procedure and detected in the second rust detection procedure are pixels forming rust. , the pixels forming rust can be detected efficiently.

Further, in the rust detection program according to the present embodiment, it is desirable that the first learning model and the second learning model undergo supervised learning using the same learning data.

Further, the rust detection program according to the present embodiment can display and control the state of rust generation on the object based on the determination result of the determination procedure.

Further, the rust detection program according to the present embodiment can calculate a numerical value indicating the degree of deterioration of the object based on the determination result of the determination procedure.

In the present embodiment, the case where the object is a bridge has been described as an example, but the present invention is not limited to this, and rust on any object such as a steel tower, a plant structure, etc. It can be applied to detection.

In addition, although the rust detection program has been mainly described in the present embodiment, it can be implemented in any form such as a rust detection system, a rust detection method, a rust detection device, and the like. When implemented as a rust detection device, a computer may execute a rust detection program to operate as a rust detection device, or hardware (ASIC, FPGA, etc.) may be used to implement the functional unit. It may be configured as follows.

In addition, each configuration illustrated in the above embodiment is a functional schematic, and does not necessarily need to be physically configured as illustrated. That is, the form of distribution/integration of each device is not limited to the illustrated one, and all or part of them can be functionally or physically distributed/integrated in arbitrary units according to various loads and usage conditions. Can be configured. Also, two or more learning models may be used.

10 rust detection device 11 communication unit 12 input unit 13 display unit 14 storage unit 14a high precision model data 14b high recall model data 14c learning pattern data 14d rust area data 14e aging evaluation data 15 control unit 15a image capturing unit 15b Pre-processing part 15c First rust detection part 15d Second rust detection part 15e Rust area determination part 15f Aging degree calculation part 21 Small unmanned aerial vehicle 22 Camera 31 CPU
32 ROMs
33 RAM
34 non-volatile memory 35 bus

Claims (9)

An image of the target object is input to the first learning model, which is a deep learning model with high reproducibility that has high expressive power for point distribution, and based on the information output from the first learning model , a first rust candidate area detection procedure for detecting a first rust candidate area in the object;
The image is input to a second learning model, which is a deep learning model with high precision and high expressive power for linear connectivity, and based on the information output from the second learning model, the a second rust candidate region detection procedure for detecting a second rust candidate region in the object;
Based on the first rust candidate area detected by the first rust candidate area detection procedure and the second rust candidate area detected by the second rust candidate area detection procedure , a determination procedure for determining a rust generation area in the object, and a rust detection program characterized by causing a computer to execute. The first learning model is a multilayer neural network that performs encoding and decoding,
2. The rust detection program according to claim 1, wherein said second learning model is a multi-layer neural network that aggregates by convolution. The first learning model and the second learning model output a probability that each pixel of the image is a pixel forming rust,
The first rust candidate region detection step detects pixels, for which the probability of each pixel output from the first learning model exceeds a first threshold , as pixels forming the first rust candidate region ,
In the second rust candidate region detection procedure, pixels whose probability for each pixel output from the second learning model exceeds a second threshold value different from the first threshold value are identified as the second rust candidate region. The rust detection program according to claim 1 or 2, wherein the rust is detected as a pixel to be formed . The determination procedure is
forming part of the first rust candidate region detected in the first rust candidate region detection procedure and of the second rust candidate region detected in the second rust candidate region detection procedure 4. The rust detection program according to any one of claims 1 to 3 , wherein the pixels forming a part are determined to be the pixels forming the rust generation area . The rust detection program according to any one of claims 1 to 4, wherein the first learning model and the second learning model are subjected to supervised learning using the same learning data. 6. The rust according to any one of claims 1 to 5, further causing a computer to execute a display control procedure for controlling the display of the rust generation area on the object based on the judgment result of the judgment procedure. detection program. 7. The computer according to any one of claims 1 to 6, further causing the computer to execute a quantification procedure for calculating a numerical value indicating the degree of aging of the object based on the determination result of the determination procedure. rust detection program. An image of the target object is input to the first learning model, which is a deep learning model with high reproducibility that has high expressive power for point distribution, and based on the information output from the first learning model , a first rust candidate region detection unit for detecting a first rust candidate region in the object;
The image is input to a second learning model, which is a deep learning model with high precision and high expressive power for linear connectivity, and based on the information output from the second learning model, the a second rust candidate region detection unit that detects a second rust candidate region in the object;
Based on the first rust candidate area detected by the first rust candidate area detection unit and the second rust candidate area detected by the second rust candidate area detection unit , and a determination unit that determines a rust generation area in the object. An image of the target object is input to the first learning model, which is a deep learning model with high reproducibility that has high expressive power for point distribution, and based on the information output from the first learning model , a first rust candidate region detection step of detecting a first rust candidate region in the object;
The image is input to a second learning model, which is a deep learning model with high precision and high expressive power for linear connectivity, and based on the information output from the second learning model, the a second rust candidate region detection step of detecting a second rust candidate region in the object;
Based on the first rust candidate area detected by the first rust candidate area detecting step and the second rust candidate area detected by the second rust candidate area detecting step , and a determination step of determining a rust-generated region in the object.

Images