WO2022097275A1

WO2022097275A1 - Information processing system

Info

Publication number: WO2022097275A1
Application number: PCT/JP2020/041543
Authority: WO
Inventors: 尚紀北村
Original assignee: 株式会社インキュビット
Priority date: 2020-11-06
Filing date: 2020-11-06
Publication date: 2022-05-12
Also published as: JPWO2022097275A1

Abstract

[Problem] To make it possible to recognize faults by considering over-detection. [Solution] This information processing system is characterized by comprising: a training model storage unit which stores a first training model for detecting abnormality from a first image obtained by capturing the abnormality and a second training model for estimating a degree of the abnormality from the first image and the abnormality information indicating the abnormality; an image input unit which receives an input of a second captured image; an abnormality detection unit which provides the second image to the first training model and detects abnormality in the second image; an estimation unit which provides the second image and abnormality information indicating the detected abnormality to the second training model, and estimates a degree of abnormality in the second image; and an output unit which outputs the degree of abnormality.

Description

Information processing system

The present invention relates to an information processing system.

In recent years, there is a machine learning technology called deep learning, and in the fields of image recognition, voice recognition, etc., the accuracy is much higher than before the technology appeared. These techniques are also being used to automate the visual inspection of infrastructure inspections. Patent Document 1 also proposes to detect cracks in roads from images.

Japanese Patent No. 6678267

In Patent Document 1, a defect portion is extracted by using a segmentation AI by a neural network, and a defect rate is obtained according to the size of the region. However, such a method using segmentation AI is very vulnerable to over-detection of region extraction, and even a slight over-detection recognizes a normal part as a defect.

The present invention has been made in view of such a background, and an object of the present invention is to provide a technique capable of recognizing a defect in consideration of over-detection.

The main invention of the present invention for solving the above-mentioned problems is an information processing system, which is a first learning model for detecting the abnormality obtained by learning a first image obtained by photographing the abnormality, and the first learning model. The learning model storage unit for storing the image of the above and the second learning model for estimating the degree of the abnormality trained from the abnormality information contained in the first image, and the input of the captured second image. An image input unit to be accepted, an abnormality detection unit that gives the second image to the first learning model and detects the abnormality information in the second image, the detected abnormality information, and the second. An image is given to the second learning model, and an estimation unit for estimating the degree of abnormality in the second image and an output unit for outputting the degree of abnormality are provided.

Other problems disclosed in the present application and solutions thereof will be clarified by the columns and drawings of the embodiments of the invention.

According to the present invention, it is possible to recognize a defect in consideration of over-detection.

It is a figure explaining the outline of the estimation process of the degree of abnormality by the analyzer 20 of this embodiment. It is a figure explaining the outline of the estimation process of the degree of cracking of a road. It is a figure which shows the hardware configuration example of the analyzer 20. It is a figure which shows the software configuration example of the analyzer 20. It is a figure explaining the operation of the analyzer 20 of this embodiment. It is a figure explaining the detection of a crack. It is a figure which shows the mode that the mesh is set in the photographed image 1. It is a figure which shows the mode that the photographed image 1 is divided into a mesh. It is a figure which shows the state of dividing the data of a crack 3 into a mesh. It is a figure which shows the state of estimating the degree of a crack. It is a figure explaining the 1st process which sets the mesh 8 by the image segmentation part 22. It is a figure which shows the state of setting the mesh 8. It is a figure explaining the 2nd process of setting the mesh 8 by the image segmentation part 22. It is a figure which shows the mode that the mesh 8 is set in the 2nd process. It is a figure explaining the 3rd process of setting the mesh 8 by the image segmentation part 22. It is a figure which shows the mode that the mesh 8 is set in the 3rd process.

<Outline of the invention>
The contents of the embodiments of the present invention will be described in a list. The present invention includes, for example, the following configuration.
[Item 1]
A first learning model for detecting the abnormality from a first image obtained by photographing the abnormality, and a second learning model for estimating the degree of the abnormality from the abnormality information indicating the abnormality and the first image. A learning model memory unit that memorizes and
An image input unit that accepts the input of the second image taken, and
An abnormality detection unit that applies the second image to the first learning model and detects the abnormality in the second image,
An estimation unit that estimates the degree of the abnormality in the second image by giving the detected abnormality information indicating the abnormality and the second image to the second learning model.
An output unit that outputs the degree of abnormality and
An information processing system characterized by being equipped with.
[Item 2]
The information processing system according to item 1.
It also has an image segmentation section that divides the image into meshes.
The second learning model is a model created by learning the first mesh obtained by dividing the first image and the abnormality information detected in the first mesh.
The image segmentation section divides the second image into the second mesh.
For each of the second meshes, the estimation unit obtains the abnormality information detected from the second image, which is included in the second mesh, and the second mesh. To estimate the degree of the anomaly for each mesh by giving it to the learning model of 2.
An information processing system featuring.
[Item 3]
The information processing system according to item 2.
The second image is a photograph of the road.
The image segmentation unit detects a white line drawn on the road from the second image and sets the position of the mesh with reference to the position of the white line.
An information processing system featuring.
[Item 4]
The information processing system according to any one of items 1 to 3.
Further equipped with a target range detection unit that detects the inspection target range from the image,
The target range detection unit detects the inspection target range from the second image, and the target range detection unit detects the inspection target range.
The abnormality detection unit applies the detected inspection target range to the first learning model, detects the abnormality information, and detects the abnormality information.
The estimation unit applies the detected inspection target range and the abnormality information to the second learning model to estimate the degree of the abnormality.
An information processing system featuring.

<Overview of the system>
Hereinafter, the analyzer 20 according to the embodiment of the present invention will be described. The analyzer 20 of the present embodiment is intended to estimate the degree of abnormality from an image, and is trying to estimate the degree of abnormality using a model obtained by machine learning an image. In general, machine learning using only a segmentation model is vulnerable to over-detection, while detection accuracy does not improve when only a classification model is used. Therefore, in the analyzer 20 of the present embodiment, estimation is performed by combining segmentation and classification. This makes it possible to perform highly accurate estimation that is resistant to over-detection. FIG. 1 is a diagram illustrating an outline of an abnormality degree estimation process by the analyzer 20 of the present embodiment. The analyzer 20 of the present embodiment is a classification that estimates the degree of abnormality (abnormality level 5) from the detector 2 for detecting the abnormal portion 3 from the captured image 1 and the captured image 1 and the abnormal portion 3. It is equipped with a vessel 4. The detector 2 includes, for example, a first learning model in which the captured image 1 is learned by using the result of the annotation specifying the abnormal portion 3 in the captured image 1 as a teacher signal. Thereby, by giving the unknown captured image 1 to the detector 2, the abnormal portion 3 can be detected from the captured image 1. Further, the classifier 4 includes a second learning model in which the captured image 1 and the abnormal portion 3 are learned by using the abnormality level 5 as a teacher signal. In the present embodiment, for the second learning model, not only the captured image 1 but also the abnormal portion 3 is learned.

In the analyzer 20 of the present embodiment, it is assumed that the degree of cracking of the road is estimated for use in the inspection of the road. FIG. 2 is a diagram illustrating an outline of an estimation process of the degree of cracking of a road. The analyzer 20 of the present embodiment gives a captured image 1 of a road image to the detector 2 to detect an abnormal portion 3 (hereinafter, also referred to as a cracked portion 3). By giving the photographed image 1 of the road to the classifier 4 together with the detected cracked portion 3, the abnormality level 5 (hereinafter, also referred to as the number of cracks 5) is estimated. In this way, when trying to estimate the degree of cracking from only the captured image 1 (or only the cracked portion 3), for example, if streaks such as the edge of a manhole are over-detected, the degree of cracking is higher than the current state. Since (the degree of deterioration of the road) is excessively evaluated, it is possible to make an estimation in consideration of over-detection by using both the captured image 1 and the cracked portion 3.

When learning the second learning model (hereinafter, also referred to as a classification model) used by the classifier 4, the category data is divided into a predetermined number of stages such as 5 cracks (for example, 1 to 10). It is possible to learn the captured image 1 and the cracked portion 3 specified by the annotation as input data using the teacher data as the teacher data, but instead of or in addition to the cracked portion 3 by the annotation as the input data, The captured image 1 may be given to the detector 2 to use the cracked portion 3 (detection value) obtained. This makes it possible to learn the overdetection of the detector 2. However, even if the estimation result of the detector 2 is not used for learning the classification model used for the classifier 4, the learning is performed so as to suppress over-detection by adding the cracked portion 3 to the captured image 1 and learning. It is possible.

<Structure of analyzer 20>
The analyzer 20 of the present embodiment may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing. The analyzer 2 may be realized as a computer directly operated by a user who intends to inspect a road, or may be realized as a server device accessed from a user terminal operated by the user.

FIG. 3 is a diagram showing a hardware configuration example of the analyzer 20. The configuration shown in the figure is an example, and may have other configurations. The analyzer 20 includes a CPU 201, a memory 202, a storage device 203, a communication interface 204, an input device 205, and an output device 206. The storage device 203 stores various data and programs, such as a hard disk drive, a solid state drive, and a flash memory. The communication interface 204 is an interface for connecting to a communication network, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone network, a wireless communication device for performing wireless communication, and the like. It is a USB (Universal Serial Bus) connector or RS232C connector for serial communication. The input device 205 is, for example, a keyboard, a mouse, a touch panel, a button, a microphone, or the like for inputting data. The output device 206 is, for example, a display, a printer, a speaker, or the like that outputs data. Each functional unit included in the analyzer 20, which will be described later, is realized by the CPU 201 reading a program stored in the storage device 203 into the memory 202 and executing the program, and each storage unit included in the analyzer 20 is the memory 202. And as part of the storage area provided by the storage device 203.

FIG. 4 is a diagram showing a software configuration example of the analyzer 20. The analyzer 20 includes a crack detector 2, a classifier 4, an area detector 6, an image input unit 21, an image division unit 22, a learning processing unit 23, an output unit 24, a learning model storage unit 25, an image storage unit 26, and annotations. A storage unit 27 is provided.

The image input unit 21 accepts the input of the captured image 1 in which the inspection target is captured. The image input unit 21 may, for example, accept input of image data as a file, or may control a camera (not shown) to acquire an image taken by the camera. Further, in the present embodiment, the image input unit 21 accepts the input of the captured image 1 for both the learning process and the inspection, but the image input unit 21 for learning and the image input unit 21 for inspection May be implemented as a separate functional part. For example, the image input unit 21 may accept input of an image file previously taken by a camera or the like at the time of learning, and may control the camera and the like at the time of inspection to take an image in real time. The image input unit 21 can register the received image as, for example, a file in the image storage unit 26.

The learning model storage unit 25 stores a learning model learned by machine learning. The learning model storage unit 25 stores each learning model of the area detection model 251, the crack detection model 252, and the classification model 253.

The learning processing unit 23 performs learning by machine learning. The learning processing unit 23 learns each learning model stored in the learning model storage unit 25. For example, the learning processing unit 23 can receive designation (annotation) of a region where an abnormality such as a crack has occurred in the captured image 1 from the user and perform learning. The learning processing unit 23 can register the received information (annotation information) related to the annotation in the annotation storage unit 27 in association with the captured image 1.

The area detection model 251 is a learning model for extracting the inspection target range from the image. The learning processing unit 23 receives an input (annotation) of an inspection target area for inspecting an abnormality in the captured image 1 for learning, and learns the information indicating the input area and the captured image 1 to detect the area. Model 251 can be updated. For example, when a median strip or a sidewalk other than the road is shown in the image, the area of the road in the captured image 1 is specified (annotation is accepted and specified so that only the road can be specified as the detection target. The region detection model 251 can be updated by learning the region and the captured image 1.

The area detector 6 detects the inspection target range in the captured image 1 for inspection. The region detector 6 can specify the inspection target region by giving the captured image 1 for inspection to the region detection model 251. It should be noted that the region detector 6 may specify the inspection target region by using, for example, information indicating the inspection target range in the image (for example, a polygonal vertex list) without using the learning model. good.

The crack detection model 252 is a learning model for detecting an abnormality from the captured image 1 by learning the captured image 1 in which the abnormality is captured, and in the present embodiment, it is a learning model for detecting cracks from the image. .. The learning processing unit 23 receives the input (annotation) of the cracked region in the captured image 1 for learning, learns the information indicating the input region and the captured image 1, and updates the crack detection model 252. Can be done. The learning processing unit 23 can also learn only the region specified by the region detector 6 in the captured image 1.

The detector 4 for detecting cracks (hereinafter referred to as crack detector 4; corresponding to the abnormality detection unit of the present invention) gives an unknown captured image 1 to the crack detection model 252 and detects an abnormality in the captured image 1. can do.

The classification model 253 is a learning model for estimating the degree of abnormality in the captured image 1 by learning the information indicating the abnormality included in the captured image 1 and the captured image 1. In the present embodiment, the classification model 253 is a learning model for detecting the degree of cracks (deterioration level according to the number of cracks). The learning processing unit 23 can update the classification model 253 by performing machine learning using the degree of cracking as teacher data and the information indicating the cracked region in the captured image 1 and the captured image 1 as input data. The information indicating the crack may be accepted by the annotation of the user, or the result of giving the captured image 1 to the crack detection model 252 may be used.

The classifier 4 (corresponding to the estimation unit of the present invention) can determine the degree of cracking by giving the photographed image 1 for inspection to the classification model 253. In the present embodiment, the captured image 1 for inspection is passed to the crack detector 2 to detect cracks, and the detection result and the captured image 1 are given to the classification model 253 to obtain the degree of cracks. ..

The image segmentation unit 22 divides the captured image 1 into meshes. The mesh size can be a predetermined value. The mesh may be a square, a rectangle, or a polygon such as a hexagon or an octagon. The image segmentation unit 22 can set a mesh of a predetermined size in the vertical and horizontal directions from the reference point (for example, the upper left corner, etc.) of the captured image 1. In the present embodiment, the mesh is set as the mesh of the road with reference to the white line of the road. That is, in the case of the captured image 1 in which the extending direction of the road (the direction in which the vehicle travels on the road) is the vertical direction, the image dividing unit 22 detects a white line from the captured image 1 and among the detected white lines. A mesh can be set for each predetermined distance in the road crossing direction (left-right direction of the image) based on the one detected at the rightmost end (or the left end) of the image. In the extending direction (longitudinal direction) of the road, for example, a reference point such as an intersection can be detected and divided so as to arrange a mesh from the reference point, or simply in the vertical direction of the image. It is also possible to set the mesh with reference to the lower end (or upper end) of (the extending direction of the road). The image segmentation unit 22 can also prevent the mesh from being set for the portion of the captured image 1 that is not the inspection target region detected by the region detector 6. In addition, the image segmentation unit 22 can divide the shape obtained by plotting the information indicating the crack region detected by the crack detector 2 on the captured image 1 with the same mesh as the mesh set in the captured image 1. ..

The classifier 4 can estimate the degree of cracking for each mesh divided by the image segmentation unit 22. That is, the captured image 1 divided into meshes and the cracked region divided into meshes can be applied to the classification model 253 for each mesh, and the degree of cracking (degree of deterioration) for each mesh can be estimated. can. This makes it possible to show the degree of cracking in mesh units on the captured image 1 of the road as shown in the example of FIG.

The output unit 24 outputs the degree of abnormality. In this embodiment, the degree of cracking (the degree of deterioration of the road according to the number of cracks) is output. Further, in the present embodiment, the classifier 4 estimates the degree of cracking for each mesh, and the output unit 24 calculates the degree of cracking for each mesh in the background image 1, for example, as shown in the lowermost part of FIG. , For example, can be displayed by color.

<Operation>
FIG. 5 is a diagram illustrating the operation of the analyzer 20 of the present embodiment.

The analyzer 20 acquires the captured image 1 for inspection (S31). As described above, the captured image 1 may accept the input of the file, or may control the camera (not shown) to acquire the captured image.

The analyzer 20 gives the captured image 1 to the region detection model 251 to detect the inspection target region from the captured image 1 (S32), and imparts the inspection target region portion of the captured image 1 to the crack detection model 252 to provide the captured image 1 to the captured image 1. Cracks are detected from (S33). FIG. 6 is a diagram illustrating the detection of cracks. As shown in FIG. 6, the detector 2 detects the crack 3 in the captured image. The detected crack 3 is superimposed and output on the captured image 1 in the example of FIG. 6, but it is possible to acquire crack data indicating only the location of the crack 3.

The analyzer 20 sets a mesh in the inspection target area of the captured image 1 (S34). FIG. 7 is a diagram showing a state in which a mesh is set in the captured image 1. As shown in FIG. 7, the image segmentation unit 22 can set the mesh 7 on the captured image. For example, the analyzer 20 can arrange the mesh from the edge of the image, but as shown in FIG. 7, the image segmentation portion 22 has a white line drawn on the road from the captured image 1. 8 can be detected and the mesh can be arranged with reference to the position of the white line 8.

The analyzer 20 divides the inspection target area of the captured image 1 into a set mesh (S35). FIG. 8 is a diagram showing how the captured image 1 is divided into meshes. As shown in FIG. 8, the image segmentation unit 22 can divide the captured image 1 into the mesh set in step S34 to create the divided image 1M. The image segmentation unit 22 may, for example, extract a portion corresponding to each mesh from the captured image 1 to create a segmented image 1M.

The analyzer 20 can divide the detected crack data (which can include the position of the crack on the captured image 1) into the mesh (S36). FIG. 9 is a diagram showing how the data of the crack 3 is divided into meshes. As shown in FIG. 9, the image segmentation unit 22 can create segmentation data 3M indicating only the data indicating the crack 3 included in the position of the mesh. The image segmentation unit 22 may divide the data of the crack 3 as an image, or extract the side of the polygon representing the crack 3 included in the mesh in the captured image 1 as the data. You may.

The analyzer 20 gives the divided image 1M obtained by dividing the captured image 1 and the divided data 3M obtained by dividing the crack data to the classification model 253 for each mesh, and estimates the degree of cracking (S37). FIG. 10 is a diagram showing a state of estimating the degree of cracking. As shown in FIG. 10, the classifier 4 gives the divided image 1M and the divided data 3M to the classification model 253, and puts the mesh in a category indicating the degree of cracking, for example, "1" or "low level". Can be classified.

As described above, the analyzer 20 of the present embodiment can estimate the degree of cracking based on the captured image 1 and the cracked portion detected from the captured image 1. By giving not only the location detected as a crack but also the captured image 1 as input data, it is possible to consider the situation in which the crack is recognized as a crack due to over-detection when estimating the degree of the crack.

FIG. 11 is a diagram illustrating a first process of setting the mesh 8 by the image segmentation unit 22. The first process is a process of simply setting the mesh 8 with reference to the white line of the road. FIG. 12 is a diagram showing how the mesh 8 is set.

The image segmentation unit 22 detects a white line from the captured image 1 (S401). For the detection process of the white line, a detection process by general image analysis can be adopted. When the white line cannot be recognized in the captured image 1 (S402: NO), the image segmentation unit 22 can consider the inside of the captured image 1 by a predetermined distance (which may be 0) as the white line (may be 0). S403).

The image segmentation unit 22 extends the white line when it detects the white line from the captured image 1 (S401: YES) and when the white line is interrupted in the middle (S404: YES) (S405). It was

In the example of FIG. 12, a state in which a broken white line is detected (A1, S401) and the white line is extended (A2, S405) is shown.

Next, the image segmentation unit 22 sets the box of the mesh 8 toward the left and top of the image with the lower right of the white line in the captured image 1 as a reference (S406). FIG. 12 shows a line (B) constituting the rectangle of the mesh 8.

The image segmentation portion 22 arranges the mesh 8s in the left and upward directions, and when the left end of the mesh 8 exceeds the white line located on the leftmost side in the captured image 1 (S407: YES), the left end of the mesh 8 is a white line. The size of the mesh 8 is adjusted so as to move to (S408). In the example of FIG. 12, the size of the mesh 8 is adjusted so that the left end (C1) of the leftmost mesh 8 is aligned with the white line (A3) at the left end. Similarly, when the upper end of the mesh 8 exceeds the upper end of the captured image 1 (or a position inside a predetermined value from the upper end), the image segmentation portion 22 moves the upper end of the mesh 8 to the upper end of the captured image 1. As such, the size of the mesh 8 can be adjusted. In the example of FIG. 12, the size of the mesh 8 is adjusted so that the upper end C2 of the uppermost mesh 8 is aligned with the upper end of the captured image 1.

As described above, the part between the white lines drawn on the road can be divided into mesh 8.

FIG. 13 is a diagram illustrating a second process of setting the mesh 8 by the image segmentation unit 22. In the second process, in addition to the first process shown in FIG. 11, the process of adjusting the setting start position of the rightmost mesh 8 according to the pixels of the white line before setting the mesh 8 in step S406 (S421). ) Has been added.

FIG. 14 is a diagram showing how the mesh 8 is set in the second process. The white line may not be a straight line due to the tilt of the camera (not shown) at the time of shooting, the fluctuation of the thickness of the white line, the shading of the white line, etc. Depending on the indicated pixels (for example, white pixels), for example, when the number of pixels including white line pixels in the mesh 8 is equal to or greater than a predetermined value, the mesh 8 is operated until the number of white line pixels is equal to or less than a predetermined value. It can be shifted to the left by one pixel. In FIG. 14, the right end (D1), which is the reference position in the left-right direction of the second rightmost mesh 8D from the bottom, is shifted in the direction of the arrow (D2) according to the pixel indicating the white line A1. It is shown that.

FIG. 15 is a diagram illustrating a third process of setting the mesh 8 by the image segmentation unit 22. In the third process, in addition to the second process shown in FIG. 13, after steps S407 and S408, the mesh 8 is continuously set on the other captured image 1 adjacent to the captured image 1 (a process of setting the mesh 8 on the other captured image 1 adjacent to the captured image 1. S431) has been added.

FIG. 16 is a diagram showing how the mesh 8 is set in the third process. As shown in FIG. 16, the mesh 8E at the upper end is set so as to extend beyond the upper end of the captured image 1 and extend to the adjacent captured image 1'. In this way, the image segmentation unit 22 can set the mesh 8 across the continuous captured images 1.

The image segmentation unit 22 arranges a plurality of captured images 1 to determine the presence or absence of the same region, and if the same region exists, arranges the same regions so that they overlap each other, and uses the arranged captured images 1 to mesh 1 Can be set. Further, at this time, the image segmentation unit 22 may create a composite image in which the arranged captured images 1 are combined, and set the mesh 8 for the created composite image.

Although the present embodiment has been described above, the above embodiment is for facilitating the understanding of the present invention, and is not for limiting the interpretation of the present invention. The present invention can be modified and improved without departing from the spirit thereof, and the present invention also includes an equivalent thereof.

For example, in the present embodiment, the analyzer 20 estimates the degree of cracking of the road, but it can be widely applied to detect some defect from the image and evaluate the risk related to the defect. For example, the analyzer 20 of the present embodiment can detect cracks in structures such as bridges and buildings by inputting captured images of structures such as bridges and buildings, and evaluate the degree of cracks. can. Further, the analyzer 20 of the present embodiment uses, for example, a map image such as an aerial photograph or a satellite photograph (including infrared data and various sensor data) as a captured image, for example, a valley that causes a landslide. It is also possible to detect the area and evaluate the risk of landslides.

Further, in the present embodiment, the classification model 253 is a learning model for categorizing the degree of cracks, but it may be a regression model for estimating the number of cracks. In this case, the classifier 4 can estimate the number of cracks.

Further, in the present embodiment, the number of cracks indicates the degree of cracks, but the present invention is not limited to this, and the depth of cracks may be learned as the degree of cracks. In this case, the data of the crack depth may be input, or the information indicating the degree according to the crack depth may be input.

Also, instead of the number of cracks, the severity of the cracks may be used as the degree of cracks. In this case, an expert can browse the image to determine the severity of the crack, and input the determined severity to learn.

Further, in the present embodiment, it was assumed that the degree of cracking of the road was estimated, but similarly, it can be applied to the case of estimating the degree of cracking of a bridge or a tunnel. In addition to cracks, it can be applied when estimating the degree of corrosion, water leakage, free lime, damage, etc. of building materials.

When inspecting a bridge, only the leg part of the bridge can be extracted as the inspection target area, and the leg part can be divided into meshes to estimate the degree of cracking.

Further, when estimating the cracks in the tunnel, the cracks may be detected directly from the photographed image 1 obtained by photographing the inner surface of the tunnel without recognizing the white line. Further, in the case of a tunnel, the reference of the mesh arranged in the longitudinal direction of the tunnel can be set at the entrance of the tunnel.

Further, in the present embodiment, the captured image is analyzed and the mesh is set, but the position of the mesh may be given.

Further, in the present embodiment, learning and inference are performed using the captured image 1 captured by the camera, but the present invention is not limited to this, and a composite image obtained by synthesizing a plurality of captured images 1 may be used. , Orthophoto corrected orthophotographed image 1 may be used.

Further, in the present embodiment, the crack detector 2 and the classifier 4 perform supervised learning in which the result of the annotation is given as a teacher signal, but at least one of the crack detector 2 and the classifier 4 is used. Learning may be performed by supervised learning. For example, at least one of the crack detector 2 and the classifier 4 learns a photographed image 1 for normal learning (no abnormality is photographed) to create a generator (for example, an autoencoder), and creates the image. A generator can be used to reproduce a normal image based on the captured image 1 for inspection, and the difference between the reproduced image and the captured image 1 for inspection can be detected as an abnormality.

2 Crack detector 4 Classifier 6 Area detector 20 Analyzer 21 Image input unit 22 Image division unit 23 Learning processing unit 24 Output unit 25 Learning model storage unit 26 Image storage unit 27 Annotation storage unit 251 Area detection model 252 Crack detection model 253 classification model

Claims

A first learning model for detecting the abnormality from a first image obtained by photographing the abnormality, and a second learning model for estimating the degree of the abnormality from the abnormality information indicating the abnormality and the first image. A learning model memory unit that memorizes and
An image input unit that accepts input of a second image whose abnormality is unknown or not, and an image input unit.
An abnormality detection unit that applies the second image to the first learning model and detects the abnormality in the second image,
An estimation unit that estimates the degree of the abnormality in the second image by giving the detected abnormality information indicating the abnormality and the second image to the second learning model.
An output unit that outputs the degree of abnormality and
An information processing system characterized by being equipped with.
The information processing system according to claim 1.
It also has an image segmentation section that divides the image into meshes.
The second learning model is a model created by learning the first mesh obtained by dividing the first image and the abnormality information detected in the first mesh.
The image segmentation section divides the second image into the second mesh.
For each of the second meshes, the estimation unit obtains the abnormality information detected from the second image, which is included in the second mesh, and the second mesh. To estimate the degree of the anomaly for each mesh by giving it to the learning model of 2.
An information processing system featuring.
The information processing system according to claim 2.
The second image is a photograph of the road.
The image segmentation unit detects a white line drawn on the road from the second image and sets the position of the mesh with reference to the position of the white line.
An information processing system featuring.
The information processing system according to any one of claims 1 to 3.
Further equipped with a target range detection unit that detects the inspection target range from the image,
The target range detection unit detects the inspection target range from the second image, and the target range detection unit detects the inspection target range.
The abnormality detection unit applies the detected inspection target range to the first learning model, detects the abnormality information, and detects the abnormality information.
The estimation unit applies the detected inspection target range and the abnormality information to the second learning model to estimate the degree of the abnormality.
An information processing system featuring.