JP7396944B2 - Pipe damage identification device, pipe damage identification method, and pipe damage identification program - Google Patents

Description

The present invention relates to a pipe damage identification device, a pipe damage identification method, and a pipe damage identification program, and particularly relates to a device for identifying damage to a pipe from identification results obtained by an image analysis program built into a computer and artificial intelligence.

Infrastructure structures are aging at an accelerating rate, and the need for maintenance, management, and renewal is said to continue to increase in the future. As the working population continues to decline, increasing the efficiency of labor related to maintenance, management, and renewal has become an urgent issue. In order to improve the efficiency of labor related to maintenance, management, and renewal, there is an increasing number of research cases applying AI-based image recognition technology, whose performance has improved significantly in recent years. By using an advanced neural network called deep learning, it is possible to achieve image recognition that is close to human intuition, which was previously difficult, and is expected to assist or replace visual inspection of damaged areas.

On the other hand, in the inspection of infrastructure structures such as pipes, for example, as in the technology described in Patent Document 1 and Patent Document 2, a photographing device such as a television camera or It was equipped with lighting equipment such as fluorescent lights, and was used to investigate damage based on images of the inside of the tube taken with a photographic device.

Japanese Patent Application Publication No. 5-346027 Japanese Patent Application Publication No. 7-216972

However, with the technology described in the above-mentioned document, workers visually inspect the captured images of the inside of the pipe, which is prone to oversight or misreading, and it is difficult to identify damage to pipes with high precision. I couldn't do it.

In order to achieve the above object, the pipe damage identification device according to the present invention has the following features:
an image acquisition unit that acquires a first inner circumferential surface image of the inner circumferential surface of the first pipe culvert;
A first ceiling plane image obtained by cutting and developing the first inner circumferential surface image from the ceiling portion of the first pipe culvert in the first inner circumferential surface image; a first planar image generation unit that generates a first bottom planar image obtained by cutting out and expanding the bottom portion of the first culvert;
a damage determination unit that determines first damage occurring in the first pipe culvert using the first ceiling plane image and the first bottom plane image;
An identifier that can identify the determined damage is given to artificial intelligence to learn the first ceiling plane image and the first bottom plane image together with the determined first damage, and a learned damage identification model is created. A model generation unit that generates
an image reception unit that receives a second inner circumferential surface image taken of the inner circumferential surface of the second pipe culvert;
a second ceiling plane image obtained by cutting and developing the second inner circumferential surface image from the ceiling portion of the second pipe in the second inner circumferential surface image; a second planar image generation unit that generates a second bottom planar image obtained by cutting out and expanding the bottom portion of the second culvert;
A damage identification unit that detects and identifies second damage occurring in the second pipe culvert using the second ceiling plane image, the second bottom plane image, and the learned damage identification model, a damage identification unit that identifies the second damage occurring in the second pipe culvert by changing a threshold value for damage detection according to the type of the second damage occurring in the second pipe culvert; and,
Equipped with

In order to achieve the above object, the pipe damage identification method according to the present invention includes:
an image acquisition step of acquiring a first inner circumferential surface image of the inner circumferential surface of the first pipe culvert;
A first ceiling plane image obtained by cutting and developing the first inner circumferential surface image from the ceiling portion of the first pipe culvert in the first inner circumferential surface image; a first planar image generation step of generating a first bottom planar image obtained by cutting out and expanding the bottom portion of the first culvert;
a damage determination step of determining first damage occurring in the first culvert using the first ceiling plane image and the first bottom plane image;
Identifying the learned damage by assigning an identifier that can identify the determined first damage and causing artificial intelligence to learn the first ceiling plane image and the first bottom plane image together with the determined first damage. a model generation step for generating a model;
an image reception step of receiving a second inner circumferential surface image taken of the inner circumferential surface of the second pipe;
a second ceiling plane image obtained by cutting and developing the second inner circumferential surface image from the ceiling portion of the second pipe in the second inner circumferential surface image; a second planar image generation step of generating a second bottom planar image obtained by cutting open and expanding the bottom portion of the second culvert;
When detecting and identifying second damage occurring in the second pipe using the second ceiling plane image, the second bottom plane image, and the learned damage identification model, the second pipe a damage identification step of identifying the second damage occurring in the second pipe by changing a threshold value for damage detection according to the type of the second damage occurring in the second pipe;
including.

In order to achieve the above object, the pipe damage identification program according to the present invention:
an image acquisition step of acquiring a first inner circumferential surface image of the inner circumferential surface of the first pipe culvert;
A first ceiling plane image obtained by cutting and developing the first inner circumferential surface image from the ceiling portion of the first pipe culvert in the first inner circumferential surface image; a first planar image generation step of generating a first bottom planar image obtained by cutting out and expanding the bottom portion of the first culvert;
a damage determination step of determining first damage occurring in the first culvert using the first ceiling plane image and the first bottom plane image;
Identifying the learned damage by assigning an identifier that can identify the determined first damage and causing artificial intelligence to learn the first ceiling plane image and the first bottom plane image together with the determined first damage. a model generation step for generating a model;
an image reception step of receiving a second inner circumferential surface image taken of the inner circumferential surface of the second pipe;
a second ceiling plane image obtained by cutting and developing the second inner circumferential surface image from the ceiling portion of the second pipe in the second inner circumferential surface image; a second planar image generation step of generating a second bottom planar image obtained by cutting open and expanding the bottom portion of the second culvert;
When detecting and identifying second damage occurring in the second pipe using the second ceiling plane image, the second bottom plane image, and the learned damage identification model, the second pipe a damage identification step of identifying the second damage occurring in the second pipe by changing a threshold value for damage detection according to the type of the second damage occurring in the second pipe;
have the computer execute it.

According to the pipe damage identification device of the present invention, pipe damage is identified using machine learning using artificial intelligence and a threshold value for damage detection. It is possible to identify damage to high-quality pipes.

It is a figure for explaining the outline of identification of pipe damage by a pipe damage identification device concerning a 1st embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram for demonstrating the structure of the pipe damage identification apparatus based on 1st Embodiment of this invention. FIG. 3 is a diagram for explaining object detection in the pipe damage identification device according to the first embodiment of the present invention. FIG. 3 is a diagram for explaining determination of damage (damage/cracks) in the pipe damage identification device according to the first embodiment of the present invention. It is another diagram for explaining the determination of damage (damage/cracks) in the pipe damage identification device according to the first embodiment of the present invention. FIG. 3 is a diagram for explaining determination of damage (intrusion water) in the pipe damage identification device according to the first embodiment of the present invention. FIG. 7 is another diagram for explaining determination of damage (intrusion water) in the pipe damage identification device according to the first embodiment of the present invention. FIG. 7 is yet another diagram for explaining determination of damage (infiltrated water) in the pipe damage identifying device according to the first embodiment of the present invention. It is a figure for explaining object detection when a threshold value is changed in the pipe damage identification device concerning a 1st embodiment of the present invention. It is a figure for explaining the determination of the threshold value using recall rate and precision rate in the pipe damage identification device concerning a 1st embodiment of the present invention. It is a figure for explaining an example of the threshold value table which the pipe damage identification device concerning a 1st embodiment of the present invention has. It is a figure for explaining an example of the damage identification table which the pipe damage identification device concerning a 1st embodiment of the present invention has. It is a figure for explaining prediction of the degree of damage by a pipe damage identification device concerning a 1st embodiment of the present invention. It is a flow chart for explaining the processing procedure of the pipe damage identification device concerning a 1st embodiment of the present invention. It is another flowchart for demonstrating the processing procedure of the pipe damage identification apparatus based on 1st Embodiment of this invention.

EMBODIMENT OF THE INVENTION Below, the form for implementing this invention will be exemplarily described in detail with reference to drawings. However, the configuration, numerical values, process flow, functional elements, etc. described in the following embodiments are merely examples, and modifications and changes may be made freely, and the technical scope of the present invention is limited to the following description. It is not intended to do so.

[First embodiment]
A pipe damage identification device 100 as a first embodiment of the present invention will be described using FIGS. 1 to 5B. The pipe culvert damage identification device 100 is used to identify damage that has occurred in a pipe culvert. FIG. 1 is a diagram for explaining an overview of identifying pipe damage by the pipe damage identification device 100 according to the present embodiment.

Here, the term "pipe culvert" is a general term for waterways, and refers to the entire waterway constructed for the purpose of water supply and drainage. Examples include water pipes, sewer pipes, drain pipes, water supply pipes, etc. In this embodiment, a sewer pipe 150 will be explained as an example of a pipe. In addition, the materials of the pipes include concrete, ceramic, iron, etc., and the types of pipes include concrete pipes, concrete fume pipes, ceramic pipes, iron pipes, etc.

Further, damage includes not only cracks, cracks, and scratches that occur in the pipe culvert, but also includes a state in which the pipe culvert is unable to exhibit its expected performance. In the following explanation, the diameter of the sewer pipe 150 is such that a worker cannot enter the sewer pipe 150 and work therein, and the explanation will be made assuming that the sewer pipe 150 has a diameter of 200 to 800 mm, for example. However, the diameter of the sewer pipe 150 is not limited to this.

In the inspection of the inside of the sewer pipe 150, an inspection is performed using an image of the inner peripheral surface of the sewer pipe 150 captured by the camera 142 of the self-propelled inspection robot 140. The worker uses a display installed outside the sewer pipe 150 to check the image captured by the camera 142 and inspects whether or not the sewer pipe 150 is damaged. The control unit 141 controls the self-propelled speed, imaging schedule, and imaging conditions of the self-propelled inspection robot 140, and controls communication with the pipe damage identification device 100 and the tablet terminal 130 carried by the worker. do. A worker operates the self-propelled inspection robot 140 using, for example, an operation application installed on the tablet terminal 130. Note that in the case of a sewer pipe 150 with a small diameter that the self-propelled inspection robot 140 cannot enter, a micro-inspection device such as a fiberscope may be used instead of the self-propelled inspection robot 140.

Then, the control unit 141 transmits the inner peripheral surface image 110 (160) captured by the camera 142 to the pipe damage identification device 100. Note that the inner circumferential surface image 110 is an image for learning by artificial intelligence, and the inner circumferential surface image 160 is an image in which damage is to be identified. Furthermore, the camera 142 may be either a wide-angle camera or an all-around camera. In the case of a wide-angle camera, the images for one revolution may be captured in multiple steps, and the inner peripheral surface image 110 (160) may be generated by connecting the captured images.

Further, the inner circumferential surface image 110 (160) is sliced at predetermined intervals (for example, 0.1 m). For example, when the space between the joints of the sewer pipe 150 is defined as one unit, the images captured for each unit (inner peripheral surface images 110 (160)) may be sliced into slices to obtain the slice images 143. Further, for example, the image captured along the entire length of the sewer pipe 150 (inner peripheral surface image 110 (160)) may be sliced into slices to obtain the slice image 143.

Next, generation of a trained damage identification model by the pipe damage identification device 100 will be explained. The pipe damage identifying device 100 cuts out and develops the sliced image 143 of the received inner peripheral surface image 110 (learning image) at the ceiling and bottom portions of the sewer pipe 150 to obtain a two-dimensional planar image. That is, the pipe damage identification device 100 develops a ceiling plane image 101 by cutting the inner circumferential surface image 110 at the ceiling portion of the sewer pipe 150 and developing the inner circumferential surface image 110 by cutting the inner circumferential surface image 110 at the bottom portion of the sewer pipe 150. An expanded bottom plane image 102 is generated.

In the ceiling plane image 101, the bottom part 111 of the sewer pipe 150 is located at the center of the strip-shaped plane image, and the ceiling parts 112 and 113 are located at both left and right ends. Similarly, in the bottom planar image 102, the ceiling portion 121 is located at the center of the strip-shaped planar image, and the bottom portions 122 and 123 are located at both left and right ends. Then, the pipe damage identification device 100 identifies damage in an image in which the ceiling plane image 101 and the bottom plane image 102 are arranged side by side, causes artificial intelligence to learn the parallel images in which damage has been identified, and creates a trained damage identification model. generate.

In the ceiling plane image 101, the bottom part 111 of the sewer pipe 150 is located at the center of the strip-shaped plane image, and the ceiling parts 112 and 113 are located at both left and right ends. Similarly, in the bottom planar image 102, the ceiling portion 121 is located at the center of the strip-shaped planar image, and the bottom portions 122 and 123 are located at both left and right ends. Then, the pipe damage identification device 100 uses the ceiling plane image 101 and the bottom plane image 102 to identify damage that has occurred in the sewage pipe 150, and uses these images (101 to 102) in which damage has been identified using artificial intelligence. to generate a trained damage identification model.

Next, damage identification by the pipe culvert damage identification device 100 will be described. The pipe damage identification device 100 generates a ceiling plane image 161 and a bottom plane image 162 for the inner circumference image 160 of the sewer pipe 170 whose damage is to be identified, similarly to the inner circumference image 110, and generates a learned damage identification model. The damage occurring in the sewer pipe 170 is identified using the following method.

At this time, in identifying damage occurring in the sewer pipe 170 in the ceiling plane image 161 and the bottom plane image 162, a threshold value is set according to the type of damage to detect and specify the damage. In other words, the pipe culvert damage identification device 100 sets a low threshold value for damage that should not be overlooked so that it can be detected without omission. However, if the set value of the threshold value is lowered, small damage that does not affect the performance or durability of the sewer pipe 170 will also be identified, and efficiency and accuracy may decrease due to excessive identification. Therefore, the pipe damage identification device 100 sets an appropriate threshold value according to the type of damage and identifies the damage that has occurred in the sewer pipe 170.

The identified damage is output to, for example, a mobile terminal such as a tablet terminal 130 owned by the worker. The worker refers to the identification results displayed on the display of the tablet terminal 130 and recognizes the damage occurring in the sewer pipe 170. Note that the pipe damage identification device 100 may output an alert depending on the type and degree of damage identified. For example, if the damage is progressing or requires immediate repair work, the pipe damage identification device 100 may output an alert using a warning sound, vibration, light, message, or the like.

FIG. 2 is a block diagram for explaining the configuration of the pipe damage identification device 100 according to this embodiment. The pipe damage identification device 100 includes an image acquisition section 201, a planar image generation section 202, a damage determination section 203, a model generation section 204, an image reception section 205, a planar image generation section 206, a damage identification section 207, an output section 208, and a memory. 209.

Here, the pipe damage identification device 100 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a network interface, and a storage. Here, the CPU is a processor for arithmetic control, and implements each functional configuration of the pipe damage identification device 100 shown in FIG. 2 by executing a program. A CPU may include multiple processors and may execute different programs, modules, tasks, threads, etc. in parallel. The ROM stores fixed data such as initial data and programs and other programs. The network interface also communicates with other devices via the network. Note that the CPU is not limited to one, and may be a plurality of CPUs or may include a GPU (Graphics Processing Unit) for image processing. Further, it is preferable that the network interface has another CPU independent of the CPU, and writes or reads transmitted and received data in the RAM area. Further, it is desirable to provide a DMAC (Direct Memory Access Controller) that transfers data between the RAM and the storage. Further, the CPU recognizes that data has been received or transferred to the RAM and processes the data. Further, the CPU prepares the processing results in the RAM, and leaves subsequent transmission or transfer to the network interface or DMAC.

RAM is a memory that the CPU uses as a temporary storage work area. The RAM has an area reserved for storing data necessary to implement this embodiment. The storage stores a database, various parameters, modules, or data or programs necessary to implement this embodiment. For example, a control program for controlling the entire pipe damage identification device 100 is stored in the storage.

Furthermore, the pipe damage identification device 100 may further include an input/output interface. A display section, an operation section, and a storage medium are connected to the input/output interface. The input/output interface may further be connected to a speaker as an audio output section, a microphone as an audio input section, or a GPS (Global Positioning System) position determination section. Note that the RAM and storage may store programs and data regarding general-purpose functions and other realizable functions that the pipe culvert damage identification device 100 has.

The image acquisition unit 201 acquires an inner peripheral surface image 110 that captures an image of the inner peripheral surface of the sewer pipe 150. The image acquisition unit 201 may acquire the inner circumferential surface image 110 from the control unit 141 of the self-propelled inspection robot 140, or may acquire it via a tablet terminal 130 owned by a worker. Note that the inner peripheral surface image 110 is an image used for later learning by artificial intelligence.

The planar image generation unit 202 generates a ceiling planar image 101 obtained by cutting the inner peripheral surface image 110 from the ceiling portion of the sewer pipe 150 in the inner peripheral surface image 110 and developing it. Similarly, the planar image generation unit 202 generates a bottom planar image 102 obtained by cutting open and developing the bottom portion of the sewer pipe 150 in the inner peripheral surface image 110.

That is, the planar image generation unit 202 first generates the sliced image 143 from the inner peripheral surface image 110 acquired by the image acquisition unit 201. Then, the plane image generation unit 202 generates a ceiling plane image 101 and a bottom plane image 102 from the sliced image 143. Note that the sliced image 143 may be generated in advance by the self-propelled inspection robot 140, and the image acquisition unit 201 may acquire the sliced image 143 generated by the self-propelled inspection robot 140.

The damage determination unit 203 determines damage occurring in the sewer pipe 150 using the ceiling plane image 101 and the bottom plane image 102. The damage determination unit 203 determines damage using images in which the ceiling plane image 101 and the bottom plane image 102 are arranged in parallel. Types of damage include breakage, cracks, water intrusion, protrusion of attachment pipes, root penetration, and mortar adhesion. Note that the damage determination unit 203 uses object detection to detect an area where damage has occurred. Faster R-CNN (Faster Region with Convolutional Neural Network) was used as the object detection algorithm.

Here, damage determination by the damage determination unit 203 will be specifically described with reference to FIGS. 3A to 3F. FIG. 3A is a diagram for explaining object detection in the pipe damage identification device 100 according to the present embodiment.

The image 301 is a ceiling plane image 101 or a bottom plane image 102 that is developed by cutting the inner peripheral surface image 110 at the ceiling portion or the bottom portion. In this way, when the inner circumferential surface image 110 is cut open at the ceiling portions 311 and 312 (or the bottom portion), if damage exists in the ceiling portion (or the bottom portion), the damage may be separated. However, in this case, the divided damage is treated as two damage rather than being integrated as one damage. That is, the damage determination unit 203 detects damage (detects an object) using the two boxes 313 and 314 for the divided damage.

Next, with reference to FIG. 3B, damage detection when the damage is breakage and cracks will be described. FIG. 3B is a diagram for explaining determination of damage (damage/cracks) in the pipe damage identification device according to the present embodiment. An image 302 shows a state in which damage or cracks have occurred on the inner circumferential surface of the sewer pipe 150. Here, the damage is a crack that occurs in the horizontal direction, and the crack is a crack that occurs in the vertical direction (circumferential direction).

When the damage occurring to the sewer pipe 150 is breakage and cracks, these damages are labeled as damage of another event. In the inner peripheral surface image 110, white or black streaks are considered to be damage or cracks. In the case of cracks, if the cracks are in both vertical and horizontal directions, or if the cracks are diagonal, they are treated as damage. For example, in the case of a vertical crack, an inclination of approximately 30 degrees or less is treated as a crack.

If the damaged or cracked area extends left and right across the joint portion 321, the damage determining unit 203 does not detect damage in one box. The damage determination unit 203 divides the box into two with the joint portion 321 as the boundary, and uses the two boxes 322 and 323 to detect two damages.

Similarly, if the damaged or cracked area extends vertically across the flowing water section 324 (the bottom of the sewer pipe 150), the damage determination unit 203 does not detect damage in one box. The damage determining unit 203 divides the box into two with the water flowing part 324 as the boundary, detects two damage using the two boxes 325 and 326, and determines the damage.

Next, damage detection when a plurality of small damages exist in a narrow area will be described with reference to FIG. 3C. FIG. 3C is another diagram for explaining determination of damage (damage/cracks) in the pipe culvert damage identification device 100 according to the present embodiment. Image 303 shows a state in which small damage or cracks have occurred in a narrow area.

When a plurality of small damages occur in a narrow range, the damage determination unit 203 uses two boxes 331 and 332, as shown in the left diagram of FIG. 3C, and does not treat the damage as a plurality of damages. As shown in the right diagram of FIG. 3C, the damage determining unit 203 surrounds a plurality of damages in one box 333 and treats them as one damage. In this way, when a plurality of small damages exist in a narrow area, by handling these damages collectively, the damages can be detected and determined efficiently.

Next, detection of damage when the damage is caused by infiltrated water will be described with reference to FIG. 3D. FIG. 3D is a diagram for explaining determination of damage (intrusion water) in the pipe damage identification device 100 according to the present embodiment. Image 304 shows a state in which infiltrated water is occurring. The upper part of the image 304 shows, from the left, a black blurred part, a brown part (soil flowing out part), and a white part (free lime), all of which are treated as infiltrated water.

Then, as shown in the lower part of the image 304, in the case of infiltrated water, the damage determination unit 203 surrounds the area from the center part 341 of the infiltrated water to the part where the same color tone continues with a box 342. However, in the case of soil or free lime, the damage determination unit 203 surrounds the non-white portion with a box 342. More specifically, the damage determining unit 203 surrounds the damage with a box 342 centering on the joint portion or damaged portion that causes intrusion water, and detects the object. For example, if intrusion water occurs due to damage, the damage determination unit 203 determines that it is damage and intrusion water.

Next, with reference to FIG. 3E, detection of damage in the case of infiltrated water where damage is difficult to determine will be described. FIG. 3E is another diagram for explaining determination of damage (infiltrated water) in the pipe culvert damage identification device 100 according to the present embodiment. The upper part of the image 305 shows a state in which the infiltrated water flows out from the left and right joint portions 352 and 353, and the boundary between the two infiltrated waters is not clear. In this case, the damage determination unit 203 surrounds the intruding water in one box 351 and treats it as one damage. In addition, as shown in the two images at the bottom of image 305, when it is difficult to distinguish between the leakage of infiltrated water and traces of running water, it is possible to distinguish between traces of infiltrated water extending from a joint or damaged part that is thought to be the source of the infiltrated water. The damage is surrounded by a box 354 based on whether or not it is connected to the water trail.

Finally, detection of small-sized intruding water will be described with reference to FIG. 3F. FIG. 3F is yet another diagram for explaining the determination of damage (intrusion water) in the pipe culvert damage identification device 100 according to the present embodiment. Image 306 shows small intrusion water. The damage determination unit 203 detects small intruding water, but for example, it detects small water near the flow channel (from its size, it is presumed to be a trace of creeping up from the flow channel) and water with a small width. shall be excluded from the scope of However, a detection target 361 is one in which the blur extends upward to some extent or is about to separate. Items that have traces of infiltrated water but are dry and have low contrast are excluded.

Returning to FIG. 2 again. The model generation unit 204 assigns an identifier (ID) that can identify the determined damage. The model generation unit 204 assigns, as identifiers, the range (size) of the damage, the name of the damage, the attributes of the damage (old, new, color), etc. to the ceiling plane image 101 and the bottom plane image 102. Note that this identifier may be directly assigned to the ceiling plane image 101 and the bottom plane image 102, or may be stored in a predetermined database or the like and read out as appropriate.

Then, the model generation unit 204 inputs the assigned identifier, the determined damage, the ceiling plane image 101, and the bottom plane image 102 to artificial intelligence (AI), and performs machine learning. When the machine learning by artificial intelligence is completed, the model generation unit 204 generates a learned damage identification model. Note that the model generation unit 204 may store the generated learned damage identification model in a predetermined storage or the like. In this case, each time a new learning image (ceiling plane image 101 and bottom plane image 102) is acquired, machine learning is performed, and a learned damage identification model is generated, the saved learned damage identification model is updated. You may also do so.

Furthermore, machine learning using artificial intelligence is performed using known algorithms. In machine learning, the loss function specifies weights and uses the reciprocal of the number of events. In addition, the model generation unit 204 increases the number of inner circumferential surface images 110 that are made to be learned by the artificial intelligence in order to improve the accuracy of machine learning by the artificial intelligence and generate a more accurate damage identification model. The model generation unit 204 obtains padded data using, for example, horizontal reversal. Furthermore, the model generation unit 204 may use transfer learning to improve the accuracy of machine learning using artificial intelligence. Here, transfer learning is a method that aims to improve model performance by transferring a trained model using a different dataset to another problem and performing partial learning. This method is expected to improve inference performance and reduce learning time, especially when training data is insufficient.

Note that recall and precision were used to evaluate the generated trained damage identification model. The recall rate is expressed as recall rate=TP/(TP+FN), and indicates the proportion of what actually appears out of what should appear as a result. In other words, it is an index regarding whether anything has been left out (an index regarding comprehensiveness). The precision rate is expressed as precision rate=TP/(TP+FP) and indicates the percentage of correct results. This is an indicator that shows the percentage of correct answers among all the results that appear (an indicator of accuracy). Note that TP=True Positive, FP=False Positive, and FN=False Negative.

The image receiving unit 205 receives an inner circumferential surface image 160 that captures an image of the inner circumferential surface of the sewer pipe 170 as a pipe. That is, the image receiving unit 205 receives the inner peripheral surface image 160 captured by the self-propelled inspection robot 140 and transmitted by the control unit 141. The inner circumferential surface image 160 is an image of the sewer pipe 170 to be inspected, and is the sewer pipe whose damage is to be identified.

The planar image generation section 206 generates a planar image similarly to the planar image generation section 202. That is, the planar image generation unit 206 generates a ceiling planar image 161 obtained by cutting and developing the inner peripheral surface image 160 from the ceiling portion of the sewer pipe 170 in the inner peripheral surface image 160. Similarly, the planar image generation unit 206 generates a bottom planar image 162 obtained by cutting open and developing the bottom portion of the sewer pipe 170 in the inner peripheral surface image 160.

That is, the planar image generation unit 206 first generates a slice image from the inner peripheral surface image 160 received by the image reception unit 205. Then, the plane image generation unit 206 generates a ceiling plane image 161 and a bottom plane image 162 from the sliced images. Note that the sliced image may be generated in advance by the self-propelled inspection robot 140, and the image reception unit 205 may acquire the sliced image generated by the self-propelled inspection robot 140.

The damage identification unit 207 uses the ceiling plane image 161, the bottom plane image 162, and the learned damage identification model to detect and identify damage occurring in the sewer pipe 170. Depending on the type of damage, the threshold value for damage detection is changed to identify damage occurring in the sewer pipe 170.

First, changes in the threshold value will be explained with reference to FIGS. 4A to 4C. FIG. 4A is a diagram for explaining object detection when the threshold value is changed in the pipe culvert damage identification device 100 according to the present embodiment. The damage identification unit 207 specifies damage in the ceiling plane image 161 and the bottom plane image 162 by setting a threshold value for the detection score. The damage identification unit 207 encloses a region in the ceiling plane image 161 and the bottom plane image 162 in which an object is thought to exist with a box. The probability that an object exists in the area surrounded by this box is defined as a detection score (upper diagram of FIG. 4A). In other words, by setting a threshold value for the detection score, it is possible to exclude boxes in which the probability of an object being present is low and exclude them from damage identification targets.

For example, as shown in the lower part of FIG. 4A, increasing the detection score threshold reduces the number of detected boxes. Therefore, if you want to prevent oversights, by lowering the threshold and detecting them, even boxes with a low probability of existence will be targeted for damage identification, thereby preventing oversights. On the other hand, in the case of damage that is not very important, by raising the threshold and detecting it, boxes with a low probability of existence are excluded from damage identification targets, reducing the number of detections and eliminating unnecessary identification of damage. Here, the box refers to a bounding box, which is a rectangular area in an image that is perfectly sized to enclose a figure (in this case, a damage). Furthermore, the detection score is the probability that "the damage in this box is the specified damage" indicated by AI for the detected damage. Detection score (probability) is information added to each box. For example, if a threshold value of ``a certain number or more'' is given for detection score, the number of boxes displayed on the screen (``display boxes'') will increase. Increase or decrease.

FIG. 4B is a diagram for explaining determination of a threshold value using a recall rate and a precision rate in the pipe culvert damage identification device 100 according to the present embodiment. 411 represents damage and cracks, 412 represents infiltrated water, 413 represents attachment pipes, and 414 represents others. Here, the vertical axis represents the recall rate and the precision rate, and the horizontal axis represents the threshold value. The trends of the detection scores described above change depending on the type of damage, and the graphs shown in FIG. 4B represent the differences in trends. Therefore, the threshold value is changed in response to requests such as wanting to reduce omissions of damage and reducing errors in specifying damage. In FIG. 4B, the threshold value is 0.3. For example, if you want to reduce the chance of missing a correct answer for each damage (for example, to make it 70% or more), a detection score that makes the recall rate 0.7 or more may be used as the threshold. In other words, in the case of FIG. 4B, by selecting 0.05 for damage and cracks 411, 0.5 for infiltrated water 412, and 0.3 for other 414, these damages will occur with the desired probability. Is displayed. In addition, regarding the attachment pipe 413, since it exceeds 0.7 in the entire area, in this case, taking into account the precision rate (for example, 95% or more), it is better to reduce incorrect answers and increase work efficiency than overlooking damage. Therefore, the threshold value can be set high (for example, 0.6).

Precision is the percentage of correct answers out of the answers given by AI. When the detection score is low, the precision rate is low because the answers include many incorrect answers. As the detection score increases, the number of incorrect answers decreases, so the precision increases. However, the possibility of missing the correct answer also increases. Recall is the ratio of correct answers given by AI to all correct answers, including answers that the AI could not find. Incorrect answers given by the AI are ignored, so if the detection score is 0, the recall rate is 100%. For example, when the detection target is damage and cracks 411, when the detection score deviates from 0, the recall rate immediately decreases, and even if the detection score is 0.1, there is a possibility that an oversight of 15% will occur.

FIG. 4C is a diagram for explaining an example of a threshold table included in the pipe damage identification device 100 according to the present embodiment. The threshold value table 403 stores threshold values 432 in association with damage types 431. Damage types 431 include breakage, cracks, infiltrated water, protrusion of attachment pipes, intrusion of tree roots, and mortar adhesion. Then, the damage identifying unit 207 refers to the threshold table 403 and sets a threshold for each damage.

Next, identification of damage by the damage identification unit 207 will be explained. The damage identification unit 207 identifies damage occurring in the sewer pipe 170 using the ceiling plane image 161, the bottom plane image 162, and the learned damage identification model. A method for identifying damage by the damage identifying unit 207 is, for example, a method of identifying damage by referring to a damage identifying table 281 shown in FIG. 4C. Note that the damage identifying unit 207 identifies damage occurring in the sewer pipe 170 using images in which the ceiling plane image 161 and the bottom plane image 162 are arranged in parallel.

The output unit 208 outputs the identified damage to a mobile terminal such as the tablet terminal 130. Further, the worker can recognize damage occurring in the sewer pipe 170 by referring to the identification results displayed on the display of the tablet terminal 130. The output unit 208 may output an alert depending on the type of damage and the degree of progress. The output unit 208 may output an alert using a warning sound, vibration, light, message, or the like.

FIG. 4D is a diagram for explaining an example of the damage identification table 291 included in the pipe damage identification device 100 according to this embodiment. Note that the damage identification table 291 is stored in the storage unit 209. The damage identification table 291 stores the pipe material 442 in association with the damage type 431. Damage types 431 include breakage, cracks, infiltrated water, protrusion of attachment pipes, intrusion of tree roots, and mortar adhesion.

Drain material 442 includes concrete and ceramic. For example, if the pipe material 442 is concrete, that is, if it is a concrete pipe, the damage types 431 may include exposed internal structure, exposed reinforcing bars, and rust stains. Then, the damage identification unit 207 refers to the damage identification table 291 and identifies the damage occurring in the sewer pipe 170. The damage types 431 included in the damage identification table 291 include damage that occurs frequently and damage that cannot be overlooked. In this way, by determining the combination of the material of the pipe and the damage that will occur in advance, it is possible to eliminate damage that cannot occur considering the material of the pipe, allowing for efficient damage identification. becomes possible.

FIG. 4E is a diagram for explaining prediction of the degree of damage by the pipe culvert damage identification device according to the present embodiment. The detection score displayed in the box shown in FIG. 4A can be used to predict the severity of damage or the urgency of repair. For example, in the box shown in FIG. 4, the damage for which the AI displays a large detection score is determined by the AI to have a high probability of being a damage. In reality, such damage has a high probability of being damage, and it may be estimated that such damage is serious or requires a high degree of urgency for repair. On the contrary, damage displaying a low detection score is likely to be a minor damage or include damage that is not a damage in the first place (incorrect answer). Therefore, as shown in the degree prediction diagram 405 in FIG. 4E, the detection score is used to classify the severity (or repair urgency) into "high," "medium," and "small." , "unnecessary", etc. Note that the classification based on the detection score is performed, for example, as 0 to 0.1, 0.1 to 0.5, 0.5 to 0.8, 0.8 to 1.0, etc.

FIG. 5A is a flowchart for explaining the processing procedure of the pipe damage identification device 100 according to the present embodiment. FIG. 5B is another flowchart for explaining the processing procedure of the pipe damage identification device 100 according to this embodiment. The flowcharts shown in FIGS. 5A and 5B are executed by an unillustrated CPU (Central Processing Unit) of the pipe culvert damage identification device 100 using ROM (Read Only Memory) and RAM (Random Access Memory). Each functional configuration of the pipe damage identification device 100 shown in FIG.

Referring to FIG. 5A, in step S501, image acquisition unit 201 acquires inner peripheral surface image 110 of sewer pipe 150 as an image for machine learning. In step S503, the planar image generation unit 202 generates a ceiling planar image 101 and a bottom planar image 102, which are developed by cutting out the inner peripheral surface image 110 from the ceiling and bottom portions of the sewer pipe 150. In step S505, the damage determination unit 203 determines damage occurring in the sewer pipe 150 using the ceiling plane image 101 and the bottom plane image 102.

In step S507, the model generation unit 204 assigns an identifier to the determined damage and inputs the determined damage together with the ceiling plane image 101 and the bottom plane image 102 to the artificial intelligence. In step S509, the model generation unit 204 determines whether machine learning using artificial intelligence has ended. If it is determined that the machine learning using artificial intelligence has not ended (NO in step S509), the model generation unit 204 continues the machine learning using artificial intelligence. If it is determined that the machine learning by artificial intelligence has ended (YES in step S509), the pipe culvert damage identification device 100 proceeds to step S511. In step S511, the model generation unit 204 generates a learned damage identification model.

Referring to FIG. 5B, in step S531, the image receiving unit 205 receives an inner circumferential surface image 160 of the inner circumferential surface of the sewer pipe 170. In step S533, the planar image generation unit 206 generates a ceiling planar image 161 and a bottom planar image 162, which are developed by cutting out the inner peripheral surface image 160 from the ceiling and bottom portions of the sewer pipe 170. In step S535, the damage identification unit 207 identifies damage occurring in the sewer pipe 170 using the ceiling plane image 161, the bottom plane image 162, and the generated learned damage identification model. The damage identification unit 207 identifies damage while changing a threshold value for damage detection depending on the type of damage.

In step S537, the output unit 208 outputs the identification result. In step S539, the pipe culvert damage identification device 100 determines whether damage identification has been completed for all received inner peripheral surface images 160. If it is determined that damage identification for all received internal peripheral surface images 160 has not been completed (NO in step S539), the pipe culvert damage identification device 100 returns to step S533. If it is determined that damage has been identified for all received internal circumferential surface images 160 (YES in step S539), the pipe culvert damage identification device 100 ends the process.

According to this embodiment, damage to pipes is identified using machine learning using artificial intelligence and a threshold value for damage detection, so damage to pipes can be identified with high accuracy while avoiding excessive detection. be able to. Further, by appropriately setting the threshold value, it is possible to accurately identify necessary damage depending on the type of damage.

[Other embodiments]
Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. The configuration and details of the present invention can be modified in various ways that can be understood by those skilled in the art within the scope of the present invention. Furthermore, systems or devices that combine the separate features included in each embodiment in any way are also included within the scope of the present invention.

Moreover, the present invention may be applied to a system composed of a plurality of devices, or may be applied to a single device. Furthermore, the present invention is also applicable when an information processing program that implements the functions of the embodiments is supplied to a system or device directly or remotely. Therefore, in order to realize the functions of the present invention on a computer, a program installed on a computer, a medium storing the program, and a WWW (World Wide Web) server from which the program is downloaded are also included in the scope of the present invention. . In particular, a non-transitory computer readable medium storing at least a program that causes a computer to execute the processing steps included in the embodiments described above is included within the scope of the present invention.

Claims (9)

an image acquisition unit that acquires a first inner circumferential surface image of the inner circumferential surface of the first pipe culvert;
A first ceiling plane image obtained by cutting and developing the first inner circumferential surface image from the ceiling portion of the first pipe culvert in the first inner circumferential surface image; a first planar image generation unit that generates a first bottom planar image obtained by cutting out and expanding the bottom portion of the first culvert;
a damage determination unit that determines first damage occurring in the first pipe culvert using the first ceiling plane image and the first bottom plane image;
Identifying the learned damage by assigning an identifier that can identify the determined first damage and causing artificial intelligence to learn the first ceiling plane image and the first bottom plane image together with the determined first damage. a model generation unit that generates a model;
an image reception unit that receives a second inner circumferential surface image taken of the inner circumferential surface of the second pipe culvert;
a second ceiling plane image obtained by cutting and developing the second inner circumferential surface image from the ceiling portion of the second pipe in the second inner circumferential surface image; a second planar image generation unit that generates a second bottom planar image obtained by cutting out and expanding the bottom portion of the second culvert;
A damage identification unit that detects and identifies second damage occurring in the second pipe culvert using the second ceiling plane image, the second bottom plane image, and the learned damage identification model, a damage identification unit that identifies the second damage occurring in the second pipe culvert by changing a threshold value for damage detection according to the type of the second damage occurring in the second pipe culvert; and,
Equipped with
The damage identification section includes:
enclosing a region where the second damage is believed to exist in a box of a predetermined size;
The detection rate of the second damage occurring in the second pipe is changed by changing the detection score of the box as the threshold for damage detection according to the type of the second damage. further comprising a detection unit that detects the second damage,
A pipe damage identification device that identifies the second damage based on a detection result by the detection unit . The model generation unit generates the learned damage identification model using an image in which the first ceiling plane image and the first bottom plane image are arranged so that the ceiling part and the bottom part correspond to each other. The pipe damage identification device according to claim 1 . The damage identification unit uses an image in which the second ceiling plane image and the second bottom plane image are arranged so that the ceiling part and the bottom part correspond to each other, and the learned damage identification model, The pipe culvert damage identification device according to claim 1 or 2, which identifies the second damage occurring in the second pipe culvert. The pipe damage identification device according to any one of claims 1 to 3 , wherein the first pipe and the second pipe are sewer pipes. The pipe culvert damage identifying device according to claim 1, wherein the damage determination unit determines the first damage occurring in the first pipe culvert using Faster R-CNN. The pipe damage identification device according to any one of claims 1 to 5, wherein the model generation unit generates the learned damage identification model using left-right reversal as padded data. The pipe damage identification device according to any one of claims 1 to 6, wherein the model generation unit generates the learned damage identification model using transfer learning. an image acquisition step of acquiring a first inner circumferential surface image of the inner circumferential surface of the first pipe culvert;
A first ceiling plane image obtained by cutting and developing the first inner circumferential surface image from the ceiling portion of the first pipe culvert in the first inner circumferential surface image; a first planar image generation step of generating a first bottom planar image obtained by cutting out and expanding the bottom portion of the first culvert;
a damage determination step of determining first damage occurring in the first culvert using the first ceiling plane image and the first bottom plane image;
Identifying the learned damage by assigning an identifier that can identify the determined first damage and causing artificial intelligence to learn the first ceiling plane image and the first bottom plane image together with the determined first damage. a model generation step for generating a model;
an image reception step of receiving a second inner circumferential surface image taken of the inner circumferential surface of the second pipe;
a second ceiling plane image obtained by cutting and developing the second inner circumferential surface image from the ceiling portion of the second pipe in the second inner circumferential surface image; a second planar image generation step of generating a second bottom planar image obtained by cutting open and expanding the bottom portion of the second culvert;
When detecting and identifying second damage occurring in the second pipe using the second ceiling plane image, the second bottom plane image, and the learned damage identification model, the second pipe a damage identifying step of identifying the second damage occurring in the second pipe by changing a threshold for damage detection according to the type of the second damage occurring in the second pipe;
including;
In the damage identification step,
enclosing a region where the second damage is believed to exist in a box of a predetermined size;
The detection rate of the second damage occurring in the second pipe is changed by changing the detection score of the box as the threshold for damage detection according to the type of the second damage. further comprising a detection step of detecting the second damage,
A pipe culvert damage identification method that identifies the second damage based on the detection result in the detection step . an image acquisition step of acquiring a first inner circumferential surface image of the inner circumferential surface of the first pipe culvert;
A first ceiling plane image obtained by cutting and developing the first inner circumferential surface image from the ceiling portion of the first pipe culvert in the first inner circumferential surface image; a first planar image generation step of generating a first bottom planar image obtained by cutting out and expanding the bottom portion of the first culvert;
a damage determination step of determining first damage occurring in the first culvert using the first ceiling plane image and the first bottom plane image;
Identifying the learned damage by assigning an identifier that can identify the determined first damage and causing artificial intelligence to learn the first ceiling plane image and the first bottom plane image together with the determined first damage. a model generation step for generating a model;
an image reception step of receiving a second inner circumferential surface image taken of the inner circumferential surface of the second pipe;
a second ceiling plane image obtained by cutting and developing the second inner circumferential surface image from the ceiling portion of the second pipe in the second inner circumferential surface image; a second planar image generation step of generating a second bottom planar image obtained by cutting open and expanding the bottom portion of the second culvert;
When detecting and identifying second damage occurring in the second pipe using the second ceiling plane image, the second bottom plane image, and the learned damage identification model, the second pipe a damage identification step of identifying the second damage occurring in the second pipe by changing a threshold value for damage detection according to the type of the second damage occurring in the second pipe;
make the computer run
In the damage identification step,
enclosing a region where the second damage is believed to exist in a box of a predetermined size;
The detection rate of the second damage occurring in the second pipe is changed by changing the detection score of the box as the threshold for damage detection according to the type of the second damage. further comprising a detection step of detecting the second damage,
A pipe damage identification program that identifies the second damage based on the detection result in the detection step .