JP7356942B2 - Pipe damage identification device, pipe damage identification method, and pipe damage identification program - Google Patents

Description

The present invention relates to a pipe damage identification device, a pipe damage identification method, and a pipe damage identification program, and particularly relates to a device for identifying damage to a pipe from identification results obtained by an image analysis program built into a computer and artificial intelligence.

Infrastructure structures are aging at an accelerating rate, and the need for maintenance, management, and renewal is said to continue to increase in the future. As the working population continues to decline, increasing the efficiency of labor related to maintenance, management, and renewal has become an urgent issue. In order to improve the efficiency of labor related to maintenance, management, and renewal, there is an increasing number of research cases applying AI-based image recognition technology, whose performance has improved significantly in recent years. By using an advanced neural network called deep learning, it is possible to achieve image recognition that is close to human intuition, which was previously difficult, and is expected to assist or replace visual inspection of damaged areas.

On the other hand, in the inspection of infrastructure structures such as pipes, for example, as in the technology described in Patent Document 1 and Patent Document 2, a photographing device such as a television camera or It was equipped with lighting equipment such as fluorescent lights, and was used to investigate damage based on images of the inside of the tube taken with a photographic device.

Japanese Patent Application Publication No. 5-346027 Japanese Patent Application Publication No. 7-216972

However, with the technology described in the above-mentioned document, workers visually inspect the captured images of the inside of the pipe, which is prone to oversight or misreading, and it is difficult to identify damage to pipes with high precision. I couldn't do it.

In order to achieve the above object, the pipe damage identification device according to the present invention has the following features:
an image acquisition unit that acquires a first inner circumferential surface image of the inner circumferential surface of the first pipe culvert;
a first planar image generation unit that generates a plurality of first planar images obtained by cutting and developing the first inner circumferential surface image at an arbitrary position of the first pipe culvert in the first inner circumferential surface image; ,
a damage determination unit that determines first damage occurring in the first culvert using the plurality of first plane images;
model generation for generating a learned damage identification model by assigning an identifier that can identify the determined first damage and causing artificial intelligence to learn the determined first damage together with the plurality of first plane images; Department and
an image reception unit that receives a second inner circumferential surface image taken of the inner circumferential surface of the second pipe culvert;
a second planar image generation unit that generates a plurality of second planar images obtained by cutting and developing the second inner circumferential surface image at an arbitrary position of the second pipe in the second inner circumferential surface image; ,
a damage identification unit that identifies second damage occurring in the second culvert using the plurality of second plane images and the learned damage identification model;
Equipped with

In order to achieve the above object, the pipe damage identification method according to the present invention includes:
an image acquisition step of acquiring a first inner circumferential surface image of the inner circumferential surface of the first pipe culvert;
a first planar image generation step of generating a plurality of first planar images obtained by cutting and developing the first inner circumferential surface image at an arbitrary position of the first pipe culvert in the first inner circumferential surface image; ,
a damage determination step of determining first damage occurring in the first culvert using the plurality of first plane images;
a model generation step of assigning an identifier that can identify the determined first damage and causing artificial intelligence to learn the determined first damage together with the first plane image to generate a learned damage identification model; ,
an image reception unit that receives a second inner circumferential surface image taken of the inner circumferential surface of the second pipe culvert;
a second planar image generation step of generating a plurality of second planar images obtained by cutting and developing the second inner circumferential surface image at an arbitrary position of the second pipe in the second inner circumferential surface image; ,
a damage identification step of identifying second damage occurring in the second culvert using the plurality of second plane images and the learned damage identification model;
including.

In order to achieve the above object, the pipe damage identification program according to the present invention,
an image acquisition step of acquiring a first inner circumferential surface image of the inner circumferential surface of the first culvert;
a first planar image generation step of generating a plurality of first planar images obtained by cutting and developing the first inner circumferential surface image at an arbitrary position of the first pipe culvert in the first inner circumferential surface image; ,
a damage determination step of determining first damage occurring in the first culvert using the plurality of first plane images;
a model generation step of assigning an identifier that can identify the determined first damage and causing artificial intelligence to learn the determined first damage together with the first plane image to generate a learned damage identification model; ,
an image reception unit that receives a second inner circumferential surface image taken of the inner circumferential surface of the second pipe culvert;
a second planar image generation step of generating a plurality of second planar images obtained by cutting and developing the second inner circumferential surface image at an arbitrary position of the second pipe in the second inner circumferential surface image; ,
a damage identification step of identifying second damage occurring in the second culvert using the plurality of second plane images and the learned damage identification model;
have the computer execute it.

According to the pipe damage identification device of the present invention, artificial intelligence can be trained efficiently and a trained model with high accuracy can be generated, so that damage to the pipe can be identified with high accuracy.

It is a figure for explaining the outline of identification of pipe damage by a pipe damage identification device concerning a 1st embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram for demonstrating the structure of the pipe damage identification apparatus based on 1st Embodiment of this invention. FIG. 3 is a diagram for explaining the distribution of occurrence positions within a pipe culvert, which are identified by the pipe culvert damage identification device according to the first embodiment of the present invention. It is a figure for demonstrating the incision position and expansion|development of the internal peripheral surface image by the plane image generation part of the pipe damage identification apparatus based on 1st Embodiment of this invention. FIG. 3 is a diagram for explaining object detection in the pipe damage identification device according to the first embodiment of the present invention. FIG. 3 is a diagram for explaining determination of damage (damage/cracks) in the pipe damage identification device according to the first embodiment of the present invention. It is another diagram for explaining the determination of damage (damage/cracks) in the pipe damage identification device according to the first embodiment of the present invention. FIG. 3 is a diagram for explaining determination of damage (intrusion water) in the pipe damage identification device according to the first embodiment of the present invention. FIG. 7 is another diagram for explaining determination of damage (intrusion water) in the pipe damage identification device according to the first embodiment of the present invention. It is a figure for explaining an example of the damage identification table which the pipe damage identification device concerning a 1st embodiment of the present invention has. It is a figure for explaining an example of the incision position table which the pipe damage identification device concerning a 1st embodiment of the present invention has. It is a flow chart for explaining the processing procedure of the pipe damage identification device concerning a 1st embodiment of the present invention. It is another flowchart for demonstrating the processing procedure of the pipe damage identification apparatus based on 1st Embodiment of this invention.

EMBODIMENT OF THE INVENTION Below, the form for implementing this invention will be exemplarily described in detail with reference to drawings. However, the configuration, numerical values, process flow, functional elements, etc. described in the following embodiments are merely examples, and modifications and changes may be made freely, and the technical scope of the present invention is limited to the following description. It is not intended to do so.

[First embodiment]
A pipe damage identification device 100 as a first embodiment of the present invention will be described using FIGS. 1 to 5B. The pipe culvert damage identification device 100 is used to identify damage that has occurred in a pipe culvert. FIG. 1 is a diagram for explaining an overview of identifying pipe damage by the pipe damage identification device 100 according to the present embodiment.

Here, the term "pipe culvert" is a general term for waterways, and refers to the entire waterway constructed for the purpose of water supply and drainage. Examples include water pipes, sewer pipes, drain pipes, water supply pipes, etc. In this embodiment, a sewer pipe 150 will be explained as an example of a pipe. In addition, the materials of the pipes include concrete, ceramic, iron, etc., and the types of pipes include concrete pipes, concrete fume pipes, ceramic pipes, iron pipes, etc.

Further, damage includes not only cracks, cracks, and scratches that occur in the pipe culvert, but also includes a state in which the pipe culvert is unable to exhibit its expected performance. In the following explanation, the diameter of the sewer pipe 150 is such that a worker cannot enter the sewer pipe 150 and work therein, and the explanation will be made assuming that the sewer pipe 150 has a diameter of 200 to 800 mm, for example. However, the diameter of the sewer pipe 150 is not limited to this.

In the inspection of the inside of the sewer pipe 150, an inspection is performed using an image of the inner peripheral surface of the sewer pipe 150 captured by the camera 142 of the self-propelled inspection robot 140. The worker uses a display installed outside the sewer pipe 150 to check the image captured by the camera 142 and inspects whether or not the sewer pipe 150 is damaged. The control unit 141 controls the self-propelled speed, imaging schedule, and imaging conditions of the self-propelled inspection robot 140, and controls communication with the pipe damage identification device 100 and the tablet terminal 130 carried by the worker. do. A worker operates the self-propelled inspection robot 140 using, for example, an operation application installed on the tablet terminal 130. Note that in the case of a sewer pipe 150 with a small diameter that the self-propelled inspection robot 140 cannot enter, a micro-inspection device such as a fiberscope may be used instead of the self-propelled inspection robot 140.

Then, the control unit 141 transmits the inner peripheral surface image 110 (160) captured by the camera 142 to the pipe damage identification device 100. Note that the inner circumferential surface image 110 is an image for learning by artificial intelligence, and the inner circumferential surface image 160 is an image in which damage is to be identified. Furthermore, the camera 142 may be either a wide-angle camera or an all-around camera. In the case of a wide-angle camera, the images for one revolution may be captured in multiple steps, and the inner peripheral surface image 110 (160) may be generated by connecting the captured images.

Further, the inner circumferential surface image 110 (160) is sliced at predetermined intervals (for example, 0.1 m). For example, when the space between the joints of the sewer pipe 150 is defined as one unit, the images captured for each unit (inner peripheral surface images 110 (160)) may be sliced into slices to obtain the slice images 143. Further, for example, the image captured along the entire length of the sewer pipe 150 (inner peripheral surface image 110 (160)) may be sliced into slices to obtain the slice image 143.

Next, generation of a trained damage identification model by the pipe damage identification device 100 will be explained. The pipe damage identification device 100 cuts out and develops the sliced image 143 of the received internal circumferential surface image 110 (learning image) at an arbitrary position of the sewer pipe 150 in the internal circumferential surface image 110, and generates a two-dimensional plane. Get the image. That is, the pipe damage identification device 100 cuts out and develops the inner peripheral surface image 110 at the ceiling, bottom, and side portions of the sewer pipe 150. A developed plane image 103 is generated.

In the ceiling expanded planar image 101, the bottom portion 111 of the sewer pipe 150 is located at the center of the strip-shaped planar image, and the ceiling portions 112 and 113, which are incision positions, are located at both left and right ends. Similarly, in the bottom developed planar image 102, the ceiling portion 121 is located at the center of the strip-shaped planar image, and the bottom portions 122 and 123, which are incision positions, are located at both left and right ends. Furthermore, in the side developed planar image 103, for example, a portion 131 corresponding to a line-symmetrical position to the incision position is located in the center of the band-shaped planar image, and both ends 132 and 133 of the incision position are located on the left and right sides of the band-shaped planar image. located at both ends. Then, the pipe damage identification device 100 uses the ceiling developed plane image 101, the bottom developed plane image 102, and the side developed plane image 103 to identify damage that has occurred in the sewer pipe 150, and The images (101 to 103) are trained by artificial intelligence to generate a trained damage identification model.

Next, damage identification by the pipe culvert damage identification device 100 will be explained. The pipe damage identifying device 100 analyzes the inner circumferential surface image 160 of the sewer pipe 170 whose damage is to be identified, as well as the inner circumferential surface image 110, a ceiling developed planar image 161, a bottom developed planar image 162, and a side developed planar image 163. generate. Then, the pipe damage identification device 100 uses the generated ceiling expanded plane image 161, bottom expanded plane image 162, and side expanded plane image 163 and the learned damage identification model to identify damage occurring in the sewer pipe 170. Identify.

The identified damage is output to, for example, a mobile terminal such as a tablet terminal 130 owned by the worker. The worker refers to the identification results displayed on the display of the tablet terminal 130 and recognizes the damage occurring in the sewer pipe 170. Note that the pipe damage identification device 100 may output an alert depending on the type and degree of damage identified. For example, if the damage is progressing or requires immediate repair work, the pipe damage identification device 100 may output an alert using a warning sound, vibration, light, message, or the like.

FIG. 2 is a block diagram for explaining the configuration of the pipe damage identification device 100 according to this embodiment. The pipe damage identification device 100 includes an image acquisition section 201, a planar image generation section 202, a damage determination section 203, a model generation section 204, an image reception section 205, a planar image generation section 206, a damage identification section 207, and an output section 208. .

Here, the pipe damage identification device 100 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a network interface, and a storage. Here, the CPU is a processor for arithmetic control, and implements each functional configuration of the pipe damage identification device 100 shown in FIG. 2 by executing a program. A CPU may include multiple processors and may execute different programs, modules, tasks, threads, etc. in parallel. The ROM stores fixed data such as initial data and programs and other programs. The network interface also communicates with other devices via the network. Note that the CPU is not limited to one, and may be a plurality of CPUs or may include a GPU (Graphics Processing Unit) for image processing. Further, it is preferable that the network interface has another CPU independent of the CPU, and writes or reads transmitted and received data in the RAM area. Further, it is desirable to provide a DMAC (Direct Memory Access Controller) that transfers data between the RAM and the storage. Further, the CPU recognizes that data has been received or transferred to the RAM and processes the data. Further, the CPU prepares the processing results in the RAM, and leaves subsequent transmission or transfer to the network interface or DMAC.

RAM is a memory that the CPU uses as a temporary storage work area. The RAM has an area reserved for storing data necessary to implement this embodiment. The storage stores a database, various parameters, modules, or data or programs necessary to implement this embodiment. For example, a control program for controlling the entire pipe damage identification device 100 is stored in the storage.

Furthermore, the pipe damage identification device 100 may further include an input/output interface. A display section, an operation section, and a storage medium are connected to the input/output interface. The input/output interface may further be connected to a speaker as an audio output section, a microphone as an audio input section, or a GPS (Global Positioning System) position determination section. Note that the RAM and storage may store programs and data regarding general-purpose functions and other realizable functions that the pipe culvert damage identification device 100 has.

The image acquisition unit 201 acquires an inner peripheral surface image 110 that captures an image of the inner peripheral surface of the sewer pipe 150. The image acquisition unit 201 may acquire the inner circumferential surface image 110 from the control unit 141 of the self-propelled inspection robot 140, or may acquire it via a tablet terminal 130 owned by a worker. Note that the inner peripheral surface image 110 is an image used for later learning by artificial intelligence.

The planar image generation unit 202 generates a plurality of planar images obtained by cutting and developing the inner circumferential surface image 110 at an arbitrary position in the inner circumferential surface image 110. The planar image generation unit 202 generates, as one of the plurality of planar images, a ceiling expanded planar image 101 obtained by cutting out and expanding the ceiling portion of the sewer pipe 150 in the inner peripheral surface image 110.

Furthermore, the planar image generation unit 202 generates, as one of the plurality of planar images, a bottom developed planar image 102 obtained by cutting open and developing the bottom portion of the sewer pipe 150 in the inner peripheral surface image 110. Further, the planar image generation unit 202 generates a side developed planar image 103, which is obtained by cutting open and developing a side portion of the sewer pipe 150 in the inner peripheral surface image 110, as one of the plurality of planar images.

In this way, the planar image generation unit 202 generates the ceiling developed planar image 101, the bottom developed planar image 102, and the side developed planar image 103 as three types of planar images.

Here, with reference to FIG. 3A, the tendency for damage to occur on the inner circumferential surface of the sewer pipe 150 will be described. FIG. 3A is a diagram for explaining the distribution of occurrence positions within a pipe culvert, which are identified by the pipe culvert damage identification device 100 according to the present embodiment. The X-axis indicates the position of damage in the left-right direction, and the Y-axis indicates the position of damage in the vertical direction.

A distribution diagram 301 shows damage and cracks, a distribution diagram 302 shows infiltrated water, a distribution diagram 303 shows protrusion of the attachment pipe, and a distribution diagram 304 shows other damage. Each point indicates the location where each damage occurred. Furthermore, locations with a high density of dots indicate a high frequency of damage occurrence.

Regarding damage, cracks, and infiltrated water (distribution diagram 301, distribution diagram 302), it can be seen that there is little distribution near the vertical center. That is, it can be seen that a large amount of water is distributed in the parts of the sewer pipe 150 other than the ceiling part and the bottom part, for example, the shoulder part, the side part, and the lower side part. Regarding the protrusion (303) of the attachment pipe, it can be seen that there is a large distribution in the vertical direction. That is, it can be seen that a large amount of water is distributed in the ceiling and bottom portions of the sewer pipe 150.

In this way, depending on the type of damage, there are characteristics in the appearance position, and if the slice image 143 or the inner circumferential surface image 110 is cut open and developed using such characteristics, the damage will be present at the position of the cut. The chances are significantly reduced. Therefore, it is possible to save time and effort in dealing with defects that occur due to division of damage, and it is therefore possible to have the artificial intelligence learn more images more quickly and efficiently.

Next, with reference to FIG. 3B, the incision position and development of the inner peripheral surface image 110 by the planar image generation unit 202 will be described. FIG. 3B is a schematic diagram for explaining the incision position and development of the inner peripheral surface image by the planar image generation unit of the duct damage identification device 100 according to the present embodiment. 3(a) shows a front view of the inner peripheral surface image 110, and FIG. 3(b) shows a perspective view of the inner peripheral surface image 110. In addition, in the figure, the thickness of the sewer pipe 150 and the like are omitted to avoid complicating the diagram.

As shown in FIG. 3B(a), the incision position in the inner peripheral surface image 110 changes depending on the type of damage. The incision position of the inner peripheral surface image 110 is divided into four regions: a ceiling portion 311 , a bottom portion 312 , a left side portion 313 , and a right side portion 314 . Note that the left side portion 313 and the right side portion 314 are in a line-symmetrical relationship with respect to the central axis of the sewer pipe 150.

As shown in FIG. 3B(b), for example, the planar image generation unit 202 cuts out the inner circumferential surface image 110 (sliced image 143) using a cutting tool such as a knife 315 on the computer. Then, the planar image generation unit 202 expands the cut inner peripheral surface image 110 in the direction of arrows (316, 317) to obtain a two-dimensional planar image. Specifically, for example, when the damage that occurs to the sewer pipe 150 is a breakage, the planar image generation unit 202 generates a bottom expanded image that is cut open and expanded at the bottom portion 312 or the left side portion 313 (right side portion 314). A planar image 102 and a side expanded planar image 103 are generated.

Returning to FIG. 2 again, damage determination by the damage determination unit 203 will be described. The damage determining unit 203 determines damage occurring in the sewer pipe 150 using the ceiling developed plan image 101, the bottom developed plan image 102, and the side developed plan image 103. Types of damage include breakage, cracks, water intrusion, protrusion of attachment pipes, root penetration, and mortar adhesion. Note that the damage determination unit 203 uses object detection to detect an area where damage has occurred. Faster R-CNN (Faster Region with Convolution Neural Network) was used as the object detection algorithm.

Here, the determination of damage by the damage determination unit 203 will be specifically explained with reference to FIGS. 4A to 4E. Detection of damage when the damage is breakage and cracking will be described with reference to FIG. 4A. FIG. 4A is a diagram for explaining determination of damage (damage/cracks) in the pipe culvert damage identification device 100 according to the present embodiment. An image 401 shows a state in which damage or cracks have occurred on the inner peripheral surface of the sewer pipe 150. Here, the damage is a crack that occurs in the horizontal direction, and the crack is a crack that occurs in the vertical direction (circumferential direction).

When the damage occurring to the sewer pipe 150 is breakage and cracks, these damages are labeled as damage of another event. In the inner peripheral surface image 110, white or black streaks are considered to be damage or cracks. In the case of cracks, if the cracks are in both vertical and horizontal directions, or if the cracks are diagonal, they are treated as damage. For example, in the case of a vertical crack, an inclination of approximately 30 degrees or less is treated as a crack.

If the damaged or cracked area extends left and right across the joint portion 411, the damage determination unit 203 does not detect damage in one box. The damage determination unit 203 divides the box into two with the joint portion 411 as the boundary, and uses the two boxes 412 and 413 to detect two damages.

Similarly, if the damaged or cracked area extends vertically across the flowing water section 414 (the bottom of the sewer pipe 150), the damage determination unit 203 does not detect damage in one box. The damage determining unit 203 divides the box into two with the water flowing part 414 as the boundary, detects two damage using the two boxes 415 and 416, and determines the damage.

Next, damage detection when a plurality of small damages exist in a narrow area will be described with reference to FIG. 4B. FIG. 4B is another diagram for explaining determination of damage (damage/cracks) in the pipe culvert damage identification device 100 according to the present embodiment. Image 402 shows a state in which small damage or cracks have occurred in a narrow area.

When a plurality of small damages occur in a narrow range, the damage determining unit 203 uses two boxes 421 and 422, as shown in the left diagram of FIG. 4C, and does not treat the damage as a plurality of damages. As shown in the right diagram of FIG. 4C, the damage determining unit 203 surrounds a plurality of damages in one box 423 and treats them as one damage. In this way, when a plurality of small damages exist in a narrow area, by handling these damages collectively, the damages can be detected and determined efficiently.

Next, with reference to FIG. 4C, detection of damage when the damage is caused by infiltrated water will be described. FIG. 4C is a diagram for explaining determination of damage (intrusion water) in the pipe culvert damage identification device 100 according to the present embodiment. Image 403 shows a state in which infiltrated water is occurring. The upper part of the image 403 shows, from the left, a black blurred part, a brown part (soil flowing out part), and a white part (free lime), all of which are treated as infiltrating water.

Then, as shown in the lower part of the image 403, in the case of infiltrated water, the damage determination unit 203 surrounds the area from the center part 431 of the infiltrated water to the part where the same color tone continues with a box 432. However, in the case of soil or free lime, the damage determination unit 203 surrounds the non-white portion with a box 432. More specifically, the damage determining unit 203 surrounds the damage with a box 432 centering on the joint portion or damaged portion that causes intrusion water, and detects the object. For example, if intrusion water occurs due to damage, the damage determination unit 203 determines that it is damage and intrusion water.

Next, with reference to FIG. 4D, a description will be given of detection of damage in the case of infiltrated water where damage is difficult to determine. FIG. 4D is another diagram for explaining determination of damage (infiltrated water) in the pipe damage identification device 100 according to the present embodiment. The upper part of the image 404 shows a state in which the infiltrated water flows out from the left and right joint portions 442 and 443, and the boundary between the two infiltrated waters is not clear. In this case, the damage determination unit 203 surrounds the intruding water in one box 441 and treats it as one damage. In addition, as shown in the two images at the bottom of image 404, when it is difficult to distinguish between the leakage of infiltrated water and traces of running water, it is possible to distinguish between traces of infiltrated water extending from a joint or damaged part that is thought to be the source of the infiltrated water. The damage is surrounded by a box 444 based on whether or not it is connected to the water trail.

Finally, detection of small-sized intruding water will be described with reference to FIG. 4E. FIG. 4E is still another diagram for explaining determination of damage (intrusion water) in the pipe culvert damage identification device 100 according to the present embodiment. Image 405 shows small infiltrated water. The damage determination unit 203 detects small intruding water, but for example, it detects small water near the flow channel (from its size, it is presumed to be a trace of creeping up from the flow channel) and water with a small width. shall be excluded from the scope of However, a detection target 451 is one in which the bleed extends upward to some extent or the bleed is about to separate. Items that have traces of infiltrated water but are dry and have low contrast are excluded.

Returning to FIG. 2 again. The model generation unit 204 assigns an identifier (ID) that can identify the determined damage. The model generation unit 204 generates identifiers such as the range (size) of the damage, the name of the damage, the attributes of the damage (old, new, color), etc. It is added to the developed planar image 103. Note that this identifier may be directly assigned to the ceiling unfolded plane image 101, the bottom unfolded plane image 102, and the side unfolded plane image 103, or may be stored in a predetermined database or the like and read out as appropriate.

Then, the model generation unit 204 inputs the assigned identifier, the determined damage, the ceiling developed plane image 101, the bottom developed plane image 102, and the side developed plane image 103 to artificial intelligence (AI). Let them learn. When the machine learning by artificial intelligence is completed, the model generation unit 204 generates a learned damage identification model. Note that the model generation unit 204 may store the generated learned damage identification model in a predetermined storage or the like. In this case, each time a new learning image (ceiling developed plane image 101, bottom developed plane image 102, and side developed plane image 103) is acquired, machine learning is performed, and a learned damage identification model is generated. The learned damage identification model may be updated.

Furthermore, machine learning using artificial intelligence is performed using known algorithms. In machine learning, the loss function specifies weights and uses the reciprocal of the number of events. In addition, the model generation unit 204 increases the number of inner circumferential surface images 110 that are made to be learned by the artificial intelligence in order to improve the accuracy of machine learning by the artificial intelligence and generate a more accurate damage identification model. The model generation unit 204 obtains padded data using, for example, horizontal reversal. Furthermore, the model generation unit 204 may use transfer learning to improve the accuracy of machine learning using artificial intelligence. Here, transfer learning is a method that aims to improve model performance by transferring a trained model using a different dataset to another problem and performing partial learning. This is a method that can be expected to improve inference performance and reduce learning time, especially when training data is insufficient.

Note that recall and precision were used to evaluate the generated trained damage identification model. The recall rate is expressed as recall rate=TP/(TP+FN), and indicates the proportion of what actually appears out of what should appear as a result. In other words, it is an index regarding whether anything has been left out (an index regarding comprehensiveness). The precision rate is expressed as precision rate=TP/(TP+FP) and indicates the percentage of correct results. This is an indicator that shows the percentage of correct answers among all the results that appear (an indicator related to accuracy). TP = True Positive, FP = False Positive. , FN=False Negative.

The image receiving unit 205 receives an inner circumferential surface image 160 that captures an image of the inner circumferential surface of the sewer pipe 170 as a pipe. That is, the image receiving unit 205 receives the inner peripheral surface image 160 captured by the self-propelled inspection robot 140 and transmitted by the control unit 141. The inner circumferential surface image 160 is an image of the sewer pipe 170 to be inspected, and is the sewer pipe whose damage is to be identified.

The planar image generation section 206 generates a planar image similarly to the planar image generation section 202. That is, the planar image generation unit 206 generates a ceiling expanded planar image 161 obtained by cutting the inner circumferential surface image 160 from the ceiling portion of the sewer pipe 170 in the inner circumferential surface image 160 and developing it. Similarly, the planar image generation unit 206 generates a bottom developed planar image 162 obtained by cutting open and developing the bottom portion of the sewer pipe 170 in the inner peripheral surface image 160. Furthermore, the planar image generation unit 206 generates a side developed planar image 163 obtained by cutting open and developing the side portion of the sewer pipe 170 in the inner peripheral surface image 160.

That is, the planar image generation unit 206 first generates a slice image from the inner peripheral surface image 160 received by the image reception unit 205. Then, the planar image generation unit 206 generates a ceiling developed planar image 161, a bottom developed planar image 162, and a side developed planar image 163 from the sliced images. Note that the sliced image may be generated in advance by the self-propelled inspection robot 140, and the image reception unit 205 may acquire the sliced image generated by the self-propelled inspection robot 140.

The damage identification unit 207 identifies damage occurring in the sewer pipe 170 using the ceiling developed plane image 161, the bottom developed plane image 162, the side developed plane image 163, and the learned damage identification model. A method for identifying damage by the damage identifying unit 207 is, for example, a method of identifying damage by referring to a damage identifying table 500 shown in FIG. 5A.

FIG. 5A is a diagram for explaining an example of a damage identification table 500 included in the pipe damage identification device 100 according to the present embodiment. The damage identification table 500 stores the pipe material 502 in association with the damage type 501. Types of damage 501 include breakage, cracks, infiltrated water, protrusion of attachment pipes, intrusion of tree roots, and mortar adhesion.

Drain material 502 includes concrete and ceramic. For example, if the pipe material 502 is concrete, that is, if it is a concrete pipe, the damage types 501 may include exposed internal structure, exposed reinforcing bars, and rust stains. Then, the damage identification unit 207 refers to the damage identification table 500 and identifies damage occurring in the sewer pipe 170. Damage types 501 included in the damage identification table 500 include damage that occurs frequently and damage that cannot be overlooked. In this way, by determining the combination of the material of the pipe and the damage that will occur in advance, it is possible to eliminate damage that cannot occur considering the material of the pipe, allowing for efficient damage identification. becomes possible.

FIG. 5B is a diagram for explaining an example of the incision position table 510 included in the duct damage identification device 100 according to the present embodiment. The incision position table 510 stores incision positions 512 in association with the injury types 501. The incision position 512 indicates a position in the inner peripheral surface images 110 and 160 at which these images are incised. The planar image generation units 202 and 206 refer to the incision position table 510 to determine the incision positions of the inner peripheral surface images 110 and 160.

Returning to FIG. 2 again. The output unit 208 outputs the identified damage to a mobile terminal such as the tablet terminal 130. Further, the worker can recognize damage occurring in the sewer pipe 170 by referring to the identification results displayed on the display of the tablet terminal 130. The output unit 208 may output an alert depending on the type of damage and the degree of progress. The output unit 208 may output an alert using a warning sound, vibration, light, message, or the like.

FIG. 6A is a flowchart for explaining the processing procedure of the pipe damage identification device 100 according to the present embodiment. FIG. 6B is another flowchart for explaining the processing procedure of the pipe damage identification device 100 according to this embodiment. The flowcharts shown in FIGS. 6A and 6B are executed by an unillustrated CPU (Central Processing Unit) of the pipe culvert damage identification device 100 using ROM (Read Only Memory) and RAM (Random Access Memory). Each functional configuration of the pipe damage identification device 100 shown in FIG.

In step S601, the image acquisition unit 201 acquires the inner peripheral surface image 110 of the sewer pipe 150 as an image for machine learning. In step S603, the planar image generation unit 202 creates a ceiling developed planar image 101, a bottom developed planar image 102, and a side developed planar image 101, a bottom developed planar image 102, and a side developed image created by cutting out the inner peripheral surface image 110 from the ceiling, bottom, and side portions of the sewer pipe 150. A planar image 103 is generated. In step S605, the damage determining unit 203 determines the damage occurring in the sewer pipe 150 using the ceiling developed plan image 101, the bottom developed plan image 102, and the side developed plan image 103.

In step S607, the model generation unit 204 assigns an identifier that can identify the determined damage, and generates an artificial intelligence along with the determined damage, the ceiling developed planar image 101, the bottom developed planar image 102, and the side developed planar image 103. Enter. In step S609, the model generation unit 204 determines whether machine learning using artificial intelligence has ended. If it is determined that the machine learning using artificial intelligence has not ended (NO in step S609), the model generation unit 204 continues the machine learning using artificial intelligence. If it is determined that the machine learning by artificial intelligence has ended (YES in step S609), the pipe culvert damage identification device 100 proceeds to step S611. In step S611, the model generation unit 204 generates a learned damage identification model.

In step S631, the image receiving unit 205 receives an inner circumferential surface image 160 obtained by capturing an image of the inner circumferential surface of the sewer pipe 170. In step S633, the planar image generation unit 206 cuts out the inner peripheral surface image 160 from the ceiling, bottom, and side portions of the sewer pipe 170 and develops a ceiling developed planar image 161, a bottom developed planar image 162, and a side developed image. A planar image 163 is generated. In step S635, damage occurring in the sewer pipe 170 is identified using the ceiling developed plane image 161, the bottom developed plane image 162, the side developed plane image 163, and the generated learned damage identification model.

In step S637, the output unit 208 outputs the identification result. In step S539, the pipe culvert damage identification device 100 determines whether damage identification has been completed for all received inner peripheral surface images 160. If it is determined that damage identification for all received internal peripheral surface images 160 has not been completed (NO in step S539), the pipe culvert damage identification device 100 returns to step S533. If it is determined that damage has been identified for all received internal circumferential surface images 160 (YES in step S539), the pipe culvert damage identification device 100 ends the process.

According to this embodiment, the way the internal circumferential surface image is cut is changed depending on the type of damage, and the artificial intelligence is trained, so that learning can be done efficiently and in a short time, and a trained damage identification model with high accuracy can be created. can be generated and damage can be identified with high accuracy.

[Other embodiments]
Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. The configuration and details of the present invention can be modified in various ways that can be understood by those skilled in the art within the scope of the present invention. Furthermore, systems or devices that combine the separate features included in each embodiment in any way are also included within the scope of the present invention.

Moreover, the present invention may be applied to a system composed of a plurality of devices, or may be applied to a single device. Furthermore, the present invention is also applicable when an information processing program that implements the functions of the embodiments is supplied to a system or device directly or remotely. Therefore, in order to realize the functions of the present invention on a computer, a program installed on a computer, a medium storing the program, and a WWW (World Wide Web) server from which the program is downloaded are also included in the scope of the present invention. . In particular, a non-transitory computer readable medium storing at least a program that causes a computer to execute the processing steps included in the embodiments described above is included within the scope of the present invention.

Claims (9)

an image acquisition unit that acquires a first inner circumferential surface image of the inner circumferential surface of the first pipe culvert;
As a plurality of first plane images obtained by cutting and developing the first inner circumferential surface image at an arbitrary position of the first pipe culvert in the first inner circumferential surface image, A first ceiling developed plane image obtained by cutting out and developing the ceiling part of the first pipe culvert, and a first bottom developed plane image obtained by cutting out the bottom part of the first pipe culvert in the first inner peripheral surface image. , and a first image that generates a first left side developed planar image and a first right side developed planar image obtained by cutting open and developing a side portion of the first pipe culvert in the first inner peripheral surface image. A planar image generation unit;
Using the first ceiling developed plane image, the first bottom developed plane image, the first left side developed plane image, and the first right side developed plane image, 1 a damage determination unit that determines damage;
model generation for generating a learned damage identification model by assigning an identifier that can identify the determined first damage and causing artificial intelligence to learn the determined first damage together with the plurality of first plane images; Department and
an image reception unit that receives a second inner circumferential surface image taken of the inner circumferential surface of the second pipe culvert;
As a plurality of second plane images obtained by cutting and developing the second inner circumferential surface image at an arbitrary position of the second pipe culvert in the second inner circumferential surface image, A second ceiling developed plane image obtained by cutting out and developing the ceiling part of the second pipe culvert, and a second bottom part obtained by cutting and developing the bottom part of the second pipe in the second internal peripheral surface image. A second developed planar image, and a second left side developed planar image and a second right side developed planar image obtained by cutting at the side portion of the second pipe in the second internal peripheral surface image. A planar image generation unit;
Using the second ceiling developed plane image, the second bottom developed plane image, the second left side developed plane image, the second right side developed plane image , and the learned damage identification model, the second a damage identification unit that identifies second damage occurring in the pipe;
Pipe damage identification device equipped with The damage determining unit includes:
identifying the first damage including at least one of infiltrating water and mortar adhesion using the first ceiling developed plan image , the first left side developed plan image , or the first right side developed plan image ;
Using the first bottom developed planar image , the first left side developed planar image , or the first right side developed planar image, detect the first bottom section including at least one of damage, cracks, protrusion of an attachment pipe, and intrusion of tree roots. The pipe culvert damage identification device according to claim 1, which identifies one damage. The damage identification section includes:
identifying the second damage including at least one of infiltrating water and mortar adhesion using the second ceiling developed plan image , the second left side developed plan image , or the second right side developed plan image ;
Using the second bottom developed plane image , the second left side developed plane image , or the second right side developed plane image, the second bottom developed plane image, which includes at least one of damage, cracks, protrusion of the attachment pipe, and tree root intrusion, is detected. 3. The pipe conduit damage identifying device according to claim 1, which identifies two types of damage. The pipe damage identifying device according to claim 1, wherein the first pipe and the second pipe are sewer pipes. 5. The pipe culvert damage identifying device according to claim 1, wherein the damage determining unit determines the first damage occurring in the first pipe using Faster R-CNN. The pipe damage identification device according to any one of claims 1 to 5, wherein the model generation unit generates the learned damage identification model using left-right reversal as the inflated data. The pipe damage identification device according to claim 1, wherein the model generation unit generates the learned damage identification model using transfer learning. an image acquisition step of acquiring a first inner circumferential surface image of the inner circumferential surface of the first pipe culvert;
As a plurality of first plane images obtained by cutting and developing the first inner circumferential surface image at an arbitrary position of the first pipe culvert in the first inner circumferential surface image, A first ceiling developed plane image obtained by cutting out and developing the ceiling part of the first pipe culvert, and a first bottom developed plane image obtained by cutting out the bottom part of the first pipe culvert in the first inner peripheral surface image. , and a first image that generates a first left side developed planar image and a first right side developed planar image obtained by cutting open and developing a side portion of the first pipe culvert in the first inner peripheral surface image. a planar image generation step;
Using the first ceiling developed plane image, the first bottom developed plane image, the first left side developed plane image, and the first right side developed plane image, 1 a damage determination step of determining damage;
model generation for generating a learned damage identification model by assigning an identifier that can identify the determined first damage and causing artificial intelligence to learn the determined first damage together with the plurality of first plane images; step and
an image reception step of receiving a second inner circumferential surface image taken of the inner circumferential surface of the second pipe;
As a plurality of second plane images obtained by cutting and developing the second inner circumferential surface image at an arbitrary position of the second pipe culvert in the second inner circumferential surface image, A second ceiling developed plane image obtained by cutting out and developing the ceiling part of the second pipe culvert, and a second bottom part obtained by cutting and developing the bottom part of the second pipe in the second internal peripheral surface image. A second developed planar image, and a second left side developed planar image and a second right side developed planar image obtained by cutting at the side portion of the second pipe in the second internal peripheral surface image. a planar image generation step;
Using the second ceiling developed plane image, the second bottom developed plane image, the second left side developed plane image, the second right side developed plane image , and the learned damage identification model, the second a damage identification step of identifying second damage occurring in the pipe;
Methods for identifying pipe damage, including: an image acquisition step of acquiring a first inner circumferential surface image of the inner circumferential surface of the first pipe culvert;
As a plurality of first plane images obtained by cutting and developing the first inner circumferential surface image at an arbitrary position of the first pipe culvert in the first inner circumferential surface image, A first ceiling developed plane image obtained by cutting out and developing the ceiling part of the first pipe culvert, and a first bottom developed plane image obtained by cutting out the bottom part of the first pipe culvert in the first inner peripheral surface image. , and a first image that generates a first left side developed planar image and a first right side developed planar image obtained by cutting open and developing a side portion of the first pipe culvert in the first inner peripheral surface image. a planar image generation step;
Using the first ceiling developed plane image, the first bottom developed plane image, the first left side developed plane image, and the first right side developed plane image, 1 a damage determination step of determining damage;
model generation for generating a learned damage identification model by assigning an identifier that can identify the determined first damage and causing artificial intelligence to learn the determined first damage together with the plurality of first plane images; step and
an image reception step of receiving a second inner circumferential surface image taken of the inner circumferential surface of the second pipe;
As a plurality of second plane images obtained by cutting and developing the second inner circumferential surface image at an arbitrary position of the second pipe culvert in the second inner circumferential surface image, A second ceiling developed plane image obtained by cutting out and developing the ceiling part of the second pipe culvert, and a second bottom part obtained by cutting and developing the bottom part of the second pipe in the second internal peripheral surface image. A second developed planar image, and a second left side developed planar image and a second right side developed planar image obtained by cutting at the side portion of the second pipe in the second internal peripheral surface image. a planar image generation step;
Using the second ceiling developed plane image, the second bottom developed plane image, the second left side developed plane image, the second right side developed plane image , and the learned damage identification model, the second a damage identification step of identifying second damage occurring in the pipe;
A pipe damage identification program that runs on a computer.