JP7320472B2 - Structure damage identification device, structure damage identification method, and structure damage identification program - Google Patents

Description

TECHNICAL FIELD The present invention relates to a structure damage identification device, a structure damage identification method, and a structure damage identification program, and more particularly to an image analysis program incorporated in a computer and an apparatus for identifying structural damage from identification results by artificial intelligence.

The deterioration of infrastructure structures is progressing at an accelerated pace, and it is said that the need for maintenance, management, and renewal will increase further in the future. As the working population continues to decline, improving the efficiency of labor related to maintenance, management and renewal has become an urgent issue. In order to improve the efficiency of labor related to maintenance, management and renewal, there are an increasing number of research cases that apply image recognition technology using AI, whose performance has improved significantly in recent years. By using advanced neural networks called deep learning, it is possible to realize image recognition that is close to human intuition, which has been difficult in the past, and is expected to assist or replace visual inspection of damaged parts.

On the other hand, in the inspection of infrastructure structures such as tubular structures, for example, as in the techniques described in Patent Document 1 and Patent Document 2, an imaging device such as a television camera is attached to the main body of the device that is put into the flow channel tube to be investigated. Equipped with lighting equipment such as fluorescent lamps, etc., and investigations such as damage were conducted based on the images taken by the imaging device inside the pipe.

JP-A-5-346027 JP-A-7-216972

However, in the technique described in the above document, an operator visually inspects the photographed image of the tubular structure. could not do

In order to achieve the above object, the structural damage identification device according to the present invention includes:
an image acquisition unit that acquires a first inner wall surface image of the inner wall surface of the first tubular structure;
A first ceiling plane image obtained by cutting the first inner wall surface image from a ceiling portion of the first tubular structure in the first inner wall surface image and developing it, and the first tubular structure in the first inner wall surface image a first bottom plane image obtained by cutting through a bottom portion of the object and a first side plane image obtained by cutting through a side portion of the first tubular structure in the first inner wall surface image; a planar image generator;
A first plane image corresponding to the attribute of the first tubular structure is selected from the first ceiling plane image, the first bottom plane image, and the first side plane image generated by the first plane image generation unit. a first selection unit for
a damage determination unit that determines a first damage occurring in the first tubular structure using the first planar image selected by the first selection unit;
A model for generating a learned damage identification model by assigning an identifier capable of identifying the determined first damage and having artificial intelligence learn a plurality of types of the first planar images together with the determined first damage. a generator;
an image reception unit that receives a second inner wall surface image of the inner wall surface of the second tubular structure;
A second ceiling plane image obtained by cutting the second inner wall image from a ceiling portion of the second tubular structure in the second inner wall image and developing the second inner wall image, and the second tubular structure in the second inner wall image A second plane generating a second bottom plane image obtained by cutting and unfolding from the bottom portion of the article and a second side plane image obtained by cutting and unfolding from the side portion of said second tubular structure. an image generator;
A second plane image corresponding to the attribute of the second tubular structure is selected from the second ceiling plane image, the second bottom plane image, and the second side plane image generated by the second plane image generation unit. a second selection unit for
a damage identification unit that identifies a second damage occurring in the second tubular structure using the second planar image selected by the second selection unit and the learned damage identification model;
provided.

In order to achieve the above object, the structural damage identification method according to the present invention includes:
an image acquisition step of acquiring a first inner wall surface image of the inner wall surface of the first tubular structure;
A first ceiling plane image obtained by cutting the first inner wall surface image from a ceiling portion of the first tubular structure in the first inner wall surface image and developing it, and the first tubular structure in the first inner wall surface image a first bottom plane image obtained by cutting through a bottom portion of the object and a first side plane image obtained by cutting through a side portion of the first tubular structure in the first inner wall surface image; a planar image generating step;
A first plane image corresponding to the attribute of the first tubular structure is selected from the first ceiling plane image, the first bottom plane image, and the first side plane image generated in the first plane image generation step. a first selection step of
a damage determination unit that determines a first damage occurring in the first tubular structure using the first planar image selected in the first selection step;
A model for generating a learned damage identification model by assigning an identifier capable of identifying the determined first damage and having artificial intelligence learn a plurality of types of the first planar images together with the determined first damage. a generation step;
an image receiving step of receiving a second inner wall surface image of the inner wall surface of the second tubular structure;
A second ceiling plane image obtained by cutting the second inner wall image from a ceiling portion of the second tubular structure in the second inner wall image and developing the second inner wall image, and the second tubular structure in the second inner wall image A second plane generating a second bottom plane image obtained by cutting and unfolding from the bottom portion of the article and a second side plane image obtained by cutting and unfolding from the side portion of said second tubular structure. an image generating step;
A second planar image corresponding to the attribute of the second tubular structure is selected from the second ceiling planar image, the second bottom planar image, and the second side planar image generated in the second planar image generating step. a second selection step of
a damage identification step of identifying a second damage occurring in the second tubular structure using the second planar image selected in the second selection step and the learned damage identification model;
including.

In order to achieve the above object, the structural damage identification program according to the present invention includes:
an image acquisition step of acquiring a first inner wall surface image of the inner wall surface of the first tubular structure;
A first ceiling plane image obtained by cutting the first inner wall surface image from a ceiling portion of the first tubular structure in the first inner wall surface image and developing it, and the first tubular structure in the first inner wall surface image a first bottom plane image obtained by cutting through a bottom portion of the object and a first side plane image obtained by cutting through a side portion of the first tubular structure in the first inner wall surface image; a planar image generating step;
A first plane image corresponding to the attribute of the first tubular structure is selected from the first ceiling plane image, the first bottom plane image, and the first side plane image generated in the first plane image generation step. a first selection step of
a damage determination unit that determines a first damage occurring in the first tubular structure using the first planar image selected in the first selection step;
A model for generating a learned damage identification model by assigning an identifier capable of identifying the determined first damage and having artificial intelligence learn a plurality of types of the first planar images together with the determined first damage. a generation step;
an image receiving step of receiving a second inner wall surface image of the inner wall surface of the second tubular structure;
A second ceiling plane image obtained by cutting the second inner wall image from a ceiling portion of the second tubular structure in the second inner wall image and developing the second inner wall image, and the second tubular structure in the second inner wall image A second plane generating a second bottom plane image obtained by cutting and unfolding from the bottom portion of the article and a second side plane image obtained by cutting and unfolding from the side portion of said second tubular structure. an image generating step;
A second planar image corresponding to the attribute of the second tubular structure is selected from the second ceiling planar image, the second bottom planar image, and the second side planar image generated in the second planar image generating step. a second selection step of
a damage identification step of identifying a second damage occurring in the second tubular structure using the second planar image selected in the second selection step and the learned damage identification model;
run on the computer.

According to the pipe damage identification device of the present invention, damage to a tubular structure is identified using machine learning based on artificial intelligence, so damage to a tubular structure can be identified with high accuracy.

It is a figure for demonstrating the outline of operation|movement of the structure damage identification apparatus which concerns on 1st Embodiment of this invention. It is a figure for demonstrating the planar image produced|generated by the structure damage identification apparatus which concerns on 1st Embodiment of this invention. BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram for demonstrating the structure of the structural damage identification apparatus which concerns on 1st Embodiment of this invention. It is a figure for demonstrating an example of the attribute table which the structure damage identification apparatus which concerns on 1st Embodiment of this invention has. It is a figure for demonstrating an example of the damage determination table which the structure damage identification apparatus which concerns on 1st Embodiment of this invention has. It is a flow chart for explaining the processing procedure of the structure damage identification device according to the first embodiment of the present invention. 7 is another flowchart for explaining the processing procedure of the structural damage identification device according to the first embodiment of the present invention;

EMBODIMENT OF THE INVENTION Below, the form for implementing this invention is demonstrated in detail with reference to drawings. However, the configuration, numerical values, process flow, functional elements, etc. described in the following embodiments are merely examples, and modifications and changes are free, and the technical scope of the present invention is limited to the following descriptions. It is not intended to

[First embodiment]
A structure damage identification device 100 as a first embodiment of the present invention will be described with reference to FIGS. 1 to 4B. A structural damage identification device 100 is used to identify damage that has occurred in a tubular structure. FIG. 1A is a diagram for explaining the outline of the operation of the structural damage identification device 100 according to this embodiment. FIG. 1B is a diagram for explaining a planar image generated by the structural damage identification device according to this embodiment.

Here, tubular structures include road tunnels, railway tunnels, mountain tunnels, undersea tunnels, utility tunnels, pipes and sewers, etc., but the purpose is to use the space created inside cylindrical or semi-cylindrical structures. It is not limited to these as long as it is a construct constructed with. Here, utility tunnels are facilities for burying lifelines such as electricity, telephones, gas, water, etc. collectively under the ground of roads and the like. In addition, culvert is a general term for water channels, and refers to the entire water channel created for the purpose of water supply and drainage. Examples include water pipes, sewer pipes, drain pipes, water supply pipes, and the like.

In this embodiment, a tunnel 120 including a road tunnel, a railroad tunnel, a mountain tunnel, etc. will be described as an example. The tunnel 120 is a horseshoe-shaped structure in cross section. The horseshoe-shaped arch portion corresponds to the inner wall of the tunnel 120 , and the portion other than the arch portion corresponds to the road surface 180 .

Materials for the tunnel 120 include concrete, pottery, iron, and the like. The material of the culvert includes concrete, pottery, iron and the like, and the types of the culvert include concrete pipes, concrete hume pipes, ceramic pipes, iron pipes and the like.

In addition, the damage includes compression, cracks, steps, floats, delamination, peeling, cracks and steps of joints, deformation, movement, subsidence, exposure of reinforcing bars, water leakage, sediment outflow, free lime, and icicles. , side ice included. In addition, damage includes floating, peeling and flaking of peanuts and coal joints, flaking, flaking, flaking and corrosion of repair materials, floating, flaking, deformation, deflection and corrosion of reinforcing materials, and corrosion of steel materials. is not limited to Furthermore, the term "damage" includes cracks, cracks, scratches, etc. occurring in the culvert, but also includes a state in which the expected performance of the culvert cannot be exhibited.

In the inspection of the wall surface inside the tunnel 120, for example, inspection using an image of the inner wall surface of the tunnel 120 captured by the camera 141 of the unmanned aerial vehicle 140 is performed. A worker operates the unmanned aerial vehicle 140 from outside or near the entrance of the tunnel 120 using a remote control application installed on the tablet terminal 130 . Note that the unmanned aerial vehicle 140 may be operated using a remote control device (remote controller) instead of using a remote control application. The remote control application controls the flight speed, imaging schedule, imaging conditions, etc. of the unmanned aerial vehicle 140 .

Here, the inner wall image is an image obtained by capturing an imaging area including the inner wall surface of the tunnel 120 and the road surface 180 with an all-around camera or a wide-angle camera. That is, the inner wall image is an image sliced along a plane perpendicular to the traveling direction of the tunnel 120 . The pitch of the slices is not particularly limited, but the pitch is selected according to the inner diameter of the tunnel 120 and the size of the damage to be identified.

Instead of the unmanned aerial vehicle 140, a self-propelled inspection robot equipped with a camera, an automobile, a train, a motorcycle, a bicycle, etc. equipped with a camera may be used. Alternatively, the user may manually push the trolley on which the camera is mounted.

Also, the unmanned aerial vehicle 140 transmits the image captured by the camera 141 to the tablet terminal 130 . The timing of image transmission is, for example, the timing at which a predetermined number of images are captured, the timing at each predetermined time interval, the timing at which a predetermined amount of images are accumulated, and the like. When transmitting an image from the unmanned aerial vehicle 140, the image compressed in a predetermined format may be transmitted. Note that the image may be directly transmitted from the unmanned aerial vehicle 140 to the structural damage identification device 100 without going through the tablet terminal 130 .

Note that the inner wall image 110 is an image for learning by artificial intelligence, and the inner wall image 160 is an image of the tunnel 120 that is subject to inspection for specific damage. Also, the camera 141 is, for example, a omnidirectional camera capable of capturing an imaging area including the wall surface and the road surface inside the tunnel 120 at once by slicing it, but it is not limited to this. For example, when a wide-angle camera is used as the camera 141, the imaging region including the inner wall surface and the road surface 180 may be captured in a plurality of times, and then these images may be combined to generate a sliced image. Note that the image captured by the camera 141 may be a still image or a moving image. In addition, the camera 141 may have an imaging function using infrared rays, ultraviolet rays, or light rays in other wavelength ranges, in addition to the imaging function using visible light.

The inner wall surface images 110 and 160 are sliced at a predetermined pitch (interval), for example, 1 m. For example, when the interior panel of the tunnel 120 is one unit, the sliced images 110 and 160 captured for each unit may be sliced to obtain the sliced image 143 . Further, for example, the inner wall surface images 110 and 160 captured over the entire length of the tunnel 120 may be sliced to obtain the sliced image 143 .

Next, generation of a learned damage identification model by the structure damage identification device 100 will be described. The structural damage identification device 100 cuts and expands the sliced image 143 of the received inner wall surface image 110 (learning image) at the ceiling, bottom and side portions of the tunnel 120 to obtain a two-dimensional planar image. . That is, the structural damage identification apparatus 100 generates a ceiling plane image 101 developed by cutting the inner wall image 110 at the ceiling of the tunnel 120, and a bottom plane image 101 developed by cutting the inner wall image 110 at the bottom of the tunnel 120. 102 is generated. Furthermore, the structural damage identification apparatus 100 generates a lateral planar image 103 by incising the inner wall surface image 110 at the lateral portion of the tunnel 120 and expanding it.

In the ceiling plane image 101, the bottom part 111 (corresponding to the road surface 180 part) of the tunnel 120 is positioned in the center of the strip-shaped ceiling plane image 101, and the ceiling parts 112 and 113 are positioned at both left and right ends. Similarly, in the bottom plane image 102, the ceiling portion 121 is positioned in the center of the band-shaped bottom plane image 102, and the bottom portions 122 and 123 are positioned at both left and right ends. Further, in the side plane image 103, for example, a portion 131 corresponding to a line-symmetrical position with respect to the incision position is located in the center of the strip-shaped inner wall surface image 110, and both ends 132 and 133 of the incision position are strip-shaped inner wall surface images. They are located at both the left and right ends of 110 . Then, the structural damage identification device 100 identifies damage occurring in the tunnel 120 using the ceiling plane image 101 , the bottom plane image 102 and the side plane image 103 . Then, the structural damage identification device 100 causes artificial intelligence to learn these images (ceiling plane image 101, bottom plane image 102, side plane image 103) in which damage has been identified, and generates a learned damage identification model. do.

Next, identification of damage by the structural damage identification device 100 will be described. The structural damage identification device 100 generates a ceiling plane image 161, a bottom plane image 162, and a side plane image 163 for the inner wall image 160 of the tunnel 120 whose damage is to be specified, similarly to the inner wall image 110. FIG. Then, the structural damage identification device 100 identifies damage occurring in the tunnel 120 using the learned damage identification model.

The identified damage is output to a portable terminal such as the tablet terminal 130 carried by the worker, for example. The worker refers to the identification result displayed on the display of the tablet terminal 130 and recognizes the damage occurring in the tunnel 120 . Note that the structural damage identification device 100 may output an alert according to the type and extent of the identified damage. For example, in the case of ongoing damage or damage that requires immediate repair work, the structural damage identification device 100 may output an alert using a warning sound, vibration, light, message, or the like.

FIG. 2 is a block diagram for explaining the configuration of the structural damage identification device 100 according to this embodiment. The structure damage identification device 100 includes an image acquisition unit 201, a plane image generation unit 202, a selection unit 203, a damage determination unit 204, a model generation unit 205, an image reception unit 206, a plane image generation unit 207, a selection unit 208, and a damage identification unit. It has a section 209 and an output section 210 .

Here, the structural damage identification device 100 has a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a network interface, and a storage. Here, the CPU is a processor for arithmetic control, and implements each functional configuration of the structural damage identification device 100 shown in FIG. 2 by executing a program. The CPU may have multiple processors and execute different programs, modules, tasks, threads, etc. in parallel. ROM stores fixed data such as initial data and programs and other programs. Also, the network interface communicates with other devices and the like via the network. Note that the number of CPUs is not limited to one, and may include a plurality of CPUs or a GPU (Graphics Processing Unit) for image processing. Moreover, it is desirable that the network interface has another CPU independent of the CPU, and writes or reads the transmission/reception data to/from the area of the RAM. It is also desirable to provide a DMAC (Direct Memory Access Controller) for transferring data between RAM and storage. Furthermore, the CPU recognizes that data has been received or transferred to the RAM and processes the data. Also, the CPU prepares the processing result in the RAM and leaves the subsequent transmission or transfer to the network interface or DMAC.

The RAM is a memory used by the CPU as a work area for temporary storage. The RAM has an area for storing data necessary for implementing the present embodiment. The storage stores databases, various parameters, modules, or data or programs necessary for implementing this embodiment. For example, the storage stores a control program for controlling the structural damage identification device 100 as a whole.

Furthermore, the structural damage identification device 100 may further include an input/output interface. A display unit, an operation unit, and a storage medium are connected to the input/output interface. The input/output interface may further be connected to a speaker as an audio output unit, a microphone as an audio input unit, or a GPS (Global Positioning System) position determination unit. The RAM or storage may store programs and data relating to general-purpose functions of the structural damage identification device 100 and other realizable functions.

The image acquisition unit 201 acquires the inner wall surface image 110 of the inner wall surface of the tunnel 120 . The image acquisition unit 201 may acquire the inner wall surface image 110 from the tablet terminal 130 carried by the worker, or may directly acquire the inner wall surface image 110 from the unmanned aerial vehicle 140 . The inner wall image 110 is an image used for learning by artificial intelligence later.

The planar image generation unit 202 generates a plurality of planar images obtained by cutting the inner wall surface image 110 at arbitrary positions on the inner wall surface image 110 and developing the cut open images. The planar image generation unit 202 generates the ceiling planar image 101 obtained by cutting the inner wall surface image 110 from the ceiling portion of the tunnel 120 in the inner wall surface image 110 and developing it. Similarly, the planar image generation unit 202 generates the bottom planar image 102 obtained by cutting the inner wall surface image 110 from the bottom portion of the tunnel 120 in the inner wall surface image 110 and developing it. Further, the planar image generation unit 202 generates the side planar image 103 obtained by cutting the inner wall surface image 110 from the side portion of the tunnel 120 in the inner wall surface image 110 and expanding it. Note that the number of side plane images 103 is not limited to one. In this way, the plane image generation unit 202 generates the ceiling plane image 101, the bottom plane image 102, and the side plane image 103 as three types of plane images.

That is, the plane image generation unit 202 first generates the sliced image 143 from the inner wall surface image 110 acquired by the image acquisition unit 201 . Then, the plane image generation unit 202 generates the ceiling plane image 101 , the bottom plane image 102 and the side plane image 103 from the sliced image 143 . Note that the sliced image 143 may be generated in the unmanned aerial vehicle 140 in advance, and the image acquisition unit 201 may acquire the sliced image 143 generated by the unmanned aerial vehicle 140 .

The selection unit 203 selects a plane image according to the attribute of the tubular structure, that is, the tunnel 120 from the ceiling plane image 101 , bottom plane image 102 and side plane image 103 generated by the plane image generation unit 202 . Here, the attributes of the tubular structure are, for example, the type, material, and use of the tubular structure. In this embodiment, the type of tubular structure is the tunnel 120 and the selection unit 203 selects a planar image suitable for identifying damage to the tunnel 120 .

In the tunnel 120, the site of damage is often the horseshoe-shaped arch portion. As for the road surface 180, damage different from the damage to the inner wall surface of the tunnel 120 occurs, so in this embodiment, the damage occurring on the road surface 180 is out of the specific target. Since the arch portion of the tunnel 120 is likely to be damaged, the side plane image 103 cut at the side portion of the tunnel 120, for example, the left and right boundary portions between the arch portion and the road surface 180 is selected. . This is because by selecting such a lateral planar image 103, the damage occurring in the arch portion can be learned by artificial intelligence using the uncut planar image by cutting the inner wall surface image 110. This is because the accuracy of learning can be improved. For example, if the tubular structure is a sewage pipe, the selection unit 203 selects the ceiling plane image 101 and the bottom plane image 102 .

As a method of selecting a plane image by the selecting unit 203, for example, there is a method of selecting with reference to an attribute table 301 shown in FIG. 3A. FIG. 3A is a diagram for explaining an example of an attribute table 301 possessed by the structural damage identification device 100 according to this embodiment. The attribute table 301 stores planar images 312 in association with attributes 311 . The attribute 311 is the attribute of the tubular structure, and includes the type, material, and usage of the tubular structure. The type is the type of tubular structure, including tunnels, common ducts, and culverts. The material is the material of the tubular structure, and includes concrete, iron, and the like. The usage is the purpose of use of tubular structures, including railroad, mountain, undersea, cable, sewage, and the like. A plane image 312 is a plane image generated by the plane image generation unit 202 and includes a ceiling plane image, a bottom plane image, and a side plane image. Then, the selection unit 203 selects the side plane image as the plane image corresponding to the attribute of the tubular structure, for example, the tunnel 120 made of concrete in this embodiment.

A damage determination unit 204 determines damage occurring in the tunnel 120 using the planar image selected by the selection unit 203 . In this embodiment, the damage determining unit 204 identifies damage occurring in the tunnel 120 using the side plane image 103 selected by the selecting unit 203 . Specifically, the damage determining unit 204 determines the damage using an image in which the two lateral plane images 103 are arranged in parallel. Types of damage include, but are not limited to, crushing, cracking, delamination, spalling, split joints, steps, deformation, movement, exposure of rebar, water intrusion, and the like. As a method of determining damage by the damage determination unit 204, for example, there is a method of determining with reference to the damage determination table 302 shown in FIG. 3B. FIG. 3B is a diagram for explaining an example of a damage determination table possessed by the structural damage identification device according to this embodiment.

The damage determination table 302 stores tubular structures 322 in association with damage types 321 . The types of damage 321 include crushing, cracking, delamination/stripping, split joints/steps, deformation/movement, exposure of reinforcing bars, and infiltration of water. The types of damage 321 are not limited to these, and examples include subsidence, sediment outflow, free lime, icicles, side ice, floating, peeling, and peeling of peanuts and cold joints, and floating, peeling, and peeling of repair materials. , corrosion, floatation, delamination, deformation, deflection, corrosion, steel corrosion, etc. of reinforcing materials. Tubular structures 322 include, but are not limited to, road tunnels, railroad tunnels, mountain tunnels, utility tunnels, conduits, and include, for example, undersea tunnels.

The damage determining unit 204 uses object detection to detect a damaged area and determine damage. As an algorithm for object detection, Faster R-CNN (Faster Region with Convolutional Neural Network) is used in this embodiment.

The model generating unit 205 assigns an identifier (ID: Identifier) that can identify the determined damage. The model generation unit 205 assigns the side plane image 103 with identifiers such as the range (size) of the damage, the name of the damage, and the attributes of the damage (old, new, color). If the ceiling plane image 101 and the bottom plane image 102 are selected by the selection unit 203, the model generation unit 205 assigns identifiers to these plane images. This identifier may be directly assigned to the selected side plane image 103, or may be stored in a predetermined database or the like and read as appropriate.

Then, the model generation unit 205 inputs the side plane image 103 together with the given identifier and the determined damage to artificial intelligence (AI) for machine learning. After the machine learning by artificial intelligence is completed, the model generation unit 205 generates a learned damage identification model. Note that the model generation unit 205 may store the generated learned damage identification model in a predetermined storage or the like. In this case, each time a new learning image (for example, the ceiling plane image 101, the bottom plane image 102 and the side plane image 103) is acquired, machine learning is performed, and a learned damage identification model is generated, it is saved. The learned damage identification model may be updated.

Machine learning by artificial intelligence is performed using known algorithms. In machine learning, the loss function is weighted and uses the reciprocal of the number of events. In addition, the model generation unit 205 increases the number of inner wall surface images 110 learned by artificial intelligence in order to improve the efficiency of machine learning by artificial intelligence and generate a more accurate damage identification model. The model generation unit 205 obtains padding data using, for example, horizontal reversal. Furthermore, the model generator 205 may use transfer learning to improve the efficiency of machine learning with artificial intelligence. Here, point-to-point learning is a technique that aims to improve the performance of a model by diverting a trained model using a different data set to another problem and performing partial learning. In particular, it is also a method that can be expected to improve inference performance and reduce learning time when training data is not sufficient.

In addition, recall and precision were used to evaluate the generated learned damage identification model. The recall is represented by recall=TP/(TP+FN), and indicates the ratio of what actually came out to what should come out as a result. That is, it is an index regarding whether there is any omission (index regarding completeness). Precision is expressed as precision=TP/(TP+FP) and indicates the percentage of correct ones. It is an index that shows how many correct answers are included in all the results (an index of accuracy). TP = true positive, FP = false positive , FN=False Negative.

The image receiving unit 206 receives an inner wall surface image 160 of the inner wall surface of the tunnel 170 . That is, the image receiving unit 206 receives and receives the inner wall surface image 160 captured by the unmanned aerial vehicle 140 from the tablet terminal 130 . The inner wall image 160 is an image of the inner wall surface of the tunnel 170 to be inspected, and is the tunnel for which damage that has occurred is desired to be specified.

The planar image generation unit 207 generates a planar image similarly to the planar image generation unit 202 . That is, the planar image generation unit 207 generates the ceiling planar image 161 obtained by cutting the inner wall surface image 160 from the ceiling portion of the tunnel 170 in the inner wall surface image 160 and developing it. Similarly, the planar image generation unit 207 generates a bottom planar image 162 obtained by cutting the inner wall surface image 160 from the bottom portion of the tunnel 170 and expanding it. Further, the plane image generation unit 207 generates a side plane image 163 obtained by cutting the inner wall surface image 160 from the side portion of the tunnel 170 and developing it. The number of side plane images 163 is not limited to one. In this way, the plane image generation unit 207 generates the ceiling plane image 161, the bottom plane image 162, and the side plane image 163 as three types of plane images.

That is, the planar image generation unit 207 first generates a sliced image from the inner wall surface image 160 received by the image reception unit 206 . Then, the plane image generation unit 207 generates a ceiling plane image 161, a bottom plane image 162, and a side plane image 163 from the sliced images. Note that the sliced image may be generated in the unmanned aerial vehicle 140 in advance, and the image receiving unit 206 may acquire the sliced image generated by the unmanned aerial vehicle 140 .

The selection unit 208 selects a plane image according to the attribute of the tubular structure, that is, the tunnel 170 from the ceiling plane image 161 , bottom plane image 162 and side plane image 163 generated by the plane image generation unit 207 . That is, the selection unit 208 selects, for example, the side plane image 163 as a plane image suitable for determining the damage of the tunnel 170 .

The damage identification unit 209 identifies damage occurring in the tunnel 170 using the side plane image 163 selected by the selection unit 208 and the learned damage identification model. As a method of identifying damage by the damage identifying unit 209, for example, there is a method of identifying with reference to the damage determination table 302 shown in FIG. 3B. Note that the damage identification unit 209 identifies damage occurring in the tunnel 170 using an image in which the two side plane images 163 are arranged in parallel and a learned damage identification model.

The output unit 210 outputs the identified damage to a portable terminal such as the tablet terminal 130 . Also, the worker can recognize the damage occurring in the tunnel 120 by referring to the identification result displayed on the display of the tablet terminal 130 . The output unit 210 may output an alert according to the type of damage and the degree of progress. The output unit 210 may output an alert such as a warning sound, vibration, light, message, or the like.

FIG. 4A is a flowchart for explaining the processing procedure of the structural damage identification device according to this embodiment. FIG. 4B is another flowchart for explaining the processing procedure of the structural damage identification device according to this embodiment. The flowcharts shown in FIGS. 4A and 4B are executed by a CPU (Central Processing Unit) (not shown) of the structural damage identification device 100 using ROM (Read Only Memory) and RAM (Random Access Memory). Each functional configuration of the structural damage identification device 100 shown in 1 is realized.

First, description will be made with reference to FIG. 4A. In step S<b>401 , the image acquisition unit 201 acquires a wall surface image of the inner wall surface of the tunnel 120 . In step S<b>403 , the planar image generation unit 202 generates the sliced image 143 from the inner wall surface image 110 . Then, the plane image generation unit 202 generates the ceiling plane image 101 , the bottom plane image 102 and the side plane image 103 from the sliced image 143 . In step S405, the selection unit 203 selects a planar image according to the attribute of the tubular structure. The selection unit 203 selects the side plane image 103 as the plane image corresponding to the tunnel 120 .

In step S407, the damage determination unit 204 determines damage occurring in the tunnel 120 using the side plane image 103 selected in step S405. In step S409, the model generation unit 205 assigns an identifier capable of identifying the damage determined in step S409, and causes the artificial intelligence to learn the side plane image 103 together with the determined damage, thereby obtaining a learned damage identification model. to generate Here, for example, an image obtained by arranging two lateral plane images 103 in parallel is used to make artificial intelligence learn. That is, artificial intelligence learns an image in which two side plane images 103, which are developed by cutting the sliced image 143 at two different side positions, are arranged in parallel. In step S411, the structural damage identification device 100 determines whether machine learning by artificial intelligence has ended. If it is determined that the machine learning has not ended (NO in step S411), the structural damage identification device 100 returns to step S403. If it is determined that the machine learning has ended (YES in step S411), the structural damage identification device 100 ends the process.

Next, description will be made with reference to FIG. 4B. In step S<b>421 , the image receiving unit 206 receives an inner wall surface image 160 of the inner wall surface of the tunnel 170 . In step S423, the planar image generation unit 207 generates a planar image in the same manner as in step S403. That is, the planar image generation unit 207 generates a ceiling planar image 161, a bottom planar image 162, and a planar image 161 obtained by cutting the inner wall surface image 160 from the ceiling, bottom, and side portions of the tunnel 170 in the inner wall surface image 160 and developing them. A side plane image 163 is generated. In this embodiment, the planar image generator 207 generates the side planar image 163 . In step S425, the selection unit 208 selects the plane image corresponding to the tunnel 170, that is, the side plane image 163 from the plane images generated in step S423. In step S427, the damage identification unit 209 uses the planar image selected in step S425, that is, the side planar image 163 and the learned damage identification model generated in step S409 to determine whether damage has occurred in the tunnel 170. Identify damage. In step S429, the structural damage identification device 100 determines whether or not the damage identification has been completed. When determining that the identification has not ended (NO in step S429), the structural damage identification device 100 returns to step S423. When determining that the identification has ended (YES in step S429), the structural damage identification device 100 proceeds to step S431. In step S431, the output unit 210 outputs the identified damage to a portable terminal such as the tablet terminal 130 or the like.

According to this embodiment, a plane image corresponding to the attribute of the tubular structure is selected and trained by artificial intelligence, so that a highly accurate model can be generated as a trained model for identifying damage. In addition, since the damage occurring in the tubular structure is identified using a highly accurate learned damage identification model, the accuracy of damage identification can be improved.

[Other embodiments]
Although the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. Also, any system or apparatus that combines separate features included in each embodiment is also included in the scope of the present invention.

Further, the present invention may be applied to a system composed of a plurality of devices, or may be applied to a single device. Furthermore, the present invention is also applicable when an information processing program that implements the functions of the embodiments is directly or remotely supplied to a system or apparatus. Therefore, in order to implement the functions of the present invention on a computer, a program installed in a computer, a medium storing the program, and a WWW (World Wide Web) server from which the program is downloaded are also included in the scope of the present invention. . In particular, non-transitory computer readable media storing programs that cause a computer to perform at least the processing steps included in the above-described embodiments are included within the scope of the present invention.

Claims (9)

an image acquisition unit that acquires a first inner wall surface image of the inner wall surface of the first tubular structure;
A first ceiling plane image obtained by cutting the first inner wall surface image from a ceiling portion of the first tubular structure in the first inner wall surface image and developing it, and the first tubular structure in the first inner wall surface image a first bottom plane image obtained by cutting through a bottom portion of the object and a first side plane image obtained by cutting through a side portion of the first tubular structure in the first inner wall surface image; a planar image generator;
A first plane image corresponding to the attribute of the first tubular structure is selected from the first ceiling plane image, the first bottom plane image, and the first side plane image generated by the first plane image generation unit. a first selection unit for
a damage determination unit that determines a first damage occurring in the first tubular structure using the first planar image selected by the first selection unit;
A model for generating a learned damage identification model by assigning an identifier capable of identifying the determined first damage and having artificial intelligence learn a plurality of types of the first planar images together with the determined first damage. a generator;
an image reception unit that receives a second inner wall surface image of the inner wall surface of the second tubular structure;
A second ceiling plane image obtained by cutting the second inner wall image from a ceiling portion of the second tubular structure in the second inner wall image and developing the second inner wall image, and the second tubular structure in the second inner wall image A second plane generating a second bottom plane image obtained by cutting and unfolding from the bottom portion of the article and a second side plane image obtained by cutting and unfolding from the side portion of said second tubular structure. an image generator;
A second plane image corresponding to the attribute of the second tubular structure is selected from the second ceiling plane image, the second bottom plane image, and the second side plane image generated by the second plane image generation unit. a second selection unit for
a damage identification unit that identifies a second damage occurring in the second tubular structure using the second planar image selected by the second selection unit and the learned damage identification model;
structure damage identification device. 2. The attributes of the first tubular structure and the attributes of the second tubular structure include at least one of the type, material and location of construction of the first tubular structure and the second tubular structure. The structure damage identification device according to . 3. The structure damage according to claim 1 or 2, wherein the first tubular structure and the second tubular structure are at least one of a railway tunnel, a road tunnel, an undersea tunnel, a common utility ditch, and a pipe culvert. Specific device. The structural damage identification device according to any one of claims 1 to 3, wherein the first damage includes compression, cracks, steps, floats, delamination, flaking, rebar exposure, water leakage, and free lime. The structure damage identification device according to any one of claims 1 to 4, wherein the damage determination unit determines the first damage occurring in the first tubular structure using Faster R-CNN. The structure damage identification device according to any one of claims 1 to 5, wherein the model generation unit generates the learned damage identification model using horizontal reversal as the padded data. The structural damage identification device according to any one of claims 1 to 6, wherein the model generation unit generates the learned damage identification model using transfer learning. an image acquisition step of acquiring a first inner wall surface image of the inner wall surface of the first tubular structure;
A first ceiling plane image obtained by cutting the first inner wall surface image from a ceiling portion of the first tubular structure in the first inner wall surface image and developing it, and the first tubular structure in the first inner wall surface image a first bottom plane image obtained by cutting through a bottom portion of the object and a first side plane image obtained by cutting through a side portion of the first tubular structure in the first inner wall surface image; a planar image generating step;
A first plane image corresponding to the attribute of the first tubular structure is selected from the first ceiling plane image, the first bottom plane image, and the first side plane image generated in the first plane image generation step. a first selection step of
a damage determination unit that determines a first damage occurring in the first tubular structure using the first planar image selected in the first selection step;
A model for generating a learned damage identification model by assigning an identifier capable of identifying the determined first damage and having artificial intelligence learn a plurality of types of the first planar images together with the determined first damage. a generation step;
an image receiving step of receiving a second inner wall surface image of the inner wall surface of the second tubular structure;
A second ceiling plane image obtained by cutting the second inner wall image from a ceiling portion of the second tubular structure in the second inner wall image and developing the second inner wall image, and the second tubular structure in the second inner wall image A second plane generating a second bottom plane image obtained by cutting and unfolding from the bottom portion of the article and a second side plane image obtained by cutting and unfolding from the side portion of said second tubular structure. an image generating step;
A second planar image corresponding to the attribute of the second tubular structure is selected from the second ceiling planar image, the second bottom planar image, and the second side planar image generated in the second planar image generating step. a second selection step of
a damage identification step of identifying a second damage occurring in the second tubular structure using the second planar image selected in the second selection step and the learned damage identification model;
a structural damage identification method comprising: an image acquisition step of acquiring a first inner wall surface image of the inner wall surface of the first tubular structure;
A first ceiling plane image obtained by cutting the first inner wall surface image from a ceiling portion of the first tubular structure in the first inner wall surface image and developing it, and the first tubular structure in the first inner wall surface image a first bottom plane image obtained by cutting through a bottom portion of the object and a first side plane image obtained by cutting through a side portion of the first tubular structure in the first inner wall surface image; a planar image generating step;
A first plane image corresponding to the attribute of the first tubular structure is selected from the first ceiling plane image, the first bottom plane image, and the first side plane image generated in the first plane image generation step. a first selection step of
a damage determination unit that determines a first damage occurring in the first tubular structure using the first planar image selected in the first selection step;
A model for generating a learned damage identification model by assigning an identifier capable of identifying the determined first damage and having artificial intelligence learn a plurality of types of the first planar images together with the determined first damage. a generation step;
an image receiving step of receiving a second inner wall surface image of the inner wall surface of the second tubular structure;
A second ceiling plane image obtained by cutting the second inner wall image from a ceiling portion of the second tubular structure in the second inner wall image and developing the second inner wall image, and the second tubular structure in the second inner wall image A second plane generating a second bottom plane image obtained by cutting and unfolding from the bottom portion of the article and a second side plane image obtained by cutting and unfolding from the side portion of said second tubular structure. an image generating step;
A second planar image corresponding to the attribute of the second tubular structure is selected from the second ceiling planar image, the second bottom planar image, and the second side planar image generated in the second planar image generating step. a second selection step of
a damage identification step of identifying a second damage occurring in the second tubular structure using the second planar image selected in the second selection step and the learned damage identification model;
A structural damage identification program that causes a computer to execute