JP6980208B2 - Structure maintenance work support system - Google Patents

Description

The present invention relates to a structure maintenance work support system used for structure maintenance work.

Infrastructure structures, such as concrete structures and tunnel structures, need to be properly maintained and managed. In particular, in maintenance work, we appropriately detect cracks, concrete peeling and floating, water leaks, etc., where the performance of the structure has deteriorated from the sound state it should be (this is called "deformation"). It is required to record and repair.

There are various methods to detect the deformation of the structure. For example, the method of visual inspection to visually inspect the structure and find the deformation, the method of tapping the structure with a predetermined instrument, and the presence or absence of the deformation by the sound. Typical methods include a tapping sound inspection method for inspecting, and an inspection method in which a structure is photographed, the photographed image data is visually inspected, and a deformed part is extracted.

For visual inspection and tapping sound inspection, it is necessary to go to the place where the structure is located and perform the work, and the work also takes time. Therefore, it is often the case that a method of photographing a structure and extracting a deformed part from the photographed image data is adopted.

In the case of the inspection method using image data, the person in charge visually observes the image data of the photographed structure to visually recognize the deformed part. Then, information indicating that there is a deformation (hereinafter referred to as "deformation information") is added to the CAD data (deformation trace data) for managing the deformation of the deformed part visually recognized by the trace work (this is referred to as "deformation information"). Is called tracing work). The deformation trace data is used for formulating a repair plan for the structure, and the actual repair work is performed based on the repair plan.

Non-Patent Document 1 shows an example of software that supports tracing work when a structure is a tunnel.

Sankyo Engineering Co., Ltd., "Crack Draw21", [online], Internet <URL: http://www.sankyo-eng.com/crackdraw.html>

When tracing the deformed part based on the image data of the photographed structure, the software of Non-Patent Document 1 or the like is used. Specifically, the image data obtained by photographing the structure is read into the software, the person in charge visually recognizes the image data displayed on the screen from the image data, and the deformed part such as a crack is identified by a mouse or the like. Register as deformation information in the deformation trace data.

In the tracing work, the person in charge visually detects the deformed part from the image data, but the structure may have stains as well as the deformed part, so it is necessary to properly judge the deformed part. Skilled personnel are required. And even a skilled person in charge has a heavy burden of tracing work. Therefore, it is required to improve the work efficiency and accuracy of tracing work. In addition to the burden of tracing work, it is expected that deformations to be extracted and work mistakes will occur in human work, and homogenization and completeness improvement by automating tracing work are required. ..

Therefore, in view of the above problems, the present inventors have invented the structure maintenance work support system of the present invention.

The first invention is a structure maintenance work support system used for structure maintenance work, and the structure maintenance work support system corresponds to the first photographed image data obtained by photographing a structure. The training data generation processing unit that generates training deformation data using the first deformation trace data to be used, and the second training image data that captures the structure are received and the generated learning deformation data is received. Based on the deformation estimation processing unit that estimates the deformation in the second captured image data based on the state data, and the deformation estimation data that estimates the deformation of the structure, the second deformation trace data is generated. It is a structure maintenance work support system that has a deformation trace data generation processing unit to generate.

By generating learning deformation data using the first captured image data and the first deformation trace data in which deformations are registered for the first captured image data, and performing deformation estimation processing based on the data, the deformation estimation processing is performed. It is possible to improve the accuracy of the deformation estimation processing based on the captured image data. In addition, since the deformation estimation data output as a result of the deformation estimation process is not clear about the presence or absence of deformation and the boundary, and the scale of the deformation area and extension cannot be estimated, the repair budget for the structure is not clear. It cannot be used for repair planning work including estimation of. Therefore, by using the structure maintenance work support system of the present invention, it is possible to generate deformation trace data used for the repair plan of the structure, so that it is possible not only to estimate the deformation location but also to formulate a smooth repair plan. It can be connected to such things.

In the above-described invention, the input of the first captured image data and the first deformation trace data is accepted, and the positions of the first captured image data and the first deformation trace data are aligned. , The first captured image data corresponding to the position of the deformation in the first deformation trace data is extracted as a deformation object, and the extracted deformation object is used to generate the deformation data for learning. It can be configured like a maintenance work support system.

In the above-mentioned invention, the learning data generation processing unit further maps the extracted deformed object on a grid having a predetermined resolution, and calculates the density of each pixel in the area of the grid. It can be configured like a structure maintenance work support system that generates deformation data for learning.

In order to generate deformation data for learning, it is advisable to carry out the processing of these inventions.

In the above-mentioned invention, the learning data generation processing unit further separates a part or all of the first photographed image data based on the environmental condition or the type of deformation of the structure, and the separation is performed. Regarding the captured image data, characteristic information including the environmental conditions of the structure or the type of deformation is added, and the processing in the learning data generation processing unit is based on the first captured image data to which the characteristic information is added. It can be configured like a structure maintenance work support system that executes.

It is known that the appearance pattern of deformation differs depending on the environmental conditions of the structure. Therefore, as in the present invention, the captured image data is separated according to the environmental conditions of the structure and the type of deformation, and the learning data is generated based on the captured image data added as the characteristic condition. The accuracy of the state estimation process can be improved.

In the above-described invention, the learning data generation processing unit further separates the generated learning deformation data according to a grid of a predetermined size, and the deformation object separates the separated grid. It can also be configured like a structure maintenance work support system that classifies according to the amount or position included.

As in the present invention, by separating the deformation data for learning for each grid and classifying the grid according to the deformation objects contained therein, the accuracy of the deformation estimation processing can be improved.

In the above-described invention, the deformation trace data generation processing unit generates binarized deformation estimation data based on the deformation estimation data or the deformation estimation data executed by the image filter processing, and the binarization is performed. In the deformed estimation data, a predetermined grid is set, grids adjacent to each other with predetermined values are grouped, and an object set based on the grouped grid is added to the layer added to the CAD data. Therefore, it can be configured like a structure maintenance work support system that generates the second deformation trace data.

In the above-described invention, the variant trace data generation processing unit further sets an object having a polygonal outer circumference of the grouped grid, and adds the set object to the layer added to the CAD data. As a result, it can be configured like a structure maintenance work support system that generates the second deformation trace data.

In the above-mentioned invention, the deformation trace data generation processing unit further calculates the area of the grouped grid, and if this area does not satisfy a predetermined condition, the group is treated as noise from the processing target. By excluding and setting an object whose outer circumference of the grid of the group not excluded from the processing target is a polygon, and adding the set object to the layer added to the CAD data, the second change It can be configured like a structure maintenance work support system that generates state trace data.

In order to generate deformation trace data, it is advisable to carry out the processing of these inventions.

The ninth invention is a structure maintenance work support system used for structure maintenance work, and the structure maintenance work support system is a photographed image data obtained by photographing a structure and a corresponding deformation. It has a learning data generation processing unit that generates training deformation data using trace data, and the learning data generation processing unit inputs the captured image data and the deformation trace data. Is received, the positions of the captured image data and the deformed trace data are aligned, the captured image data corresponding to the deformed position in the deformed trace data is extracted as a deformed object, and the extracted deformed object is extracted. It is a structure maintenance work support system that uses it to generate deformation data for learning.

By generating learning deformation data using the first captured image data and the first deformation trace data in which deformations are registered for the first captured image data, and performing deformation estimation processing based on the data, the deformation estimation processing is performed. It is possible to improve the accuracy of the deformation estimation processing based on the captured image data.

The tenth invention is a structure maintenance work support system used for structure maintenance work, and the structure maintenance work support system is a photographed image data obtained by photographing a structure and a corresponding deformation. It has a learning data generation processing unit that generates training deformation data using trace data, and the learning data generation processing unit uses the captured image data as part or all of the structure. The captured image is separated based on the environmental condition or the type of deformation of the above, and the characteristic information including the environmental condition or the type of the deformation of the structure is added to the separated captured image data, and the captured image to which the characteristic information is added is added. It is a structure maintenance work support system that generates deformation data for learning using data and the corresponding deformation trace data.

It is known that the appearance pattern of deformation differs depending on the environmental conditions of the structure. Therefore, as in the present invention, the captured image data is separated according to the environmental conditions of the structure and the type of deformation, and the learning data is generated based on the captured image data added as the characteristic condition. The accuracy of the state estimation process can be improved.

The eleventh invention is a structure maintenance work support system used for structure maintenance work, and the structure maintenance work support system is a photographed image data obtained by photographing a structure and a corresponding deformation. It has a learning data generation processing unit that generates training deformation data using trace data, and the learning data generation processing unit has a predetermined size of the generated learning deformation data. It is a structure maintenance work support system that separates according to the grid and classifies the separated grid according to the amount or position of the deformed object.

As in the present invention, by separating the deformation data for learning for each grid and classifying the grid according to the deformation objects contained therein, the accuracy of the deformation estimation processing can be improved.

The twelfth invention is a structure maintenance work support system used for structure maintenance work, and the structure maintenance work support system is based on deformation estimation data for estimating deformation in a structure. It has a deformation trace data generation processing unit that generates deformation trace data, and the deformation trace data generation processing unit is based on the deformation estimation data or the deformation estimation data that has been subjected to image filtering processing. A layer that generates binarized deformation estimation data, sets a predetermined grid in the binarized deformation estimation data, groups grids adjacent to each other with predetermined values, and adds them to the CAD data. This is a structure maintenance work support system that generates deformation trace data by adding objects set based on the grouped grid.

The deformation estimation data output as a result of the deformation estimation processing cannot be used as it is for a structure repair plan or the like. Therefore, by using the structure maintenance work support system of the present invention, it is possible to generate deformation trace data used for the repair plan of the structure. Can be connected to.

The structure maintenance work support system of the first invention can be realized by loading and executing the information processing program of the present invention in a computer. That is, for learning, the computer used for the maintenance work of the structure is used to generate the deformation data for learning by using the first photographed image data obtained by photographing the structure and the corresponding first deformation trace data. The data generation processing unit accepts the input of the second photographed image data of the structure, and estimates the deformation in the second photographed image data based on the generated deformation data for learning. This is a structure maintenance work support program that functions as a deformation trace data generation processing unit that generates a second deformation trace data based on the deformation estimation data that estimates the deformation of the structure.

The structure maintenance work support system of the ninth invention can be realized by loading and executing the information processing program of the present invention in a computer. That is, the computer used for the maintenance work of the structure is used as a learning data generation processing unit that generates training deformation data using the photographed image data obtained by photographing the structure and the corresponding deformation trace data. In the information processing program to function, the learning data generation processing unit accepts the input of the captured image data and the deformed trace data, and determines the position of the captured image data and the deformed trace data. In addition, a structure maintenance work support program that extracts captured image data corresponding to the position of the deformation in the deformation trace data as a deformation object and generates deformation data for learning using the extracted deformation object. Is.

The structure maintenance work support system of the tenth invention can be realized by loading and executing the information processing program of the present invention in a computer. That is, the computer used for the maintenance work of the structure is used as a learning data generation processing unit that generates training deformation data using the photographed image data obtained by photographing the structure and the corresponding deformation trace data. In the information processing program to function, the learning data generation processing unit separates a part or all of the captured image data based on the environmental condition or the type of deformation of the structure, and the separated imaging is performed. For image data, characteristic information including the environmental conditions of the structure or the type of deformation is added, and the photographed image data to which the characteristic information is added and the corresponding deformation trace data are used for learning deformation. It is a structure maintenance work support program that generates data.

The structure maintenance work support system of the eleventh invention can be realized by loading and executing the information processing program of the present invention in a computer. That is, the computer used for the maintenance work of the structure is used as a learning data generation processing unit that generates training deformation data using the photographed image data obtained by photographing the structure and the corresponding deformation trace data. An information processing program to function, the learning data generation processing unit separates the generated learning deformation data according to a grid of a predetermined size, and the separated grid is separated into a deformation object. This is a structure maintenance work support program that classifies data according to the amount or location of the data.

The structure maintenance work support system of the twelfth invention can be realized by loading and executing the information processing program of the present invention in a computer. That is, information processing that causes the computer used for the maintenance work of the structure to function as a deformation trace data generation processing unit that generates deformation trace data based on the deformation estimation data that estimates the deformation in the structure. In the program, the deformation trace data generation processing unit generates the deformation estimation data binarized based on the deformation estimation data or the deformation estimation data executed by the image filter processing, and the binarization is performed. A predetermined grid is set in the deformed estimation data, grids adjacent to each other with predetermined values are grouped, and an object set based on the grouped grid is added to the layer added to the CAD data. It is a structure maintenance work support program that generates deformation trace data.

By using the structure maintenance work support system of the present invention, it is possible to generate learning data (learning deformation data) for performing deformation estimation processing for estimating a deformation portion from captured image data. Then, by learning the deformation estimation process based on the deformation data for learning, the accuracy of the deformation estimation process based on the captured image data can be improved.

In addition, since the deformation estimation data output as a result of the deformation estimation process is not clear about the presence or absence of deformation and the boundary, and the scale of the deformation area and extension cannot be estimated, it is a structure repair plan as it is. It cannot be used for such purposes. Therefore, by using the structure maintenance work support system of the present invention, it is possible to generate deformation trace data used for repair planning work including estimation of the repair budget of the structure, so that only estimation of the deformation part can be performed. However, it can be linked to the formulation of a smooth repair plan.

It is a figure which shows an example of the whole structure of the structure maintenance work support system of this invention schematically. It is a figure which shows an example of the hardware composition of the computer which executes the structure maintenance work support system of this invention schematically. It is a flowchart which shows an example of the processing process in the learning data generation processing part. It is a flowchart which shows an example of the processing process in the transformation trace data generation processing part. It is a figure which shows an example of the photographed image data which is input to the learning data generation processing unit. It is a figure which shows an example of the transformation trace data which is input to the learning data generation processing part. It is a figure which shows an example of the learning transformation data output by a learning data generation processing unit. It is a figure which shows an example of the deformation estimation data output by a deformation estimation processing unit. It is a figure which shows an example when the photographed image data and the deformation trace data are aligned. It is a figure which shows an example of the case where it is divided for each grid. It is a figure which shows an example which shows that the processing is performed for each grid. It is a figure which shows an example of the photographed image data which accepts an input in a deformation estimation processing unit. This is an example of the deformation estimation data as a result of performing the deformation estimation processing in the deformation estimation processing unit on the captured image data of FIG. 12. It is a figure which shows an example which created the deformation trace data from the deformation estimation data output by a deformation estimation processing unit. It is a figure which shows typically an example of the processing which adds characteristic information to the photographed image data input to a learning data generation processing unit. It is a figure which shows typically an example of the process which divides the deformation data for learning into each grid and classifies according to the deformation. It is a figure which shows the other example of the process of dividing the deformation data for learning into each grid and classifying according to the deformation.

FIG. 1 shows an example of the overall system configuration of the structure maintenance work support system 1 used for the structure maintenance work of the present invention, and an example of the computer hardware configuration used in the structure maintenance work support system 1 of the present invention. It is shown in FIG.

In the structure maintenance work support system 1 of the present invention, in performing the deformation estimation processing for estimating the deformation based on the captured image data, an image recognition engine (deformation estimation processing unit 21) that performs the estimation processing is used. Generate learning data (learning deformation data) for learning. Further, based on the deformation estimation data generated by the deformation estimation processing in the deformation estimation processing unit 21, the deformation trace data (described later) which is the data including the deformation information is generated.

The structure maintenance work support system 1 is realized by a computer such as a server or a terminal. The computer includes an arithmetic unit 70 such as a CPU that executes arithmetic processing of a program, a storage device 71 such as a RAM or a hard disk for storing information, a display device 72 such as a display, and an input device 73 for inputting information. It has a communication device 74 for communicating the processing result of the arithmetic unit 70 and the information stored in the storage device 71. When the computer is provided with a touch panel display, the input device 73 and the display device 72 may be integrally configured. Touch panel displays are often used in portable communication terminals such as tablet computers and smartphones, but are not limited to these.

The touch panel display is a device in which the functions of the display device 72 and the input device 73 are integrated in that input can be performed directly on the display with a predetermined input device (such as a pen for a touch panel) or a finger.

The control terminal 2 of the structure maintenance work support system 1 may be realized by one computer, but its function may be realized by a plurality of computers. The computer in this case may be, for example, a cloud server.

Further, each means in the structure maintenance work support system 1 of the present invention has only logically distinguished functions, and may be physically or substantially the same area.

The structure maintenance work support system 1 includes a control terminal 2. The structure maintenance work support system 1 has a learning data generation processing unit 20, a deformation estimation processing unit 21, and a deformation trace data generation processing unit 22.

The learning data generation processing unit 20 learns the deformation estimation processing in the deformation estimation processing unit 21, which will be described later, by using the photographed image data obtained by photographing the structure and the deformation trace data corresponding to the photographed image data. Generate training deformation data, which is training data for the purpose of learning. The deformation trace data input to the learning data generation processing unit 20 is preferably deformation trace data generated based on the deformation specified by the person in charge who visually recognizes the captured image data, but is limited thereto. Instead, for example, it may be automatically generated deformation trace data. The deformation trace data is data that includes the correct answer as a deformation when the deformation estimation process is performed on the captured image data. FIG. 5 shows an example of captured image data input to the learning data generation processing unit 20, and FIG. 6 shows an example of deformation trace data.

As for the deformation trace data, the deformation visually recognized by the person in charge based on the photographed image data obtained by photographing the structure is divided into layers for each type of deformation and stored in the corresponding part in the developed view or the like. CAD data and so on. Therefore, the CAD data includes information indicating the type of deformation for each layer. When the structure is a tunnel, a development view of the upper floor, inner wall, side wall, etc. of the tunnel is created by CAD data or the like. Then, the deformation identified by visually recognizing the photographed image data is written as deformation information in the corresponding portion of the developed view and managed as deformation trace data.

The learning data generation processing unit 20 receives inputs of captured image data and corresponding deformation trace data, and aligns the respective data. Then, the captured image data and the deformation information of the portion specified as the deformation in the deformation trace data are extracted as the deformation object. Then, each of the extracted deformed objects is mapped on the grid generated at the specified resolution. Furthermore, by performing a predetermined density calculation for each pixel of the grid, deformation data for learning is generated. The deformation data for learning is image data to be used for learning the deformation estimation process for estimating whether or not the corresponding position is a deformation point in the captured image data, and is, for example, a heat map. The estimation result of whether or not it is deformed is expressed by the concentration. FIG. 7 shows an example of deformation data for learning, which is a binary heat map.

The deformation estimation processing unit 21 executes deformation estimation processing for determining whether or not there is a deformation in the structure based on the captured image data. The estimation process at this time can be performed using deep learning, but is not limited to that. Specifically, in the structure, the input of the photographed image data at the place where the presence or absence of deformation is estimated is accepted, and based on the photographed image data and the learning deformation data generated by the learning data generation processing unit 20. ， Generates deformation estimation data for captured image data that has received input. FIG. 8 shows an example of deformation estimation data in which the presence or absence of deformation is estimated based on the captured image data. Deformation estimation data, like a heat map, expresses the estimation result of whether or not it is deformed depending on the concentration.

The deformation trace data generation processing unit 22 generates deformation trace data based on the deformation estimation data generated by the deformation estimation processing unit 21. Specifically, an image filter such as a morphology operation is applied to the deformation estimation data. A grid of a predetermined size is set for the deformation trace data after applying the image filter. Then, the binarization process is performed for each grid, and the binarized deformation estimation data is calculated. Then, for the binarized deformation estimation data, a grid in which the value of "1" with the deformation object is adjacent is grouped. The area of the group is calculated by calculating the number of grids classified into the same group for each group, and the group whose area is less than the predetermined area is excluded from the processing target as noise. By polygonizing the outer circumference of the remaining group and creating it as an object, and adding this object as a new layer, deformation trace data is generated.

Next, an example of the processing process of the structure maintenance work support system 1 of the present invention will be described with reference to the flowcharts of FIGS. 3 and 4.

First, an example of a processing process for generating learning deformation data in the learning data generation processing unit 20 will be described. The learning deformation data generated by the learning data generation processing unit 20 generates data for learning for the deformation estimation processing in the deformation estimation processing unit 21, preferably the deformation estimation processing using deep learning. It is a process to do.

First, the operator causes the control terminal 2 to read the photographed image data obtained by photographing the structure and the deformation trace data corresponding to the photographed image data. Then, the learning data generation processing unit 20 accepts the input of the captured image data and the deformation trace data (S100).

The learning data generation processing unit 20 aligns the captured image data received in S100 with the deformed trace data (S110). That is, the positions are aligned so that the positions of the captured image data and the developed view in the deformed trace data are the same or almost the same. An example of this is shown in FIG. FIG. 9A is captured image data for which input is received by the learning data generation processing unit 20, and FIG. 9B is deformation trace data corresponding to FIG. 9A. Then, FIG. 9 (c) is the image data as a result of the alignment of FIGS. 9 (a) and 9 (b). By aligning, the position of the deformation in the captured image data can be specified, so the captured image data corresponding to the deformation position in the deformation trace data and the deformation information are used as a set of deformation objects. Extract (S120).

In the image data of the same or almost the same vertical and horizontal size as the captured image data or the deformation trace data, a grid generated at a predetermined resolution, for example, 100 × 100 pixels, 200 × 200 pixels, 500 × 500 pixels. create. Then, each of the extracted deformed objects is mapped (aligned) on the grid (S130). The resolution of the grid is lower than that of the captured image data. For example, as shown in FIG. 10, a plurality of grids of 100 × 100 pixels are created in the image data having the same size as the captured image data, and each deformed object is mapped to the corresponding position. Of the grid, the positions that have vertical and horizontal fractions may be trimmed and deleted, or the grid may have only fractions.

By mapping the deformed object on the grid in this way, it is possible to map the captured image data corresponding to the position specified as the deformed in the deformed trace data. Then, the learning data generation processing unit 20 performs density calculation for each pixel of the grid, and generates heat-mapped learning deformation data (S140).

As a method of calculating the concentration, a plurality of methods can be used. Further, the method may be different depending on whether the deformed object is a line segment or a closed curve.

As the first method, the density calculation of the deformed object represented by the line segment is performed by applying an expansion process to the line segment to give it a thickness, and drawing it at the level of the original resolution of the captured image data. I do. Then, in the resolution of the deformation data for learning, the ratio occupied by the line segment region in the pixel is set as the shading value of the pixel.

As a second method, the density calculation of the deformed object represented by the closed curve fills the inside of the closed curve at the original resolution of the captured image data, and the ratio of the line segment area in the pixel at the resolution of the heat map. Set as the shade value of the pixel.

As a third method, the density calculation that can be used for both the deformed object represented by a line segment and the deformed object represented by a closed curve applies the target deformed object to the pixels on the grid (the target deformed object is applied to the pixels on the grid. It is binarized depending on whether or not it intersects. In such a case, it is set to "1", and if it is not applied, it is set to "0".

When performing the density calculation, for example, in a grid of 100 × 100 pixels, the density calculation process is executed while shifting the pixels up and down and left and right by a predetermined unit, for example, one pixel at a time. FIG. 11 schematically shows this.

FIG. 7 is an example of heat-mapped learning deformation data generated by the learning data generation processing unit 20 as described above. FIG. 7 shows a case where the third method is used for the concentration calculation, and when the first method and the second method are used for the concentration calculation, different learning deformation data can be generated.

Further, since the CAD data includes information indicating the type of deformation of the deformed object, it is possible to specify which type of deformed object it is as the learning deformation data.

By generating the learning deformation data as described above, it is possible to generate the learning data for the deformation estimation processing in the deformation estimation processing unit 21. The learning data generation processing unit 20 generates a large number of learning deformation data by inputting a large number of captured image data and deformation trace data. As a result, the accuracy of the deformation estimation processing using the deep learning in the deformation estimation processing unit 21 can be improved.

It is preferable, but not limited to, the photographed image data obtained by photographing the structure (the photographed image data is different from the photographed image data input for generating the training deformation data by the learning data generation processing unit 20). In the case of estimating the deformation in the structure in the same photographed image data), the operator causes the control terminal 2 to read the photographed image data obtained by photographing the structure. Then, the deformation estimation processing unit 21 accepts the input of the photographed image data obtained by photographing the structure. For example, it is assumed that FIG. 12 is captured image data for which input is accepted by the deformation estimation processing unit 21.

The deformation estimation processing unit 21 is a region of the grid of the captured image data that has received the input in a grid of a predetermined size (preferably a grid of a size used in the learning data generation processing unit 20). The deformation estimation process for estimating whether or not there is a deformation is executed. In the deformation estimation processing, whether or not there is a deformation in the grid area based on the similarity with the learning deformation data by deep learning using the learning deformation data generated by the learning data generation processing unit 20. Estimate.

That is, similarly to FIG. 11, the deformation estimation processing unit 21 uses a grid of a predetermined size, for example, a grid of 100 × 100 pixels, as a predetermined unit in the captured image data, for example, one pixel at a time. While moving up, down, left and right, the process of determining the degree of similarity with the learning deformation data and estimating whether or not there is a deformation is executed.

The deformation estimation processing in the deformation estimation processing unit 21 is not limited to the above-mentioned method, but whether or not it is a deformation location while moving a grid of a predetermined size as described above one pixel at a time up and down and left and right. By executing the estimation process, the deformation estimation data as shown in FIG. 13 can be obtained. The deformation estimation data is, for example, a heat map, and its density indicates the possibility that the pixel is deformed or not.

Since the deformation estimation data generated by the deformation estimation processing unit 21 is a heat map, it cannot be used as it is for the maintenance work of the structure. That is, in order to manage the structure in the same manner as in the conventional case, it is desired to convert it into CAD data suitable for it, but since the heat map is image data, it cannot be converted into CAD data as it is. Therefore, the operator performs a predetermined operation to execute a process of converting the deformation estimation data generated by the deformation estimation processing unit 21 into deformation trace data such as CAD data.

The processing process when the deformation trace data generation processing unit 22 generates the deformation trace data based on the deformation estimation data will be described with reference to the flowchart of FIG. Further, this series of processes is schematically shown in FIG.

First, the operator causes the control terminal 2 to read the deformation estimation data to be processed, for example, the deformation estimation data of FIG. 13 (FIG. 14A). Then, the deformation trace data generation processing unit 22 accepts the input of the deformation estimation data (S200).

Then, the deformation trace data generation processing unit 22 executes image filter processing on the deformation estimation data that has received the input (S210). The image filter processing includes, for example, image filter processing by morphology calculation, but is not limited thereto. In addition, the image filtering process does not have to be executed.

Then, a grid of a predetermined size, for example, 100 × 100 pixels is set for the deformation estimation data or the deformation estimation data for which the image filter processing is executed, and the processing of S230 and S240 described later for each grid is set. Is executed (S220).

That is, the deformation trace data generation processing unit 22 calculates statistics such as an average value and a maximum value for the pixels in the grid (S230). Then, "1" is registered when the calculated statistic is included in the predetermined threshold range, and "0" is registered when the calculated statistic is not included (S240). A lower limit value is required as a threshold value, but an upper limit value may or may not be set.

In this way, the deformation trace data generation processing unit 22 predeterminedly defines a grid of a predetermined size in the deformation estimation data or the deformation estimation data for which the image filter processing is executed, as in FIG. The processing of S230 and S240 is executed while moving the unit, for example, one pixel at a time, up, down, left, and right. Then, when the processing of S230 and S240 is completed for all the grids, binarized deformation estimation data (heat map data) can be generated.

The deformation trace data generation processing unit 22 groups the upper, lower, left, and right grids in which the values of "1" are adjacent to the binarized deformation estimation data as described above (S250). Then, the number of grids separated into the same group is calculated for each group, and the area of that group is calculated. If the calculated area of the group is less than a predetermined area, the group is regarded as noise and removed from the processing target (S260). On the other hand, if the calculated area of the group is equal to or larger than the predetermined area, that group is set as the processing target. FIG. 14 (b) is an example of a group set as a processing target.

By the above processing, the noise group is removed, so the remaining group becomes a deformed object. Therefore, the deformation trace data generation processing unit 22 sets the remaining group as an object whose outer circumference is a polygon (S270). Then, a new layer to which the layer name is given is added to the CAD data, and the object set in S270 is added to the layer as a deformed object (S280). As a result, the deformation object is added to the deformation trace data which is CAD data. FIG. 14 (c) shows an example of an object added as a deformed object.

By executing the above processing, it is possible to generate deformation trace data including the deformation object as a layer from the deformation estimation data.

Next, as a different embodiment of the structure maintenance work support system 1 of the present invention, the accuracy may be improved by further processing when generating the deformation data for learning.

For example, as the first process, when the structure is a tunnel, the captured image data includes three surfaces (surfaces other than the surface on which the railroad track is laid) constituting the tunnel.

For example, as shown in FIG. 15, one image data includes three surfaces, a middle wall, an upper floor, and a side wall. On the other hand, it is known that the appearance pattern of deformation differs between the tunnel wall surface (side surface) and the tunnel upper floor (ceiling). For example, deformations such as water leakage and runoff lime tend to have a bell-shaped shape on the tunnel wall surface, whereas crack deformations tend to occur on the tunnel upper floor in the longitudinal direction or the transverse direction depending on the structure and surrounding environment. In this way, the deformations that appear there tend to differ depending on the physical characteristics of the structure. Therefore, if it can be distinguished from the learning deformation data for the wall surface and the learning deformation data for the upper floor, it will be further used for learning. It is hoped that the accuracy of deep learning will be improved by the deformation data.

That is, part or all of the captured image data is separated and rotated according to the local environmental conditions of the structure (for example, information indicating the position of the middle wall, upper floor, side wall, etc.) and the type of deformation of the detection target. Read it by flipping it upside down. In addition, for the separated captured image data, by adding characteristic information according to the local environmental conditions and the type of deformation of the detection target, such as the captured image data on the upper floor of the tunnel and the captured image data on the side wall of the tunnel, each captured image. It can be modified data for learning according to the characteristics of the data.

In the second process, after the learning data generation processing unit 20 generates the learning deformation data, the grid in the learning deformation data is divided according to the amount and position of the deformation object. Alternatively, the direction of the data for learning may be aligned by rotating the data. FIG. 16 schematically shows this. Further, depending on whether or not the deformation object is included in a specific position such as the central part of the training deformation data, it may be distinguished whether the training data is a negative example group or a positive example group. .. FIG. 17 schematically shows this.

In FIG. 16, the learning deformation data is divided into grids of a predetermined size. The size of the grid at this time is preferably the same size as the grid used in S130. Then, for each grid, there is no deformation (total negative case group), there is a deformation on the lower side of the grid (lower normal case group), there is a deformation on the upper side of the grid (upper normal case group), and the grid. Classify into grids that have deformations on the right or left side of the grid (left and right regular cases group), deformations on the entire surface of the grid (full-scale regular cases group), and grids other than the above (other groups).

Also in FIG. 17, similarly to FIG. 16, the learning deformation data is divided into grids of a predetermined size. Then, the grids are classified into grids with deformation (normal group) in the center of each grid and no deformation (negative case group).

By classifying the training deformation data for each grid or adjusting the direction of the divided data in this way, the accuracy of deep learning using the learning deformation data in the deformation estimation processing unit 21. Can be improved.

By using the structure maintenance work support system 1 of the present invention, it is possible to generate learning data (learning deformation data) for performing deformation estimation processing for estimating a deformation portion from captured image data. .. Then, by learning the deformation estimation process based on the deformation data for learning, the accuracy of the deformation estimation process based on the captured image data can be improved.

In addition, the deformation estimation data output as a result of the deformation estimation processing cannot be used as it is for the repair plan of the structure. Therefore, by using the structure maintenance work support system 1 of the present invention, it is possible to generate deformation trace data used for the repair plan of the structure. It can be connected to such things.

1: Structure maintenance work support system 2: Control terminal 20: Learning data generation processing unit 21: Deformity estimation processing unit 22: Deformation trace data generation processing unit 70: Arithmetic unit 71: Storage device 72: Display device 73 : Input device 74: Communication device

Claims (17)

It is a structure maintenance work support system used for structure maintenance work.
The structure maintenance work support system is
A learning data generation processing unit that generates learning deformation data using the first photographed image data obtained by photographing a structure and the corresponding first deformation trace data.
A deformation estimation processing unit that accepts the input of the second captured image data obtained by photographing the structure and estimates the deformation in the second captured image data based on the generated deformation data for learning.
A deformation trace data generation processing unit that generates a second deformation trace data based on the deformation estimation data that estimates the deformation of the structure, and
A structure maintenance work support system characterized by having. The input of the first captured image data and the first deformation trace data is accepted.
Align the position of the first captured image data with the first deformed trace data,
The first captured image data corresponding to the position of the deformation in the first deformation trace data is extracted as a deformation object.
Generate learning deformation data using the extracted deformation object,
The structure maintenance work support system according to claim 1. The learning data generation processing unit further
Map the extracted deformed object on a grid with a predetermined resolution, and map it.
By performing density calculation for each pixel in the area of the grid, deformation data for training is generated.
The structure maintenance work support system according to claim 2, characterized in that. The learning data generation processing unit further
A part or all of the first captured image data is separated based on the environmental condition or the type of deformation of the structure.
Characteristic information including the environmental conditions of the structure or the type of deformation is added to the separated photographed image data.
The process in the learning data generation processing unit is executed based on the first captured image data to which the characteristic information is added.
The structure maintenance business support system according to any one of claims 1 to 3, characterized in that. The learning data generation processing unit further
The generated deformation data for learning is separated according to a grid of a predetermined size.
The separated grid is classified according to the amount or position of the deformed object.
The structure maintenance business support system according to claim 2 or 3 , characterized in that. The deformation trace data generation processing unit is
Generate binarized deformation estimation data based on the deformation estimation data or the deformation estimation data obtained by performing image filtering.
In the binarized deformation estimation data, a predetermined grid is set, and grids adjacent to each other with predetermined values are grouped.
By adding an object set based on the grouped grid to the layer added to the CAD data, the second deformation trace data is generated.
The structure maintenance business support system according to any one of claims 1 to 5, characterized in that. The deformed trace data generation processing unit further
Set an object whose outer circumference of the grouped grid is a polygon, and set it.
By adding the set object to the layer added to the CAD data, the second deformation trace data is generated.
The structure maintenance work support system according to claim 6. The deformed trace data generation processing unit further
The area of the grouped grid is calculated, and if this area does not satisfy the predetermined conditions, the group is excluded from the processing target as noise.
Set an object whose polygon is the outer circumference of the grid of the group that is not excluded from the processing target.
By adding the set object to the layer added to the CAD data, the second deformation trace data is generated.
The structure maintenance work support system according to claim 6. It is a structure maintenance work support system used for structure maintenance work.
The structure maintenance work support system is
It has a learning data generation processing unit that generates learning deformation data using captured image data obtained by photographing a structure and the corresponding deformation trace data.
The learning data generation processing unit is
Accepting the input of the captured image data and the deformation trace data,
Align the captured image data with the deformed trace data,
The captured image data corresponding to the position of the deformation in the deformation trace data is extracted as a deformation object.
Generate learning deformation data using the extracted deformation object,
A structure maintenance work support system characterized by this. It is a structure maintenance work support system used for structure maintenance work.
The structure maintenance work support system is
It has a learning data generation processing unit that generates learning deformation data using captured image data obtained by photographing a structure and the corresponding deformation trace data.
The learning data generation processing unit is
Part or all of the captured image data is separated based on the environmental conditions of the structure or the type of deformation.
Characteristic information including the environmental conditions of the structure or the type of deformation is added to the separated photographed image data.
The learning deformation data is generated by using the photographed image data to which the characteristic information is added and the corresponding deformation trace data.
A structure maintenance work support system characterized by this. It is a structure maintenance work support system used for structure maintenance work.
The structure maintenance work support system is
It has a learning data generation processing unit that generates learning deformation data using captured image data obtained by photographing a structure and the corresponding deformation trace data.
The learning data generation processing unit is
The generated deformation data for learning is separated according to a grid of a predetermined size.
The separated grids are classified according to the amount or position of the deformed objects.
A structure maintenance work support system characterized by this. It is a structure maintenance work support system used for structure maintenance work.
The structure maintenance work support system is
It has a deformation trace data generation processing unit that generates deformation trace data based on deformation estimation data that estimates deformation in a structure.
The deformation trace data generation processing unit is
Generate binarized deformation estimation data based on the deformation estimation data or the deformation estimation data obtained by performing image filtering.
In the binarized deformation estimation data, a predetermined grid is set, and grids adjacent to each other with predetermined values are grouped.
Deformation trace data is generated by adding an object set based on the grouped grid to the layer added to the CAD data.
A structure maintenance work support system characterized by this. Computers used for structure maintenance work,
A learning data generation processing unit that generates learning deformation data using the first shot image data obtained by shooting a structure and the corresponding first deformation trace data.
Deformation estimation processing unit, which accepts the input of the second captured image data obtained by photographing the structure and estimates the deformation in the second captured image data based on the generated deformation data for learning.
Deformation trace data generation processing unit that generates a second deformation trace data based on the deformation estimation data that estimates the deformation of the structure,
A structure maintenance work support program characterized by functioning as. Computers used for structure maintenance work,
It is an information processing program that functions as a learning data generation processing unit that generates learning deformation data using captured image data obtained by photographing a structure and the corresponding deformation trace data.
The learning data generation processing unit is
Accepting the input of the captured image data and the deformation trace data,
Align the captured image data with the deformed trace data,
The captured image data corresponding to the position of the deformation in the deformation trace data is extracted as a deformation object.
Generate learning deformation data using the extracted deformation object,
A structure maintenance work support program characterized by this. Computers used for structure maintenance work,
It is an information processing program that functions as a learning data generation processing unit that generates learning deformation data using captured image data obtained by photographing a structure and the corresponding deformation trace data.
The learning data generation processing unit is
Part or all of the captured image data is separated based on the environmental conditions of the structure or the type of deformation.
Characteristic information including the environmental conditions of the structure or the type of deformation is added to the separated photographed image data.
The learning deformation data is generated by using the photographed image data to which the characteristic information is added and the corresponding deformation trace data.
A structure maintenance work support program characterized by this. Computers used for structure maintenance work,
It is an information processing program that functions as a learning data generation processing unit that generates learning deformation data using captured image data obtained by photographing a structure and the corresponding deformation trace data.
The learning data generation processing unit is
The generated deformation data for learning is separated according to a grid of a predetermined size.
The separated grids are classified according to the amount or position of the deformed objects.
A structure maintenance work support program characterized by this. Computers used for structure maintenance work,
An information processing program that functions as a deformation trace data generation processing unit that generates deformation trace data based on deformation estimation data that estimates deformation in the structure.
The deformation trace data generation processing unit is
Generate binarized deformation estimation data based on the deformation estimation data or the deformation estimation data obtained by performing image filtering.
In the binarized deformation estimation data, a predetermined grid is set, and grids adjacent to each other with predetermined values are grouped.
Deformation trace data is generated by adding an object set based on the grouped grid to the layer added to the CAD data.
A structure maintenance work support program characterized by this.

Images