JP2017167969A - Damage extraction system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a damage extraction system that provides an extraction result of damage as appropriate data contents in consideration of information processing capability and a communication environment of a terminal at a side provided with the extraction result of the damage of a structure.SOLUTION: A damage extraction system includes: a user terminal 10 operated by an investigator who checks if damage has occurred to a structure; and a damage extraction server 20 that extracts damage of the structure on the basis of information received from the user terminal 10 via a network 100, and provides the user terminal 10 with its extraction result.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a damage extraction system, and more particularly to a damage extraction system that extracts damage such as cracks generated in a concrete asphalt structure.

In recent years, aged deterioration of concrete and asphalt structures (hereinafter simply referred to as structures) constructed during the period of high growth has been regarded as a problem.
When these structures are damaged, such as cracks, the structure collapses and the road surface is excessively uneven, leading to a serious accident. Therefore, the inspection of these structures has become important.
However, on the other hand, the structure covers a wide area such as a road, a bridge, or a tunnel, and the inspection work of the structure is limited only by human labor of the worker.

As a conventional technique for solving the above problems, a concrete structure maintenance management system disclosed in Patent Document 1 has been proposed.
In this concrete structure maintenance and management system, the measuring instrument measures the initial crack of the structure by optical distance measurement, and when the measuring terminal sends the measurement information to the server via the network, the server is based on the measurement information. Analyzes the deterioration prediction of structures, etc., and sends the analysis results and crack information to the installer terminal via the network.

JP2015-138467

However, in the concrete structure maintenance management system of Patent Document 1, when the server transmits the analysis result of the structure to the builder terminal, the information processing ability of the builder terminal and the communication environment are not considered at all. Therefore, there is a problem that trouble may occur in the display processing of the analysis result on the contractor terminal side.
In particular, when the contractor terminal is an information processing terminal other than a PC wired to the network, it may take a long time to display the analysis result or the display itself may not be possible depending on the data size. .

  The present invention has been made in view of the above problems, and considering the information processing ability and communication environment of the receiving terminal on the extraction result of the damage of the structure, the extraction result of the damage is appropriately determined. The object is to provide a damage extraction system that provides data contents.

  In order to achieve such an object, the present invention provides a user terminal operated by a worker who performs an inspection work on a structure, a damage extraction server that is connected to the user terminal via a network and extracts damage caused to the structure, The user terminal transmits a captured image of the structure to the damage extraction server, and the damage extraction server is damage that has occurred in the structure based on the captured image. A damage candidate image indicating an image estimated to be generated is generated, the generated damage candidate image is transmitted to the user terminal, and the damage candidate image transmitted to the user terminal is combined with the photographed image based on the model information of the user terminal. It is characterized by determining whether or not.

  According to the damage extraction system of the present invention, the damage extraction server adds a damage score indicating the possibility of actual damage to the damage candidate image and transmits the damage candidate image to the user terminal. .

  Further, according to the damage extraction system of the present invention, the damage extraction server transmits the damage candidate image to the user terminal based on the model information without synthesizing the damage candidate image, and the user terminal recognizes the damage candidate image as damaged. The threshold value of the damage point used as a reference is stored, and when the damage candidate image is received from the damage extraction server, the damage candidate image to be displayed is determined based on the threshold value of the damage point, and the captured image stored in the own device The damage candidate images determined in the above are synthesized and displayed.

  Further, according to the damage extraction system of the present invention, when the user terminal inputs the threshold value of the damage score, the user terminal determines a damage candidate image to be displayed based on the input threshold value of the damage score, and stores the damage candidate image to be stored in its own device. The damage candidate image determined on the photographed image is synthesized and displayed.

  According to the damage extraction system of the present invention, the damage extraction server is characterized in that the number of damage points is reduced based on the model information.

  According to the damage extraction system of the present invention, the damage extraction server transmits only damage candidate images having a predetermined damage score or more to the user terminal based on the model information.

  Further, according to the damage extraction system of the present invention, the damage extraction server stores a threshold of damage points that is a reference when recognizing damage candidate images as damage, and combines them based on the threshold of damage points. A damage candidate image is determined, the determined damage candidate image is combined with the captured image received from the user terminal, and the combined image is transmitted to the user terminal.

  According to the damage extraction system of the present invention, when the user terminal inputs the threshold value of the damage score, the user terminal transmits the threshold value of the damage score to the damage extraction server, and the damage extraction server receives the threshold value of the damage score. A damage candidate image to be synthesized is determined based on the threshold value of the received damage score, the determined damage candidate image is synthesized with a captured image received from the user terminal, and the synthesized image is transmitted to the user terminal. To do.

  In addition, the present invention includes a user terminal operated by a worker who performs a structure inspection work, and a damage extraction server that is connected to the user terminal via a network and extracts damage caused to the structure. In this damage extraction system, the user terminal transmits a plurality of photographed images of the same structure to the damage extraction server, and the damage extraction server uses the common photographed images of the plurality of structures received from the user terminal. An image is extracted, and a connected image obtained by connecting a plurality of captured images to each other so as to superimpose the extracted common image is generated. Based on the model information of the user terminal, the resolution of the generated connected image is adjusted, and the resolution is set. The adjusted connected image is transmitted to the user terminal.

  It should be noted that any combination of the above-described constituent elements, or those obtained by replacing the constituent elements and expressions of the present invention with each other among methods, apparatuses, systems, computer programs, recording media storing computer programs, and the like are also included in the present invention. It is effective as an embodiment of

  According to the present invention, a user terminal transmits a captured image of a structure to a damage extraction server, and the damage extraction server generates a damage candidate image indicating an image estimated to be damage that has occurred in the structure based on the captured image. Generate and transmit the generated damage candidate image to the user terminal, and determine whether the damage candidate image to be transmitted to the user terminal is combined with the captured image based on the model information of the user terminal. In consideration of the information processing capability and communication environment of the user terminal, it is possible to provide the damage extraction result with appropriate data contents.

It is a figure which shows the structure of the damage extraction system in the 1st Embodiment of this invention. It is a figure which shows the structure of the user terminal 10 in the 1st Embodiment of this invention. It is a figure which shows the structure of the damage extraction server 20 in the 1st Embodiment of this invention. It is a figure which shows each database stored in the information storage part 22 of the damage extraction server 20 in the 1st Embodiment of this invention. It is a figure which shows an example of a data structure of user terminal DB221 in the 1st Embodiment of this invention. It is a figure which shows an example of the data structure of imaging | photography DB222 in the 1st Embodiment of this invention. It is a figure which shows an example of the data structure of damage candidate DB223 in the 1st Embodiment of this invention. It is a figure which shows an example of a data structure of damage score conversion TB226 in the 1st Embodiment of this invention. It is a sequence chart which shows the flow of the registration operation | movement in the 1st Embodiment of this invention. It is a sequence chart which shows the flow of the damage extraction operation | movement by the damage extraction system in the 1st Embodiment of this invention. It is a sequence chart which shows the flow of the damage extraction operation | movement by the damage extraction system in the 1st Embodiment of this invention. It is a sequence chart which shows the flow of the damage extraction operation | movement by the damage extraction system in the 1st Embodiment of this invention. It is a sequence chart which shows the flow of the damage extraction operation | movement by the damage extraction system in the 1st Embodiment of this invention. It is a figure which shows an example of the change of the damage candidate image displayed by the change of the threshold value of the damage score in the 1st Embodiment of this invention. It is a figure which shows an example of the change of the damage candidate image displayed by the change of the threshold value of the damage score in the 1st Embodiment of this invention. It is a figure which shows an example of the change of the damage candidate image displayed by the change of the threshold value of the damage score in the 1st Embodiment of this invention. It is a sequence chart which shows the flow of the production | generation operation | movement of the connection image in the 1st Embodiment of this invention. It is a figure which shows the production | generation method of the connection image in the 1st Embodiment of this invention. It is a figure which shows the production | generation method of the connection image in the 1st Embodiment of this invention.

(Outline of the first embodiment)
The damage extraction system according to the embodiment of the present invention is, for example, a road management company that checks whether or not roads or other concrete structures or asphalt structures (hereinafter simply referred to as “structures”) are damaged. Or the like uses the user terminal 10 to photograph a concrete structure or asphalt structure, and transmits the photographed image to the damage extraction server 20. The damage extraction server 20 uses the photographed image to detect the structure. Damage is extracted and information indicating the damage extraction result is provided to the user terminal 10.
When the damage extraction server 20 provides the user terminal 10 with information on the result of extracting damage to the structure, the damage extraction server 20 adjusts the content of the information to be provided based on the model information of the user terminal 10, thereby providing the entire system. It is possible to efficiently disperse the information processing load in and to provide information on the damage extraction result smoothly.

(Configuration of the first embodiment)
(1) Overall Configuration of Damage Extraction System FIG. 1 is a diagram showing a configuration of a damage extraction system according to the first embodiment of the present invention.
As shown in the figure, the damage extraction system includes a user terminal 10 operated by an investigator who checks whether or not a structure is damaged, and information received from the user terminal 10 via the network 100. A damage extraction server 20 that extracts damage of the structure based on the above and provides the extraction result to the user terminal 10 is configured.

(2) Configuration of User Terminal 10 The user terminal 10 is an information processing apparatus such as a camera with a communication function operated by a worker, a mobile terminal such as a smartphone or a tablet terminal, a PC, or a robot having a photographing / communication function. Then, the structure is photographed, the photographed image is transmitted to the damage extraction server 20, and the damage extraction result of the structure generated based on the photographed image is received from the damage extraction server 20 and displayed. .

FIG. 2 is a diagram illustrating a configuration of the user terminal 10 according to the first embodiment of the present invention.
As shown in the figure, a user terminal 10 is composed of a CPU or the like, and controls the operation of the entire user terminal 10, and an information storage unit 12 that stores input information, information received via a network, and the like. A communication unit 13 that transmits and receives information via a network, a display unit 14 that includes a display and displays information on a screen, an operation unit 15 that includes a key and a mouse and inputs information, and the like. And an imaging unit 16 that performs the imaging.

As described above, the user terminal 10 is various information processing apparatuses such as a PC, a camera, or a mobile terminal.
Hereinafter, each information processing terminal will be divided into cases according to which information processing terminal is configured, and each configuration and operation will be described below.

(2-a) When the user terminal 10 is a PC and a camera The user terminal 10 is configured, for example, by connecting a camera to a desktop or notebook PC connected to a network 100 such as the Internet.
The PC may have a wireless communication function, but may be configured to be connectable to the Internet using a wired communication function. In this case, high-capacity and high-speed data communication can be expected.
When the user terminal 10 is configured by a PC and a camera as described above, a worker takes a picture of a structure using the camera and records it on an information recording medium such as a memory of the camera body or an SD card inserted in the camera. The photographed image is stored.
Thereafter, the worker connects the camera to the PC with a USB cable or the like, transfers the captured image from the camera to the PC, and stores it in a hard disk or the like in the PC. This transfer method is not limited to this method, and for example, wireless communication may be used.
Then, the PC transmits the captured image to the damage extraction server 20 via the network 100.
The damage extraction server 20 extracts a damage candidate image based on the captured image, and transmits the damage candidate image to the PC via the network 100 in association with position information indicating the damage position and the number of damage points.
When the PC receives the damage candidate image from the damage extraction server 20, the PC generates and displays a composite image (extraction result image) by superimposing the received damage candidate image on the captured image stored in the own apparatus.
The operator can limit the damage candidate images displayed on the photographed image to be equal to or higher than the threshold value by operating the PC and inputting the threshold value of the damage score.
As the threshold value of the damage score, for example, any one of 100 levels from 1 to 100 can be selected. Since damage candidate images less than the threshold are not superimposed on the captured image, the lower the threshold set, the more damage candidate images are displayed.

(2-b) When the user terminal 10 is a camera The user terminal 10 is configured by a camera that can be connected to the network 100 using a wireless communication function, for example. In this case, it is assumed that the camera has a communication function but does not have an image composition function.
In this case, the worker photographs the structure using the camera, and records the captured image in an information recording medium such as a memory of the camera body or an SD card inserted in the camera.
Thereafter, the worker connects the camera to the network 100 using the wireless communication function, and transmits the captured image to the damage extraction server 20 via the network 100.
The damage extraction server 20 extracts a damage candidate image based on the captured image, generates an extraction result image obtained by superimposing the damage candidate image on the captured image, and transmits it to the camera via the network 100.
When the camera receives the extraction result image from the damage extraction server 20, the camera stores it in its own machine and displays it on the screen.
In this way, in the case where the user terminal 10 is a camera with low information processing capability or no image composition function, even if only a damage candidate image is received, such as a PC, damage extraction is superimposed on the captured image. Since the resulting image cannot be displayed, the damage extraction server 20 performs image synthesis processing.

(2-c) When the user terminal 10 is a portable terminal The user terminal 10 is comprised by portable terminals, such as a smart phone and a tablet terminal which can be connected to the networks 100, such as the internet, for example. Although this portable terminal has an image composition function, the information processing capability is slightly lower than that of the PC, and the communication environment may be slightly inferior.
In this case, the worker photographs the structure using the photographing function provided in the portable terminal, and the photographed image is recorded on an information recording medium such as a memory of the portable terminal main body or an SD card inserted in the portable terminal. Store.
Thereafter, the mobile terminal transmits the captured image to the damage extraction server 20 via the network 100.
The damage extraction server 20 extracts a damage candidate image based on the captured image, and transmits the damage candidate image to the mobile terminal via the network 100 in association with position information indicating the damage position.
When the portable terminal receives the damage candidate image from the damage extraction server 20, the portable terminal generates and displays an extraction result image by superimposing the received damage candidate image on the captured image stored in the own device.

  As described above, when the user terminal 10 is a mobile terminal, the information processing capability may be lower than that of the PC. By transmitting only damage candidate images (for example, 20 or more) that are equal to or greater than the threshold of the number of damage points requested or set from the worker side), it is possible to reduce the information processing load on the portable terminal.

Further, the damage extraction server 20 can set the damage score of each damage candidate image not to 100 levels from 1 to 100, but can set the damage score at a smaller level to each damage candidate image and transmit it to the portable terminal.
For example, 1 to 10 is 1, 11 to 20 is 2,... 91 to 100 is 10, one level for every 10 damage points, and a total of 10 levels are set for each damage candidate image. Can also be sent to mobile terminals.
When the portable terminal receives the damage candidate image in which any one of the above-mentioned 10 damage points is set, the threshold is set to any one of 1 to 10, and the damage candidate image superimposed on the photographed image is based on the damage score. Can be selected arbitrarily.
As described above, for example, an image can be combined like a mobile terminal, but when an extraction result image is displayed on the user terminal 10 with low information processing capability, the number of damage points is reduced and the damage candidate image is carried. By transmitting to the terminal, it is possible to reduce the information processing load on the portable terminal.

(3) Configuration of Damage Extraction Server 20 The damage extraction server 20 is an information processing apparatus managed by a road management company to which a worker belongs, and is on the captured image of the structure received from the user terminal 10 described above. Based on the feature points, a damage candidate image that is an image that is estimated to be damage such as a crack generated in the structure is generated, and the damage candidate image or the like is transmitted to the user terminal 10.
When generating the damage candidate image, the damage extraction server 20 first divides the captured image into minute sections. This minute section is formed, for example, by dividing a photographed image into a matrix of vertical m cells × horizontal n cells (m, n is an integer equal to or greater than 1).
The damage extraction server 20 calculates, for each minute section, a damage score that is a score representing the possibility that each of the extracted damage candidate images is actual damage, and sets the damage candidate image for each minute section. Associate.

  As described above, the damage score is a score representing the high possibility that the damage candidate image is actually damaged. For example, for the damage candidate image extracted based on the lightness or color tone of the photographed image, the lightness Alternatively, it is calculated based on the color tone, the shape of the damage candidate image, or the like.

Here, an example of calculation of the damage score will be described.
For example, the raw score is calculated based on the following formula (1).
Element number of damage points = aX1 + bX2 + cX3 +. . . Formula (1)
The above a, b, c,... Are arbitrary constants.
For example, X1 is a lightness difference between the damage candidate images, X2 is an average lightness of the entire captured image from which the damage candidate images are extracted, and X3 is an index representing the linearity of the damage candidate images.
In the above formula, as an example, if a = −1, b = 1, and c = 1, the damage score becomes higher as the lightness X1 of the damage candidate image is lower (the darker the dark), and the damage candidate image becomes larger. Although it can be estimated that there is a high possibility of representing actual damage, if the average brightness X2 of the entire captured image is high, it can be said that the brightness of not only the damage candidate image but also the entire captured image is low (dark), so the damage score is low. Calculated.
In addition to the damage such as cracks, the photographed image of the structure may include non-damage such as molds and other material traces. Generally, since the trace of a concrete formwork has high linearity, when the linearity X3 of the said damage candidate image is high, it is highly likely that the damage candidate image is not damaged. Therefore, in that case, the damage score is calculated low.
In addition, the calculation example of the damage score is merely an example as described above, and the calculation method is not limited to this method.

When the damage extraction server 20 calculates the score of the damage score as described above, the damage extraction server 20 calculates the score of the damage from the score based on the damage score conversion TB 226 described later.
The number of damage points is expressed in a predetermined number of stages. Depending on the model of the user terminal 10 (model information in the user terminal DB 221), for example, it may be expressed in 100 stages from 1 to 100, or in 10 stages from 1 to 10. .

FIG. 3 is a diagram showing a configuration of the damage extraction server 20 according to the first embodiment of the present invention.
As shown in the figure, the damage extraction server 20 is configured based on a control unit 21 configured by a CPU or the like that controls the operation of the entire damage extraction server 20, a captured image of the structure received from the user terminal 10, and the captured image. The information storage unit 22 stores various information and the like, and the communication unit 23 transmits and receives information via a network.

FIG. 4 is a diagram showing each database stored in the information storage unit 22 of the damage extraction server 20 according to the first embodiment of the present invention.
As shown in the figure, the information storage unit 22 of the damage extraction server 20 includes a user terminal database (DB) 221 that manages information related to the user terminal 10 operated by the worker, and the damage extraction server 20 receives from the user terminal 10. A photographed image database (DB) 222 that manages the photographed images of the structure, a damage candidate image database (DB) 223 that manages the damage candidate images extracted based on the photographed images of the structure, and represented by 0 to 100 A damage point conversion table (TB) 226 for converting the damage points to be converted into damage points having a specified number of stages.
The information storage unit 22 stores captured images, damage candidate images, extraction result images, and the like in addition to the databases 221 to 223 and the table 226.

FIG. 5 is a diagram illustrating an example of a data configuration of the user terminal DB 221 according to the first embodiment of the present invention.
The user terminal DB 221 is a database for managing model information such as the type and quality of information transmitted from the damage extraction server 20 to the user terminal 10 for each user terminal 10.
As shown in the figure, the user terminal DB 221 determines whether or not the damage candidate image can be synthesized by the user terminal 10, the number of stages of the damage points used in the user terminal 10, and the damage candidate image when estimating the damage candidate image as actual damage. Each of the threshold value of the damage score serving as a reference for the reference, the damage score serving as a reference when transmitting the damage candidate image to the user terminal 10 (transmission damage score), and the upper limit of the resolution of the image displayed on the user terminal 10, Management is performed in association with the user terminal ID unique to the user terminal 10.

FIG. 6 is a diagram illustrating an example of a data configuration of the imaging DB 222 according to the first embodiment of the present invention.
The shooting DB 222 is a database for managing shooting information related to a shot image received by the damage extraction server 20 from the user terminal 10.
As shown in the figure, the shooting DB 222 sets a shooting time, a shooting position represented by latitude / longitude information, and the user terminal ID of the user terminal 10 that has transmitted the shot image as a shooting image ID unique to each shooting image. Manage in association.

FIG. 7 is a diagram illustrating an example of a data configuration of the damage candidate DB 223 according to the first embodiment of this invention.
The damage candidate DB 223 is a database for managing damage candidate images extracted by the damage extraction server 20 based on the feature points of the captured image received from the user terminal 10.
As shown in the figure, the damage candidate DB 223 includes a captured image ID of the captured image from which the damage candidate image is extracted, a raw score of the damage score related to the damaged candidate image, and a captured time of the captured image from which the damaged candidate image is extracted. And the positional information of the minute section where the damage candidate image is present in the captured image from which the damage candidate image is extracted, and the user terminal ID of the user terminal 10 that has transmitted the captured image from which the damage candidate image has been extracted. Management is performed in association with the damage candidate image ID unique to the candidate image.

FIG. 8 is a diagram illustrating an example of a data configuration of the damage point conversion TB 226 according to the first embodiment of this invention.
The damage score conversion TB 226 is a table for converting the raw score of the damage score calculated by the damage extraction server 20 into the damage score of the number of stages requested by the user terminal 10.
As shown in the figure, the damage score conversion TB 226 is configured such that the score of damage points, the score of damage when the score is expressed in 100 levels, and the score of damage when the score is expressed in 10 levels are mutually equal. Correspondingly recorded.
It should be noted that the number of damage points 10, 100 is merely an example, and the present invention is not limited to this.

(Operation of the first embodiment)
(1) Model information registration operation Before the operator performs damage extraction of a structure, the operator registers the model information of the user terminal 10 in advance, which is suitable for the information processing capability of the user terminal 10 and the communication environment. The extraction result image can be displayed on the user terminal 10 by the method described above.
FIG. 9 is a sequence chart showing a flow of registration operation in the first embodiment of the present invention.
Hereinafter, the description will be made along this drawing.

First, the worker operates the operation unit 15 of the user terminal 10 and inputs model information (step S101).
The user terminal 10 stores the input model information in the information storage unit 12 (step S102) and transmits it to the damage extraction server 20 (step S103).

Upon receiving the model information from the user terminal 10, the damage extraction server 20 stores it in the information storage unit 22 (step S104).
This completes the registration operation of the model information.

In this manner, by registering the model information in the user terminal 10 and the damage extraction server 20 in advance, the damage extraction server 20 can extract the extraction result image by a method suitable for the information processing capability of the user terminal 10 and the communication environment. Will be able to provide.
For example, when the user terminal 10 is composed of a PC and a camera and the information processing capability and communication capability are high, the user terminal 10 executes the extraction result image generation processing, and the user terminal 10 has a communication function. When the information processing capability and communication capability cannot be expected so much as in the case of only a camera, the damage extraction server 20 executes the extraction result image generation process instead of the user terminal 10 to generate the extraction result image. It is possible to efficiently distribute the burden on the entire system.

  In addition, when the user terminal 10 has a lower information processing capability and communication function than a PC, such as a mobile terminal, the information processing and communication of the user terminal 10 are performed by thinning out and reducing the number of damage points. The burden can be reduced.

(2) Damage Extraction Operation FIGS. 10 to 13 are sequence charts showing the flow of damage extraction operation by the damage extraction system in the first embodiment of the present invention.
The operation of the damage extraction system that extracts the damage of the building based on the image of the building photographed by the worker and provides the worker with information on the damage of the building in accordance with this figure. Proceed with the explanation.

First, a worker photographs a structure using the user terminal 10 (imaging unit 16) (step S201).
The user terminal 10 provides shooting information in which a shooting image of the structure generated at the time of shooting is associated with a shooting image ID unique to the shooting image, a shooting time, and a shooting position (latitude and longitude information, etc.). The information is stored in the information storage unit 12 (step S202).

In order for the user terminal 10 to transmit a captured image to the damage extraction server 20 and to receive provision of a damage extraction result, a user terminal ID or device information unique to the user terminal 10 is transmitted to the damage extraction server 20 together with the captured image. There is a need to.
Therefore, here, the user terminal 10 determines whether or not the operator has selected to transmit the user terminal ID unique to the user terminal 10 to the damage extraction server 20 by operating the operation unit 15 (step S203). .
When transmission of the user terminal ID is selected (step S203 / Yes), the user terminal 10 reads out the user terminal ID unique to the own device stored in the information storage unit 22, and the read out user terminal ID Is associated with the photographing information (step S204), and the photographing information associated with the user terminal ID is transmitted to the damage extraction server 20 (step S205).

On the other hand, when it is selected not to transmit the user terminal ID (step S203 / No), the worker operates the operation unit 15 to input device information (step S206).
In this case, the user terminal 10 associates the input device information with the photographing information (step S207), and transmits the photographing information associated with the user terminal ID to the damage extraction server 20 (step S205).

  When the damage extraction server 20 receives the shooting information associated with the user terminal ID or the device information from the user terminal 10, the damage extraction server 20 stores the received shooting information in the shooting DB 222 (step S208).

Next, the damage extraction server 20 determines whether or not a user terminal ID is included in the received shooting information (step S209).
Here, when it is determined that the user terminal ID is included (step S209 / Yes), the damage extraction server 20 refers to the user terminal DB 221 and obtains device information associated with the user terminal ID. Extract (step S210).
On the other hand, if it is determined that the user terminal ID is not included in the received shooting information (step S209 / No), device information is extracted from the shooting information (step S211).

Next, the damage extraction server 20 divides the image area of the photographed image included in the photographing information into a plurality of minute sections, detects feature points in each minute section, and based on the detected feature points, damage points are detected. A damage candidate image estimated to be likely is extracted (step S212).
The extraction process of the damage candidate image based on the feature point is, for example, an image based on a general feature point such as extracting a set of pixel regions having lightness lower (darker) than the surrounding lightness by a predetermined value or more as a damage candidate image. Since it is the same as the extraction process, its detailed description is omitted.

  Next, the damage extraction server 20 writes the damage candidate information, the shooting time, and position information on the shot image together with the extracted damage candidate image into the damage candidate DB 223 for each minute section as damage candidate information (step S213).

Next, the damage extraction server 20 measures the width and length of each damage candidate image on the basis of the number of pixels and the position information constituting the extracted damage candidate image, and calculates the total cracks in each minute section. A crack rate representing the length is calculated, and the measured or calculated information is written in the damage candidate DB 223 in association with the corresponding minute section (step S214).
The method for measuring the width and length of the image is also the same as the conventional general method, and the description thereof is omitted.

  Next, the damage extraction server 20 calculates the damage score for the damage candidate image extracted for each micro section, and writes the damage score calculated for each micro section in the damage candidate DB 223 for each micro section (step S215). .

Next, the damage extraction server 20 refers to the “transmission information type” in the device information extracted in step S210 or S211, and determines whether or not to transmit the damage score to the user terminal 10 (step S216).
Here, when it is determined that the damage score is transmitted to the user terminal 10 in the “transmission information type” of the device information (step S216 / Yes), the damage extraction server 20 determines the “number of damage score steps” of the device information. ”, It is determined whether or not a number of stages (for example, 10 stages) smaller than the original number of stages (for example, 100 stages) is designated as the number of damage points (step S217).
If it is determined that a number of stages smaller than the original number of stages is designated (step S217 / Yes), the damage extraction server 20 designates the number of damage points based on the damage point conversion table 226. The damage extraction server 20 transmits the damage candidate images for each minute section and the converted damage points to the user terminal 10 as damage candidate information (step S219).
On the other hand, if it is determined that the number of stages smaller than the original number of stages is not designated (step S217 / No), the damage extraction server 20 does not directly convert the number of stages of damage points, Damage candidate information includes a damage candidate image for each minute section, a captured image ID of the captured image from which the damage candidate image is extracted, position information on the captured image of the damage candidate image, and a damage score of the damage candidate image. To the user terminal 10 (step S219).
At this time, the damage extraction server 20 refers to the “transmission damage score” in the user terminal DB 221 and does not transmit a damage candidate image having a damage score lower than the transmission damage score to the user terminal 10.

When receiving the damage candidate information from the damage extraction server 20, the user terminal 10 extracts the captured image ID from the damage candidate information, and reads the captured image of the captured image ID from the information storage unit 12 (step S231).
Next, the user terminal 10 extracts a threshold value for the number of damage points from the set device information stored in the information storage unit 12 (step S232).
Then, the user terminal 10 compares the damage score included in the received damage candidate information with the threshold value of the extracted damage score (step S233), and for a minute section having a damage score equal to or greater than the threshold value (step S233). / Yes), based on the position information included in the damage candidate information, the damage candidate image of the minute section is superimposed on the read-out photographed image and synthesized (step S234).
On the other hand, for a minute section having a value lower than the threshold, the damage candidate image is not superimposed on the captured image.
The user terminal 10 repeats these processes in all minute sections (step S235), generates an extraction result image (step S236), and displays it on the display unit 14 (step S237).

In the displayed extraction result image, the operator compares the actual damage location on the captured image with the damage candidate image, and determines that the display of the damage candidate image is necessary, the operator operates the operation unit 15. Then, a threshold value for the number of damage points is newly input (step S238).
For example, if the worker determines that the actual damage location is not displayed as a damage candidate image in the extraction result image on the display unit 14, the damage score registered in the information storage unit 12 as device information The threshold value of the damage point lower than the threshold value is input, and the number of displayed damage candidate images is increased to approach the actual damage state of the building.
On the other hand, if the worker determines that an extraction result image that is not actually a damaged part is displayed as a damage candidate image, the number of damage points registered in the information storage unit 12 as device information A threshold value of damage points higher than the threshold value is input to reduce the number of displayed damage candidate images so as to approach the actual damage status of the building.

  As described above, when a new threshold value for damage points is input (step S238 / Yes), the user terminal 10 proceeds to step S233, and similarly extracts based on the newly input threshold value for damage points. Generate the image again.

When the damage extraction server 20 refers to the “type of transmission information” in the device information in step S216 and determines that the damage score is not transmitted to the user terminal 10 (step S216 / No), the damage extraction server 20 The server 20 extracts a threshold value for the number of damage points included in the device information (step S251).
Next, the damage extraction server 20 compares the damage score of each minute section with the threshold value of the extracted damage score (step S252), and for the minute section of the damage score equal to or greater than the threshold value (step S252 / Yes). ) Based on the position information of the damage candidate image, the damage candidate image of the minute section is superimposed on the captured image received from the user terminal 10 and synthesized (step S253).
On the other hand, for a minute section having a value lower than the threshold, the damage candidate image is not superimposed on the captured image.
The damage extraction server 20 repeats these processes in all minute sections (step S254), generates an extraction result image (step S255), and transmits it to the user terminal 10 (step S256).

  When receiving the extraction result image from the damage extraction server 20, the user terminal 10 displays it on the display unit 14 (step S257).

  In the displayed extraction result image, the operator compares the actual damage location on the captured image with the damage candidate image, and determines that the display of the damage candidate image is necessary, the operator operates the operation unit 15. Then, a new threshold value for damage points is input (step S258 / Yes), and the user terminal 10 transmits the input threshold value for damage points to the damage extraction server 20 (step S259).

When the damage extraction server 20 receives a new threshold value for damage points (step S260 / Yes), the damage extraction server 20 proceeds to step S252 and again generates an extraction result image based on the received threshold value for damage points.
This is the end of the operation.

  As described above, the damage extraction server 20 provides the user terminal 10 with information related to the damage of the structure based on the model information reflecting the information processing capability of the user terminal 10 and the contents of the communication environment. The information processing load can be efficiently distributed as a whole system, and the user terminal 10 can easily display an image representing damage to the structure.

Further, as described above, the user terminal 10 can select a damage candidate image to be displayed superimposed on the captured image by changing the threshold value of the damage score.
FIGS. 14-16 is a figure which shows an example of the change of the damage candidate image displayed by the change of the threshold value of the damage score in the 1st Embodiment of this invention.
In these figures, the square frame represents the extraction result image, the thick curve in the frame is cracked, and the thin straight line represents the trace of the concrete mold.

FIG. 14 shows an example of an extraction result image that is initially displayed on the user terminal 10.
The worker confirms the extraction result image shown in FIG. 14 and compares the damage candidate image on the extraction result image (photographed image) with the actual damage portion. As a result, many formwork traces that are not actual damage are displayed. If you want to delete them, enter a higher damage score threshold than the current one.
In this case, as shown in FIG. 15, a damage candidate image having a damage point lower than a newly input threshold value is not displayed, and an image close to actual damage can be acquired.

On the other hand, as a result of checking the extraction result image of FIG. 14 and comparing the damage candidate image on the extraction result image (photographed image) with the actual damage portion, the worker does not display the actual damage. If an additional value is to be displayed, a threshold value for damage points lower than the current one is input.
In this case, as shown in FIG. 16, damage candidate images having a damage number lower than the previous threshold value and greater than or equal to the newly input threshold value are additionally displayed, and an image in a state close to actual damage can be acquired. it can.

(3) Checking operation of structural secular change As described above, the damage extraction server 20 stores information such as a photographing time and a photographing position in association with a photographed image in the photographing DB 222 as photographing information.
Further, the damage extraction server 20 stores each damage candidate image together with its width, length, and crack rate in the damage candidate DB 223 in association with the photographing time, position information, and the like.
When the operator operates the operation unit 15 of the user terminal 10 to input a captured image ID, a capturing time, position information, and the like and transmits the information to the damage extraction server 20, the damage extraction server 20 selects the corresponding captured image or damage. The candidate image is transmitted to the user terminal 10. At this time, the damage extraction server 20 can also transmit to the user terminal 10 photographed images or damage candidate images of the same position information at different photographing times.

When receiving the captured image or the damage candidate image from the damage extraction server 20, the user terminal 10 displays the image on the display unit 14.
By referring to the displayed image, the worker can easily observe the secular change of damage such as a crack in the structure.

(4) Image Concatenation Operation The user terminal 10 transmits photographing information including a plurality of photographed images of the same structure to the damage extraction server 20, and extracts a joint image (panorama image) obtained by concatenating the plurality of photographed images. It can be acquired from the server 20.
FIG. 17 is a sequence chart showing the flow of a linked image generation operation according to the first embodiment of the present invention.
Hereinafter, the description will be made along this drawing.

  Since the process until the damage extraction server 20 receives the shooting information including a plurality of shot images from the user terminal 10 and extracts the model information is the same as the process of steps S201 to S211 of the damage extraction operation process described above, Description is omitted.

The damage extraction server 20 extracts feature points (cracks, etc.) of a plurality of captured images included in the captured information received from the user terminal 10 (step S301).
Next, the damage extraction server 20 collates a plurality of extracted feature points and detects a common feature point among these feature points (step S302).
For this common feature point detection processing, a general detection method based on brightness, color tone, or the like may be used.

Next, the damage extraction server 20 connects the captured images so as to overlap the extracted common feature points by enlarging, reducing, rotating, or the like, and generates a connected image (step S303).
Since processing such as enlargement / reduction / rotation when the two images are connected is the same as the conventional general image processing method, detailed description thereof will be omitted.

18 and 19 are diagrams showing a method for generating a connected image according to the first embodiment of the present invention.
As shown in FIG. 18, the feature points of the two captured images 1 and 2 are extracted, and the captured images of the captured images 1 and 2 overlap each other. When extracted, the damage extraction server 20 connects the right end portion of the captured image 1 and the left end portion of the captured image 2 so as to superimpose these common feature points, as shown in FIG.
The damage extraction server 20 connects the captured images having such a common area, and generates a connected image in which the entire structure is displayed.

Here, it returns to FIG. 17 again and advances description.
As described above, when the damage extraction server 20 generates a linked image, it detects resolution information from the model information, and determines whether the resolution of the generated linked image is different from the resolution specified by the resolution information. Judgment (step S304), if different (step S304 / Yes), the generated linked image is converted to the resolution specified in the resolution information (step S305), and the resolution-converted linked image is converted to the user terminal. 10 (step S306).
Note that the resolution conversion method is the same as the conventional general image resolution conversion method, and thus detailed description thereof is omitted.

  On the other hand, when the resolutions are not different (step S304 / No), the damage extraction server 20 transmits the generated connected image to the user terminal 10 without performing resolution conversion processing (step S306).

When the user terminal 10 receives the connected image, the user terminal 10 displays it on the display unit 14 (step S307).
The operation is thus completed.

Thus, the worker can easily acquire the entire image of the building with a resolution corresponding to the information processing capability of the user terminal 10.
For example, when the user terminal 10 is a device having a relatively high information processing capability such as a PC, a high-resolution linked image having a large data size can be displayed as it is.
On the other hand, when the user terminal 10 is a camera or the like having a communication function, a low-resolution linked image with a small data size is displayed and used to check whether there is a region that has been forgotten to be photographed in the structure. be able to.

(5) CAD drawing generation operation The damage candidate image extracted from the above-described photographed image is represented by a set of points (pixels).
When this damage candidate image is reflected in a CAD drawing, a set of these points is connected to form vector data.
When displaying the damage candidate image on the CAD drawing, the worker uses the user terminal 10 to transmit a photographed image or the like requesting the CAD drawing to the damage extraction server 20.
When the damage extraction server 20 receives the captured image, the damage extraction server 20 executes the above-described damage extraction operation to extract the damage candidate image, and the connection relationship between the points constituting the extracted damage candidate image and the luminance around the point The points are automatically connected from the information, converted into vector data, and the damage candidate image converted into vector data is transmitted to the user terminal 10.
The user terminal 10 displays the damage candidate image converted into vector data on a CAD drawing.
As described above, in this embodiment, it is possible to easily draw and use the damage candidate image as a CAD drawing.

(6) Report Creation Operation As described above, in the damage extraction operation, damage candidate images are managed by the damage candidate DB 223.
In response to a request from the user terminal 10, the damage extraction server 20 can extract a damage candidate image managed in the damage candidate DB 223 and automatically create a structure inspection report.

The information storage unit 22 of the damage extraction server 20 stores a report data template in which report items common to all workers are set from a typical example of an inspection report.
The information storage unit 22 stores and sets report items that are different for each worker and report items that are expressed in different ways even for the same item.
Items that are different for each worker are registered before the report is created by the worker inputting them using the user terminal 10 and transmitting them to the damage extraction server 20.
By being configured in this way, the damage extraction server 20 creates a report template unique to the worker, and automatically creates an inspection report by pouring damage candidate information and the like into the template.
In addition, a damage candidate image can be combined with the inspection report. The damage candidate image to be combined with the inspection report is the scale setting, the automatic setting of the scale to fit in the frame on the inspection report, the line type and the line width describing the crack, etc. It can be registered in advance in the damage extraction server 20 in the same manner.

  As described above, when the worker creates the inspection report of the structure, it can be automatically created with the contents registered in advance in the damage extraction server 20, so the inspection report of the layout and contents desired by the worker can be created. It becomes possible to create a book easily.

(7) Learning Operation for Damage Detection The user terminal 10 displays the extraction result image on the display unit 14 as described above.
If the worker confirms the displayed extraction result image and compares the damage candidate images such as cracks shown in the extraction result image with the actual damage location, if there is a difference between the two, extraction is performed. The result image can be corrected.

The operator uses the operation unit 15 (mouse or other device) to specify a predetermined location on the extraction result image, and draws a damage image that is actually present on the extraction result image and adds it. The deletion result image is corrected by specifying deletion of a damage candidate image of damage that does not actually exist.
The user terminal 10 stores the correction information indicating the correction in the information storage unit 12 and then transmits the correction information to the damage extraction server 20.

  When the damage extraction server 20 receives the correction information from the user terminal 10, the damage extraction server 20 performs correction (addition / deletion of damage candidate images) along the correction information on the corresponding extraction result image, and extraction after the correction is performed. The result image, the damage candidate image, and the like are stored in the information storage unit 22.

Next, the damage extraction server 20 corrects the calculation formula for the score of damage points based on the correction information.
For example, the equation for calculating the number of damage points is the following equation (1).
Element number of damage points = aX1 + bX2 + cX3 +. . . Formula (1)
Here, the damage extraction server 20 changes the constants a, b, and c so as to be closest to the damage candidate image having the content indicated in the correction information.
For example, if deletion of a certain damage candidate image is specified in the correction information, the damage extraction server 20 changes the value of “a” from “−1” to “−1. Change to 2 ”.

  As described above, the damage extraction server 20 has a function of appropriately correcting and learning the calculation formula of the score of damage points based on the correction information received from the user terminal 10, so that a more optimal damage result can be obtained. It is possible to effectively increase the detection accuracy so as to detect.

(Summary of the first embodiment)
As described above, according to the first embodiment of the present invention, the damage extraction server 20 transmits the damage candidate image and the damage score to the user terminal 10 based on the model information of the user terminal 10. Or whether to generate an extraction result image and transmit it to the user terminal 10, the load on the entire damage extraction system for causing the user terminal 10 to display an image indicating the damage of the concrete structure or the asphalt structure is efficient. The image is well dispersed and the image can be easily displayed on the user terminal 10.
Therefore, a road management company or the like can manage and maintain a concrete structure or an asphalt structure very easily.

  Further, according to the present embodiment, when the damage candidate images are generated, the damage score indicating the possibility that each damage candidate image is actual damage is calculated. The damage candidate image can be easily deleted or added as compared with the actual captured image.

  Further, according to the present embodiment, the damage extraction server 20 converts the number of damage points based on the model information and transmits it to the user terminal 10, so that the user terminal 10 performs the damage candidate image generation process. It becomes possible to remarkably reduce the load.

  In addition, according to the present embodiment, the damage extraction server 20 does not transmit damage candidate images less than a predetermined damage score (transmission damage score) to the user terminal 10, so that the user terminal 10 performs a damage candidate image generation process. It is possible to remarkably reduce the load at the time.

  Further, according to the present embodiment, when the damage extraction server 20 transmits a connected image (panoramic image) generated by connecting a plurality of captured images of the same structure to the user terminal 10, based on the model information. Since the resolution of the connected image is adjusted, it is possible to provide an appropriate image according to the information processing capability, communication environment, or specifications of the user terminal 10.

The user terminal 10 and the damage extraction server 20 are realized mainly by a program loaded on the CPU and the memory. However, it is also possible to configure this apparatus or server by a combination of any other hardware and software, and the high degree of freedom of design will be easily understood by those skilled in the art.
When the user terminal 10 or the damage extraction server 20 is configured as a software module group, the program is recorded on a recording medium such as an optical recording medium, a magnetic recording medium, a magneto-optical recording medium, or a semiconductor, It may be loaded from a recording medium, or may be loaded from an external device connected via a predetermined network.

  The above-described embodiment is an example of a preferred embodiment of the present invention. The embodiment of the present invention is not limited to this, and various modifications can be made without departing from the scope of the present invention. It becomes possible to do.

For example, the damage extraction system in the present embodiment can be applied to any concrete structure, asphalt structure or other structure, such as road surfaces, bridges, guardrails, median strips, tunnels, dikes, dams. Applicable to any structure such as.
Moreover, the imaging | photography part 16 of the user terminal 10 may be comprised with the camera and portable terminal which an operator operates as above-mentioned, and may be mounted in a vehicle, a ship, etc. For example, the photographing unit 16 may be mounted on a vehicle, and a road surface may be photographed while traveling to check for road surface damage.

  Moreover, according to this Embodiment, although the example which an operator image | photographs a structure using the camera or portable terminal which comprises the user terminal 10 was demonstrated, it is also possible to image | photograph with another method.

DESCRIPTION OF SYMBOLS 10 User terminal 11,21 Control part 12,22 Information storage part 13,23 Communication part 14 Display part 15 Operation part 16 Shooting part 100 Network 221 User terminal DB
222 Shooting DB
223 Damage candidate DB
226 Damage point conversion TB

Claims (9)

A damage extraction system comprising a user terminal operated by a worker who inspects a structure, and a damage extraction server connected to the user terminal via a network and extracting damage generated in the structure Because
The user terminal transmits a captured image of a structure to the damage extraction server,
The damage extraction server generates a damage candidate image indicating an image presumed to be damage caused in the structure based on the captured image, and transmits the generated damage candidate image to the user terminal.
A damage extraction system that determines whether or not the damage candidate image to be transmitted to the user terminal is combined with the captured image based on model information of the user terminal.   The damage extraction system according to claim 1, wherein the damage extraction server adds a damage score indicating the possibility of actual damage to the damage candidate and transmits the damage candidate image to the user terminal. . The damage extraction server transmits the damage candidate image to the user terminal without being combined with the captured image based on the model information,
The user terminal stores a threshold value of the damage score that is a reference for recognizing the damage candidate image as damaged. When the damage candidate image is received from the damage extraction server, the user terminal is based on the threshold value of the damage score. The damage extraction system according to claim 2, wherein the damage candidate image to be displayed is determined, and the determined damage candidate image is combined with the captured image stored in the own device and displayed.   When the user terminal inputs a threshold value of the damage score, the user terminal determines the damage candidate image to be displayed based on the input threshold value of the damage score, and the determined damage is stored in the captured image stored in the own device. 4. The damage extraction system according to claim 3, wherein candidate images are synthesized and displayed.   5. The damage extraction system according to claim 3, wherein the damage extraction server reduces the number of stages of the damage points based on the model information.   The damage according to any one of claims 3 to 5, wherein the damage extraction server transmits only damage candidate images having a predetermined number of damage points or more to the user terminal based on the model information. Extraction system.   The damage extraction server stores a threshold value of the damage point that is a reference for recognizing the damage candidate image as damaged, and determines the damage candidate image to be synthesized based on the threshold value of the damage point, The damage extraction system according to claim 2, wherein the determined damage candidate image is combined with a captured image received from the user terminal, and the combined image is transmitted to the user terminal. The user terminal, when the threshold value of the damage score is input, transmits the threshold value of the input damage score to the damage extraction server,
Upon receiving the damage score threshold, the damage extraction server determines the damage candidate image to be synthesized based on the received damage score threshold, and determines the determined damage candidate in the captured image received from the user terminal. 8. The damage extraction system according to claim 7, wherein the images are combined and the combined image is transmitted to the user terminal. A damage extraction system comprising a user terminal operated by a worker who inspects a structure, and a damage extraction server connected to the user terminal via a network and extracting damage generated in the structure Because
The user terminal transmits a plurality of captured images of the same structure to the damage extraction server,
The damage extraction server extracts a common image of a plurality of captured images of the structure received from the user terminal, and combines a plurality of captured images connected to each other so as to overlap the extracted common images. A damage extraction system that generates, adjusts the resolution of the generated connected image based on the model information of the user terminal, and transmits the connected image with the adjusted resolution to the user terminal.

Images