JP6534001B2 - Damage extraction system - Google Patents

Description

  The present invention relates to a damage extraction system, and more particularly to a damage extraction system for extracting a damage such as a crack generated in a concrete-asphalt structure.

In recent years, aging deterioration of a concrete-asphalt structure (hereinafter, simply referred to as a structure) constructed in the high growth period has been regarded as a problem.
Inspection of these structures is becoming important because damage such as cracks occurs in these structures, causing the structures to collapse and excessive unevenness on the road surface, leading to serious accidents.
However, on the other hand, the structure covers a wide area such as a road, a bridge or a tunnel, and the inspection work of the structure is limited only by the human labor of the worker.

As a prior art for solving the above problems, the concrete structure maintenance management system which patent document 1 discloses is proposed.
In this concrete structure maintenance management system, the measuring device measures an initial crack of the structure by optical ranging, and when the measuring terminal transmits the measurement information to the server via the network, the server is based on the measurement information. It analyzes about degradation prediction of a structure etc., and transmits the analysis result and the information of a crack to a builder terminal via a network.

JP 2015-138467

However, in the concrete structure maintenance management system of Patent Document 1, when the server transmits the analysis result of the structure, etc. to the builder terminal, the information processing capability of the builder terminal and the communication environment are not considered at all. Therefore, there is a problem that the display processing of the analysis result on the side of the builder terminal may be disturbed.
In particular, when the builder terminal is an information processing terminal other than a PC wired connected to the network, depending on the size of the data, it may take a long time to display the analysis result or may not be able to display itself. .

  The present invention has been made in view of the above problems, and the extraction result of the damage is appropriately determined in consideration of the information processing capability of the terminal receiving the provision and the communication environment. It aims to provide a damage extraction system provided by data content.

In order to achieve such an object, the present invention provides a user terminal operated by a worker who inspects a structure, and a damage extraction server connected to the user terminal via a network and extracting damage to the structure. A damage extraction system configured to include: a damage extraction server that is a damage candidate that indicates an image estimated to be damage to a structure based on a captured image of the structure received from a user terminal; Whether an image is generated for each minute section of the captured image, model information of the user terminal is acquired based on the information included in the captured image, and a damage candidate image generated based on the acquired model information is transmitted to the user terminal or determines, as a result, if it is determined to send the damaged candidate image, the generated damage candidate image, denoted by the damage score indicating the likelihood that the actual damage to transmit to the user terminal, Yu The terminal transmits the photographed image to the damaged extraction server, enter the threshold value in relation to the standard damage score in recognizing damage candidate image damage and, when receiving damage candidate image from damage extraction server, enter damaged It is characterized in that the damage candidate image to be displayed is determined based on the score threshold, and the determined damage candidate image is synthesized and displayed at the photographed image corresponding position stored in the own machine .

  Further, according to the damage extraction system in the present invention, the damage extraction server is characterized by reducing the number of stages of damage points based on the model information.

  The damage extraction system according to the present invention is characterized in that the damage extraction server transmits only damage candidate images having a predetermined damage score or more to the user terminal based on the model information.

Further, the present invention is configured to have a user terminal operated by a worker who inspects a structure, and a damage extraction server which is connected to the user terminal via a network and extracts damage to the structure. Damage extraction system, wherein the damage extraction server is configured to capture a damage candidate image showing an image estimated to be damage caused to the structure based on the captured image of the structure received from the user terminal; The model number is generated for each minute section, and the damage candidate image is given a damage score serving as a reference for recognizing damage, and model information of the user terminal is acquired based on the information included in the captured image. Based on the information, it is determined whether or not the damage candidate image generated is to be transmitted to the user terminal, and as a result, when it is determined not to transmit the damage candidate image, the damage score and received from the user terminal The damage candidate image to be synthesized is determined based on the threshold value of the flaw score, the damage candidate image determined at the corresponding position of the captured image received from the user terminal is synthesized, and the synthesized image is transmitted to the user terminal The present invention is characterized in that the threshold value of the damage score is input, the photographed image and the threshold value of the input damage score are transmitted to the damage extraction server, and when the synthesized image is received from the damage extraction server, the synthesized image is displayed.

  In the present invention, any combination of the above-described components, and one in which the components and expressions of the present invention are mutually replaced among a method, an apparatus, a system, a computer program, a recording medium storing a computer program, etc. It is effective as an aspect of

  According to the present invention, the user terminal transmits the photographed image of the structure to the damage extraction server, and the damage extraction server indicates the damage candidate image indicating the image estimated to be the damage caused to the structure based on the photographed image. The damage candidate image to be generated and generated is transmitted to the user terminal, and it is determined whether the damage candidate image to be transmitted to the user terminal is to be combined with the captured image based on the model information of the user terminal. It becomes possible to provide the extraction result of the said damage by an appropriate data content in consideration of the information processing capability of a user terminal, and a communication environment about the extraction result of.

It is a figure showing the composition of the damage extraction system in a 1st embodiment of the present invention. It is a figure which shows the structure of the user terminal 10 in the 1st Embodiment of this invention. It is a figure which shows the structure of the damage extraction server 20 in the 1st Embodiment of this invention. It is a figure which shows each database stored in the information storage part 22 of the damage extraction server 20 in the 1st Embodiment of this invention. It is a figure showing an example of data composition of user terminal DB221 in a 1st embodiment of the present invention. It is a figure which shows an example of a data structure of imaging | photography DB222 in the 1st Embodiment of this invention. It is a figure which shows an example of a data structure of damage candidate DB223 in the 1st Embodiment of this invention. It is a figure which shows an example of a data structure of damage score conversion TB226 in the 1st Embodiment of this invention. It is a sequence chart which shows the flow of the registration operation in a 1st embodiment of the present invention. It is a sequence chart which shows a flow of damage extraction operation by a damage extraction system in a 1st embodiment of the present invention. It is a sequence chart which shows a flow of damage extraction operation by a damage extraction system in a 1st embodiment of the present invention. It is a sequence chart which shows a flow of damage extraction operation by a damage extraction system in a 1st embodiment of the present invention. It is a sequence chart which shows a flow of damage extraction operation by a damage extraction system in a 1st embodiment of the present invention. It is a figure which shows an example of a change of the damage candidate image displayed by the change of the threshold value of the damage score in the 1st Embodiment of this invention. It is a figure which shows an example of a change of the damage candidate image displayed by the change of the threshold value of the damage score in the 1st Embodiment of this invention. It is a figure which shows an example of a change of the damage candidate image displayed by the change of the threshold value of the damage score in the 1st Embodiment of this invention. It is a sequence chart which shows the flow of generation operation of a connected picture in a 1st embodiment of the present invention. It is a figure which shows the production | generation method of the connection image in the 1st Embodiment of this invention. It is a figure which shows the production | generation method of the connection image in the 1st Embodiment of this invention.

(Outline of the First Embodiment)
The damage extraction system according to the embodiment of the present invention is, for example, a road administrator who checks whether or not the road or other concrete structure or asphalt structure (hereinafter simply referred to as “structure”) is damaged. The worker or the like photographs the concrete structure or asphalt structure using the user terminal 10, transmits the photographed image to the damage extraction server 20, and the damage extraction server 20 calculates the structure of the structure based on the photographed image. The damage is extracted, and information indicating the damage extraction result is provided to the user terminal 10.
When the damage extraction server 20 provides the user terminal 10 with the information on the extraction result of the damage of the structure, the damage extraction server 20 adjusts the content of the information to be provided based on the model information of the user terminal 10 to obtain the entire system. It becomes possible to disperse | distribute the load of the information processing in efficiently, and to provide the information of the said damage extraction result smoothly.

(Configuration of the first embodiment)
(1) Overall Configuration of Damage Extraction System FIG. 1 is a diagram showing the configuration of a damage extraction system according to a first embodiment of the present invention.
As shown in the figure, the damage extraction system uses the user terminal 10 operated by the investigator who checks whether the structure is damaged or not, and the information received from the user terminal 10 via the network 100. It comprises comprising the damage extraction server 20 which extracts the damage of the said structure based on and provides the extraction result to the user terminal 10. FIG.

(2) Configuration of User Terminal 10 The user terminal 10 is a camera with a communication function operated by a worker or the like, a portable terminal such as a smartphone or tablet terminal, or an information processing apparatus such as a PC or a robot having a photographing / communication function. The structure is photographed, and the photographed image is transmitted to the damage extraction server 20, and the damage extraction result of the structure generated based on the photographed image is received from the damage extraction server 20 and displayed. .

FIG. 2 is a diagram showing the configuration of the user terminal 10 in the first embodiment of the present invention.
As shown in the figure, the user terminal 10 includes a control unit 11 configured of a CPU or the like to control the overall operation of the user terminal 10, and an information storage unit 12 for storing input information, information received via a network, and the like. A communication unit 13 that transmits and receives information via a network, a display unit 14 configured by a display or the like to display information on a screen, an operation unit 15 configured by a key or a mouse to input information or the like, a structure And a photographing unit 16 for photographing.

The user terminal 10 is, as described above, various information processing apparatuses such as a PC, a camera or a portable terminal.
Hereinafter, which information processing terminal the user terminal 10 is configured of are divided into cases, and each configuration and operation will be described below.

(2-a) When the User Terminal 10 is a PC and a Camera For example, the user terminal 10 is configured by connecting a camera to a desktop or notebook PC connected to a network 100 such as the Internet.
Although the PC may have a wireless communication function, it may be configured to be connectable to the Internet using a wired communication function. In this case, large-capacity, high-speed data communication can be expected.
When the user terminal 10 is configured by the PC and the camera as described above, the worker shoots a structure using the camera, and uses the memory of the camera body or an information recording medium such as an SD card inserted in the camera. The captured image is stored.
After that, the worker connects the camera to the PC with a USB cable or the like, transfers the photographed image from the camera to the PC, and stores the image in a hard disk or the like in the PC. The transfer method is not limited to the method, and wireless communication may be used, for example.
Then, the PC transmits the photographed image to the damage extraction server 20 via the network 100.
The damage extraction server 20 extracts a damage candidate image based on the photographed image, and transmits the damage candidate image to the PC via the network 100 in association with position information indicating the damage position and the number of damage.
When the PC receives the damage candidate image from the damage extraction server 20, the PC superimposes the received damage candidate image on the photographed image stored in its own machine to generate and display a composite image (extraction result image).
The worker can limit the damage candidate image displayed on the photographed image to one having the threshold value or more by operating the PC and inputting the threshold value of the damage score.
The threshold value of the damage score can be selected from, for example, 1 to 100 levels. As damage candidate images below the threshold are not superimposed on the captured image, the lower the set threshold, the more damage candidate images are displayed.

(2-b) When the User Terminal 10 is a Camera The user terminal 10 is configured of, for example, a camera that can be connected to the network 100 using a wireless communication function. In this case, the camera has a communication function but does not have an image combining function.
In this case, the worker photographs the structure using the camera, and records the photographed image on the memory of the camera body or an information recording medium such as an SD card inserted in the camera.
Thereafter, the worker connects the camera to the network 100 using the wireless communication function, and transmits the photographed image to the damage extraction server 20 via the network 100.
The damage extraction server 20 extracts a damage candidate image based on the captured image, generates an extraction result image obtained by superimposing the damage candidate image on the captured image, and transmits the extracted result image to the camera via the network 100.
When the camera receives the extraction result image from the damage extraction server 20, it stores it in its own device and displays it on the screen.
As described above, in the case of a camera in which the user terminal 10 has a low information processing capability or does not have an image combining function, damage extraction is performed even if only a damage candidate image is received like a PC. Since the resulting image can not be displayed, the damage extraction server 20 performs image composition processing.

(2-c) When the User Terminal 10 is a Portable Terminal The user terminal 10 is configured of, for example, a portable terminal such as a smartphone or a tablet terminal that can be connected to the network 100 such as the Internet. Although this portable terminal has an image combining function, the information processing capability may be slightly lower than that of the PC, and the communication environment may be slightly inferior.
In this case, the worker photographs the structure using the photographing function provided in the portable terminal, and the photographed image is stored in the memory of the portable terminal body or an information recording medium such as an SD card inserted in the portable terminal. Store.
Thereafter, the portable terminal transmits the photographed image to the damage extraction server 20 via the network 100.
The damage extraction server 20 extracts a damage candidate image based on the captured image, and transmits the damage candidate image to the portable terminal via the network 100 in association with position information indicating the damage position.
When the portable terminal receives the damage candidate image from the damage extraction server 20, the portable terminal superimposes the received damage candidate image on the photographed image stored in its own device, and generates and displays an extraction result image.

  Note that, as described above, when the user terminal 10 is a portable terminal, the information processing capacity may be lower than that of a PC, so the damage extraction server 20 selects one of the 1 to 100 damage points on the portable terminal side By transmitting only damage candidate images (for example, 20 or more) equal to or more than the threshold value of damage points requested or set from the worker side) to the portable terminal, the load of information processing in the portable terminal can be reduced.

In addition, the damage extraction server 20 can set the damage score of each damage candidate image not to the 100 levels of 1 to 100 but to the damage candidate image and transmit the damage score to the portable terminal.
For example, 1 to 10 is 1, 1 to 20 is 2,... 91 to 100 is 10, and the damage score is one step per 10 damage points, and a total of 10 damage scores are set for each damage candidate image. Can also be sent to the mobile terminal.
When the portable terminal receives the damage candidate image in which the damage score of any of the 10 levels is set, the threshold value is set to any one of 1 to 10, and the damage candidate image superimposed on the photographed image is based on the damage score. Can be selected arbitrarily.
Thus, for example, when displaying an extraction result image with the user terminal 10 having low information processing ability although image synthesis is possible like a portable terminal, the number of stages of the damage score is reduced to carry the damage candidate image By transmitting to the terminal, the load of information processing in the portable terminal can be reduced.

(3) Configuration of Damage Extraction Server 20 The damage extraction server 20 is an information processing apparatus managed by a road management company or the like to which a worker belongs, and is on the photographed image of the structure received from the user terminal 10 described above. The damage candidate image, which is an image estimated to be damage such as a crack generated in the structure, is generated based on the feature points of (1), and the damage candidate image or the like is transmitted to the user terminal 10.
When generating the damage candidate image, the damage extraction server 20 first divides the photographed image into minute sections. The minute section is formed, for example, by dividing the photographed image into a matrix of vertical m mass × horizontal n mass (m, n: an integer of 1 or more).
The damage extraction server 20 calculates a damage score, which is a score representing the height of the possibility that each of the extracted damage candidate images is the actual damage, for each of the minute sections, and generates a damage candidate image for each of the minute sections. Match.

  The damage score is, as described above, a score representing the likelihood that the damage candidate image is actually damaged, and, for example, the lightness of the damage candidate image extracted based on the lightness or color tone of the captured image Alternatively, it is calculated based on the color tone, the shape of the damage candidate image, and the like.

Here, an example of calculation of the damage score will be described.
For example, the raw score of the damage score is calculated based on the following equation (1).
Damage score score = aX1 + bX2 + cX3 +... . . Formula (1)
The above a, b, c, ... are arbitrary constants.
For example, X1 is the lightness difference of the damage candidate image, X2 is the average lightness of the entire photographed image from which the damage candidate image is extracted, and X3 is an index representing the linearity of the damage candidate image.
In the above equation, if a = -1, b = 1 and c = 1, for example, the lower the lightness X1 of the damage candidate image (the darker), the higher the damage score, and the damage candidate image It can be estimated that the possibility of representing actual damage is high, but if the average brightness X2 of the entire captured image is high, not only the damage candidate image but also the entire captured image has low brightness (dark), so the damage score is low. It is calculated.
Further, in the photographed image of the structure, in addition to damage such as a crack, a non-damage such as a mark of a formwork or other material may be reflected. Generally, since the trace of the concrete formwork is highly linear, it is highly possible that the damage candidate image is not damaged if the linearity X3 of the damage candidate image is high. Therefore, the damage score is calculated low in that case.
In addition, the calculation example of the said damage score is an example to the last as mentioned above, and a calculation method is not limited to this method.

When the damage extraction server 20 calculates the raw score of the damage score as described above, the damage extraction server 20 calculates the damage score from the raw score based on the damage score conversion TB 226 described later.
The damage score is represented by a predetermined number of stages, and is represented by, for example, 100 stages of 1 to 100 or 10 stages of 1 to 10 depending on the model of the user terminal 10 (model information of the user terminal DB 221). .

FIG. 3 is a diagram showing the configuration of the damage extraction server 20 according to the first embodiment of this invention.
As shown in the figure, the damage extraction server 20 includes a control unit 21 configured of a CPU or the like to control the overall operation of the damage extraction server 20 and a photographed image of a structure received from the user terminal 10 and the photographed image. The information storage unit 22 stores various information and the like, and the communication unit 23 that transmits and receives information via the network.

FIG. 4 is a view showing each database stored in the information storage unit 22 of the damage extraction server 20 according to the first embodiment of this invention.
As shown in the figure, the information storage unit 22 of the damage extraction server 20 receives a user terminal database (DB) 221 for managing information related to the user terminal 10 operated by the worker, and the damage extraction server 20 received from the user terminal 10 A captured image database (DB) 222 for managing captured images of the structure, and a damage candidate image database (DB) 223 for managing damage candidate images extracted based on the captured image of the structure And a damage score conversion table (TB) 226 for converting the number of damage scores into the specified number of stages of damage scores.
The information storage unit 22 stores a photographed image, a damage candidate image, an extraction result image, and the like in addition to the respective databases 221 to 223 and the table 226.

FIG. 5 is a diagram showing an example of a data configuration of the user terminal DB 221 in the first embodiment of the present invention.
The user terminal DB 221 is a database for managing, for each user terminal 10, model information such as the type and quality of information that the damage extraction server 20 transmits to the user terminal 10.
As shown in the figure, the user terminal DB 221 estimates whether or not the damage candidate image can be synthesized by the user terminal 10, the number of stages of damage points used in the user terminal 10, and the damage candidate image as actual damage. The threshold value of the damage score serving as a reference for the above, the damage score serving as a reference for transmitting the damage candidate image to the user terminal 10 (transmission damage score), and the upper limit of the resolution of the image displayed on the user terminal 10 It manages in association with the user terminal ID specific to the user terminal 10.

FIG. 6 is a view showing an example of the data configuration of the imaging DB 222 according to the first embodiment of the present invention.
The imaging DB 222 is a database for managing imaging information related to a captured image that the damage extraction server 20 receives from the user terminal 10.
As shown in the figure, the shooting DB 222 sets the shooting time, the shooting position represented by latitude / longitude information, etc., and the user terminal ID of the user terminal 10 that has sent the shooting image to the shooting image ID unique to each shooting image. Manage in association.

FIG. 7 is a diagram showing an example of a data configuration of the damage candidate DB 223 according to the first embodiment of this invention.
The damage candidate DB 223 is a database for managing the damage candidate image extracted based on the feature point of the captured image received from the user terminal 10 by the damage extraction server 20.
As shown in the figure, the damage candidate DB 223 includes the photographed image ID of the photographed image from which the damage candidate image is extracted, the raw score of the number of damage points related to the damage candidate image, and the photographing time of the photographed image from which the damage candidate image is extracted. , Position information of the minute section in which the damage candidate image is present in the photographed image from which the damage candidate image is extracted, and the user terminal ID of the user terminal 10 that transmitted the photographed image from which the damage candidate image is extracted; It manages in association with the damage candidate image ID unique to the candidate image.

FIG. 8 is a view showing an example of the data configuration of the damage score conversion TB 226 according to the first embodiment of the present invention.
The damage point conversion TB 226 is a table for converting the raw points of the damage points calculated by the damage extraction server 20 into damage points of the number of stages requested by the user terminal 10.
As shown in the figure, in the damage score conversion TB 226, the raw score of the damage score, the damage score when the raw score is represented in 100 steps, and the damage score when the raw score is represented in 10 steps are mutually different. It is matched and recorded.
Note that the number 10, 100 of damage points is merely an example, and the present invention is not limited to this.

(Operation of the first embodiment)
(1) Operation of registering model information The worker registers model information of the user terminal 10 in advance before extracting damage to a structure, which is suitable for the information processing capability of the user terminal 10 and the communication environment. The extraction result image can be displayed on the user terminal 10 by the above method.
FIG. 9 is a sequence chart showing the flow of the registration operation in the first embodiment of the present invention.
The following description will be made along the figure.

First, the worker operates the operation unit 15 of the user terminal 10 to input model information (step S101).
The user terminal 10 stores the input model information in the information storage unit 12 (step S102), and transmits it to the damage extraction server 20 (step S103).

When receiving the model information from the user terminal 10, the damage extraction server 20 stores the information in the information storage unit 22 (step S104).
This is the end of the model information registration operation.

Thus, by registering model information in the user terminal 10 and the damage extraction server 20 in advance, the damage extraction server 20 extracts the extraction result image by a method suitable for the information processing capability of the user terminal 10 and the communication environment. Will be able to offer.
For example, when the user terminal 10 is configured of a PC and a camera, and the information processing ability and the communication ability are high, the user terminal 10 executes an extraction result image generation process, and the user terminal 10 has a communication function. When the information processing ability and the communication ability can not be expected so much as in the case of only the camera, the damage extraction server 20 instead of the user terminal 10 executes the generation process of the extraction result image, thereby generating the extraction result image It is possible to efficiently distribute the burden of the entire system involved in

  In addition, when the user terminal 10 is lower in information processing ability and communication function than a PC like a portable terminal, it is related to information processing and communication of the user terminal 10 by thinning out and reducing the number of stages of damage points. The burden can be reduced.

(2) Damage Extraction Operation FIGS. 10 to 13 are sequence charts showing the flow of the damage extraction operation by the damage extraction system according to the first embodiment of the present invention.
In the following, along the figure, the operation of the damage extraction system which extracts the damage of the structure based on the image of the structure taken by the worker and provides the information on the damage of the structure to the worker side Advance the explanation.

First, the worker shoots a structure using the user terminal 10 (the shooting unit 16) (step S201).
The user terminal 10 associates the shooting image ID unique to the shooting image, the shooting time, and the shooting position (latitude / longitude information etc.) with the shooting image of the structure generated at the time of shooting. It stores in the information storage part 12 (step S202).

In order to transmit the photographed image to the damage extraction server 20 and receive provision of the extraction result of damage, the user terminal 10 transmits the user terminal ID or device information unique to the user terminal 10 to the damage extraction server 20 together with the photographed image. There is a need to.
Therefore, here, the user terminal 10 determines whether or not the worker operates the operation unit 15 to transmit the user terminal ID unique to the user terminal 10 to the damage extraction server 20 (step S203). .
If transmission of the user terminal ID is selected (Yes at Step S203), the user terminal 10 reads the user terminal ID unique to the own machine stored in the information storage unit 22, and reads the read user terminal ID Are associated with the imaging information (step S204), and imaging information associated with the user terminal ID is transmitted to the damage extraction server 20 (step S205).

On the other hand, when it is selected not to transmit the user terminal ID (Step S203 / No), the operator operates the operation unit 15 to input the device information (Step S206).
In this case, the user terminal 10 associates the input device information with the imaging information (step S207), and transmits the imaging information in which the user terminal ID is associated to the damage extraction server 20 (step S205).

  When the damage extraction server 20 receives from the user terminal 10 the imaging information associated with the user terminal ID or the device information, the damage extraction server 20 stores the received imaging information in the imaging DB 222 (step S208).

Next, the damage extraction server 20 determines whether the received imaging information includes the user terminal ID (step S209).
Here, when it is determined that the user terminal ID is included (step S 209 / Yes), the damage extraction server 20 refers to the user terminal DB 221 and determines the device information associated with the user terminal ID. It extracts (Step S210).
On the other hand, when it is determined that the user terminal ID is not included in the received imaging information (step S209 / No), device information is extracted from the imaging information (step S211).

Next, the damage extraction server 20 divides the image area of the photographed image included in the photographing information into a plurality of minute sections, detects a feature point in each minute section, and detects a damage point based on the detected feature points. The damage candidate image estimated to be will be extracted (step S212).
The extraction processing of a damage candidate image based on this feature point is, for example, an image based on a general feature point, such as extracting a set of pixel regions whose brightness is lower (darker) by a predetermined value or more than surrounding lightness. Since the process is the same as the extraction process, the detailed description is omitted.

  Next, the damage extraction server 20 writes in the damage candidate DB 223 as damage candidate information for each minute section together with the extracted damage candidate image, shooting timing, and position information on the captured image (step S213).

Next, the damage extraction server 20 measures the width and length of each damage candidate image based on the number of pixels constituting the extracted damage candidate image and the position information, and the size of the crack in each minute section. The crack rate representing the length is calculated, and the measured or calculated information is written into the damage candidate DB 223 in association with the corresponding minute section (step S214).
The method of measuring the width and length of the image is also the same as the conventional general method, and thus the description thereof is omitted.

  Next, the damage extraction server 20 calculates a damage score for the damage candidate image extracted for each minute section, and writes the damage score calculated for each minute section in the damage candidate DB 223 for each minute section (step S215). .

Next, the damage extraction server 20 refers to “type of transmission information” in the device information extracted in step S210 or S211, and determines whether to transmit the damage score to the user terminal 10 (step S216).
Here, when it is determined that the damage score is to be transmitted to the user terminal 10 in the “type of transmission information” of the device information (step S216 / Yes), the damage extraction server 20 determines that “the number of damage score stages” in the device information. It is determined whether or not the number of steps (eg, 10 steps) smaller than the original number of steps (eg, 100 steps) is designated as the number of damage points (step S 217).
Here, when it is determined that the number of steps smaller than the original number of steps is specified (step S 217 / Yes), the damage extraction server 20 specifies the number of damage points based on the damage points conversion table 226. The damage extraction server 20 transmits the damage candidate image and the converted damage score for each minute section as damage candidate information to the user terminal 10 (step S219).
On the other hand, when it is determined that the number of stages smaller than the original number of stages is not specified (Step S217 / No), the damage extraction server 20 does not execute conversion of the number of stages of damage scores as it is, Damage candidate information of a damage candidate image for each minute section, a photographed image ID of a photographed image obtained by extracting the damage candidate image, position information on the photographed image of the damage candidate image, and the number of damage points of the damage candidate image And transmit to the user terminal 10 (step S219).
At this time, the damage extraction server 20 refers to the “transmission damage score” in the user terminal DB 221 and does not transmit damage candidate images of damage scores lower than the transmission damage score to the user terminal 10.

When the user terminal 10 receives damage candidate information from the damage extraction server 20, the user terminal 10 extracts a photographed image ID from the damage candidate information, and reads a photographed image of the photographed image ID from the information storage unit 12 (step S231).
Next, the user terminal 10 extracts the threshold value of the number of damage points from the set device information stored in the information storage unit 12 (step S232).
Then, the user terminal 10 compares the damage score included in the received damage candidate information with the threshold of the extracted damage score (step S233), and for the minute section of the damage score equal to or more than the threshold (step S233). / Yes), based on the position information included in the damage candidate information, the damage candidate image of the minute section is superimposed and synthesized on the read-out captured image (step S234).
On the other hand, for a minute section having a value lower than the threshold value, the damage candidate image is not superimposed on the photographed image.
The user terminal 10 repeats these processes for all minute sections (step S235), generates an extraction result image (step S236), and displays the image on the display unit 14 (step S237).

The operator operates the operation unit 15 when it is determined that the display of the damage candidate image is necessary by comparing the actual damage part on the photographed image with the damage candidate image in the displayed extraction result image. The threshold value of the damage score is newly input (step S238).
For example, when the worker determines that the actual damaged part is not displayed as a damage candidate image in the extraction result image on the display unit 14, the number of damage points registered in the information storage unit 12 as device information The damage score threshold lower than the threshold is input to increase the number of damage candidate images displayed to approximate the actual damage status of the building.
On the other hand, when it is determined that the worker in the extraction result image does not actually display the damaged part as the damage candidate image, the number of damage points registered in the information storage unit 12 as the device information The damage score threshold higher than the threshold is input to reduce the number of damage candidate images displayed to approximate the actual damage status of the building.

  As described above, when a new damage score threshold is input (step S238 / Yes), the user terminal 10 proceeds to step S233, and similarly extracts the extraction result based on the newly input damage score threshold. Perform image generation again.

When the damage extraction server 20 determines that the damage score is not to be transmitted to the user terminal 10 with reference to “type of transmission information” in the device information in step S216 (step S216 / No), the damage extraction is performed. The server 20 extracts the threshold value of the damage score included in the device information (step S251).
Next, the damage extraction server 20 compares the damage score of each of the minute sections with the threshold of the extracted damage score (step S252), and for the minute section of the damage score equal to or more than the threshold (step S252 / Yes And) superimposing and synthesizing the damage candidate image of the minute section on the photographed image received from the user terminal 10 based on the position information of the damage candidate image (step S253).
On the other hand, for a minute section having a value lower than the threshold value, the damage candidate image is not superimposed on the captured image.
The damage extraction server 20 repeats these processes in all minute sections (step S254), generates an extraction result image (step S255), and transmits the image to the user terminal 10 (step S256).

  When the user terminal 10 receives the extraction result image from the damage extraction server 20, the user terminal 10 displays the image on the display unit 14 (step S257).

  The operator operates the operation unit 15 when it is determined that the display of the damage candidate image is necessary by comparing the actual damage part on the photographed image with the damage candidate image in the displayed extraction result image. The threshold value of the damage score is newly input (step S258 / Yes), and the user terminal 10 transmits the threshold value of the damage score thus input to the damage extraction server 20 (step S259).

When the damage extraction server 20 newly receives the threshold for the damage score (step S260 / Yes), the process proceeds to step S252, and similarly generates the extraction result image again based on the received threshold for the damage score.
This is the end of the operation.

  As described above, the damage extraction server 20 provides the information related to the damage to the structure to the user terminal 10 based on the model information in which the information processing capability of the user terminal 10 and the content of the communication environment are reflected. The load of information processing can be efficiently distributed as the entire system, and the user terminal 10 can easily display an image representing damage to a structure.

Further, as described above, the user terminal 10 can select a damage candidate image to be superimposed and displayed on the captured image by changing the threshold value of the damage score.
FIGS. 14-16 is a figure which shows an example of a change of the damage candidate image displayed by the change of the threshold value of the damage number | score in the 1st Embodiment of this invention.
In these figures, the rectangular frame shows the extraction result image, the thick curve in the frame shows a crack, and the thin straight line shows the trace of the concrete form.

FIG. 14 shows an example of the extraction result image displayed first on the user terminal 10.
The worker confirms the extraction result image of FIG. 14 and as a result of comparing the damage candidate image on the extraction result image (photographed image) with the actual damage location, a lot of mold frame traces that are not actual damage are displayed. If you want to delete them, enter the damage score threshold higher than the current one.
In this case, as shown in FIG. 15, a damage candidate image having a damage score lower than the newly input threshold is not displayed, and an image in a state close to the actual damage can be acquired.

On the other hand, the worker confirms the extraction result image of FIG. 14 and as a result of comparing the damage candidate image on the extraction result image (photographed image) with the actual damage location, the actual damage is not displayed. If you want to add and display, enter the threshold of damage score lower than the current one.
In this case, as shown in FIG. 16, damage candidate images having damage scores lower than the previous threshold and not less than the newly input threshold are additionally displayed, and an image in a state close to the actual damage may be acquired. it can.

(3) Operation of Checking Aging of Structure As described above, the damage extraction server 20 associates information such as the imaging time and imaging position with the captured image in the imaging DB 222 and stores the information as imaging information.
In addition, the damage extraction server 20 stores the damage candidate images together with their width, length, and crack rate in the damage candidate DB 223 in association with the imaging time, the position information, and the like.
When the worker operates the operation unit 15 of the user terminal 10 to input the photographed image ID, the photographing time, the position information, etc. and transmit it to the damage extraction server 20, the damage extraction server 20 detects the corresponding photographed image or damage The candidate image is transmitted to the user terminal 10. At this time, the damage extraction server 20 can also transmit, to the user terminal 10, photographed images or damage candidate images of different photographing timings of the same position information.

When the user terminal 10 receives a photographed image or a damage candidate image from the damage extraction server 20, the user terminal 10 displays the image on the display unit 14.
By referring to the displayed image, the worker can easily observe the secular change of damage such as a crack in the structure.

(4) Image connection operation The user terminal 10 transmits shooting information including a plurality of shot images in the same structure to the damage extraction server 20, and extracts a damage of a connected image (panoramic image) in which the plurality of shot images are linked. It can be acquired from the server 20.
FIG. 17 is a sequence chart showing a flow of linked image generation operation according to the first embodiment of the present invention.
The following description will be made along the figure.

  The process until the damage extraction server 20 receives the shooting information including the plurality of shot images from the user terminal 10 and extracts the model information is the same as the process of steps S201 to S211 of the above-described damage extraction operation process, I omit explanation.

The damage extraction server 20 extracts feature points (cracks and the like) of a plurality of photographed images included in the photographing information received from the user terminal 10 (step S301).
Next, the damage extraction server 20 collates the plurality of extracted feature points and detects a common feature point among the feature points (step S302).
For this common feature point detection process, a general detection method based on lightness, color tone or the like may be used.

Next, the damage extraction server 20 generates a connected image by linking each captured image by scaling, rotating, etc. so as to overlap the extracted common feature points (step S303).
The processing such as enlargement / reduction and rotation when connecting the two images is the same as that of a conventional general image processing method, and thus the detailed description thereof is omitted.

FIGS. 18 and 19 are diagrams showing a method of generating a concatenated image in the first embodiment of the present invention.
As shown in FIG. 18, the feature points of the two captured images 1 and 2 are extracted, and since the regions captured by the captured images 1 and 2 overlap, the feature points common to both captured images 1 and 2 are extracted. If extracted, the damage extraction server 20 connects the right end portion of the captured image 1 and the left end portion of the captured image 2 so as to superimpose these common feature points, as shown in FIG.
The damage extraction server 20 connects the photographed images having such a common area to generate a connected image in which the entire structure is displayed.

Here, it returns to FIG. 17 again and advances explanation.
As described above, when the damage extraction server 20 generates a connected image, it detects resolution information from model information, and determines whether the resolution of the generated connected image is different from the resolution specified by the resolution information. Judge (step S304), and if different (step S304 / Yes), convert the generated concatenated image into the resolution specified in the resolution information (step S305), and convert the resolution-converted concatenated image to the user terminal It transmits to 10 (step S306).
The method of converting the resolution is the same as the conventional method of converting the resolution of the image, and thus the detailed description thereof is omitted.

  On the other hand, when the two resolutions are not different (step S304 / No), the damage extraction server 20 transmits the generated concatenated image to the user terminal 10 without performing the resolution conversion process (step S306).

When receiving the connected image, the user terminal 10 displays it on the display unit 14 (step S307).
The operation ends here.

Thus, the worker can easily acquire the entire image of the building at a resolution according to the information processing capability of the user terminal 10.
For example, when the user terminal 10 is a device having a relatively high information processing capability, such as a PC, it is possible to display a high resolution connected image having a large data size as it is.
On the other hand, when the user terminal 10 is a camera or the like having a communication function, it is used to display a low-resolution connected image with a small data size and to check whether there is an area in the structure which has not been captured. be able to.

(5) Generation Operation of CAD Drawing The damage candidate image extracted from the above-described captured image is represented by a set of points (pixels).
When the damage candidate image is to be reflected on a CAD drawing, a set of these points is connected to form vector data.
When displaying the damage candidate image on the CAD drawing, the worker uses the user terminal 10 to transmit a photographed image or the like requiring CAD drawing to the damage extraction server 20.
When the damage extraction server 20 receives the photographed image, the damage extraction server 20 executes the above-described damage extraction operation to extract a damage candidate image, and the point-to-point connection relationship between points and the luminance around the point constitute the extracted damage candidate image. From the information, points are automatically connected to each other to be vector data, and the damage candidate image converted into vector data is transmitted to the user terminal 10.
The user terminal 10 displays the damage candidate image converted into vector data on the CAD drawing.
As described above, in the present embodiment, a damage candidate image can be easily drawn and used as a CAD drawing.

(6) Report Creation Operation As described above, in the damage extraction operation, the damage candidate image is managed by the damage candidate DB 223.
In response to a request from the user terminal 10, the damage extraction server 20 can extract damage candidate images managed in the damage candidate DB 223 and automatically create a structure inspection report.

The information storage unit 22 of the damage extraction server 20 stores a template of report data in which a report item common to all workers is set from the typical example of the inspection report.
Further, in the information storage unit 22, a report item different for each worker or a report item represented by a different method even if the same item is set and stored for each worker.
The items that are different for each worker are entered by the worker using the user terminal 10 and transmitted to the damage extraction server 20, and are registered before the report creation.
By being configured in this manner, the damage extraction server 20 creates a report template unique to the worker, and automatically creates an inspection report by pouring damage candidate information or the like into the report template.
In addition, damage candidate images can be synthesized on the inspection report. The damage candidate image to be synthesized in this inspection report is set by the scale, automatically set the scale to fit within the frame on the inspection report, and the type and width of lines describing cracks etc. Similarly, it can be registered in advance in the damage extraction server 20 by using it.

  As described above, when a worker prepares an inspection report for a structure, it can be automatically created with the contents registered in advance in the damage extraction server 20, so the inspection report for the layout and contents that he / she desires It is possible to easily create a book.

(7) Learning Operation of Damage Detection As described above, the user terminal 10 displays the extraction result image on the display unit 14.
The worker confirms the displayed extraction result image, and as a result of comparing the damage candidate image such as a crack shown in the extraction result image with the actual damage location, if there is a difference between the two, the extraction is performed. The resulting image can be corrected.

The operator designates a predetermined location on the extraction result image using the operation unit 15 (mouse or other device), draws a damage image on the extraction result image and adds a damage location actually present Or modify the extraction result image by designating deletion of damage candidate images of damage which are not actually present.
The user terminal 10 temporarily stores correction information indicating this correction in the information storage unit 12 and then transmits the correction information to the damage extraction server 20.

  When the damage extraction server 20 receives the correction information from the user terminal 10, the damage extraction server 20 performs correction (addition / deletion of damage candidate image) on the corresponding extraction result image according to the correction information, and the extraction after the correction The result image, the damage candidate image and the like are stored in the information storage unit 22.

Next, the damage extraction server 20 corrects the calculation formula of the damage score score based on the correction information.
For example, the calculation formula of the raw score of the damage score is set as the following formula (1) as described above.
Damage score score = aX1 + bX2 + cX3 +... . . Formula (1)
Here, the damage extraction server 20 changes the above constants a, b, and c so as to be closest to the damage candidate image of the content indicated by the correction information.
For example, when deletion of a certain damage candidate image is designated in the correction information, the damage extraction server 20 sets the value of a from “−1” to “−1.” So that the damage of the damage candidate image is not detected. Change to 2 ".

  As described above, the damage extraction server 20 has a function to appropriately correct and calculate the calculation formula of the raw score of the damage score based on the correction information received from the user terminal 10, so that the damage result is more optimal. It is possible to effectively increase the detection accuracy to detect.

(Summary of the first embodiment)
As described above, according to the first embodiment of the present invention, the damage extraction server 20 transmits the damage candidate image and the damage score to the user terminal 10 based on the model information of the user terminal 10 Since it is determined whether the extraction result image is generated and transmitted to the user terminal 10, the load in the entire damage extraction system for displaying the image showing the damage of the concrete structure or the asphalt structure on the user terminal 10 is efficient It becomes possible to disperse | distribute well and to display the said image easily in the user terminal 10. FIG.
Therefore, a road management company etc. can perform management and maintenance of a concrete structure or an asphalt structure very easily.

  Further, according to the present embodiment, when damage candidate images are generated, damage points indicating the possibility that each damage candidate image is an actual damage are calculated. As compared with the actual captured image, it is possible to easily delete or add the damage candidate image.

  Further, according to the present embodiment, the damage extraction server 20 converts the number of stages of the damage score based on the model information and transmits it to the user terminal 10, so that the user terminal 10 generates damage candidate images. It is possible to significantly reduce the load on

  Further, according to the present embodiment, the damage extraction server 20 does not transmit damage candidate images less than a predetermined damage score (transmission damage score) to the user terminal 10, so that the user terminal 10 generates damage candidate images. It is possible to significantly reduce the load on the occasion.

  Further, according to the present embodiment, the damage extraction server 20 transmits a connected image (panoramic image) generated by connecting a plurality of photographed images of the same structure to the user terminal 10 based on the model information. Since the resolution of the connected image is adjusted, it is possible to provide an appropriate image according to the information processing capability, communication environment or specification of the user terminal 10.

The user terminal 10 and the damage extraction server 20 described above are mainly realized by a CPU and a program loaded in a memory. However, it is also possible to configure this device or server by any other combination of hardware and software, and the high degree of freedom of design is easily understood by those skilled in the art.
When the user terminal 10 or the damage extraction server 20 is configured as a software module group, the program is recorded on a recording medium such as an optical recording medium, a magnetic recording medium, a magneto-optical recording medium, or a semiconductor. It may be loaded from a recording medium, or may be loaded from an external device connected via a predetermined network.

  The above embodiment is an example of the preferred embodiment of the present invention, and the embodiment of the present invention is not limited to this, and various modifications can be made without departing from the scope of the present invention. It is possible to

For example, the damage extraction system in the present embodiment is applicable to any concrete structure, asphalt structure or other structure, and the road surface of a road, a bridge, a guardrail, a median divider, a tunnel, a dike, a dam It can be applied to any structure such as
In addition, as described above, the photographing unit 16 of the user terminal 10 may be configured by a camera or a mobile terminal operated by a worker, or may be mounted on a vehicle, a ship, or the like. For example, the imaging unit 16 may be mounted on a vehicle, and the road surface may be photographed while traveling to check whether the road surface is damaged.

  Furthermore, according to the present embodiment, an example has been described in which the worker shoots a structure using a camera or a mobile terminal that constitutes the user terminal 10, but it is also possible to shoot using other methods.

DESCRIPTION OF SYMBOLS 10 user terminal 11, 21 control part 12, 22 information storage part 13, 23 communication part 14 display part 15 operation part 16 imaging part 100 network 221 user terminal DB
222 Shooting DB
223 Damage candidate DB
226 Damage point conversion TB

Claims (4)

A damage extraction system comprising: a user terminal operated by a worker who inspects a structure; and a damage extraction server connected to the user terminal via a network and extracting damage caused to the structure And
The damage extraction server is
Based on the captured image of the structure received from said user terminal, damage candidate image showing an image that is estimated to be damage caused to the structure generated for each small section of the captured image, the captured image The model information of the user terminal is acquired based on the information included in the?, And whether or not the generated damage candidate image is to be transmitted to the user terminal is determined based on the acquired model information, and as a result, the damage is determined When it is determined to transmit the candidate image, the generated damage candidate image is transmitted to the user terminal with a damage score indicating possibility of actual damage,
The user terminal is
When the photographed image is transmitted to the damage extraction server, a threshold value of the damage score serving as a reference when recognizing the damage candidate image as damage is input, and the damage candidate image is received from the damage extraction server, The damage candidate image to be displayed is determined based on the threshold value of the damage score input, and the determined damage candidate image is synthesized and displayed at the corresponding position of the photographed image stored in the own machine. Damage extraction system. The damage extraction system according to claim 1 , wherein the damage extraction server reduces the number of stages of the damage score based on the model information. The damage extraction system according to claim 1 or 2 , wherein the damage extraction server transmits only damage candidate images having a predetermined damage score or more to the user terminal based on the model information. A damage extraction system comprising: a user terminal operated by a worker who inspects a structure; and a damage extraction server connected to the user terminal via a network and extracting damage caused to the structure And
  The damage extraction server is
  Based on the photographed image of the structure received from the user terminal, a damage candidate image indicating an image estimated to be damage to the structure is generated for each minute section of the photographed image, and the damage is generated. The candidate image is given a damage score serving as a reference for recognizing damage, the model information of the user terminal is acquired based on the information included in the captured image, and the generation is performed based on the acquired model information. It is determined whether to transmit the damaged candidate image to the user terminal, and as a result, when it is determined not to transmit the damaged candidate image, the damage score and the threshold value of the damage score received from the user terminal The damage candidate image to be synthesized is determined, and the determined damage candidate image is synthesized at the corresponding position of the photographed image received from the user terminal, and the synthesized image is used as the user To send to the end,
  The user terminal is
Input the threshold value of the damage score,
  A damage extraction system, comprising: transmitting the photographed image and the threshold value of the input damage score to the damage extraction server; and receiving the synthesized image from the damage extraction server, displaying the synthesized image.

Images