JP6807093B1 - Inspection system and management server, program, crack information provision method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an inspection system and a management server by an unmanned aerial vehicle suitable for inspection of a concrete revetment. According to the present invention, a specialized crack learning model capable of identifying a specific type of crack appearing on a concrete bank, and a plurality of cracks other than the specific type of crack appearing on the concrete bank. A learning model storage unit that stores a non-specialized learning model that can identify cracks in a species, and a statistical information storage unit that stores a range in which the cracks of the specific species frequently appear as statistical information in a concrete bank. , A detection target range setting unit that sets a different detection target range based on the statistical information as a detection target range to which the specialized learning model and the non-specialized learning model are applied in a photographed image taken by an unmanned vehicle. The crack information output unit that outputs the first crack information and the second crack information obtained by applying the specialized learning model and the non-specialized learning model to each detection target range, and the first crack information output unit. Provided is an inspection system including a detection result output unit that outputs detection result information including crack information and a second crack information. [Selection diagram] FIG. 11

Description

The present invention relates to a concrete revetment inspection system by an air vehicle, a management server, a program, and a method for providing crack information.

In recent years, flying objects such as drones and unmanned aerial vehicles (UAVs) (hereinafter collectively referred to as "unmanned aerial vehicles") have begun to be used in industry. Under these circumstances, Patent Document 1 discloses a system in which a flying object sequentially photographs an imaged object at a plurality of preset waypoints.

Japanese Unexamined Patent Publication No. 2014-089160

Here, in the conventional inspection, especially the inspection of the concrete revetment, the personnel on board the ship manually take pictures, etc., and a large amount of man-hours and load are required especially in the part where there is an obstacle such as a floating matter recovery net or a bridge. It was hanging. It would be very useful if a system could be constructed to inspect such concrete revetments with an unmanned aerial vehicle.

However, it cannot be said that the inspection system for dealing with factors such as the location peculiar to the concrete revetment is sufficiently constructed at present.

The present invention has been made in view of such a background, and an object of the present invention is to provide an inspection system and a management server by an unmanned aerial vehicle suitable for inspection of a concrete revetment.

The main inventions of the present invention for solving the above problems are a specialized crack learning model capable of identifying a specific type of crack appearing on a concrete bank, and a crack other than the specific type of crack. A learning model storage unit that stores a non-specialized learning model that can identify multiple types of cracks that appear on the concrete bank, and a range of high frequency of appearance of the specific types of cracks on the concrete bank is stored as statistical information. Detection that sets different detection target ranges based on the statistical information as the detection target range to which the specialized learning model and the non-specialized learning model are applied in the statistical information storage unit and the captured image taken by the unmanned flying object. The target range setting unit and the crack information output unit that outputs the first crack information and the second crack information obtained by applying the specialized learning model and the non-specialized learning model to each detection target range, respectively. An inspection system including a detection result output unit that outputs detection result information including the first crack information and the second crack information.

According to the present invention, it is possible to provide an inspection system and a management server by an unmanned aerial vehicle, which is particularly suitable for inspection of a concrete revetment.

It is a figure which shows the whole structure of the inspection system which concerns on embodiment of this invention. It is a figure which shows the system structure of the inspection system which concerns on embodiment of this invention. It is a block diagram which shows the hardware configuration of the management server of FIG. It is a block diagram which shows the hardware configuration of the user terminal of FIG. It is a block diagram which shows the hardware composition of the flying object of FIG. It is a block diagram which shows the function of the management server of FIG. This is an example of a cracked image according to an embodiment of the present invention. This is an example of the detection result for each crack type according to the embodiment of the present invention. This is an example of a detection result by specialized learning according to the embodiment of the present invention. This is an example of a crack detection image and a detection result image according to the embodiment of the present invention. It is a flowchart of the inspection system which concerns on embodiment of this invention.

The contents of the embodiments of the present invention will be described in a list. The inspection system and the management server according to the embodiment of the present invention have the following configurations.
[Item 1]
A specialized crack learning model that can identify specific types of cracks that appear on concrete shores, and multiple types of cracks that appear on concrete shores that are other than the specific types of cracks. A learning model storage unit that stores non-specialized learning models,
In the concrete revetment, a statistical information storage unit that stores the range in which the specific type of cracks frequently appear as statistical information,
In the captured image taken by the unmanned aerial vehicle, the detection target range setting unit that sets a different detection target range based on the statistical information as the detection target range to which the specialized learning model and the non-specialized learning model are applied.
A crack information output unit that outputs the first crack information and the second crack information obtained by applying the specialized learning model and the non-specialized learning model to each detection target range, and
A detection result output unit that outputs detection result information including the first crack information and the second crack information, and a detection result output unit.
Inspection system equipped with.
[Item 2]
The specific type of crack is a crack generated by surface deterioration due to wind waves.
The inspection system according to claim 1, wherein the inspection system is characterized in that.
[Item 3]
The range in which the specific type of cracks frequently appear is within -1 m or more from the ground in the concrete revetment.
The inspection system according to claim 1 or 2.
[Item 4]
The range in which the specific type of cracks frequently appear is within the range of -2 m or more from the ground in the concrete revetment.
The inspection system according to claim 1 or 2.
[Item 5]
A specialized crack learning model that can identify specific types of cracks that appear on concrete shores, and multiple types of cracks that appear on concrete shores that are other than the specific types of cracks. A learning model storage unit that stores non-specialized learning models,
In the concrete revetment, a statistical information storage unit that stores the range in which the specific type of cracks frequently appear as statistical information,
In the captured image taken by the unmanned aerial vehicle, the detection target range setting unit that sets a different detection target range based on the statistical information as the detection target range to which the specialized learning model and the non-specialized learning model are applied.
A crack information output unit that outputs the first crack information and the second crack information obtained by applying the specialized learning model and the non-specialized learning model to each detection target range, and
A detection result output unit that outputs detection result information including the first crack information and the second crack information, and a detection result output unit.
Management server with.
[Item 6]
In the management server program
The management server
A specialized crack learning model that can identify specific types of cracks that appear on concrete shores, and multiple types of cracks that appear on concrete shores that are other than the specific types of cracks. A learning model storage unit that stores non-specialized learning models,
In the concrete revetment, a statistical information storage unit that stores the range in which the specific type of cracks frequently appear as statistical information,
In the captured image taken by the unmanned aerial vehicle, the detection target range setting unit that sets a different detection target range based on the statistical information as the detection target range to which the specialized learning model and the non-specialized learning model are applied.
A crack information output unit that outputs the first crack information and the second crack information obtained by applying the specialized learning model and the non-specialized learning model to each detection target range, and
A detection result output unit that outputs detection result information including the first crack information and the second crack information, and a detection result output unit.
A program for the management server that is characterized by being functioning as a function.
[Item 7]
Specialized learning model and non-specialized for the detection target range set based on the statistical information that stores the range where the appearance frequency of cracks of a specific type is high in the photographed image of the concrete revetment taken by the unmanned aircraft. It is a detection result information providing method for providing detection result information including first crack information and second crack information obtained by applying a learning model to each detection target range from a management server.
The specialized crack learning model is a learning model capable of identifying a specific type of crack that appears on a concrete revetment.
The non-specialized crack learning model is a learning model that is a crack other than the specific type of crack and can identify a plurality of types of cracks appearing on the concrete revetment.
A method for providing detection result information.

<Details of the embodiment>
Hereinafter, embodiments of an unmanned aerial vehicle inspection system according to an embodiment of the present invention and a management server for implementing the inspection system will be described. In the accompanying drawings, the same or similar elements are given the same or similar reference numerals and names, and duplicate description of the same or similar elements may be omitted in the description of each embodiment. In addition, the features shown in each embodiment can be applied to other embodiments as long as they do not contradict each other.

<Overall configuration>
As shown in FIG. 1, in the inspection system of the present embodiment, the concrete revetment is photographed by the unmanned vehicle 4 flying on the flight path set in advance by the user, and the concrete revetment is photographed based on the photographed image. It detects cracks.

In the past, a ship was floated on the water surface (for example, the sea such as a bay) as shown in Fig. 1, and the state of cracks in the concrete revetment was manually checked by the boarding personnel, but in this inspection system, As described above, cracks are confirmed using the captured image taken by the unmanned aircraft 4. In particular, as a result of diligent studies, it was found that in cracks in such an environment, the types of cracks that are difficult to detect by normal machine learning can be identified, and the tendency of the specific cracks to easily occur is found. Therefore, it is possible to confirm the occurrence of cracks accurately by using the specialized learning model learned only for the relevant type, especially for a specific part of the captured image.

<System configuration>
As shown in FIG. 2, the inspection system according to the present embodiment includes a management server 1, a user terminal 2, and an unmanned aerial vehicle 4. The management server 1, the user terminal 2, and the unmanned aerial vehicle 4 are connected to each other so as to be able to communicate with each other via the network NW. The configuration shown is an example, and is not limited to this.

<Management server 1>
FIG. 3 is a diagram showing a hardware configuration of the management server 1. The illustrated configuration is an example, and may have other configurations.

As shown in the figure, the management server 1 is connected to the user terminal 2 and the unmanned aerial vehicle 4 to form a part of the system. The management server 1 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

The management server 1 includes at least a processor 10, a memory 11, a storage 12, a transmission / reception unit 13, an input / output unit 14, and the like, and these are electrically connected to each other through a bus 15.

The processor 10 is an arithmetic unit that controls the operation of the entire management server 1, controls the transmission and reception of data between each element, and performs information processing and the like necessary for application execution and authentication processing. For example, the processor 10 is a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit), and executes each information processing by executing a program or the like for the system stored in the storage 12 and expanded in the memory 11. ..

The memory 11 includes a main memory composed of a volatile storage device such as a DRAM (Dynamic Random Access Memory) and an auxiliary memory composed of a non-volatile storage device such as a flash memory or an HDD (Hard Disk Drive). .. The memory 11 is used as a work area or the like of the processor 10, and also stores a BIOS (Basic Input / Output System) executed when the management server 1 is started, various setting information, and the like.

The storage 12 stores various programs such as application programs. A database storing data used for each process may be built in the storage 12. Further, the storage unit 130 described later may be provided in a part of the storage area.

The transmission / reception unit 13 connects the management server 1 to the network NW. The transmission / reception unit 13 may be provided with a short-range communication interface of Bluetooth (registered trademark) and BLE (Bluetooth Low Energy).

The input / output unit 14 is an information input device such as a keyboard and a mouse, and an output device such as a display.

The bus 15 is commonly connected to each of the above elements and transmits, for example, an address signal, a data signal, and various control signals.

<User terminal 2>
The user terminal 2 shown in FIG. 4 also includes a processor 20, a memory 21, a storage 22, a transmission / reception unit 23, an input / output unit 24, and the like, which are electrically connected to each other through a bus 25. Since the functions of each element can be configured in the same manner as the management server 1 described above, detailed description of each element will be omitted.

<Unmanned aerial vehicle 4>
FIG. 5 is a block diagram showing a hardware configuration of the unmanned aerial vehicle 4. The flight controller 41 can have one or more processors such as a programmable processor (eg, a central processing unit (CPU)).

Further, the flight controller 41 has a memory 411 and can access the memory. Memory 411 stores logic, code, and / or program instructions that the flight controller can execute to perform one or more steps. Further, the flight controller 41 may include sensors 412 such as an inertial sensor (accelerometer, gyro sensor), GPS sensor, proximity sensor (for example, rider) and the like.

Memory 411 may include, for example, a separable medium such as an SD card or random access memory (RAM) or an external storage device. The data acquired from the cameras / sensors 42 may be directly transmitted and stored in the memory 411. For example, still image / moving image data taken by a camera or the like may be recorded in the internal memory or an external memory, but the present invention is not limited to this, and at least the management server 1 or the management server 1 or the internal memory may be recorded from the camera / sensor 42 or the internal memory via the network NW. It may be recorded in any one of the user terminals 2. The camera 42 is installed on the unmanned aerial vehicle 4 via the gimbal 43.

The flight controller 41 includes a control module (not shown) configured to control the state of the unmanned aerial vehicle 4. For example, the control module adjusts the spatial arrangement, velocity, and / or acceleration of an unmanned vehicle 4 with 6 degrees of freedom (translational motion x, y and z, and rotational motion θ _x , θ _y and θ _z ). Therefore, the propulsion mechanism (motor 45, etc.) of the unmanned vehicle 4 is controlled via the ESC44 (Electric Speed Module). The propeller 46 is rotated by the motor 45 supplied from the battery 48 to generate the lift of the unmanned aerial vehicle 4. The control module can control one or more of the states of the mounting unit and the sensors.

The flight controller 41 is configured to transmit and / or receive data from one or more external devices (eg, transmitter / receiver (propo) 49, terminal, display device, or other remote control). It is possible to communicate with the unit 47. The transmitter / receiver 49 can use any suitable communication means such as wired communication or wireless communication.

For example, the transmission / reception unit 47 uses one or more of a local area network (LAN), a wide area network (WAN), infrared rays, wireless, WiFi, a point-to-point (P2P) network, a telecommunications network, and cloud communication. can do.

The transmission / reception unit 47 transmits and / or receives one or more of the data acquired by the sensors 42, the processing result generated by the flight controller 41, the predetermined control data, the user command from the terminal or the remote controller, and the like. be able to.

Sensors 42 according to this embodiment may include inertial sensors (accelerometers, gyro sensors), GPS sensors, proximity sensors (eg, riders), or vision / image sensors (eg, cameras).

<Management server function>
FIG. 6 is a block diagram illustrating the functions implemented in the management server 1. In the present embodiment, the management server 1 includes a communication unit 110, a flight path generation unit 120, a storage unit 130, a specialized crack learning unit 140, a non-specialized crack learning unit 150, a detection target range setting unit 160, and a crack. It includes a crack information output unit 170 and a detection result output unit 180. Further, the storage unit 130 includes various databases of a parameter information storage unit 132, a flight log storage unit 134, a learning model storage unit 136, and a statistical information storage unit 138, and the learning model storage unit 136 is a specialized crack learning model storage unit. A unit 1362 and a non-specialized crack learning model storage unit 1364 are included.

The communication unit 110 communicates with the user terminal 2 and the unmanned aerial vehicle 4. The communication unit 110 also functions as a reception unit that receives flight requests from the user terminal 2.

The flight path generation unit 120 uses parameter information (for example, waypoint position (latitude / longitude and altitude), flight speed, minimum flight altitude, waypoint arrangement interval, shooting angle, shooting angle, and imaging) of the parameter information storage unit 132. A new flight path may be automatically generated by referring to the overlap rate of images, etc.), or information about some or all waypoints may be manually set to generate the flight path. Good.

The specialized crack learning unit 140 machine-learns a crack image related to a specific crack as teacher data, and generates a specialized crack learning model. The specific cracks referred to here are, for example, cracks generated by surface deterioration due to wind waves (hereinafter referred to as "wind wave cracks"). Then, for example, a plurality of first crack images related to wind wave cracks are machine-learned as teacher data, a specialized crack learning model specialized for wind wave cracks is generated, and a specialized crack learning model is used as a learning model. It is stored in the storage unit 136. The specialized crack learning model is not limited to the specialized crack learning unit 140 in the management server 1, and may be created outside the management server 1, and the specialized crack learning model is stored from the outside. It may be stored in unit 1362.

Here, the problem of the detection rate peculiar to cracks in the concrete revetment will be described by taking as an example the case where crack detection is performed on an image such as cracks AC illustrated in FIG. 7. In particular, the crack C is a crack generated by surface deterioration due to wind waves. The types of cracks AB are not mentioned here, but they are different types of cracks from crack C.

As shown in FIG. 8, when machine learning was performed using a plurality of crack images of which the type of crack is not particularly limited as teacher data, the detection rate of cracks AB was improved by performing additional learning. However, especially for crack C, the detection rate did not improve sufficiently even if additional learning was performed.

Therefore, when a specialized crack learning model was generated in which the crack image corresponding to the crack C was trained as teacher data, the detection rate could be improved by performing the specialized learning as shown in FIG. found. In this system, in view of this result, the crack detection rate is improved by using the above-mentioned specialized learning model.

The non-specialized crack learning unit 150 machine-learns a plurality of second crack images in the concrete bank as teacher data, generates a non-specialized crack learning model, and learns the non-specialized crack learning model. Model storage unit 136 Remember in. The non-specialized crack learning model is not limited to the non-specialized crack learning unit 150 in the management server 1, and may be created outside the management server 1, and the non-specialized crack learning model is stored from the outside. It may be stored in unit 1364. Further, the second crack image is not particularly limited, and may be, for example, a crack image in which the type of crack is not limited, but other than the above-mentioned first crack image (image of wind wave crack). If it is a crack image, it is possible to generate a non-specialized crack learning model in which the difference between them becomes clear.

The detection target range setting unit 160 sets different detection target ranges as detection target ranges for detecting cracks using the above-mentioned specialized crack learning model and non-specialized crack learning model. For example, as a detection target range for detecting cracks using a non-specialized crack learning model, the entire revetment is set as a detection target range, while a non-special range for detecting cracks using a specialized crack learning model. A range narrower than the detection target range to which the cracking learning model is applied is set as the detection target range. The detection target range to which the specialized crack learning model is applied is set with reference to statistical information 138 indicating the range in which a specific type of crack (for example, wind wave crack) of a specific type of concrete revetment appears frequently. Further, it is used so that crack detection is performed on a captured image in which a part where a specific type of crack occurs frequently is removed, for example, a part where a specific type of crack occurs frequently is removed. You may.

An example of statistical information on the occurrence of wind wave cracks on a concrete revetment is, for example, information on the range in which wind wave cracks are likely to occur on a revetment. In constructing this system, research shows that wind wave cracks are likely to occur on the water surface side, and that the frequency of wind wave cracks is high in the range of -1 m or more from the ground, especially in the range of -2 m or more from the ground. It is known. Then, based on the statistical information, the crack detection accuracy is further improved by performing crack detection based on the specialized crack learning model for the processed image in which the part where the crack occurrence frequency is high is left. Is possible.

The crack information output unit 170 uses the above-mentioned specialized crack learning model and non-specialized crack learning model to obtain the first crack information and the second crack information (for example, a still image) from each detection target range. Etc.) is output.

The detection result output unit 180 is a detection image as each crack information output by the crack information output unit 170 (for example, a detection image using the (b) non-specialized crack learning model of FIG. 10 and (c) specialization. The detection image of the detection image using the crack learning model) is generated, and these detection images are superposed to generate the detection result (for example, the detection result image of FIG. 10D), and the detection result is obtained. The detection result is output to the user terminal 2.

<Example of inspection method>
An inspection method by the inspection system according to the present embodiment will be described with reference to FIG.

First, the management server 1 receives the input of predetermined parameter information by the start of the inspection, and stores the parameter information in the parameter information storage unit 132 (SQ101).

Next, the management server 1 generates a flight path (SQ102). For example, a new flight path is automatically generated by referring to the parameter information of the parameter information storage unit 132, or the user sets and generates the flight path.

Next, the unmanned aircraft 4 takes a picture of the concrete revetment at the waypoint based on the flight path and the waypoint information, and the management server 1 stores the information (for example, the taken image) acquired by the picture in the flight log storage unit 134. It is stored in (SQ103).

Next, the management server 1 sets the detection target range based on the above-mentioned statistical information by the detection target range setting unit 160. (SQ104).

Next, the management server 1 uses the specialized crack learning model and the non-specialized crack learning model by the crack information output unit 170 for the detection target range to obtain the first crack information from the captured image. And the second crack information (for example, still image) is output (SQ105).

Next, the management server 1 generates detection images (particularly, crack positions) as each crack information detected by the crack information output unit 170 (for example, as illustrated in FIG. 10). , (B) a detection image using a non-specialized crack learning model and (c) a detection image of a detection image using a specialized crack learning model), and a detection result (for example, FIG. 10) by superimposing these detected images. (D) Crack detection result image) exemplified in 1) is generated and output to the user terminal 2 (SQ106).

As described above, the present invention can provide an inspection system and a management server by an unmanned aerial vehicle suitable for inspection of a concrete revetment without undergoing inspection work by a ship or the like. In particular, as shown in FIG. 10, the cracks in the concrete revetment are composed of a plurality of types of cracks due to multiple factors, and this inspection system can detect the crack shape in the concrete revetment more accurately. It becomes.

The above-described embodiments are merely examples for facilitating the understanding of the present invention, and are not intended to limit the interpretation of the present invention. It goes without saying that the present invention can be modified and improved without departing from the spirit thereof, and the present invention includes an equivalent thereof.

1 Management server 2 User terminal 4 Unmanned aerial vehicle

Claims (7)

A specialized learning model that can identify a specific type of crack that appears on a concrete revetment, and a non- specialized model that can identify multiple types of cracks that appear on a concrete revetment that are cracks other than the specific type of crack. A learning model storage unit that stores the learning model,
In the concrete revetment, a statistical information storage unit that stores the range in which the specific type of cracks frequently appear as statistical information,
In the captured image taken by the unmanned aerial vehicle, the detection target range setting unit that sets a different detection target range based on the statistical information as the detection target range to which the specialized learning model and the non-specialized learning model are applied.
A crack information output unit that outputs the first crack information and the second crack information obtained by applying the specialized learning model and the non-specialized learning model to each detection target range, and
A detection result output unit that outputs detection result information including the first crack information and the second crack information, and a detection result output unit.
Inspection system equipped with. The specific type of crack is a crack generated by surface deterioration due to wind waves.
The inspection system according to claim 1, wherein the inspection system is characterized in that. The range in which the specific type of cracks frequently appear is within -1 m or more from the ground in the concrete revetment.
The inspection system according to any one of claims 1 or 2. The range in which the specific type of cracks frequently appear is within the range of -2 m or more from the ground in the concrete revetment.
The inspection system according to any one of claims 1 or 2. A specialized learning model that can identify a specific type of crack that appears on a concrete revetment, and a non- specialized model that can identify multiple types of cracks that appear on a concrete revetment that are cracks other than the specific type of crack. A learning model storage unit that stores the learning model,
In the concrete revetment, a statistical information storage unit that stores the range in which the specific type of cracks frequently appear as statistical information,
In the captured image taken by the unmanned aerial vehicle, the detection target range setting unit that sets a different detection target range based on the statistical information as the detection target range to which the specialized learning model and the non-specialized learning model are applied.
A crack information output unit that outputs the first crack information and the second crack information obtained by applying the specialized learning model and the non-specialized learning model to each detection target range, and
A detection result output unit that outputs detection result information including the first crack information and the second crack information, and a detection result output unit.
Management server with. In the management server program
The management server
A specialized learning model that can identify a specific type of crack that appears on a concrete revetment, and a non- specialized model that can identify multiple types of cracks that appear on a concrete revetment that are cracks other than the specific type of crack. A learning model storage unit that stores the learning model,
In the concrete revetment, a statistical information storage unit that stores the range in which the specific type of cracks frequently appear as statistical information,
In the captured image taken by the unmanned aerial vehicle, the detection target range setting unit that sets a different detection target range based on the statistical information as the detection target range to which the specialized learning model and the non-specialized learning model are applied.
A crack information output unit that outputs the first crack information and the second crack information obtained by applying the specialized learning model and the non-specialized learning model to each detection target range, and
A detection result output unit that outputs detection result information including the first crack information and the second crack information, and a detection result output unit.
A program for the management server that is characterized by being functioning as a function. Specialized learning model and non-specialized for the detection target range set based on the statistical information that stores the range where the appearance frequency of cracks of a specific type is high in the photographed image of the concrete revetment taken by the unmanned aircraft. It is a detection result information providing method for providing detection result information including first crack information and second crack information obtained by applying a learning model to each detection target range from a management server.
The specialized learning model is a learning model that can identify specific types of cracks that appear on concrete revetments.
The non-specialized learning model is a learning model that is a crack other than the specific type of crack and can identify a plurality of types of cracks appearing on the concrete revetment.
A method for providing detection result information.

Images