KR102100496B1 - System and method for finding crack in concrete using drone and laser scanner - Google Patents

Abstract

The present invention relates to crack inspection of a concrete structure (10) using a drone (20) and a laser scanner (30). Surface shape information of the structure (10) is collected by the laser scanner (30), and at the same time, an image for the same inspection point is obtained by a first light (41) and a second light (42) which are mounted on the drone (20) and emit visible light of different wavelengths and a camera (40). Accurate identification of a crack portion of the structure (10) is possible by binarizing and difference-processing the image. The present invention can resolve a problem of conventional computer image processing-based crack inspection, thereby drastically improving the accuracy compared to computer image processing-based concrete structure crack inspection.

Description

System and method for crack inspection of concrete structures using drones and laser scanners {SYSTEM AND METHOD FOR FINDING CRACK IN CONCRETE USING DRONE AND LASER SCANNER}

The present invention relates to a crack inspection of a concrete structure (10) using a drone (20) and a laser scanner (30), the surface shape information of the structure (10) through the laser scanner (30) At the same time as collecting, the image for the same inspection point is acquired through the first illumination 41 and the second illumination 42 and the camera 40 mounted on the drone 20 and emitting visible light of different wavelengths, and By binarization and differential treatment, it is possible to accurately grasp the crack portion of the structure 10.

The most basic and important inspection in the concrete structure inspection is the crack inspection that identifies the crack condition of the structure, and the size, shape, and speed of the crack are used as key indicators to determine the structural integrity of the structure. .

The traditional method of crack inspection of concrete structures is a method by which a technician inspects the structure to be inspected by visually observing the occurrence and shape of cracks, and periodically photographing the cracks to estimate the progress of cracks. Short-range access to structures to be inspected and inspected on site is inevitable.

Therefore, if the extension of the structure is long or short-distance access such as a large pier is difficult, it takes not only a long time for crack inspection, but also requires a large amount of manpower and equipment, and serious inconvenience due to traffic blockage when inspecting road facilities. I had no choice but to.

Accordingly, a technique such as Patent Publication No. 2003-83359 has been developed and utilized by introducing a computer image processing technique to crack inspection of concrete structures, which facilitates the speed and convenience of inspection. Through computer image processing, it is possible to quickly detect cracks in structures.

Conventional computer image processing-applied concrete structure crack inspection technology including published patent No. 2003-83359 basically captures the surface of the concrete structure to obtain an image that is a digital image, and processes the obtained image by processing it using a computer algorithm. By determining whether or not cracks are present in the site, numerical values for uniformly increasing or decreasing the imaging information obtained from the camera from the light section to the dark part according to light and dark are given, and this is binarized. ) Process to detect dark areas that contrast sharply with the background, which is mainly the list.

That is, by setting a numerical value of 0 to 255 commonly used in gray scale for each pixel of the raw imaging information acquired through the camera, to the darkest part and the brightest part By quantifying the contrast of each pixel by assigning 0 and 225, respectively, by performing binarization to collectively assign a single value to pixels above and below a certain reference value, for example, the average value is averaged by grayscale values of all pixels. After setting the reference value, the imaging information is processed in such a way that 0 is assigned to a pixel above the reference value and 1 is assigned to a pixel below the reference value.

In this binarized imaging information, 1 is assigned to a pixel where a crack portion, which is a dark part that is clearly contrasted with a background, is given, and 0 is given to a pixel where a surface of a non-cracked portion, which is a relatively bright part, is photographed. Through this, the computer can grasp whether cracks are generated or not.

On the other hand, in the binarization process, the numerical values assigned to pixels 1 and 0 exemplified above are only logical values assigned for computational convenience in dividing information, and are equivalent to values of 0 to 225 set in general grayscale. Instead of 1 and 0, various numbers such as 0 and 225 may be applied.

Through the conventional computer image processing-applied concrete structure crack inspection technology, including the aforementioned Patent Publication No. 2003-83359, it has been possible to secure the speed, convenience, and safety of crack inspection, but such conventional technology is basically captured information. It is performed in such a way that it is simply divided into a bright part and a dark part, and the dark part is regarded as a crack part.

That is, it is a malfunction that detects a low lightness compared to the periphery as a dark part that contrasts with the background of the light, and detects it as a crack formation site, as a recessed part or a foreign material with a low brightness compared to the non-cracked surface of the structure that is the background.

Therefore, unnecessary processing such as generating a warning or storing image information separately by simply mistaken the place where the foreign material is deposited and not storing the image is carried out, resulting in a problem of reduced inspection efficiency and erosion of computational resources. Became.

The imaging information in video inspection of long structures such as bridges is composed of thousands of to tens of thousands of high-quality images. A considerable amount of computational resources are inevitably consumed in the processing of these imaging information. In contrast, considering that the raw image capturing information of the crack portion is stored in a separate storage storage device, the process of recording the raw image capturing information by mistaken the non-cracked portion as the crack portion causes a serious waste of computational resources.

On the other hand, in the conventional concrete structure crack inspection technology applied with computer image processing, it is necessary to grasp the shape of the structure and the location of the cracks only with the captured image, which is plane information. Not only was it difficult to grasp, there was also a problem that it was difficult to specify a cracked structure during continuous inspection of multiple structures.

That is, in the prior art, only individual images captured in a state in which the overall shape information of the structure is excluded are provided, so that there are significant difficulties in specifying the structure as well as the location of cracks in the structure.

In view of the above-mentioned problems, the present invention was devised to provide the overall external information of the structure to be inspected along with the imaging image, and to prevent the miscalculation of cracks due to foreign matter, etc., mounted on the unmanned flight chain drone 20 The first light (41) and the second light (42) that emit visible light of different wavelengths as illumination directed toward the structure (10) to be inspected, and the structure (10) by irradiating laser light to the structure (10) to be inspected A laser scanner 30 for detecting surface shape information of the camera, a camera 40 mounted on the drone 20 to photograph the structure 10, a camera 40 mounted on the drone 20, and a first light ( 41) and the controller 45 for controlling the second lighting 42, an information device having a built-in memory, connected to the laser scanner 30 and the controller 45, and the surface shape information and camera of the structure 10 ( 30) The image information captured by is stored in the storage device, and the same The first binary image 81 and the second binary image are binarized by binarizing the first image 71 photographed when the first light 41 emits light at the point and the second image 72 photographed when the second light 42 emits light. Concrete structure using drone and laser scanner including image processor 50 that generates image 82 and differentiates first binary image 81 and second binary image 82 to generate differential image 90 It is a crack inspection system.

In addition, the laser scanner 30 detects the surface shape information of the structure 10 to be inspected and inputs it to the image processor 50 and the surface shape information is stored in the storage device of the image processor 50. The first illumination 41 of the first illumination 41 and the second illumination 42 mounted on the 20 and emitting visible light of different wavelengths is turned on by the controller 45, the first illumination 41 ) In the light emitting state, the controller 45 operates the camera 40 to photograph the structure 10 to obtain the first image 71, and the controller 45 receives the first image 71 from the image processor 50 ), The image processor 50 stores the first image 71 in the storage device, the first light 41 is turned off by the controller 45 and the second light 42 is lit , In the light emission state of the second light 42, the controller 45 operates the camera 40 to capture the structure 10 to be inspected, thereby obtaining the second image 72, and the controller 45 The second image 72 is transmitted to the image processor 50, and the image processor 50 stores the second image 72 in the storage device, the first image 71 and the second image stored in the memory device The image processor 50 processes 72, but the first and second images 71 and the second image are given by logical values of 1 and 0, respectively, for pixels below the brightness average value for all pixels and pixels exceeding the brightness average value. 72) generating a first binary image 81 and a second binary image 82, respectively, the image processor 50 logic for each pixel of the first binary image 81 and the second binary image 82 It is a method for inspecting cracks in concrete structures using drones and laser scanners, which includes the step of generating a difference image (90) by calculating the values in a logical product.

Through the present invention, it is possible to solve the problem of the conventional computer image processing-based crack inspection to misinterpret the foreign material as a crack, thereby dramatically improving the accuracy of crack inspection based on computer image processing.

In particular, it is possible to easily specify the structure to be inspected by providing the surface shape information of the casting obtained through laser scanning along with the captured image information for crack identification.

In addition, it is possible to improve the efficiency of crack inspection of concrete structures based on image processing by preventing unnecessary waste of computational resources due to miscalculation of cracks.

1 is an explanatory diagram of the performance of the present invention
2 is an explanatory diagram of the crack inspection situation of the bridge structure to which the present invention is applied.
3 is a system block diagram of the present invention
4 is an explanatory diagram of communication methods between components of the present invention;
5 is an example of a step-by-step image of the present invention

Hereinafter, preferred embodiments of a crack inspection system and method for a concrete structure using a drone and a laser scanner according to the present invention will be described in detail with reference to the drawings.

The terms or words used below should not be interpreted as being limited to ordinary or dictionary meanings, and based on the principle that the inventor can appropriately define the concept of terms to explain his or her invention in the best way. It should be interpreted as meanings and concepts consistent with the technical spirit of the present invention.

The concrete structure crack inspection system using image processing according to the present invention is for inspecting whether a crack (C) is issued to a concrete structure 10, such as various buildings and bridges, and as shown in FIG. 1, the inspection target structure 10 A laser scanner 30 that is installed around and extracts surface shape information of the structure 10 by irradiating laser light to the structure 10, and a drone 20 that is an unmanned air vehicle flying around the structure 10 to be inspected. , These are composed of an image processor 50 and the like to receive image information and surface shape information of the structure from the drone 20 and the laser scanner 30, respectively.

In the present invention, the surface shape information of the structure 10 acquired through the laser scanner 30 is reflected by the laser light irradiated from the laser scanner 30 from the surface of the structure 10, through which the laser scanner 30 is It is established by extracting the three-dimensional coordinates of the surface of the structure 10, and is stored in the storage device of the image processor 50 together with the acquired image information of the camera 40, which will be described later. It is treated as image information and surface shape information.

Therefore, the user of the application system of the present invention can check the image information and the surface shape information recorded at the same time, so that in the crack inspection through the image information, the structure or the inspection point on the structure corresponding to the image information is easily available through the surface shape information. Can be specified.

On the other hand, the actual discrimination as to whether or not a crack occurs in the present launch is performed through the processing of image information acquired by the camera 40. As shown in FIG. 2, an unmanned aerial vehicle flying around the structure 10 to be inspected An image photographing the surface of the structure 10 is obtained by the camera 40 mounted on the in drone 20.

In addition, in the present invention, the first light 41 and the second light 42, which are selectively synchronized by being connected to the camera 40, are configured. In addition, as shown in FIGS. 3 and 4, the camera 40 ), The controller 45 connected to the first lighting 41 and the second lighting 42, the controller 45, the image processor 50, and the like.

In the present invention, the controller 45 connected to the camera 40, the first lighting 41 and the second lighting 42 controls the camera 40, the first lighting 41 and the second lighting 42. At the same time, wireless communication with the image processor 50 is also performed. In these controllers 45 and the image processor 50, wireless communication modules satisfying the same communication protocol are mounted.

That is, as shown in FIG. 4, the above-described laser scanner 30 and the image processor 50 are wired connected, while the controller 45 mounted on the drone 20 and the image processor 50 are wirelessly connected. In this way, the structure 10 generated by the laser scanner 30 installed on the ground is input to the image processor 50 through wired communication, while being transmitted from the controller 45 of the drone 20 of the flight chain. The image information is transmitted to the image processor 50 through wireless communication.

As shown in FIG. 2, the first and second lights 41 and 42 are installed so as to face the same direction as the camera 40 to illuminate the surface of a structure 10 such as a bridge, which is an object. These first and second lights 41 and 42 are configured to irradiate visible light of different wavelengths.

For example, a method in which the first light 41 emits visible light having a red system wavelength and the second light 42 emits visible light having a yellow system wavelength may be applied, and these first lights 41 and second lighting ( 42) are not lit at the same time, but are alternately lit, so that the object structure 10 is not irradiated with illumination of different wavelengths at the same time.

As described above, the visible light emitted from the first light 41 and the second light 42 has different wavelengths, that is, colors, and these different colors are preferably colors that are clearly distinguished. In the following description, for convenience of explanation and ease of understanding, the case where the first light 41 and the second light 42 emit red light and yellow light, respectively, as illustrated above is applied.

In the present invention, when the drone 20 is temporarily stopped at any one inspection point of the structure 10, that is, when the inspection start command is input from the inspector in the state of hovering or low-speed flight, the controller 45 firstly operates. Second, after lighting the first light 41, the camera 40 is operated to obtain the first image 71, which is the raw imaging information for the structure 10, which is the illuminated object, and obtain the first image ( 71) is wirelessly transmitted to the image processor 50, and the third first light 41 is turned off.

The inspection start command of the inspector may be input through an operation of a wireless remote controller connected to the controller 45, and the transmitted first image 71 is stored in the memory of the image processor 50.

As such, immediately after the storage of the image processor 50 of the first image 71 and the extinguishing of the first light 41 are completed, the same process is performed for the second light 42 at the same point.

That is, the controller 45 first, after the second lighting 42 is turned on, secondly, by operating the camera 40, the second image 72, which is the raw imaging information for the structure 10, which is the illuminated object. To obtain, wirelessly transmit to the image processor 50, and turn off the third second light 42.

In this way, the image processor 50 inspects the first image 71 which is the image under the red light illumination of the first light 41 and the second image 72 which is the image under the yellow light illumination of the second lighting 42. (10) It is possible to secure the same image capturing area.

Although the above-described process is composed of a number of steps, such as the first and second lights 41 and 42 are sequentially turned on and off, and two cameras 40 are taken, this series of procedures is performed by the controller ( In addition to being carried out by the camera 40 and the first light 41 and the second light 42, which are automatically controlled by 45), the camera 40, which is the controlled chain, is also an optical device, and is capable of instantaneous operation. Bar, it can be carried out in a short amount of time that the inspector does not experience.

In particular, the above-described series of procedures may be completed within a momentary time of about a few tenths of a second, so long time hovering is not required, and can be performed smoothly even in a low-speed flight state, not a stationary flight.

As such, when the storage of the second image 72 and the extinguishing of the second lighting 42 are completed and the shooting for a single point is completed, the inspector moves the drone 20 to the next inspection point and repeats the above-described process. This process is repeatedly performed so that the first image 71 and the second image 72 for the entire inspection section can be secured at predetermined intervals with respect to the entire inspection section, that is, no missing portion is generated.

Meanwhile, the image processor 50 processes the first image 71 and the second image 72 for each inspection point stored in its memory, such as a desktop computer or a laptop computer equipped with a predetermined image processing program. It may be composed of a computer, and is equipped with a wireless module that performs wireless communication with the controller 45 as described above.

That is, the image processor 50 in the present invention is a computer equipped with a program that performs computational processing on image information, and performs wireless communication with the controller 45 of the drone 20, and the laser scanner 30. And wired connection to collect information.

Computational image processing performed in the image processor 50 of the present invention includes binarization processing, and the binarization processing is performed on the first image 71 and the second image 72 acquired for each inspection point. The first technique 71 and the second image 72 are binarized by the same technique, which is basically obtained for each of the first images 71 and the second image acquired at one inspection point. Only the processing of the image 72 will be described.

In addition, for easy understanding, the subsequent processing will be described by taking the case where a mud foreign matter F and a crack C of red color are formed on the light gray concrete structure 10 as shown in FIG. 5.

For binarization, the image processor 50 first calculates the average brightness value of all pixels of the first image 71 photographed when the first light 41 is emitted.

The first image 71 includes a plurality of pixels, and each pixel is provided with brightness information by light reflected from the surface of the structure 10 in which the visible light emitted from the first light 41 is a subject, the image processor 50 ) Sums the brightness information of all the pixels and divides them by the number of pixels to calculate the average brightness of the first image 71.

As described above, since the first light 41 emits red light, a large amount of visible light of the first light 41 is reflected from the mud foreign matter F of a reddish color, as well as the surface of the light grayscale concrete structure 10 close to white. On the other hand, since the light irradiated into the crack C is not reflected in the crack C portion of the concrete structure 10, a dark portion is formed.

Therefore, the average brightness value of the first image 71 is smaller than the brightness information of the pixels on the surface of the light gray concrete structure 10 and the pixels of the mud foreign matter (F), that is, dark, rather than the brightness information of the pixels in the crack (C) region. In other words, it becomes bright.

The image processor 50 also processes the second image 72 photographed when the second light 42 emits light in the same manner as above, but the second light 42 emits yellow light, and the light gray concrete On the surface of the structure 10, a large amount of reflected light is formed, whereas in the mud foreign matter (F) and crack (C) areas, the reflected light is poorly formed.

Therefore, the average brightness value of the second image 72 is smaller than the brightness information of the surface pixels of the light gray concrete structure 10, that is, it is dark, and is larger than the brightness information of the pixels of the mud foreign matter (F) portion and the pixels of the crack (C) region. That is, it becomes bright.

As a result, in the first image 71 photographed during the red light emission of the first light 41 and the second image 72 photographed during the yellow light emission of the second light 42, the crack C portion is The first image 71 and the second image 72 are both detected as dark areas, but the foreign matter (F) region is detected as the first image 71, while the second image 72 is detected as the dark region. From the point of view, the first image 71 and the second image 72 are identified as cracks C only for the portion that is detected as the dark part, and only in either the first image 71 or the second image 72 The part detected as the dark part is regarded as a foreign material (F).

When the above-described process is explained in terms of logical operations performed in computational processing, after calculating the brightness average value, the image processor 50 assigns a logical value of 1 to a pixel to which brightness information below the brightness average value, which is a reference value, is given as a logical value and processed as a dark part. Then, a logic value of 0 is assigned to a pixel to which brightness information exceeding the brightness average value is assigned and processed as a list.

Therefore, in the first image 71, the logic value of 0 is assigned to the pixels of the concrete structure 10 part and the mud foreign matter F part, and is processed as a list, and the logic value of 1 is limited to the pixels of the crack C part. It is given and processed as a dark part. In the second image 72, pixels of the structure 10 part are assigned a logic value of 0 and processed as a light part, but 1 is applied to the pixels of the foreign material (F) part and the crack (C) part. The logical value of is given and treated as a dark section.

The first binary image 81 and the second binary image 82 generated by binarizing the first image 71 and the second image 72 are illustrated in FIG. 5.

When the binarization process is completed, the image processor 50 performs differential processing to generate a differential image 90.

In the difference processing, a logical value for each pixel of the first binary image 81 and a logical value for each pixel of the second binary image 82 are logical multiplied, that is, if the two logical values are the same, the logical value is maintained. If different, a difference image 90 is generated by calculating in a way that 0 is assigned as a logical value.

Therefore, after the difference processing is performed, the concrete structure 10 portion of the first binary image 81 and the concrete structure 10 portion of the second binary image 82 are 0 by 0∧0 = 0, which is a logical product operation. The logical value of is maintained, and in the foreign matter (F) portion of the first binary image 81 and in the foreign matter (F) portion of the second binary image 82, the logical value of 0 by the logical multiplication operation 0∧1 = 0 Is given, and in the crack (C) portion of the first binary image 81 and the crack (C) portion of the second binary image 82, a logical value of 1 is maintained by a logical product operation of 1∧1 = 1. .

As a result, as shown in FIG. 5, only the crack (C) portion is displayed on the difference image 90 that is finally generated through the above-described binarization and difference processing, and the foreign substance (F) portion is not displayed.

Meanwhile, the image processor 50 may additionally perform processing such as storing the difference image 90 in a storage medium in the image processor 50 or outputting a warning sound when a crack (C) portion occurs in the difference image 90. have.

In the prior art introduced above, the foreign material (F) part or the boundary between the foreign material (F) and the structure 10 may be mistaken for the crack (C), but in the present invention, the foreign material (F) part is ignored and the crack (C ) Only the part can be clearly distinguished, and according to the present invention, the accuracy of crack (C) inspection using computer image processing can be dramatically improved.

As described above, although the present invention has been described by limited embodiments and drawings, the present invention is not limited by this, and the technical idea of the present invention and the following will be described by those skilled in the art to which the present invention pertains. Of course, various modifications and variations can be made within the scope of the claims to be described, and the above-described embodiments can be variously combined.

10: structure
20: drone
30: laser scanner
40: camera
41: Article 1
42: Article 2
45: controller
50: image processor
71: first video
72: second video
81: first binary image
82: second binary image
90: difference video

Claims (2)

A first light 41 and a second light 42 that are mounted on the drone 20 which is an unmanned aerial vehicle and which emit visible light of different wavelengths as illumination directed to the structure 10 to be inspected;
A laser scanner 30 that detects surface shape information of the structure 10 by irradiating laser light to the structure 10 to be inspected;
A camera 40 mounted on the drone 20 to photograph the structure 10;
A controller 45 mounted on the drone 20 to control the camera 40, the first lighting 41 and the second lighting 42;
As an information device with a built-in storage device, it is connected to the laser scanner 30 and the controller 45, and the surface shape information of the structure 10 and the image information captured by the camera 30 are stored in the storage device, and the same inspection point The first binary image 81 and the second binary image by binarizing the first image 71 captured when the first light 41 emits light and the second image 72 captured when the second light 42 emits light Cracking the concrete structure using drones and laser scanners, including an image processor 50 that generates 82 and differentiates the first binary image 81 and the second binary image 82 to generate a differential image 90 Inspection system.
The laser scanner 30 detects the surface shape information of the structure 10 to be inspected and inputs it to the image processor 50, and the surface shape information is stored in the storage device of the image processor 50:
A step in which the first light 41 of the first light 41 and the second light 42 that is mounted on the unmanned air vehicle drone 20 and emits visible light of different wavelengths is turned on by the controller 45;
In the light-emitting state of the first light 41, the controller 45 operates the camera 40 to photograph the structure 10 to obtain the first image 71, and the controller 45 obtains the first image 71 Transmitting to the image processor 50, the image processor 50 storing the first image 71 in the storage device;
A step in which the first light 41 is turned off and the second light 42 is lit by the controller 45;
In the light emission state of the second lighting 42, the controller 45 operates the camera 40 to photograph the structure 10 to be inspected, thereby obtaining the second image 72, and the controller 45 generates the second image ( 72) is transmitted to the image processor 50, and the image processor 50 stores the second image 72 in the storage device;
The image processor 50 processes the first image 71 and the second image 72 stored in the storage device, but logical values of 1 and 0 for pixels below the brightness average value for all pixels and pixels exceeding the brightness average value, respectively. Generating the first binary image 81 and the second binary image 82 from the first image 71 and the second image 72, respectively, by giving this;
A drone and a laser scanner comprising an image processor 50 generating a differential image 90 by calculating logical values of each pixel of the first binary image 81 and the second binary image 82 in a logical multiplication method Concrete structure crack inspection method using.

Images