WO2023038187A1

WO2023038187A1 - Unmanned aerial vehicle employing lidar sensor for measuring crack thickness of structure, and computer program and crack measurement and management system using same for performing crack measurement and management method for structure

Info

Publication number: WO2023038187A1
Application number: PCT/KR2021/014062
Authority: WO
Inventors: 이관희; 손창섭; 김재홍; 전세원
Original assignee: 경남도립거창대학산학협력단
Priority date: 2021-09-07
Filing date: 2021-10-13
Publication date: 2023-03-16

Abstract

An embodiment may provide an unmanned aerial vehicle employing a LiDAR sensor for measuring a crack thickness of a structure, the unmanned aerial vehicle comprising: a body; a camera location control module connected to the body; a camera installed in the camera location control module; a LiDAR location control module connected to the body: and a LiDAR sensor which is installed in the LiDAR location control module and detects information of a distance to an inspection point to determine a size of a crack of an inspection structure detected in a photographed image detected by the camera, wherein the camera is changed in photographing location by the camera location control module and the LiDAR sensor is changed in location by the LiDAR location control module.

Description

An unmanned aerial vehicle to which a lidar sensor is applied for measuring the thickness of cracks in a structure, and a computer program for measuring and managing cracks in a structure using the same, and a system for measuring and managing cracks

The present invention relates to an unmanned aerial vehicle to which a lidar sensor for measuring the thickness of cracks in a structure is applied, a computer program for performing a method for measuring and managing cracks in a structure using the same, and a system for measuring and managing cracks.

Currently, the use of unmanned vehicles (drones) for facility inspections is increasing. It can inspect areas that are difficult for people to access, eliminating inspection blind spots, ensuring the safety of workers and increasing work efficiency. When using drones to inspect facilities such as bridges, there are advantages in reducing labor and inspection costs, but there are operational difficulties such as flight stability and flight time. Large and heavy aircraft are resistant to drafts but have short flight times due to their weight, while small and light aircraft have long flight times but are more sensitive to wind. In addition, mechanical vibrations generated by drones do not generally interfere with flight, but may cause a decrease in the clarity of inspection images and difficulty in maneuvering. In addition, drones that are widely used for facility inspection are GNSS/INS combined navigation, and their dependence on satellite navigation is very high. When these drones are located under bridge facilities such as roads or railways, signal quality of satellite navigation may deteriorate or reception may be difficult. In order to introduce drones for bridge inspection, localization of satellite shadow areas is a major issue to be resolved. In addition, along with the development of drone technology for safety inspection of structures such as bridges, interest in the development of programs for detecting defects in bridges by analyzing images taken by drones is also increasing.

An object of the present invention is to detect cracks in a structure by analyzing an image of a structure to be inspected taken by an unmanned aerial vehicle, and to manage crack information for each structure.

Embodiments according to the present invention detect defects in the appearance of an inspection structure from camera-photographed images, determine the size of defects based on relative distance information between inspection points, and improve flight accuracy even in satellite shadow areas. The purpose.

The embodiment body; a camera position control module connected to the main body; a camera installed in the camera position control module; LiDAR position control module connected to the main body; And a lidar sensor installed in the lidar position control module and detecting distance information to an inspection point in order to determine the size of a crack in an inspection structure detected in a photographed image detected by the camera; A camera whose shooting position is changed by the camera position control module, and the lidar sensor can provide an unmanned air vehicle to which a lidar sensor is applied for measuring the thickness of a crack in a structure whose position is changed by the lidar position control module. there is.

In another aspect, presence or absence of cracks and cracks in a commercially-captured image based on photographed image information generated by photographing a plurality of different inspection points in the inspection structure in a hovering state and distance information of each of the plurality of different inspection points. It is possible to provide an unmanned air vehicle to which a lidar sensor for measuring the crack thickness of a structure that determines the size information of the structure is applied.

On the other hand, providing an unmanned air vehicle equipped with a lidar sensor for measuring the thickness of cracks in a structure whose altitude is changed based on the shape information of the surface of the inspection structure and the distance information to the inspection point measured by the lidar sensor can do.

On the other side, a GNSS (Global Navigation Satellite System) module further includes, and the current position based on the shape information of the surface of the inspection structure and the measurement information of the lidar sensor in the shaded area that is the lower area of the inspection structure A preset inspection path for estimating information, detecting current location information by the GNSS module in a non-shaded area other than the shaded area, and inspecting the inspection structure in the shaded area includes the non-shaded area a plurality of times. It is possible to provide an unmanned aerial vehicle to which a lidar sensor for measuring the thickness of cracks in a structure set to pass through is applied.

In another aspect, an internal crack inspection module installed in the main body to apply a physical impact to the inspection point to determine whether or not there is a crack inside the inspection point of the inspection structure; further comprising, the internal crack inspection module inspects An unmanned aerial vehicle with lidar sensor for measuring the crack thickness of a structure including an extension link extending to a point and a contact portion that reciprocates in the longitudinal direction of the extension link and applies a physical impact to the test point by contacting the test point. can do.

In another aspect, a crack in a structure further comprising a canopy covering the extension link and the contact portion in response to the extension of the extension link in order to increase the reception efficiency of the sound by the physical impact received through the microphone installed on the extension link. It is possible to provide an unmanned aerial vehicle to which a lidar sensor for thickness measurement is applied.

In another aspect, a computer program for performing a method of measuring and managing cracks in a structure to be inspected is executed in a terminal and in response to selection of any one of a plurality of affordances in a main menu affordance, different sub-menu affordances and Displays a work content display area and/or an auxiliary user interface matched to the submenu affordance, and in response to selection of a structure and crack setting affordance in the main menu affordance, a structure registration affordance and an inspection part registration affordance in the submenu affordance In response to selection of any one of the structure registration affordance and the inspection part registration affordance, the information of the structure to be inspected is registered, and the information of at least one inspection part constituting the structure to be inspected and the inspection period information are registered and inspected. Registering the captured image of the part, analyzing the registered captured image, detecting and displaying cracks in the captured image, and based on distance information between the subject and the UAV in the captured image measured by the UAV A computer program for measuring and managing cracks in a structure may be provided by storing information on a recording medium that detects and displays information on the length of the detected crack and the thickness of the crack.

In another aspect, in response to selection of any one of the structure registration affordance and the inspection part registration affordance, information of the structure to be inspected is registered, information of at least one inspection part constituting the structure to be inspected and inspection period information are registered, and When a photographed image of an inspection part is registered, a crack map with the structure as an upper element and the inspection part of each structure as a lower element is set and stored in the display area in the work contents display area, and is stored in a recording medium for measurement and management of cracks in the structure. A computer program for performing can be provided.

On the other hand, a recording medium that displays information on a registered structure and a plurality of inspection parts of the registered structure, and determines whether or not to display an inspection request based on an inspection cycle of the selected inspection part in response to a selection of a plurality of displayed inspection parts. It can be stored in and provide a computer program for performing the crack measurement and management method of the structure.

In another aspect, the captured image is displayed, cracks in the captured image are detected by comparing and analyzing color values between pixels based on a set color value or within a crack range specified by a user, and an image displaying the detected crack is displayed. A computer program for performing a method of measuring and managing cracks in a structure may be provided by being stored in a recording medium for displaying.

In another aspect, a crack pixel corresponding to a crack is detected in the photographed image, a crack pixel group is set by grouping adjacent crack pixels, and a virtual center point is set for each of the crack pixels in any one crack pixel group, , Searching for each of a plurality of arbitrary crack pixels, setting a virtual pixel direction line connecting the virtual center points of each of the plurality of crack pixels, Set virtual first lines connecting the center points to each other, detect the longest line segment among the set virtual first lines, set the length of the detected line segment as the length of the crack, and correspond to the length of the crack A plurality of virtual pixel thickness direction lines are set orthogonal to the detected virtual line segment and pass through the virtual center point of the crack pixel, and a virtual second line segment connecting the virtual center points among the plurality of pixel thickness direction lines to each other A computer program for measuring and managing cracks in a structure may be provided by being stored in a recording medium in which the length of the set second line segment is set as the thickness of the crack.

On the other hand, if the number of crack pixel groups has a separation distance less than a preset value, they are set as one crack, and the separation space between crack pixels belonging to different crack pixel groups having a separation distance less than the preset value is stored in a recording medium in which at least one chukka crack pixel is inserted based on the thickness value of the crack pixel in each group, and spaced apart crack pixel groups are connected to each other to set them as one pixel group, and a method for measuring and managing cracks in a structure is stored. A computer program to perform may be provided.

On the other hand, detecting crack pixels corresponding to cracks in the photographed image, and connecting the center points of the leftmost and uppermost start point crack pixels and the lower right and lowermost end point crack pixels to each other A partial crack line segment In step 1, a horizontal crack pixel group region with the largest number of crack pixels in contact with each other in the horizontal direction is detected on the path passing through the partial crack line segment, and the center point of the leftmost crack pixel in the horizontal crack pixel group and the maximum A horizontal crack line segment connecting the center points of the right crack pixels is set, and as a second step, a plurality of cracks located in the right direction based on the center point of the rightmost crack pixel and the center point of the rightmost crack pixel of the horizontal crack line. A line segment satisfying a predetermined condition is detected from among virtual lines connecting the center points of the pixels to each other, the detected line segment is set as the next partial crack segment, and the end point of the generated crack segment is repeated by repeating steps 1 and 2 above. When the crack pixel and the end-point crack pixel located at the right side and the bottom of the bottom match, a path passing through at least a part of the line segment is set according to a preset condition, and the set path is stored in a recording medium for setting the length of the crack to determine the structure of the structure. A computer program for performing the crack measurement and management method may be provided.

The embodiment analyzes images taken from an unmanned aerial vehicle to extract cracks, manage crack state information for each structure, build a crack map for each structure by making the extracted cracks into a database, and It is possible to detect cracks that exist on multiple areas and cracks whose direction changes irregularly, and to determine the danger of cracks by detecting information on the length and width of the detected cracks. It can detect cracks by integrating cracks systematically, provide information on the length and width of cracks, make it possible to grasp the inspection status according to the set cycle for several structures and inspection parts of structures, and analyze the captured images to It automatically detects cracks and analyzes crack size information.

In addition, the efficiency of visual inspection is increased by using the video taken by the unmanned aerial vehicle, and the data captured efficiently in a short time is accumulated to manage the maintenance and life cycle cost (LCC) of structures requiring inspection. It is possible to secure the safety and economic feasibility of the structure at the same time, and various data of the structure can be used for future construction planning and operation. In addition, the embodiment protects the airframe from external impact, enables posture control and stable flight after a collision, can provide even small crack images at high resolution to enable external inspection, and is difficult for humans to access due to the nature of the structure environment. It can be used to inspect plant facilities such as dangerous and limited space facilities, marine infrastructure, and civil structures. In addition, by estimating the position of the unmanned aerial vehicle based on the relative distance between the inspection point by the lidar sensor and the unmanned aerial vehicle in the satellite shadow area where the signal quality of satellite navigation may deteriorate or reception may be difficult, the flight path of the unmanned aerial vehicle even in the satellite shadow area accuracy can be improved. In addition, it is possible to provide an unmanned air vehicle and a system including the same capable of detecting internal defects such as internal cracks as well as external defects of the inspection structure detected from the photographed image.

1A is a diagram illustrating a system according to some examples for inspecting a bridge. Figure 1b schematically shows the inspection of the state of various types of structures using an unmanned aerial vehicle. 2 shows an entire unmanned aerial vehicle according to an embodiment of the present invention, FIG. 3 shows a part of an unmanned aerial vehicle according to an embodiment of the present invention, and FIG. 4 shows an exemplary block diagram of an unmanned aerial vehicle structure, 5 schematically shows the display of 3D model data of an inspection object on a terminal, FIG. 6 is for explaining setting an inspection area in the 3D model data, and FIG. 7 is an unattended inspection area model. It schematically shows that the flight path of the air vehicle is set, and FIGS. 8 and 9 are for explaining that the test path is set so that the unmanned air vehicle passes through the non-shaded area multiple times, and FIG. 10 is between the unmanned air vehicle and the test subject. This is to explain the flight height of the UAV according to the distance difference of , Figure 11 schematically depicts the UAV inspecting the inspection point, and Figure 12 shows the inspection path so that the UAV passes through the unshaded area multiple times 13 and 14 show an unmanned aerial vehicle moving along an inspection path and inspecting a bridge, and FIG. 15 schematically illustrates an unmanned aerial vehicle according to various embodiments of the present invention. 16 is a schematic diagram for explaining a method of inspecting whether or not the inside of an inspection point is cracked using an internal crack inspection module, and FIG. 17 is an inner part further including a contact part cover according to another embodiment of the present invention. It schematically shows the crack inspection module.

18 illustrates a main user interface among user interfaces displayed when a computer program for performing a method for measuring and managing cracks in a structure stored in a recording medium is executed on a terminal. 19 shows a main user interface including a first auxiliary user interface displayed when a computer program for performing a crack measurement and management method of a structure stored in a recording medium is executed on a terminal, and FIG. 20 illustrates a structure stored in a recording medium 21 and 22 show a main user interface including a second auxiliary user interface displayed when a computer program for performing a crack measurement and management method is executed on a terminal, and FIGS. 21 and 22 measure and manage cracks of structures stored in a recording medium. As a main user interface displayed when a computer program for performing the method is executed on a terminal, a main user interface for displaying crack information monitoring information is shown, and FIGS. 23 to 26 show a method for measuring and managing cracks in a structure stored in a recording medium. It shows a main user interface including a third auxiliary user interface displayed when a computer program for execution is executed on a terminal, and FIGS. 27 and 28 are schematic diagrams for explaining a method for detecting the length of a crack in an original image, 29 is a schematic diagram for explaining a method for detecting the thickness of a crack in an original image, and FIGS. 30 and 31 describe a method for detecting the total length of a crack when the number of crack pixels in a crack pixel group greater than or equal to a preset value exists. 32 and 33 are for explaining a method of detecting the total length of cracks in a form different from FIGS. 30 and 31 when the number of crack pixels in the crack pixel group is greater than a preset value, and FIG. 34 is for explaining a method of processing crack pixel groups spaced apart by less than a preset value as one crack, and FIG. It shows the main user interface including the fourth auxiliary user interface.

1A is a diagram illustrating a system according to some examples for inspecting a bridge. Figure 1b schematically shows the inspection of the state of various types of structures using an unmanned aerial vehicle. 2 shows the entire unmanned aerial vehicle according to an embodiment of the present invention, and FIG. 3 shows a part of the unmanned aerial vehicle according to an embodiment of the present invention.

Referring to FIGS. 1A-3 , system 10 includes an unmanned aerial vehicle 30 that can move (fly) around a structure requiring periodic inspection. In this example, the drone 30 is a rotorcraft. The structure being audited is illustrated as a bridge 20, but system 10 includes power lines, power generation facilities, power grids, dams, levees, stadiums, large buildings, large antennas and water treatment facilities, oil refineries. It is equally well suited for use in the inspection of a wide range of other structures, including but not limited to, chemical processing plants, high rises, and infrastructure and monorail support structures associated with trains. Additionally, system 10 may be particularly suitable for use in the underside of bridges or inside large buildings such as manufacturing facilities and warehouses. Virtually any structure that would be difficult, expensive, or too dangerous to inspect by a person controlling the inspected object or a platform carrying the inspected object could potentially be inspected using the system shown in FIG. 1 .

In addition, the system 10 may include a terminal 200 that is a computing device for detecting cracks on the captured image by analyzing the captured image of the unmanned aerial vehicle 30 . In various embodiments, system 10 may further include an unmanned aerial vehicle controller 40 . Also, the terminal 200 may include a portable device 200a and a computing device 200b such as a desktop.

In addition, the unmanned aerial vehicle 30 may have a main body 100a capable of protecting the aircraft from external impact.

The unmanned air vehicle 30 is preferably a rotorcraft due to its ability to hover and move at extremely slow speeds. The vertical take-off and landing capability of remote controlled drone 30 may be when operating inside structures or facilities, such as manufacturing plants, warehouses, etc., or having many tall structures (eg, chimneys) closely packed together. This can be very advantageous when inspecting complex facilities such as chemical processing plants or oil refineries. The ability to hover or move only vertically allows the remotely controlled drone 30 to fly close to and inspect large vertical structures, such as vertical support columns of bridges, antennas or vertical surfaces of dams.

4 shows an exemplary block diagram of an unmanned aerial vehicle structure.

The unmanned aerial vehicle 30 may include an application platform and a flight platform. The application platform may process signals for driving the unmanned aerial vehicle 30 and providing services by wirelessly interworking with other electronic devices, for example, the remote control station 40 or an external controller equipped with a remote controller function. The flight platform may control the overall flight of the unmanned aerial vehicle 30 by including a flight control algorithm and/or a navigation algorithm.

One or more application processors (eg AP) 110, wireless communication module 120, memory 130, sensor module 140, thrust generating device 150, camera module 160, audio module 170 , an indicator 180, a power management module 190, and a battery 191.

The application processor 110 may control a plurality of hardware or software components connected to the application processor 110 by driving an operating system or an application program as part of an application platform, and perform various data processing and operations. can The application processor 110 may further include a graphic processing unit (GPU) and/or an image signal processor. The application processor 110 may include at least some of the elements shown in FIG. 4 . The application processor 110 may load and process commands or data received from at least one of the other components into a volatile memory, and store resultant data in a non-volatile memory. The application processor 110 may control the thrust generating device 150 and/or the camera module 160 according to programs stored in the wireless communication module 120 and/or the memory 130 .

The wireless communication module 120 may be disposed inside the housing or connected to the housing. The wireless communication module 120 may include, for example, a cellular module 121, a WiFi module 122, a Bluetooth module 123, a Global Navigation Satellite System (GNSS) module 124, and an RF module 125. there is.

The cellular module 121 may provide, for example, a voice call, a video call, a text service, or an Internet service through a communication network. According to various embodiments, the cellular module 121 may perform at least some of the functions that the application processor 110 may provide. According to various embodiments, the cellular module 121 may include a communication processor. According to various embodiments, at least some (eg, two or more) of the cellular module 121, the WiFi module 122, the Bluetooth module 123, or the GNSS module 124 are one integrated chip (IC) or IC package may be included in The RF module 125 may transmit and receive communication signals (eg, RF signals), for example. The RF module 125 may include, for example, a transceiver, a power amp module (PAM), a frequency filter, a low noise amplifier (LNA), or an antenna. According to another embodiment, at least one of the cellular module 121, the WiFi module 122, the Bluetooth module 123, or the GNSS module 124 may transmit and receive RF signals through a separate RF module.

The memory 130 may include, for example, an internal memory 131 or an external memory 132 . The built-in memory 131 includes, for example, volatile memory (eg, DRAM, SRAM, or SDRAM, etc.), non-volatile memory (eg, OTPROM (one time programmable ROM), PROM, EPROM, EEPROM, mask ROM, flash ROM) , a flash memory, a hard drive, or a solid state drive (SSD).The external memory 132 may include a flash drive, for example, a compact flash (CF) or secure digital (SD). ), Micro-SD, Mini-SD, xD (extreme digital), MMC (multi-media card) or memory stick, etc. The external memory 132 is functionally connected to the unmanned aerial vehicle 30 through various interfaces. can be physically or physically connected.

The sensor module 140 may measure a physical quantity or detect an operating state of the unmanned aerial vehicle 30 and convert the measured or sensed information into an electrical signal. In addition, the sensor module 140 may measure the distance between the unmanned aerial vehicle 30 and an object in front, and may estimate its own location information based on the distance. In various embodiments, the physical quantity detected through the sensor module 140 may be used as information necessary for flight control of the unmanned aerial vehicle 30 . The sensor module 140 includes, for example, a Lidar sensor 141, a gyro sensor 142, an air pressure sensor 143, a compass sensor 144, an acceleration sensor 145, an ultrasonic sensor 146 , a temperature sensor 147, a humidity sensor 148, and an illuminance sensor 149. Additionally or alternatively, sensor module 24 may further include, for example, control circuitry for controlling one or more sensors included therein. In some embodiments, the unmanned aerial vehicle 30 further includes a processor configured to control the sensor module 140 as part of or separately from the application processor 110 so that the application processor 110 is in a sleep state. During this time, the sensor module 149 can be controlled. In another embodiment, the sensor module 140 may measure a physical quantity or detect an operating state of the unmanned aerial vehicle 30 and provide the measured or sensed information to the thrust generating device 150, and the thrust generating device 150 ) may control the flight of the unmanned aerial vehicle 30 based on the provided information. For example, the sensor module 140 and the thrust generating device 150 may at least partially constitute part of a flight platform. In various embodiments, the position of the lidar sensor 141 may be controlled by the lidar position control module 141a.

The thrust generating device 150 may include a plurality of microprocessor units (MPUs) 151 , a plurality of driving circuits 152 and a plurality of motors 153 .

The navigation circuit unit 154 may generate a signal for controlling the motor 153 based on a control signal provided from the application processor 110 and various physical quantities provided through the sensor module 140 . The microprocessor unit (MPU) 151 and the drive circuit 152 drive the motor 153 according to the control signal of the navigation circuit unit 154 to generate thrust and/or lift necessary for the flight of the unmanned aerial vehicle 30. can make it

The camera module 160 may include a camera 161 , a camera position control module 162 , and a camera position control motor 163 . The camera module 160 is, for example, a device capable of capturing still images and moving images. In various embodiments, camera 161 may include one or more image sensors, lenses, image signal processors (ISPs), or flashes. In various embodiments, camera 161 is a still camera (color and/or black and white) for obtaining still images, a video camera for obtaining color and/or black and white video or infrared still images or infrared rays of parts of bridge 11. It may include an infrared camera to obtain video. The camera position control module 162 maintains a constant attitude or direction of the camera 161 when the unmanned aerial vehicle 30 is shaken due to the influence of the vibration of the thrust generating device 150 or the surrounding air current, and the camera 161 maintains a shake-free image. can be taken. In the unmanned aerial vehicle 30, the camera 110 may be installed in the camera position control module 162 in a detachable form to the camera position control module 162 installed on the upper side of the main body 100a.

In addition, the camera position control module 162 may adjust the position or shooting direction of the camera 161 by using the power of the camera position control motor 163, and may control the camera 161 based on a control signal from the application processor 110. ) can be controlled.

Meanwhile, in the unmanned aerial vehicle 30, the lidar sensor 141 may be installed in the lidar position control module 141a in a detachable form to the lidar position control module 141a connected to the camera position control module 102. .

The lidar sensor 141 emits a signal toward a portion of the bridge 11. A signal from the lidar sensor 141 collides with a part of the bridge 11 so that information on the position of the unmanned aerial vehicle 30 can be obtained.

The audio module 170 may convert a sound and an electrical signal in both directions. The audio module 170 may process sound information input or output through the speaker 171 and/or the microphone 172.

The indicator 180 may display a specific state of the unmanned aerial vehicle 30 or a part thereof, for example, a booting state or a charging state.

The power management module 190 may manage power of the unmanned aerial vehicle 30 , for example. According to one embodiment, the power management module 190 may include a power management integrated circuit (PMIC), a charger IC, or a battery or fuel gauge. A PMIC may have a wired and/or wireless charging method. The wireless charging method includes, for example, a magnetic resonance method, a magnetic induction method, or an electromagnetic wave method, and may further include an additional circuit for wireless charging, for example, a coil loop, a resonance circuit, or a rectifier. there is. The battery gauge may measure, for example, the remaining capacity of the battery 191, voltage, current, or temperature during charging. The battery 191 may include, for example, a rechargeable battery and/or a solar battery. In various embodiments, the unmanned aerial vehicle 30 may be a tethered drone powered by wire.

The system shown in FIG. 1A may further include a remote control station 40 for sending and receiving wireless communications to and from the drone 30 . The remote control station 40 may include a terminal, a transceiver and an antenna. In various embodiments, the terminal, transceiver and antenna may be configured as independent devices. The transceiver may communicate with the antenna to enable communication between the terminal and the unmanned aerial vehicle 30 .

The onboard system of the unmanned aerial vehicle 30 is configured according to one or more different stored inspection paths digitally represented by flight planning data stored in a non-transitory tangible computer readable storage medium (not shown in FIG. 1A). It may further include a guidance and control hardware and software system (not shown in FIG. 1A ) capable of implementing flight plans. The onboard system controls the orientation of the unmanned aerial vehicle 30 and assists in the execution of a flight plan according to a pre-programmed inspection path stored in memory, such as position estimation information based on lidar sensing information or a global satellite navigation system (GNSS). For example, a global positioning system/inertial navigation system (GPS/INS) may be applied. An onboard antenna (not shown in FIG. 1 ) and a radio transceiver enable two-way radio electromagnetic waves communications with the remote control station 40 .

An unmanned aerial vehicle 30 of the type shown in FIG. 1A can be upgraded with the ability to obtain scale and point-to-point distance information for objects under inspection. Unmanned aerial vehicle 30 may be provided with onboard sensors and processing techniques to provide discrete or continuous measurements of the scale of a target object or distances between points on a target object.

FIG. 5 schematically illustrates displaying 3D model data of an object to be tested on a terminal, and FIG. 6 is for explaining setting an examination area in 3D model data. And, Figure 7 schematically shows that the flight path of the unmanned aerial vehicle is set on the inspection area model. In addition, FIGS. 8 and 9 are for explaining that an inspection path is set so that an unmanned aerial vehicle passes through a non-shaded area a plurality of times.

Referring to FIG. 5 , the terminal 200 may display 3D model data 11m of a bridge, which is an exemplary inspection target shown in the drawing.

The terminal 200 may change the area or display direction of the displayed 3D bridge model 11m in response to selection of the displayed 3D bridge model 11m. Illustratively, the terminal 200 may move, rotate, enlarge, or reduce the displayed 3D bridge model 11m on the screen based on a touch input and a movement direction of the input touch.

Referring further to FIG. 6 , the terminal 200 may set an inspection area in response to selection of a plurality of points of the displayed 3D bridge model 11m (a).

The terminal 200 may display (b) the examination area model 11im based on the selected plurality of points. As an example, a partial area of the lower part of the bridge 11 is selected and displayed as the inspection area model 11im.

The terminal 200 may detect surface shape information of the inspection object (examination structure) based on pre-stored 3D model data of the inspection object based on the inspection area model 11im.

The terminal 200 relates to the shape information of the surface including height information (d) of all points on the surface of the test object, substructure information (u) such as piers or bridges, and the size of the unmanned aerial vehicle 30 stored in advance. A flight path 100p of the unmanned aerial vehicle 30 may be set on the inspection area model 11im based on the unmanned aerial vehicle shape information.

Illustratively, as shown in FIG. 7, a flight path 100p in which the unmanned aerial vehicle 30 flies in a zigzag form in a transverse direction along the longitudinal axis of the inspection area model 11im may be set.

As shown in FIGS. 8 and 9 , the flight path 100p may be set such that the unmanned aerial vehicle 30 passes through the non-shaded area nsr multiple times. Illustratively, the lower side of the inspection area model 11im (for example, the lower space of the bridge 11) may be an area in which signal quality of satellite navigation may deteriorate or reception may be difficult. A shaded area sr may be set as the area.

The terminal 200 may set the unmanned aerial vehicle 30 to pass through the unshaded area nsr multiple times when flying in a zigzag pattern in the horizontal direction along the longitudinal axis of the inspection area model 11im.

As shown by way of example, the terminal 200 inspects a partial area of the surface of the test object while the unmanned aerial vehicle 30 departs from the first non-shaded area nsr1 and flies to the shaded area sr, and After getting out of the area sr, passing through the second non-shaded area nsr2, entering the shaded area sr again to inspect another part of the surface of the object to be inspected, and after leaving the shaded area sr, the first The flight path 100p may be set in such a way that it passes through the non-shaded area nsr1 and then enters the shaded area sr again.

After setting the flight path 100p, the terminal 200 sets the location search method switching start point information tp that initiates the change of the location search method of the unmanned aerial vehicle 30 on the flight path 100p. As shown above, the test path (inp) can be finally set.

The location search method switching start point information tp may be a boundary area between the shaded area sr and the non-shaded area nsr or its vicinity.

10 is for explaining the flight height of the unmanned aerial vehicle according to the distance difference between the unmanned aerial vehicle and the test object. And, FIG. 11 schematically depicts an unmanned aerial vehicle inspecting an inspection point.

For convenience of description of the invention, an example is described in which an unmanned aerial vehicle flies along the inspection path shown in FIG. 9 .

4, 9, 10, and 11, the unmanned aerial vehicle 30 located in the non-shaded area determines the altitude when entering the shaded area based on its position information detected by the GNSS module 124. You can decide and enter the shaded area with the determined altitude. In addition, the unmanned aerial vehicle 30 can move while changing the altitude based on the distance information between the unmanned aerial vehicle 30 detected by the lidar sensor 141 and the bridge 11 on the shadow area while moving along the inspection route. there is.

The unmanned aerial vehicle 30 entering the shaded area directs the lidar sensor 141 toward the upper direction of the unmanned aerial vehicle 30 by the lidar position control module 141a, and the sensing information of the lidar sensor 141 Based on this, it is possible to detect distance information between the unmanned aerial vehicle 30 and a structure on the upper side of the unmanned aerial vehicle 30 .

When the unmanned aerial vehicle 30 flies along the inspection path (flight in the transverse direction of the bridge 11 according to the illustrated example), the distance between the unmanned aerial vehicle 30 and the inspection point maintains a distance value within a predetermined range can fly Accordingly, when the height of each test point is changed, the altitude of the unmanned aerial vehicle 30 may be changed accordingly. For example, when the height of the inspection point increases, the unmanned aerial vehicle 30 may ascend and fly, and when the height of the inspection point decreases again, the unmanned aerial vehicle 30 may descend.

In addition, at point a of the test points, the unmanned aerial vehicle 30 on the point a on the test route has a movement distance to be moved in the transverse direction equal to or greater than a preset value, and at point b, the unmanned air vehicle 30 on the point b on the test route Different flight plans can be established by classifying the case where is less than the set value of the travel distance to be moved in the transverse direction. Point b is an inspection point with increased height, and it is a point where the inspection point can be inspected only by the unmanned aerial vehicle 30 inspecting the inspection point after ascending flight and flying down again, and point a is the transverse direction of the bridge occupying more than point b Since the length of is long, the unmanned air vehicle 30 ascends and then inspects the corresponding point while flying horizontally so that the accurate inspection of the corresponding point is possible.

While the unmanned aerial vehicle 30 moves in the transverse direction of the bridge 11 and inspects the inspection point, the unmanned aerial vehicle 30 detects the distance information between the unmanned aerial vehicle 30 and the inspection point detected by the lidar sensor 141 Also, the unmanned aerial vehicle 30 may estimate its position based on the shape information of the surface of the inspection area. And, after flying into the unshaded area, the unmanned aerial vehicle 30 may determine its position through the GNSS module 124 again. Then, the unmanned aerial vehicle 30 re-enters the shadow area and based on the distance information between the unmanned aerial vehicle 30 detected by the lidar sensor 141 and the inspection point and the shape information of the surface of the inspection area, the unmanned aerial vehicle ( 30) can estimate its position and inspect the state of the bridge 11.

In addition, the unmanned aerial vehicle 30 photographs the inspection point of the bridge 11 through the camera 161 to generate photographed image information, and at the same time, the photographed inspection point detected through the lidar sensor 141 and the unmanned aerial vehicle ( 30) can generate distance information between them. In addition, the unmanned aerial vehicle 30 may store and/or transmit photographed image information and distance information between the photographing point and the unmanned aerial vehicle 30 to the terminal 200 . In addition, the terminal 200 may analyze the captured image to determine whether defects such as cracks exist in the captured image. At this time, the terminal 200 may generate information on the size of a combination such as a crack in the captured image based on information on the distance between the unmanned aerial vehicle 30 and the capturing point.

12 is for explaining another example in which an inspection path is set so that an unmanned aerial vehicle passes through a non-shaded area a plurality of times. 13 and 14 show that the unmanned aerial vehicle inspects the bridge while moving along the inspection path.

Referring to FIG. 12, the unmanned aerial vehicle 30 departs from the non-shaded area nsr2 and enters the shaded area sr, which is the lower space of the bridge 11, and then inspects the inspection point while flying along the inspection point. Then, a first flight path (100p1) returning to the adjacent non-shaded area (nsr2) is set, and the unmanned aerial vehicle 30 enters the shaded area (sr) again from its current position on the non-shaded area (nsr2) and then inspects After inspecting the non-shaded area (nsr2), the second flight path (100p2) returning to the adjacent non-shaded area (nsr2) is set, and the unmanned aerial vehicle 30 returns to the shaded area from the current position on the non-shaded area (nsr2). After entering (sr), a third flight path 100p3 returning to the adjacent non-shaded area nsr1 after inspecting an untested inspection area may be set. In addition, an inspection path may be set based on the first and third flight paths 100p1, 100p2, and 100p3. Illustratively, in the first and second flight paths 100p1 and 100p2, the unmanned aerial vehicle 30 departs from the second unshaded area nsr2 and returns to the second unshaded area nsr2, , In the third flight path 100p3, the unmanned aerial vehicle 30 may start from the second unshaded area nsr2 and return to the first unshaded area nrs1 adjacent to the position at the end point after the end of the test.

In addition, the terminal 200 sets the flight paths 100p1, 100p2, and 100p3, and then sets the location search method switching start point information (tp) for starting to switch the location search method of the unmanned aerial vehicle 30 on the flight path 100p. You can set the final inspection path (inp) by setting .

In various embodiments, as shown in FIG. 13 , when the unmanned aerial vehicle 30 flies along an inspection point on the shaded area sr, it may move along a point with a height within a predetermined range. That is, in each of the first to third flight paths 100p1, 100p2, and 100p3, the heights of points to be inspected on the path along which the unmanned aerial vehicle 30 moves may be points having heights within a predetermined range a1 to a2. . In various embodiments, the altitude of the unmanned aerial vehicle 30 may be maintained when the unmanned aerial vehicle 30 flies over an inspection point having a height within a predetermined range (a1 to a2). That is, the altitude of the unmanned aerial vehicle 30 may be changed correspondingly when the inspection point is outside the predetermined range (a1 to a2). In addition, as shown exemplarily in FIG. 14, the unmanned aerial vehicle 30 moves along the inspection path and captures the inspection point to generate a photographed image and generates distance information between the unmanned aerial vehicle 30 and the captured inspection point. It can be stored and/or transmitted to the terminal 200. In addition, the unmanned aerial vehicle 30 generates a photographed image by photographing an inspection point in the left oblique direction as in (a) at an arbitrary position on the inspection route, and photographs an inspection point in the upward direction as in (b) A photographed image may be generated, and as in (c), a photographed image may be generated by photographing an inspection time point in a right oblique direction. The captured image may be displayed on the terminal 200 . A captured image may include a plurality of cracks CR1 , CR2 , and CR3 . The terminal 21 may additionally display crack size information or detected crack risk information based on distance information between the photographed inspection point and the unmanned aerial vehicle 30 .

15 schematically illustrates an unmanned aerial vehicle according to various embodiments of the present invention. And, Figure 16 is a schematic diagram for explaining a method of inspecting whether or not the inside of the inspection point is cracked using the internal crack inspection module. 17 schematically shows an internal crack inspection module further including a contact part cover according to another embodiment of the present invention.

4, 15 and 16, the unmanned aerial vehicle 30 according to various embodiments of the present invention may further include an internal crack inspection module 300. The internal crack inspection module 300 may be installed in the body 100a of the unmanned aerial vehicle 30.

The internal crack inspection module 300 is connected to the body 100a and is located inside the housing link 310 and the housing link 310 rotatable in various directions based on the point connected to the body 100a, and the housing link 310 An extension link 320 that can be inserted into or drawn from the housing link 310 and extendable, and a contact portion 330 that can reciprocate along the length of the extension link 320, as well as a plurality of motors (not shown) for driving them. can include In various embodiments, the audio module 170 may be installed at the end of the extension link 320 . In various embodiments, a microphone 172 may be installed at the end of the extension link 320 . The position of the extension link 320 is adjusted while the unmanned aerial vehicle 30 hovers over an arbitrary point of the inspection structure such as the bridge 11, and the extension link 320 moves from the housing link 310 toward the inspection point. It can be drawn out in the direction and extended. In addition, the contact portion 330 in the extension link 320 may apply a physical impact to the inspection point while reciprocating a plurality of times along the length direction of the extension link 320 . In addition, the unmanned aerial vehicle 30 may detect sound information received through the microphone 171 installed on the extension link 320 . In various embodiments, the unmanned aerial vehicle 30 detects sound information received through the microphone 171 before reciprocating driving of the contact unit 330, and detects sound information received through the reciprocating movement of the contact unit 330, and checks the test point and the contact unit 330. Sound information detected by mutual impact can be detected. In addition, the unmanned aerial vehicle 30 may transmit the detected sound information to the terminal 200 . The terminal 200 may analyze the received sound information and determine whether there is a crack inside the inspection point by analyzing sound obtained by removing noise according to ambient sound information from sound information detected by an impact applied to the inspection point.

In the embodiment, the presence or absence of cracks can be investigated by tapping the piers and the back of the bridge with a metal rod and using the difference between sound and vibration. In detail, when inspecting the state of the bridge 11, it is possible to determine whether or not there is an internal crack in the concrete through the sound generated by tapping the concrete pier with a metal rod by bringing the unmanned aerial vehicle 30 close to the inspection point of the pier and the bridge.

Referring further to FIG. 17 , according to various embodiments, the internal crack inspection module 300 may further include a contact part cover part 340 . The contact part cover part 340 may have an umbrella type shape. The contact part cover part 340 is installed in a plurality of the canopy 342 and the outer surface of the canopy 342 spaced apart from each other, and has a predetermined elasticity and maintains the shape of the canopy 342. ) Will include a hooking part 345 formed on one side, a plurality of spreaders 341 located outside the canopy 342, one side connected to the hooking part 345, and the other side connected to the extension link 320. can The hooking part 345 is inserted into the hooking part receiving groove 342 formed on one side of the spreader 341 and can reciprocate from one side of the spreader 341 along the longitudinal direction of the spreader 341 within a predetermined range. The hooking part receiving groove 342 and the hooking part 345 help to smoothly unfold the folded canopy 342 according to the movement of the spreader 341 and fold it back to the opposite side. Before the extension link 320 is extended from the housing link 310 , the canopy 342 is positioned to surround a portion of the housing link 310 . Also, when the extension link 320 extends from the housing link 310, the plurality of spreaders 341 spreads the canopy 342 while moving along the extension link 320, and the extension link 320 has a predetermined extension length. When the length exceeds the length, the canopy 342 wraps the extension link 320, and as the extension length of the extension link 320 increases, the area of the canopy 342 covering the extension link 320 increases. That is, the canopy 342 changes from a folded state in one direction to an unfolded state, and then changes to a folded state in the opposite direction to cover the extension link 320 and the contact portion 330 . In addition, when the contact unit 330 reciprocates multiple times along the longitudinal direction of the extension link 320 and applies a physical impact to the inspection point, the canopy 342 surrounds the contact unit 330, thereby preventing damage from the outside. By minimizing the introduction of noise into the microphone 171, the reception efficiency of the collision sound of the inspection point detected by the microphone 171 may be increased.

18 illustrates a main user interface among user interfaces displayed when a computer program for performing a method for measuring and managing cracks in a structure stored in a recording medium is executed on a terminal.

After the computer program 100 stored in the recording medium is executed on the terminal 200, the user's information is entered and the computer program 100 goes through a log-in procedure, and the main user interface 210 as shown in FIG. 18 is displayed. can be displayed

The main user interface 210 may include a work content display area 211 , a main menu affordance 212 , an integrated information display area 213 , and a display information control affordance 214 . The work content display area 211 may include a work content display area 211a for each menu and a crack information display area 211b that are displayed separately from each other. The display information control affordance 214 may include a first display information control affordance 214a and a second display information control affordance 214b. The terminal 200 removes the integrated information display area 213 displayed in response to selection of the first display information control affordance 214a (eg, a touch input or a click by an external device such as a mouse) or the removed The integrated information display area 213 can be displayed again. In addition, the terminal 200 may remove the displayed crack information display area 211b in response to the selection of the second display information control affordance 214b or display the previously removed crack information display area 211b again.

The main menu affordance 212 may include a crack information console affordance, a structure and crack setting affordance, a crack information analysis affordance, a record inquiry affordance, an adjustment affordance, and a system setting affordance. The terminal 200 may change submenu information displayed in the submenu affordance 215 in response to selection of any one of a plurality of affordances in the main menu affordance 212 . The terminal 200 may display the view all affordance and the view list affordance, which are secondary menu information displayed as the secondary menu affordance 215 in response to selection of the crack information console affordance. The terminal 200 may display structure registration affordance and inspection part registration affordance, which are auxiliary menu information displayed as the auxiliary menu affordance 215 in response to the selection of the structure and crack setting affordance. The terminal 200 displays the information displayed in the work content display area 211 in response to selection of any one of the affordances displayed as sub-menu information, such as in response to selection of any one of the displayed affordances of full view and list view affordances. can be changed In addition, the terminal 200 may control whether to display the crack map in the crack information display area 211b in response to selection of the measuring point map on the main user interface 210 . In addition, the terminal 200 may display the crack map grade view affordance 217 while displaying the crack map in response to the selection of the measuring point map. The terminal 200 responds to selection of at least one of a normal grade, a caution grade, and a risk grade in the crack map grade view affordance 217, and displays an item corresponding to the selected grade from among other items in the crack map, or displays the selected grade You can control whether only the corresponding item is displayed.

19 illustrates a main user interface including a first auxiliary user interface displayed when a computer program for performing a method for measuring and managing cracks in a structure stored in a recording medium is executed on a terminal.

18 and 19, the terminal 200 responds to selection of 'structure and crack setting affordance', which is one of the main menu affordances 212, and 'structure 'Registration Affordance' and 'Inspection Part Registration Affordance' may be displayed, and the terminal 200 displays previously matched information in response to selection of any one of 'Structure Registration Affordance' and 'Inspection Part Registration Affordance' in the work content display area. (211). The terminal 200 may display a task content display area 211a for each menu including the first auxiliary user interface 221 for registering a structure in response to selection of 'structure registration affordance'. A first auxiliary user interface 221 may be displayed in the work content display area 211a for each menu. Through the first auxiliary user interface 221, the user registers the structure by adding image information including the name of the structure, the location of the structure, a picture of the structure, or crack information of a partial area of the structure, or the information of the registered structure is modified or deleted. can In detail, the terminal 200 may store information input to the first auxiliary user interface 221 in response to selection of the additional affordance in the first auxiliary user interface 221 . Information on registered structures may be listed and displayed on the work content display area 211a for each menu.

20 illustrates a main user interface including a second auxiliary user interface displayed when a computer program for performing a method for measuring and managing cracks in a structure stored in a recording medium is executed on a terminal.

18 to 20, the terminal 200 displays a work content display area 211a for each menu including a second auxiliary user interface 222 for registering an inspection part in response to selection of 'inspection part registration affordance'. can be displayed The user may input information about an ID for identifying an inspection part in a structure, a location of the inspection part, an inspection method, and an inspection period through the second auxiliary user interface 222 . In addition, the user may match and store the information input to the second auxiliary user interface 222 with a pre-registered structure. In detail, the terminal 200 may store information input to the second auxiliary user interface 222 in response to selection of an additional affordance in the second auxiliary user interface 222 . Information on inspection parts of each of the registered structures may be listed and displayed on the work content display area 211a for each menu. In addition, the terminal 200 may arrange information on each of the registered structures and each of the lower inspection parts in a tree form and display the information on the crack information display area 211b. In response to selection of any one item of tree information displayed in the crack information display area 211b, the terminal 200 displays information on a structure matched with the selected item and an inspection part of the structure in a work contents display area 211a for each menu. can be displayed on In detail, in the embodiment, in order to analyze and register cracks, they may be classified into a cracked structure and an inspection part of the structure, analyzed and stored. And, it can be saved in the order of structure, inspection part and crack. For example, Geochang Bridge divides the crack target into Geochang Bridge piers and Geochang Bridge upper slabs, each of which is set as a structure, and then an inspection part can be set after each structure part.

Illustratively, structures and inspection parts can be set in the order shown in Table 1 below.

예 1Example 1
거창대교 > 거창대교 교각 > Geochang Bridge > Geochang Bridge Pier >	교각1 > 교각1 크랙 Pier 1 > Pier 1 crack
	교각2 > 교각2 크랙Pier 2 > Pier 2 Crack
	교각n > 교각n 크랙Pier n > Pier n crack
예 2example 2
거창대교 > 거창대교 상판 >Geochang Bridge > Geochang Bridge Deck >	상판1 > 상판1 크랙 Top board 1 > Top board 1 crack
	상판2 > 상판2 크랙Top 2 > Top 2 crack
	상판n > 상판n 크랙Top board n > Top board n crack

21 and 22 show a main user interface that displays crack information monitoring information as a main user interface displayed when a computer program for performing a method for measuring and managing cracks in a structure stored in a recording medium is executed on a terminal. Referring to FIGS. 18, 21, and 22, after setting the structure and cracks as described above, the terminal 200 responds to the selection of 'crack information console affordance', which is one of the main menu affordances 212, as described in FIG. 18. While returning to the main user interface 210, crack information monitoring information may be displayed in the work content display area 211a for each menu. The terminal 200 displays a crack information image item in response to selection of 'view all affordance', which is one of the auxiliary menu affordances 215 (FIG. 21), and displays a list of crack information in response to selection of 'list view affordance'. can be displayed (FIG. 22). The crack information image item may include the inspection part name set in the structure crack setting, the structure inspection part image analysis result, the crack image according to the structure inspection part image analysis result, and the registration date information of the structure inspection part image analysis result. In addition, the terminal 200 may display an image showing the state of the inspection part so that the user can easily determine the status of the inspection part in the work content display area 211a for each menu in response to 'selection of view all affordance'. there is. In addition, the terminal 200 responds to selection of any one of a plurality of crack information image items displayed on the work content display area 211a for each menu, and if the selected item is a normal inspection part, it is displayed in green, and the inspection time If more than this set time interval has elapsed, it is displayed in gray, and if there is a part where the inspection time has elapsed, it can be displayed as “inspection request”. And this display can be made through the notification window 216 . In addition, the terminal 200 may display a list of crack information so that the state of inspection parts for each structure can be monitored in a list mode in the work content display area 211a for each menu in response to 'selection of list view affordance'. . When a user clicks a label at the top of the list, the user can search in descending or ascending order based on the clicked column. FIGS. 23 to 26 show a computer program for performing a method of measuring and managing cracks in a structure stored in a recording medium on a terminal. It shows the main user interface including the third auxiliary user interface displayed when executed.

Referring to FIG. 23 , the terminal 200 may display a third auxiliary user interface 223 in response to selection of 'crack information analysis affordance', which is one of the main menu affordances 212 . The terminal 200 displays information on a plurality of registered structures in response to selection of 'structure selection affordance' in the crack analysis target selection area 223a displayed on the third auxiliary user interface 223, and displays the plurality of displayed structures. In response to the selection of any one of them, inspection part information matched to the selected structure can be read. Then, the terminal 200 displays at least one inspection part information list matched to the structure selected in the selection of 'inspection part selection affordance' in the crack analysis target selection area 223a displayed on the third auxiliary user interface 223, In response to selection of any one inspection part in the displayed inspection part information list, image information matched with the selected inspection part and stored may be called and displayed. According to the illustrated example, Geochang Bridge was selected as the structure and Pier 1 was selected as the inspection part. The terminal 200 may display size information of the original image and distance information between a camera mounted on the unmanned aerial vehicle 30 and an inspection point on the image at the time of capturing the image. According to the illustrated example, the size of the original image is 2000x1500 (number of pixels), and the photographing distance of the image is displayed as 300 cm. In various embodiments, the resolution of the image to be analyzed for cracks may be lowered in order to increase data processing speed in which the terminal 200 analyzes and detects cracks on the image. Illustratively, the terminal 200 may convert the size of the original image and display it as the first image p1. The terminal 200 may display the first image p1 after generating the first image p1 by reducing the size of the original image. According to the illustrated example, a first image p1 having a size of 800x537 (number of pixels) obtained by reducing the size of an original image may be generated and displayed. The terminal 200 may detect cracks in the first image p1, detect cracks corresponding to the detected cracks in the original image, and then detect length and width information of the cracks detected in the original image. The user can directly set R, G, and B values set in the crack color value setting area 223b. Unlike this, the user selects 'color detection affordance' in the crack color value setting area 22b, detects the color value of the pixel corresponding to the selected point in the displayed first image p1, and responds to the detected color value. It is possible to automatically assign crack color values to the R, G, and B values of That is, as shown exemplarily in FIG. 24 , the terminal 200 detects the color value of the pixel corresponding to the selected point in the displayed first image p1 and the R, G, and B values corresponding to the detected color value. The crack color value can be displayed with . In addition, the user can designate the range of cracks. The user may select one of 'affordances related to manual trajectory display, area display, and display erasure' in the crack range designation area 223c. As shown exemplarily in FIG. 25 , the terminal 200 may allow the user to manually display the crack trajectory in the first image p1 in response to selection of 'trajectory manual display area affordance'. In addition, the terminal 200 may allow the user to manually display the crack area in the first image p1 in response to the selection of 'area display affordance'. In addition, the terminal 200 may remove the previously displayed trajectory or region from the first image p1 in response to the selection of the 'display erase affordance'.

Referring to FIG. 26 , the terminal 200 detects cracks by analyzing pixels corresponding to the set color values in the first image p1 in response to selection of 'crack analysis affordance' in the third auxiliary user interface 223. can do. And, the terminal 200 may display the detected crack as the second image p2. In addition, the terminal 200 may detect cracks through comparative analysis of color values of pixels within a crack range selected by the user and display the detected cracks as the second image p2 . In addition, the terminal 200 may automatically detect cracks based on analysis of color values of pixels in the first image p1 in response to selection of 'auto crack analysis affordance' in the third auxiliary user interface 223. may be In addition, the terminal 200 may analyze the original image and display information on the length of cracks detected, information on the average width of cracks, and information on the number of cracks. As exemplarily indicated in the drawing, the total length of cracks is 3,036.5 mm, the average width of cracks is 6,8 mm, and the number of cracks (dots) having a predetermined length or more is indicated as two. In addition, the terminal 200 may store the analyzed crack information in response to selection of 'storage affordance' to form a database. In addition, the terminal 200 may store crack grade information, crack registration ID information, and crack description information input to the third auxiliary user interface 223 .

27 and 28 are schematic diagrams for explaining a method of detecting the length of a crack in an original image.

Referring to FIG. 27 , the terminal 200 may detect crack pixels cp.x corresponding to the crack detected in the first image p1 from among a plurality of pixels in the original image op. The terminal 200 may group crack pixels cp.x with each other. The terminal 200 may set the crack pixel cp.x as one group when there is a crack pixel cp.x among 8 pixels closest to any one crack pixel cp.x. According to the illustrated example, first and second crack pixel groups cpg1 and cpg2 exist. The terminal 200 calculates the number of crack pixels (cp.x) in each crack pixel group (cpg1, cpg2) to obtain a preset number of crack pixels (cp.x) in each crack pixel group (cpg1, cpg2). If it is less than, the length of the crack may be estimated according to the first crack length detection algorithm.

Hereinafter , the first crack length detection algorithm will be described in detail.

The terminal 200 may set a virtual center point (cp.x_p) to each of the crack pixels (cp.x) in any one crack pixel group. In addition, a virtual pixel direction line pd connecting the virtual center point cp.x_p for each of the plurality of crack pixels cp.x is searched while searching for each of the plurality of crack pixels cp.x. can be set For example, the terminal 200 provides a virtual center point (cp.x_p) of one crack pixel (cp.x) or a virtual center point (cp.x_p) of one crack pixel (cp.x) and a crack adjacent thereto. It is possible to set a pixel direction line (pd) in eight directions passing through the center point of the pixel (cp.x). The terminal 200 establishes pixel direction lines pd in eight directions for each of the crack pixels cp.x, and sets a virtual center point cp among points constituting the pixel direction lines pd in eight directions. .x_p) can be set with virtual lines connecting each other. In addition, a line segment having the longest length among set virtual segments may be detected, and the length of the detected segment may be set as the length of the crack.

Referring to FIG. 28, cracks exist in two areas on the original image op, the longest virtual line segment in each area is detected, and the length of the detected virtual line segment (cr.lg) can be set as the length of the crack.

29 is a schematic diagram for explaining a method of detecting the thickness of a crack in an original image.

Referring to FIGS. 27 and 29 , the terminal 200 crosses the detected virtual line segment corresponding to the length of the crack in FIG. 28 and passes through the virtual center point cp.x_p of the crack pixel cp.x. A plurality of virtual pixel thickness direction lines ptd can be set. In addition, virtual line segments connecting virtual center points cp.x_p among the plurality of pixel thickness direction lines ptd may be set. In addition, a line segment having the longest length among the set segments may be detected, and the length of the detected segment may be set as the maximum thickness of the crack. In addition, the shortest line segment among virtual line segments connecting the virtual center points (cp.x_p) may be detected, and the length of the detected line segment may be set as the minimum thickness of the crack. If there is a line ptd in the pixel thickness direction passing only the virtual center point (cp.x_p) of one crack pixel, the minimum thickness of a crack is the width of the crack pixel in the diagonal direction or the width in the horizontal or vertical direction. Length can be a minimum thickness. For example, if the pixel thickness direction line (ptd) is a diagonal line, the minimum thickness is the width of the crack pixel in the diagonal direction, and the pixel thickness direction line (ptd) corresponds to the width or height of the original image (op) of a rectangle. In the case of non-slanting lines, the minimum thickness may be the length of the horizontal or vertical width of the crack pixel.

30 and 31 are for explaining a method of detecting the total length of a crack when the number of crack pixels in a crack pixel group equal to or greater than a preset value exists.

Referring to FIG. 30 , the terminal 200 may estimate the length of a crack according to the second crack detection algorithm when it is determined that there are crack pixels (cp.x) greater than or equal to a predetermined value in a crack pixel group.

The terminal 200 1) sets the leftmost crack pixel (cp.x) among the uppermost crack pixels (cp.x) as the starting point crack pixel (cp.xs), and sets the rightmost crack pixel (cp.xs). Among (cp.x), the rightmost crack pixel (cp.x) may be set as the end-point crack pixel (cp.xe). In addition, a first partial crack line segment pcr.l1 connecting the center points of the start crack pixel cp.xs and the end crack pixel cp.xe to each other may be set. Next, 2) in the horizontal direction from among the crack pixels cp.x through which the first partial crack segment pcr.l1 passes (excluding the start crack pixel and the end crack pixel of the first partial crack segment pcr.l1) A horizontal crack pixel group area, which is an area with the largest number of crack pixels (pc.x) in contact with each other, may be detected. If a plurality of areas having the same number of crack pixels exist, an area adjacent to the viewpoint crack pixel cp.xs may be detected. Subsequently, a second partial crack line segment (pcr.l2) may be set to connect the center point of the leftmost crack pixel (start point) and the rightmost (end point) crack pixel center point in the corresponding region. Also, the second partial crack line segment pcr.l2 may be defined as a horizontal crack line segment. Next, 3) a plurality of crack pixels (cp.x) located in the right direction based on the center point of the rightmost (end point) crack pixel of the second partial crack segment (pcr.l2) and the center point of the rightmost crack pixel. A line segment satisfying the following condition 1-2 may be detected from among virtual lines connecting center points to each other, and the detected line segment may be set as a third partial crack line segment (pcr.l3).

[부분 크랙 선분 설정 조건 1][Partial crack segment setting condition 1]

1-1. “After setting the horizontal or vertical crack segment, when setting the next partial crack segment, the horizontal crack pixel group area that the horizontal or vertical crack segment passes through is not detected.”

1-2. “+1 point is calculated for every line segment passing through one cracked pixel (cp.x) and -0.5 point is calculated each time it passes through one non-cracked pixel (ncp.x), and the line segment with the highest total is detected. ”

Next, 4) Among the crack pixels pc.x through which the third partial crack segment pcr.l3 passes (excluding the start crack pixel and the end crack pixel of the third partial crack segment pcr.l3) in the horizontal direction Detects the horizontal crack pixel group area, which is the area with the largest number of crack pixels (pc.x) in contact with each other, and connects the center point of the leftmost crack pixel and the rightmost crack pixel center point in the area. A line segment (pcr.l4) can be set, and this is the same method as the process of 2) above. In addition, the fourth partial crack line segment pcr.l4 may be defined as a horizontal crack line segment.

Next, 5) connected from the center point of the rightmost (end point) crack pixel of the fourth partial crack line segment (pcr.l4) to the center point of rightmost crack pixels (cp.x) of the rightmost (end point) crack pixel. Among the virtual lines, a line segment satisfying the above condition 1-2 may be detected, and the detected line segment may be a fifth partial crack segment (pcr.l5), in the same manner as the process of 3) above.

Next, 6) among the crack pixels pc.x through which the fifth partial crack segment pcr.l5 passes (excluding the start crack pixel and the end crack pixel of the fifth partial crack segment pcr.l5) in the horizontal direction Detects a horizontal crack pixel group area, which is an area with the largest number of crack pixels (pc.x) in contact with each other, and detects a case where the number of crack pixels (pc.x) in the area is less than a preset value, and a fifth partial crack line segment (pcr. The process of generating the partial crack line segment is terminated in accordance with condition 2, which is a case in which the end point crack pixel of l5) and the end point crack pixel (cp.xe) of the first partial crack line segment (pcr.l1) coincide with each other. The preset value here is a case where the number of crack pixels is three.

[부분 크랙 선분 설정 조건 2][Partial crack segment setting condition 2]

“When the number of crack pixels in the horizontal crack pixel group area is less than 3, and the end point of the last partial crack segment coincides with the end point of the first partial crack segment”

Next, 7-1) a path extending from the viewpoint crack pixel cp.xs of the first partial crack line segment pcr.l1 to the first extension point ep1 is formed. The first extension point satisfies Condition 1 below.

[경로 설정 조건 1][Route setting condition 1]

If the point at which the partial crack line segment and the horizontal crack line segment meet is the center point of an arbitrary crack pixel, the center point is the extension point. Otherwise, the center point of the horizontal crack line segment is the extension point.

According to the illustrated example, the partial crack line segment in condition 2 is the first partial crack line segment (pcr.l1), and the horizontal crack line segment is the second partial crack segment (prc.l2).

Next, 7-2) When condition 2 is not satisfied, a path is extended from the first extension point ep1 to the end point crack pixel of the horizontal crack line segment.

[경로 설정 조건 2][Route setting condition 2]

2-1. “If the partial crack segment created after the horizontal crack segment is a horizontal or vertical crack segment, the next extension point is the midpoint of the horizontal or vertical crack segment”

2-2. “If the partial crack line segment created after the horizontal crack segment is a horizontal or vertical crack segment, and there is a horizontal crack segment set after the horizontal or vertical crack segment that intersects the horizontal or vertical crack segment, the end point of the horizontal crack segment set later extended to the center point of the crack pixel, otherwise to the center point of the crack pixel end point of the horizontal or vertical crack line segment”

Next, the route is extended according to the process of 7-3) 7-1) described above. According to the illustrated example, since the point where the third partial crack line segment prc.l3 and the fourth partial crack line segment pcr.l4 meet is not the center point of the crack pixel cp.x, the fourth partial crack line segment ( pcr.l4) becomes the second extension point ep2. Accordingly, a path extends from the center point of the end point crack pixel of the second partial crack line segment prc.l3 to the second extension point ep2.

Next, 7-4) Since the fifth partial crack segment (pcr.l5) generated after the fourth partial crack segment (pcr.l4) corresponding to the horizontal crack segment does not satisfy the path setting condition 2-1, A path extends from the second extension point ep2 to the center point of the end point crack pixel of the fourth partial crack line segment pcr.l4.

Next, 7-5) The end point of the fourth partial crack segment pcr.l4 extends from the center point of the crack pixel along the fifth partial crack segment pcr.l5 to the end point of the first partial crack segment pcr.l1. A path is extended to the crack pixel (cr.xe). Since the path has reached the end point of the first partial crack line segment pcr.l1 and the center point of the crack pixel cr.xe, path setting is ended. In addition, the terminal 200 may estimate the total length of the path detected in the above-described manner as the length of the crack (cr.lg).

When the method described in FIG. 31 is applied to the crack pixel group according to FIG. 31 , the length of the crack (cr.lg) can be estimated.

32 and 33 are for explaining a method of detecting the total length of cracks in a form different from that of FIGS. 30 and 31 when the number of crack pixels in the crack pixel group is equal to or greater than a preset value.

Referring to FIG. 32, the terminal 200 1) sets a starting crack pixel (cp.xs) and an ending crack pixel (cp.xe), and connects their center points to a first partial crack line segment (pcr.l1). ) can be set. Next, 2) the horizontal crack pixel group region is detected within the crack pixel (pc.x) emf through which the first partial crack line segment (pcr.l1) passes, and the leftmost crack pixel and rightmost crack pixel in the region are detected. A second partial crack line (pcr.l2), which is a horizontal crack line connecting center points, may be set.

Next, 3) the center point of each of the crack pixels cp.x located in the right direction based on the center point of the rightmost crack pixel of the second partial crack line pcr.l2 and the second partial crack line segment pcr.l2 ), a line segment satisfying the above-described 'partial crack line segment setting condition 1-2' is detected from among virtual line segments connecting the center points of the rightmost crack pixels, and the detected line segment is a third partial crack line segment (pcr.l3 ) can be set. Here, the third partial crack line (pcr,l3) is a vertical crack line, which corresponds to the vertical crack line referred to in 'route setting condition 2'. This satisfies the partial crack line segment setting condition 1-1.

Therefore, 4) Since the third partial crack segment pcr.l3 corresponds to the vertical crack line segment, again, based on the end point crack pixel of the third partial crack segment pcr.l3, the third partial crack segment pcr.l3 ), among virtual line segments connected from the end point crack pixel to the center point of the crack pixels (cp.x) located in the right direction, a line segment satisfying the above-described 'partial crack line segment setting condition 1-2' is detected, and the detected line segment is removed. It can be set as a 4-part crack line segment (pcr.l4).

Next, 5) Among the crack pixels cp.x through which the fourth partial crack segment pcr.l4 passes (excluding the start crack pixel and the end crack pixel of the fourth partial crack segment pcr.l4) in the horizontal direction It is possible to detect a horizontal crack pixel group region with the largest number of crack pixels (cp.x) in contact with each other, and set a fifth partial crack line segment (pcr.l5) connecting the center points of the crack pixels (cr.x) in the corresponding region. there is.

Next, 6) Since the fifth partial crack segment (pcr.l5) corresponds to a horizontal crack segment, the fifth partial crack segment (pcr.l5) based on the end point crack pixel of the fifth partial crack segment (pcr.l5) Among the virtual line segments connected from the end point crack pixel to the center point of the crack pixels (cp.x) located in the right direction and the center point of the end point crack pixel of the fifth partial crack line segment (pcr.l5), the above-described 'partial crack line segment setting condition 1-2' ', and the detected line segment may be set as the sixth partial crack line segment (pcr.l4).

Next, 7) cracks in the horizontal direction in an area where the number of crack pixels (cr.x) in contact with the horizontal direction is the largest among the crack pixels (cr.x) that the sixth partial crack segment (pcr.l4) passes through. When the number of pixels pc.x is less than the preset value, the end point crack pixel of the sixth partial crack line pcr.l6 and the end point crack pixel cp.xe of the first partial crack line pcr.l1 are mutually exclusive. In the case of coincidence, since 'partial crack line segment setting condition 2' is satisfied, the process of generating the partial crack line segment is terminated.

Next, a path creation process will be described. 8-1) A path extending from the viewpoint crack pixel cp.xs of the first partial crack line segment pcr.l1 to the first extension point is formed. According to the following 'path setting condition 1', the first extension point becomes the center point of the second partial crack line segment pcr.l2.

Next, 8-2) Since it corresponds to the case where the above-mentioned 'route setting condition 2' is satisfied, the path is extended from the first extension point to the center point of the third partial crack line pcr.l3, which is the vertical crack line segment.

Next, 8-3) Since the above-mentioned path setting condition 2-2 is satisfied, the fifth partial crack intersects the third partial crack line segment (pcr.l3) from the center point of the third partial crack line segment (pcr.l3). A path is extended to the center point of the end point crack pixel of the line segment pcr.l5.

Next, 8-4) Since the end point of the fifth partial crack line segment (pcr.l5) corresponds to the end point of the first partial crack line segment (pcr.l1), the end point of the fifth partial crack line segment (pcr.l5) is of the crack pixel. It is extended to the center point, and the path setting is finished.

The terminal 200 may estimate the total length of the path detected in the above manner as the length of the crack (cr.lg).

When the method described in FIG. 33 is applied to the crack pixel group according to FIG. 33 , the length of the crack (cr.lg) can be estimated.

34 is for explaining a method of processing crack pixel groups spaced apart by less than a preset value as one crack.

Referring to FIG. 34 , the terminal 200 may set crack pixel groups cpg1 , cpg2 , and cpg3 by detecting crack pixels cp.x corresponding to cracks on the original image op. In various embodiments, when the number of crack pixels in a crack pixel group is less than a preset value, the corresponding group may be ignored, eg, the corresponding crack pixel may be determined as a non-crack pixel.

The terminal 200 may group the effective crack pixel groups cpg1 and cpg2 into one crack pixel group when the distance between them is less than a preset value. Further, the terminal 200 detects the length of cracks in each of the first and second crack pixel groups, and calculates the sum of the lengths of each crack and the crack pixels in one group and the crack pixels in another group (these are in different groups). The total length of cracks can be set as the result of summing the spaced distances between the cracks that belong to each other but are located closest to each other). In addition, the terminal 200 calculates the average value of the thickness of cracks detected from crack pixels in one group that belong to different groups but are positioned closest to each other and the thickness of cracks detected from crack pixels in the other group, in a group and another group. It can be set as the thickness of cracks at spaced points.

In various embodiments, the terminal 200 sets an average value of the thickness of cracks detected from crack pixels in one group belonging to different groups but positioned closest to each other and the thickness of cracks detected from crack pixels in another group as the thickness. An additional crack pixel that does not exist may be added on the group and spaced apart areas of the other group.

In various embodiments, the terminal 200 sets a partial crack line segment connecting the center line of the start crack pixel and the center point of the end crack pixel, and the thickness at one point of the partial crack line segment corresponding to the separation area is the additional crack pixel. An area where additional crack pixels are inserted may be determined to match the thickness of the group within a preset range.

35 illustrates a main user interface including a fourth auxiliary user interface displayed when a computer program for performing a method for measuring and managing cracks in a structure stored in a recording medium is executed on a terminal.

Referring to FIG. 35 , the terminal 200 may display a fourth auxiliary user interface 224 in response to selection of 'record inquiry affordance' from the main menu affordance 212 .

The fourth auxiliary user interface 224 provides a registered structure, an inspection part of the corresponding structure, and a crack ID. Crack images, inspection points, crack information, and analysis results can be listed and displayed. In addition, the terminal 200 provides an interface capable of selectively displaying all or part of the registered structure or all or part of the inspection part of the structure, and displays information in which the registration date matches the inquiry period input by the user. You can also provide an interface that allows

Embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium.

The present invention can be used in the field of unmanned aerial vehicles.

Claims

main body;

a camera position control module connected to the main body;

a camera installed in the camera position control module;

LiDAR position control module connected to the main body; and

A lidar sensor installed in the lidar position control module and detecting distance information to an inspection point in order to determine the size of a crack of an inspection structure detected in a photographed image detected by the camera; includes,

The camera position is changed by the camera position control module, and the position of the lidar sensor is changed by the lidar position control module.

An unmanned aerial vehicle with a lidar sensor for measuring the crack thickness of a structure.
According to claim 1,

Based on the photographed image information generated by photographing a plurality of different inspection points in the inspection structure in the hovering state and the distance information of each of the plurality of different inspection points, information on the presence or absence of cracks and the size of cracks in the image is obtained. to decide

An unmanned aerial vehicle with a lidar sensor for measuring the crack thickness of a structure.
According to claim 1,

The altitude is changed based on the shape information of the surface of the inspection structure and the distance information to the inspection point measured by the lidar sensor.

An unmanned aerial vehicle with a lidar sensor for measuring the crack thickness of a structure.
According to claim 3,

It further includes; a GNSS (Global Navigation Satellite System) module;

Estimating current position information based on shape information of the surface of the inspection structure and measurement information of the lidar sensor in the shaded area, which is a lower area of the inspection structure,

Detecting current location information by the GNSS module in a non-shaded area other than the shaded area,

A preset inspection path for inspecting the inspection structure in the shaded area is set to pass through the non-shaded area a plurality of times.

An unmanned aerial vehicle with a lidar sensor for measuring the crack thickness of a structure.
According to claim 1,

An internal crack inspection module installed in the main body to apply a physical impact to the inspection point in order to determine whether or not there is a crack inside the inspection point of the inspection structure; further comprising,

The internal crack inspection module includes an extension link extending to the inspection point and a contact portion that reciprocates in the longitudinal direction of the extension link and applies a physical impact to the inspection point by contacting the inspection point.

An unmanned aerial vehicle with a lidar sensor for measuring the crack thickness of a structure.
According to claim 1,

Further comprising a canopy covering the extension link and the contact portion in response to the extension of the extension link in order to increase the reception efficiency of the sound by the physical impact received through the microphone installed in the extension link

An unmanned aerial vehicle with a lidar sensor for measuring the crack thickness of a structure.
A computer program for performing a method of measuring and managing cracks of a structure to be inspected,

Executed in a terminal to display different sub-menu affordances and a task content display area and/or auxiliary user interface matched to the sub-menu affordance in response to selection of any one of a plurality of affordances in the main menu affordance;

Displaying structure registration affordance and inspection part registration affordance in the sub menu affordance in response to selection of structure and crack setting affordance in the main menu affordance,

In response to selection of either the structure registration affordance or the inspection part registration affordance, the information of the structure to be inspected is registered, the information of at least one inspection part constituting the structure to be inspected and the inspection period information are registered, and the inspection part is photographed. registration of images,

The registered captured image is analyzed to detect and display cracks in the captured image, and the length of the crack detected based on the distance information between the UAV and the target in the captured image measured by the UAV and the crack detects and displays the thickness information of

A computer program stored in a recording medium to perform the method of measuring and managing cracks in a structure.
According to claim 7,

In response to selection of either the structure registration affordance or the inspection part registration affordance, the information of the structure to be inspected is registered, the information of at least one inspection part constituting the structure to be inspected and the inspection period information are registered, and the inspection part is photographed. When registering an image,

Setting and displaying a crack map with structures as upper elements and inspection parts of each structure as lower elements in the work contents display area

A computer program stored in a recording medium to perform the method of measuring and managing cracks in a structure.
According to claim 7,

Displaying information on a registered structure and a plurality of inspection parts of the registered structure, and determining whether or not to display an inspection request based on the inspection cycle of the selected inspection part in response to the selection of a plurality of displayed inspection parts

A computer program stored in a recording medium to perform the method of measuring and managing cracks in a structure.
According to claim 7,

display the captured image;

Detecting cracks in the photographed image by comparing and analyzing color values between pixels based on a set color value or within a crack range specified by the user, and displaying an image displaying the detected cracks

A computer program stored in a recording medium to perform the method of measuring and managing cracks in a structure.
According to claim 7,

Detecting crack pixels corresponding to cracks in the photographed image;

grouping adjacent crack pixels to establish a crack pixel group;

Setting a virtual center point for each of the crack pixels in any one crack pixel group;

While searching for each of a plurality of arbitrary crack pixels, setting a virtual pixel direction line connecting virtual center points for each of the plurality of crack pixels;

Setting virtual first lines connecting virtual center points of different crack pixels among points constituting the pixel direction line;

Detecting the longest line segment among the set virtual first lines and setting the length of the detected line segment as the length of the crack;

Setting a plurality of virtual pixel thickness direction lines perpendicular to the detected virtual line segment corresponding to the length of the crack and passing through the virtual center point of the crack pixel;

Setting the length of the set second line segment to the thickness of the crack by setting a second virtual line segment connecting virtual center points among the plurality of pixel thickness direction lines to each other

A computer program stored in a recording medium to perform the method of measuring and managing cracks in a structure.
According to claim 11,

If the number of crack pixel groups have a separation distance less than the preset value, set them as one crack,

Based on the thickness value of the crack pixels in each group, at least one chukka crack pixel is inserted into the spaced space between crack pixels belonging to different crack pixel groups with a distance less than the preset value to form spaced apart crack pixel groups. which are connected to each other to form a single pixel group.

A computer program stored in a recording medium to perform the method of measuring and managing cracks in a structure.
According to claim 7,

Detecting crack pixels corresponding to cracks in the photographed image;

Set a partial crack line segment that interconnects the center point of the leftmost and uppermost start point crack pixel and the lower right and lower end crack pixel center point,

In step 1, a horizontal crack pixel group region with the largest number of crack pixels contacting each other in the horizontal direction is detected on the path passing through the partial crack line segment, and the center point of the leftmost crack pixel and the rightmost crack pixel in the horizontal crack pixel group are detected. Set horizontal crack segments interconnecting the center points of

In step 2, a predetermined condition is determined among virtual lines connecting the center point of the rightmost crack pixel of the horizontal crack segment and the center point of a plurality of crack pixels located in the right direction based on the center point of the rightmost crack pixel. Detect a satisfied line segment, set the detected line segment as the next partial crack line segment,

When the end point crack pixels of the crack line segment generated by repeating steps 1 and 2 above match the end point crack pixels located on the right side and bottom of the line segment, a path passing through at least a part of the line segment is set according to a preset condition, and the set path to the length of the crack

A computer program stored in a recording medium to perform the method of measuring and managing cracks in a structure.