KR20200067286A - 3D scan and VR inspection system of exposed pipe using drone - Google Patents

Abstract

The present invention relates to 3D scanning and VR inspecting system of an exposed pipe using a drone, which secures an image of an exposed pipe by using a photographing device mounted on a drone, generates a 3D image by inputting the secured image to a modeling device, and enables a user to conveniently and safely check a state of the exposed pipe by checking the generated 3D image through a VR device. According to the present invention, the 3D scanning and VR inspecting system of an exposed pipe using a drone comprises: one or more inspection drones; a photographing device mounted on the drones and photographing an exposed pipe installed in a bridge at various angles to generate a plurality of input images; one or more control devices for controlling the drones and the photographing device; a modeling device installed with a modeling program for generating a 3D image by using the input images; and a VR device for realizing the 3D image as virtual reality to provide the same for a user.

Description

3D scan and VR inspection system of exposed pipe using drone}

The present invention relates to a system that enables to check the state of an exposed pipe installed in a building, a bridge, etc. using drones and VR technology, and more specifically, secures an image of the exposed pipe using a shooting device mounted on the drone. Then, the secured image is input to the modeling device to generate a 3D image, and the generated 3D image can be checked through a VR device, so that the user can comfortably and safely check the status of the exposure piping using a drone. It is about 3D scan of pipe and VR inspection system.

In general, exposed pipes installed in buildings, bridges, etc. should be repaired if leaks are found through periodic inspections, but it is difficult for workers to access due to various reasons such as the location, height, or length of the pipes installed. It is difficult to inspect smoothly, and workers may be exposed to danger.

Conventional inventions related to piping inspection include “Pipe Monitoring System and Piping Monitoring Method” of Korean Patent Registration No. 10-1731450 and “Ultra large structure ultrasonic inspection method using drone” of Korean Patent Publication No. 10-2018-0025413. Korean Patent Application Publication No. 10-2018-0036299, “Real-time monitoring device for remote gas leakage and piping exterior inspection using drones” and Korean Registered Patent Publication No. 10-1858009 “Drone for bridge inspection” have been proposed and released have.

The Republic of Korea Patent Registration No. 10-1731450, "Pipe Monitoring System and Piping Monitoring Method," the first piping sensor device capable of generating and transmitting measurement data on the integrity of the piping, and the location where the measurement data of the first piping sensor can be received. It is configured to include a mobile node that moves to and moves it to a location where it can be transmitted to a monitoring server, thereby reducing the installation cost and maintenance cost of the sensor device, and inventing a system and method that can increase the monitoring efficiency of piping This has been proposed, and in the Korean Patent Application Publication No. 10-2018-0025413, “Method for Ultrasound Inspection of Large Structures Using Drones”, it is easy to change the laser pulse irradiation point toward the structure by adjusting the flight position of the drone equipped with the laser reflector. The invention has been proposed for a method configured to easily acquire ultrasonic surface vibration of a large high-rise structure using a non-contact measuring means mounted on a drone.

In addition, in the Republic of Korea Patent Publication No. 10-2018-0036299, "a real-time monitoring device for remote gas leakage and piping appearance inspection using a drone," the laser gas detector and the leak gas thermal image data to obtain the leak gas data to the drone. The invention is proposed to be equipped with a thermal image acquisition camera, etc. to be acquired, and to efficiently and accurately perform gas leak and pipe appearance inspection even in environments where there is a space limitation in pipe inspection such as high-rise apartments or large bridges. In the "Patent Inspection Drone" of the Republic of Korea Patent Publication No. 10-1858009, the horizontal protection guard and the vertical protection guard installed in the drone body can protect the drone in case of collision with foreign objects and check the bridge An invention has been proposed for a device capable of securing a clear image by minimizing the obstruction of the camera.

However, the above-described prior arts have a problem in that it is difficult to freely and closely inspect by the user because the exposure pipe must be checked only with limited images collected by shooting, and the method using a gas detector using a laser or a camera for acquiring a thermal image is exposed. Since it is possible to check the damage only after the gas leaks from the pipe, there is a problem that the condition just before the breakage cannot be grasped, and since measurement is impossible, there is a problem that the size of the damaged part and the displacement due to the deformation of the pipe cannot be measured. There is a need for a solution to this problem.

On the other hand, VR (Virtual Reality) refers to a virtual reality implemented by devices such as computers, which can be experienced by users through VR devices, which are display devices, and actively introduced into various fields such as games, medicine, and military fields. It is used and is the latest technology that is expected to be further developed in the future.

Republic of Korea Registered Patent Publication No. 10-1731450 (2017. 04. 24) Republic of Korea Patent Publication No. 10-2018-0025413 (2018. 03. 09) Republic of Korea Patent Publication No. 10-2018-0036299 (2018. 04. 09) Republic of Korea Registered Patent Publication No. 10-1858009 (2018. 05. 17)

The 3D scanning and VR inspection system of the exposure pipe using the drone according to the present invention is a technique proposed to solve the problems of the conventional invention as described above,

The exposed piping requires periodic inspection because the shape and properties of the material may change due to aging and external impacts, but there are problems that workers cannot access due to various reasons such as the location, height, or length of the piping.

Since it is necessary to check the exposure piping with only the limited images collected by shooting, a problem occurred freely and closely by the user.

The method using a gas detector using a laser or a camera for acquiring a thermal image can confirm the damage only after the gas leaks from the exposed pipe, so that the state immediately before the breakdown cannot be grasped.

Since there is a problem in that it is impossible to measure the displacement due to the size of the damaged part and the deformation of the pipe since measurement is not possible, a solution to this is presented.

The objective is to secure the 3D and 2D shape data of the exposed pipe at regular intervals to secure big data regarding the deterioration process of the pipe.

The 3D scanning and VR inspection system of the exposure pipe using the drone according to the present invention is intended to realize the above object,

One or more inspection drones; A photographing device mounted on the drone 100 and photographing exposure pipes installed on a bridge or the like at various angles to generate a plurality of input images; At least one control device capable of controlling the drone and the photographing device; A modeling device in which a modeling program for generating a 3D image using the input image is installed; A VR device implementing the 3D image as a virtual reality and providing it to a user; It provides a 3D scan and VR inspection system of the exposure pipe using a drone, characterized in that comprises a.

In addition, the modeling device includes a second generating unit for generating a 2D image using the generated 3D image; A database for storing the 3D image and the 2D image; A comparison analysis unit that compares a plurality of 2D images with each other and analyzes a state change of the exposure pipe; Characterized in that it comprises a.

3D scanning and VR inspection system of the exposure pipe using the drone according to the present invention,

Realization of the actual shape and state of the exposed pipe precisely photographed using the shooting device mounted on the drone, and allowing the user to check it through the VR device, so that the user is comfortable, safe, and closely exposed on the ground. It was possible to check, and the effect was made to be able to grasp both the damage and the state immediately before the damage,

3D model implemented with the same size and length as the actual value of the exposed pipe is configured to be enlarged, reduced, moved, and rotated, so that the exact location of the damaged area can be measured and sized. The measurable effect occurred,

The 3D and 2D shape data of the exposed piping was secured at regular intervals, so that the deterioration state and progress of the piping could be identified.

1 is a configuration diagram of a 3D scan and VR inspection system of an exposure pipe using a drone according to the present invention.
Figure 2 (a) is an exemplary view showing an embodiment of the drone of the 3D scan and VR inspection system of the exposure pipe using the drone according to the present invention to fly and the imaging device generates an input image.
Figure 2 (b) is an exemplary view showing another embodiment of the drone of the 3D scan and VR inspection system of the exposure pipe using the drone according to the present invention to fly and the imaging device generates an input image.
3 is a detailed configuration diagram of a modeling device that is one component of a 3D scan and VR inspection system of exposure piping using a drone according to the present invention.
4(a) to 4(h) are exemplary views showing various embodiments of inspecting a pipe using a VR device of a 3D scan and VR inspection system using a drone according to the present invention.
5 is a further detailed configuration diagram of a modeling device that is a component of a 3D scan and VR inspection system of exposure piping using a drone according to the present invention.

The present invention relates to a system that can check the state of the exposed piping installed in buildings, bridges, etc. using drones and VR technology,

One or more inspection drones 100; A photographing device 110 mounted on the drone 100 and photographing exposure pipes installed on a bridge or the like at multiple angles to generate a plurality of input images; At least one control device 120 capable of controlling the drone and the photographing device; A modeling device 130 in which a modeling program for generating a 3D image using the input image is installed; A VR device 140 that implements the 3D image as a virtual reality and provides it to a user; It relates to a 3D scan and VR inspection system of the exposure pipe using a drone, characterized in that comprises a.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

First, as shown in FIG. 1, the 3D scan and VR inspection system of the exposure pipe using the drone according to the present invention is characterized by including one or more inspection drones 100.

Generally, a drone refers to an unmanned air vehicle of a multicopter type that is provided with a plurality of guide frames extending outward from the center of the main body, and a rotorcraft is provided at the ends of each guide frame. Since it is possible to stop the flight after approaching the inspection target while controlling the speed during the moving flight, there is an advantage that it is possible to perform a precise inspection operation compared to an unmanned aerial vehicle composed of a fixed wing.

Therefore, the drone 100 secures the image of the exposed pipe, generates a 3D image with the secured image, and enables the user to check the generated 3D image in virtual reality, so that the user can comfortably and safely check the state of the exposed pipe. It is the most suitable device to achieve the object of the present invention to enable the operation environment or the type and size of the equipment to be mounted, a dual copter composed of two rotor blades, a tricopter composed of three rotor blades, and four rotor blades. The quadcopter may be composed of a hexacopter composed of six rotor blades or an octocopter composed of eight rotor blades.

In addition, the drone 100 may be operated alone, but a plurality of entities may be simultaneously operated for rapid operation, and the method of operation is manually controlled by a control signal transmitted from the following control device 120 Way.

Therefore, the drone 100 should be provided with a transmitting and receiving device for receiving control signals and the like transmitted from the outside, and transmitting various information, and a drone control unit for controlling the drone 100 by a control signal should be provided. .

In addition, the drone 100 may be equipped with a GPS device that collects GPS coordinate information for the purpose of identifying and recording the current location, and for preventing collision with external objects such as a building, bridge, or drone 100 A distance sensor for sensing a distance from an external object may be provided for use.

In addition, as shown in Figure 1, the 3D scan and VR inspection system of the exposure pipe using the drone according to the present invention is mounted on the drone 100, and photographed at various angles exposed pipes installed in buildings, bridges, etc. Characterized in that it comprises a photographing device 110 for generating an input image.

The photographing device 110 is referred to as a name of a camera or the like and is a known device capable of providing a user with a real-time image for controlling the drone 100 and photographing a subject, and exposing the image to a 3D image generated as a photographed image It is preferable that a device having excellent performance capable of photographing an image having a high quality such as high resolution is provided to enable a close inspection of the product.

At this time, it is obvious that the photographing device 110 should be configured with a size and weight that does not cause obstacles to the flight of the drone 100, and it is fastened to a gimbal to prevent shaking of the image being photographed and to adjust the shooting angle. It is preferable that it is mounted on the drone 100 in the old state.

In addition, in general, considering that the exposed pipe is mainly installed on both sides of the building and is installed on both sides or the bottom of the bridge, the imaging device 110 may be mounted on the side or the bottom of the drone 100, It can also be mounted on the top of the drone 100, it should be able to rotate at a certain angle to adjust the shooting angle.

In addition, the photographing device 110 may be configured to be detachable so as to change the mounting position of the drone 100 as needed, a plurality of pieces are distributed to each position of the drone 100 so as to be selectively driven as needed Can be configured.

2(a) and 2(b) are exemplary views showing an embodiment in which the drone 100 flies along the exposure pipe and the imaging device 110 generates an input image. Both examples should be mixed.

The input image is a still image taken from various angles of the whole and each part of the exposed pipe installed in a building, a bridge, etc., and serves as a material for generating a 3D image of the exposed pipe. It may be included, and some of the surrounding landscape may be included.

These input images are immediately stored in the photographing device 110 and can be copied or transmitted through a mobile storage medium or a wired connection after recovery of the drone 100, but generated to enable rapid creation of a 3D image of the exposed pipe It can be sent out as soon as possible.

The method of controlling the photographing device 110 is a manual control method by a control signal transmitted from the following control device 120, and accordingly a photographing device control unit that receives the control signal and controls the photographing device 110 should be provided. A separate transmission/reception device for receiving a control signal and transmitting an input image may be provided, but it may be configured to use a transmission/reception device provided in the drone 100.

In addition, as shown in Figure 1, the 3D scan and VR inspection system of the exposure pipe using the drone according to the present invention includes the drone 100 and one or more control devices 120 capable of controlling the photographing device 110 It characterized in that the configuration.

The control device 120 is a known control device capable of simultaneously or separately controlling the drone 100 and the photographing device 110 by a wireless communication method, and is used for controlling the drone 100 and the photographing device 110 It may be a dedicated terminal used as, but may be configured as a user terminal, such as a smart phone with a control application installed.

That is, when the control device 120 is configured as a dedicated terminal, the control device 120 is provided with a transmitting and receiving device for two-way communication with a transmitting and receiving device provided on the drone 100 side, and displays the captured image Thus, a display device provided to a user is provided, an input unit for driving or stopping the drone 100 may be provided, an input unit for controlling the flight of the drone 100 may be provided, and the photographing device 110 is provided. An input unit for driving or stopping driving may be provided, an input unit for photographing an image may be provided, and an input unit for zooming in or zooming out the image to be photographed may be provided. It can be composed of either a switch or a stick.

In addition, when the control device 120 is composed of a user terminal, such as a smart phone with a control application installed, the drone 100 is driven or stopped on the screen of the user terminal, and the flight of the drone 100 is controlled. A button for driving or stopping the photographing device 110, photographing an image, and zooming in or zooming out the photographed image may be displayed.

Therefore, the user can simultaneously control the drone 100 and the photographing device 110 using the control device 120, control the drone 100, and stop the flight in a certain position, and then photograph the shooting device 110 ) May be repeated, and two users may respectively control the drone 100 with one control device 120 and the imaging device 110 with the other control device 120. .

In addition, the control device 120 may control the gimbal to which the photographing device 110 is fastened to adjust the photographing angle of the photographing device 110.

In addition, as described above, since the drone 100 may be operated in plural at the same time, the control device 120 may simultaneously operate the plurality of drones 100 and the photographing device 110 mounted to each of the drones 100. Alternatively, a plurality of pieces may be provided for the purpose of controlling at a time.

In addition, the control device 120 may include a caution function that generates and displays a warning message on a display device or a screen when the distance to the external object sensed by the distance sensor is less than or equal to a preset distance. Through the drone 100 can prevent collisions with buildings, bridges, or exposed piping, and when multiple drones 100 are operated at the same time, can prevent collisions with other drones 100. It may also be implemented in a manner that generates a vibration in the device 120 or emits a notification sound.

In addition, as shown in FIG. 1, the 3D scan and VR inspection system of the exposure pipe using the drone according to the present invention includes a modeling device 130 in which a modeling program for generating a 3D image using the input image is installed. It is characterized by being configured.

The modeling device 130 is a device that receives a plurality of input images generated by the photographing device 110 and generates a single 3D image, and it is easy to input and output information using wired and wireless communication methods and modeling program It is preferable to be composed of a known computer that can be installed, but it may be configured as a dedicated terminal.

The 3D image is an image modeled to check the entire shape of the exposed pipe at various angles according to the user's choice and to enlarge each part.It is provided in the virtual reality to the user through the following VR device 140 and the building , Compared to the exposure pipe inspection method using the input image, which is a still image of various parts of the exposed pipe installed at various angles, it has the effect of improving the inspection efficiency.

That is, in order to check the exposure piping using only the input image, which is a still image of all and each part of the exposure piping, the user must check a plurality of input images individually and determine the location of the exposure piping corresponding to the input image where the problem is found. Although it is not efficient due to problems such as the need to check again, the inspection method of the exposure pipe using the 3D image solves all of these problems, so that the user can check the exposure pipe quickly, accurately, and easily.

Therefore, the modeling program may be a program produced by a user of the present invention directly developed or commissioned by a user of the present invention having a function capable of generating a single 3D image by integrating a plurality of input images, and available for a fee or free of charge. It can be a known program.

Specifically, as shown in FIG. 3, the modeling device 130 inputs a plurality of input images, extracts 121 extracting one or more feature points from the input multiple input images, and references the extracted feature points. It characterized in that it is configured to include an image matching unit 122 for performing image registration and a first generation unit 123 for generating a 3D image by synthesizing a plurality of image-matched input images.

The extraction unit 121 extracts unique feature points that appear in the shape of each subject from buildings, bridges, and exposed pipes included in the input image.

That is, since the shape and size of buildings, bridges, and exposed pipes included in the input image are differently expressed according to the angle to be photographed and the distance from the photographing device 110, the extraction unit 121 is limited to the shape and size of the subject. By extracting unique features that are not, each location image is captured so that the actual location can be grasped.

In addition, the drone may be equipped with a GPS device that collects GPS coordinate information, and each input image may have GPS coordinate information generated at the time of generation, and accordingly, the GPS coordinate information included in each input image may be input. It can be used to determine the location of the image.

The image matching unit 122 may use any one input image among a plurality of input images taken in different coordinate systems by the movement of the drone 100 and the change in the shooting angle of the photographing device 110 as a reference image, As a component of the modeling device 120 that implements the same effect as being shot in the same coordinate system by matching different input images to the reference image, the matching of the coordinate system includes one or more feature points extracted from the extraction unit 121. It is used as a standard.

The first generation unit 123 synthesizes a plurality of input images matched by the image matching unit 122 to generate one 3D image. To generate the 3D image, express an image and extract a 3D position. A method such as beam speed adjustment using a collinear condition, which is a basic condition for the control, may be used.

In addition, the modeling device 130 is characterized in that a conversion program capable of converting the format of the 3D image is installed.

If the format of the 3D image generated by the modeling device 130 is different from the format that the VR device 140 can execute, the conversion program is a 3D image if the 3D image is not compatible with the VR device 140 below. As a program for solving this by converting the format of, it may be a program produced by a user of the present invention directly or developed and commissioned, or a publicly available program that is available for a fee or for free.

In addition, the conversion program may be a method in which a plurality of programs are linked to change the format of the 3D image generated by the photographing device 110 to a format usable by the following VR device 140.

In addition, as shown in FIG. 1, the 3D scan and VR inspection system of the exposure pipe using the drone according to the present invention is configured to include the VR device 140 that provides the user with the 3D image as a virtual reality. It is characterized by.

The VR device 140 is a known device that provides a user with a virtual reality implemented by a device such as a computer, and is configured to be worn on the head so that the user's gaze does not face the head mounted display (HMD) or face mounted display It may also be referred to as (FMD), and may be provided with a stereo speaker, a microphone, and the like to enhance realism.

In addition, the VR device 140 is configured to allow a user to enlarge, reduce, move, and rotate the 3D image using a gesture control method, wherein the gesture used is a smartphone using two fingers. It can be variously set by gestures such as gestures used to enlarge and reduce an image displayed on the screen.

Therefore, the VR device 140 should further include a device such as a rear camera for recognizing a user's gesture and reflecting it in a virtual reality, a controller for a user to grip and recognize motion.

4(a) to 4(h) are exemplary views illustrating various embodiments in which the user enlarges and reduces, moves, and rotates the exposure pipe by using the VR device 140.

In addition, the VR device 140 is a method for displaying a damaged portion, etc., found during the inspection of the exposure pipe, characterized in that the user is configured to specify a part of the 3D image using a gesture control method. , As a method of specifying a part of a 3D image, various methods such as a method of changing a part of a color differently from a color of another part and a method of leaving a mark on a part may be used.

In addition, in order to increase the efficiency of the exposure pipe inspection, a plurality of the VR devices 140 are also provided so that multiple users can simultaneously check the exposure pipe, and the plurality of VR devices 140 are connected to each other by wireless communication. If a part of the 3D image is specified by one VR device 140, the part specified by the other VR device 140 can be identified at the same time, thereby preventing the efficiency drop due to overlapping check.

In addition, the 3D scan and VR inspection system of the exposed pipe using the drone according to the present invention is characterized in that it is possible to easily compare and analyze the front and rear states of the pipe by generating a 2D image based on the parasitic 3D image. It enables related users to quickly and easily grasp the status of changes in the exposure pipe over a long period of time.

Specifically, as shown in FIG. 5, the modeling device 130 includes a second generator 134 for generating a 2D image using the generated 3D image, and a database for storing the 3D image and the 2D image It characterized in that it comprises a (135), a comparison analysis unit 136 for analyzing the shape change of the exposure pipe by comparing a plurality of 2D images with each other.

The second generation unit 134 generates an overall side view of multiple angles, each of which is a 2D image, from a plurality of 3D images generated at intervals of a predetermined time. It is possible to further generate partial side views of the angle.

In addition, the database 135 stores the 3D image generated by the modeling device 130 so that it can be transmitted to the VR device 140, and 2D generated by the second generation unit 134. By saving the image, it can be used as comparative data to analyze the change in the shape of the exposure pipe.

In addition, the comparison and analysis unit 136 analyzes the shape change of the exposure pipe by overlapping and comparing a plurality of 2D images generated from a plurality of 3D images generated according to time intervals of a predetermined period on the same plane.

Therefore, in the plurality of 2D images to be compared, the same exposure pipe should be expressed at the same angle and the same distance. Accordingly, the second generation unit 134 composes each of the plurality of 3D images before generating the 2D image. All the backs should be identical.

In addition, the comparison analysis unit 136 may cause the second generation unit 134 to regenerate the entire side view of various angles when the comparison analysis fails due to problems such as size and sharpness of the entire side view, which is a 2D image. In addition, it is possible to additionally generate partial side views of various angles.

The embodiments introduced above are provided as examples, so that the technical spirit of the present invention can be sufficiently transmitted to a person having ordinary knowledge in the technical field to which the present invention belongs, and the present invention is applied to the embodiments described above. It is not limited and may be embodied in other forms.

In order to clearly describe the present invention, parts irrelevant to the description are omitted in the drawings, and in the drawings, the width, length, and thickness of components may be exaggerated or reduced for convenience.

In addition, the same reference numbers throughout the specification indicate the same components.

100: drone
110: photographing device
120: control device
130: modeling device
131: extraction unit 132: image matching unit
133: first generation unit 134: second generation unit
135: database 136: comparative analysis unit
140: VR device

Claims (6)

One or more inspection drones 100;
A photographing device 110 mounted on the drone 100 and photographing exposure pipes installed on a bridge or the like at multiple angles to generate a plurality of input images;
One or more control devices 120 capable of controlling the drone 100 and the photographing device 110;
A modeling device 130 in which a modeling program for generating a 3D image using the input image is installed;
A VR device 140 that implements the 3D image as a virtual reality and provides it to a user; 3D scanning and VR inspection system of the exposure pipe using a drone, characterized in that comprises a.
According to claim 1,
The modeling device 130,
A plurality of input images, and an extraction unit 131 for extracting one or more feature points from the input plurality of input images;
An image matching unit 132 that performs image matching based on the extracted feature points;
A first generation unit 133 for synthesizing a plurality of image-matched input images to generate one 3D image; 3D scanning and VR inspection system of the exposure pipe using a drone, characterized in that comprises a.
According to claim 1,
The modeling device 130,
3D scanning and VR inspection system of exposure pipe using a drone, characterized in that a conversion program capable of converting the format of the 3D image is installed.
According to claim 1,
The 3D scanning and VR inspection system of the exposure pipe using the drone,
3D scan and VR inspection system of exposed piping using drones, which can easily compare and analyze the front and rear states of piping by generating 2D images based on the generated 3D images.
The method of claim 4,
The 3D scanning and VR inspection system of the exposure pipe using the drone,
Modeling device 130,
A second generator 134 for generating a 2D image using the generated 3D image;
A database 135 for storing the 3D image and the 2D image;
A comparison analysis unit 136 for comparing a plurality of 2D images with each other and analyzing the shape change of the exposure pipe; 3D scanning and VR inspection system of the exposure pipe using a drone, characterized in that comprises a.
According to claim 1,
The VR device 140,
3D scanning and VR inspection system of exposure piping using a drone, characterized in that the user is configured to enlarge, reduce, move, and rotate the 3D image using a gesture control method.

Images