KR101783585B1 - Method for managing plant facilities - Google Patents

Abstract

A plant facility management method using a mobile object is disclosed. According to an aspect of the present invention, there is provided a method of controlling a moving object, comprising: determining a maximum photographing distance of an object to be imaged by an image capturing device implemented in a moving object; Acquiring a three-dimensional model of the object to be imaged by the moving object; Setting a non-movable region of the moving object by extending the three-dimensional model; The moving object scanning the imaging object on the setting movement route; Sensing the depth information on the surface to be inspected of the plant facility by the image pickup device; And acquiring information about a crack in the surface to be scanned based on the depth information, wherein the setting movement route is a position in the image capture area excluding the non-movable area, And the setting course is set to either the first route or the second route depending on the type of photographing, the first route is set along the outline so as to maintain the same distance as the outline of the object to be imaged, Wherein the route is set to a straight line distance based on a point at the farthest distance from the outline of the object to be picked up with reference to the object to be picked up and when the second route is set, And the image pickup device determines the estimated focal distance based on the movement of the moving object in the second route, Wherein the image pickup device is configured to perform imaging of the object to be picked up in consideration of the focal distance, and the surface to be imaged is an image picked up by the image pick- When the surface to be imaged is a plane free from bending and perpendicular to the straight line, the image capturing apparatus is provided with information on the minimum distance between the center point of the lens and the surface to be imaged, Wherein the position of the four vertexes of the surface to be inspected is determined based on the optical system characteristic information of the image sensing unit, and when the surface to be inspected is not the bend-free plane and is not perpendicular to the straight line, On the basis of the optical characteristic information and the angle information of the imaging axis of the imaging unit Wherein the image capturing device determines the positions of the four vertexes of the surface to be inspected based on the depth measurement array when the surface to be inspected is a plane on which the curvature exists, And the presence or absence of a crack in the surface to be etched may be determined based on each of the plurality of depth information.

Description

[0001] METHOD FOR MANAGING PLANT FACILITIES [0002]

[0001] The present invention relates to a plant facility management method, and more particularly, to a method of managing plant facilities using a moving object based on object information of a plant facility.

The plant industry can be a device or a factory facility that supplies equipment that can produce products such as power, gas, oil, and fresh water, or that supplies raw materials or energy to the factory building industry to make physical and chemical effects. These plant factories are made up of complex facilities, which are exposed to the external environment and can be damaged, requiring management.

On the other hand, the drone, a representative of unmanned vehicles, is an unmanned aerial vehicle that can be controlled by a remote controller or a controller on the ground. The drones were developed for military use in the 1960s during the Cold War era of the United States and the Soviet Union. The drones, which were used only for military purposes in the Middle East Gulf and Kosovo conflicts by the 1990s, are expanding their application fields due to the development of wireless communication technologies.

Recently, the use area of drones is expanding not only for military use but also for transportation, logistics, aerial photography, agriculture and security.

US defense industry consulting firm Till Group expects the drones market to grow at an annual average of 35%, to $ 12.5 billion in 2023. An official of the Korea Aerospace Research Institute said, "The technology development is being carried out in order to enhance the economic efficiency, safety and efficiency of the aircraft such as environment friendly, unmanned, and miniaturized." The UAV developed and used for military purposes is used for communication relay, And aerial photographing, "he said.

In fact, companies are trying to utilize drones for unmanned delivery services, and they are making moves to use drones at the national level to investigate the accident sites. In addition to this, the drones are also used in the broadcasting business to shoot 'Heli-cam' which can capture the scenes of movies and sports shoots.

On the other hand, the method of acquiring depth information in an image can be divided into a passive method and an active method. The manual method is a method of acquiring depth information from a two-dimensional image acquired based on stereo photographing and multi-view photographing. The active method is a method of directly measuring depth information in an image based on a device for direct depth information measurement such as a depth camera or a laser scanner.

As stereoscopic three-dimensional space is recognized by using human's two-eye parallax (binocular parallax), stereo matching by stereo matching method is a method of extracting three-dimensional information using a camera sensor capable of receiving only two- to be. Depth information can be extracted based on the acquired stereo image with two cameras spaced apart spatially. The principle of stereo matching is to find an arbitrary pattern of an image of one camera on another camera image matching the same point on the three dimensional space and then to find coordinates on the three dimensional space by using a triangle method on corresponding points.

Another method for extracting depth information from an image is to use a time-of-flight (TOF) depth camera to acquire depth information from an image. The distance measurement from the depth camera is made on the principle of traveling time. The basic principle is to find the depth value by calculating the phase shift of the time that the light is reflected by irradiating light such as infrared ray. Thus, the measured distance may be proportional to the time of the light traveling from the camera to the object. To measure depth information, a light wall hits a 3D object and the depth information of the object in the image can be measured based on the waveform of the reflected object shape from the object surface.

The present invention relates to the management of facility facilities of plant factories by utilizing drone.

KR 10-2009-0111046

An object of the present invention is to provide a plant facility management method using a moving object.

According to an aspect of the present invention, there is provided a method of controlling a moving object, comprising: determining a maximum photographing distance of an object to be imaged by an image capturing device implemented in a moving object; Acquiring a three-dimensional model of the object to be imaged by the moving object; Setting a non-movable region of the moving object by extending the three-dimensional model; The moving object scanning the imaging object on the setting movement route; Sensing the depth information on the surface to be inspected of the plant facility by the image pickup device; And acquiring information about a crack in the surface to be scanned based on the depth information, wherein the setting movement route is a position in the image capture area excluding the non-movable area, And the setting course is set to either the first route or the second route depending on the type of photographing, the first route is set along the outline so as to maintain the same distance as the outline of the object to be imaged, Wherein the route is set to a straight line distance based on a point at the farthest distance from the outline of the object to be picked up with reference to the object to be picked up and when the second route is set, And the image pickup device determines the estimated focal distance based on the movement of the moving object in the second route, Wherein the image pickup device is configured to perform imaging of the object to be picked up in consideration of the focal distance, and the surface to be imaged is an image picked up by the image pick- When the surface to be imaged is a plane free from bending and perpendicular to the straight line, the image capturing apparatus is provided with information on the minimum distance between the center point of the lens and the surface to be imaged, Wherein the position of the four vertexes of the surface to be inspected is determined based on the optical system characteristic information of the image sensing unit, and when the surface to be inspected is not the bend-free plane and is not perpendicular to the straight line, On the basis of the optical characteristic information and the angle information of the imaging axis of the imaging unit Wherein the image capturing device determines the positions of the four vertexes of the surface to be inspected based on the depth measurement array when the surface to be inspected is a plane on which the curvature exists, And the presence or absence of a crack in the surface to be etched may be determined based on each of the plurality of depth information.

Also, the image capturing device performs modeling on the surface to be fitted using a combination model for a shape to be fitted to the surface to be inspected based on the plurality of depth information, wherein the shape to be fitted is a plane, a spherical surface, Stairs or spirals.

Further, the surface to be etched is divided into a plurality of divided surfaces to be etched based on the depth measurement array, and the presence or absence of the cracks can be determined for each of the plurality of divided surfaces to be etched.

In addition, when there is a non-imaging region in which the curvature exists on the surface to be imaged and depth information on a part of the surface to be imaged due to the curvature is not obtained, the moving object has a first portion for acquiring information on the non- Wherein the second moving route is a region in which the image sensing area is located in the image sensing area of the image sensing area, It can be included in the face on the space that is the furthest distance from the target object.

The moving object scans the object to be imaged together with a plurality of other moving objects. When the moving object is a master moving object, the moving object divides the imaging area into a plurality of divided imaging areas, Each of the divided imaging regions may be assigned to each of the other plurality of moving bodies.

In addition, when a first divided surface to be scanned having cracks is found in the divided surface to be scanned, the moving body sets a third moving path for performing a secondary scan on the first divided surface to be scanned, Wherein the fourth moving path detects the information on the crack on the re-divided surface to be imaged by re-segmenting the first divided surface to be retouched on the moving path into a plurality of re-divided surfaces to be imaged, And may be included in a plane on the space.

In addition, the non-movable area may be set by taking into account global positioning system (GPS) errors and control errors with respect to the moving object.

According to an embodiment of the present invention, a plant facility management method using a mobile object can be provided.

1 is a conceptual diagram illustrating a plant facility management method using a moving object according to an embodiment of the present invention.
FIG. 2 is a conceptual diagram illustrating the setting of a non-movable area according to an embodiment of the present invention.
3 is a conceptual diagram illustrating a method of setting a setting travel route on a plant facility management method using a moving object according to an embodiment of the present invention.
4 is a conceptual diagram showing a method of picking up an object to be picked up in a setting travel route according to an embodiment of the present invention.
5 is a conceptual diagram illustrating a method of acquiring image information for an object to be imaged using a plurality of moving objects according to an embodiment of the present invention.
6 is a conceptual diagram showing a method of setting a divided imaging area according to an embodiment of the present invention.
7 is a conceptual diagram illustrating a networking method between a master moving body and a servant moving body according to an embodiment of the present invention.
FIG. 8 is a conceptual view showing a video imaging device and a surface to be imaged according to an embodiment of the present invention.
FIG. 9 is a conceptual view showing a video imaging device and a surface to be inspected different from the embodiment of the present invention. FIG.
FIG. 10 is a conceptual diagram illustrating a method for acquiring subject surface information based on a similar figure according to an embodiment of the present invention.
11 is a conceptual diagram illustrating a method for acquiring subject surface information according to an embodiment of the present invention.
12 is a conceptual diagram illustrating a method of acquiring target surface information of a moving object according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Referring to the accompanying drawings, the same or corresponding components are denoted by the same reference numerals, .

It is also to be understood that the terms first, second, etc. used hereinafter are merely reference numerals for distinguishing between identical or corresponding components, and the same or corresponding components are defined by terms such as first, second, no.

In addition, the term " coupled " is used not only in the case of direct physical contact between the respective constituent elements in the contact relation between the constituent elements, but also means that other constituent elements are interposed between the constituent elements, Use them as a concept to cover each contact.

Also, in the context of a computer program, the suffix "module" and / or "part" for a component may be part of a computer program for causing a computer to function as a means of a particular function, It can be a part. For example, the module A may be interpreted as a computer program for making the computer function as the means A or a computer program for realizing the function A in the computer. As a method, a "step" can be implemented and executed as a computer program in a computer. Also, "module" and / or "part"

On the other hand, an application refers to a set of computer programs designed to perform a specific task, which is also referred to as an application program. The user can add the related function by installing the application according to the embodiment of the present invention in his electronic device.

The user's electronic device in which the application is installed is composed of a CPU, RAM, ROM, storage device such as a computer, a tablet PC, and a smart phone, and is an environment in which the entire system is controlled by a graphical operating system such as Windows, iOS, Android, Especially for smartphones that can send and receive phone calls and texts to registered contacts.

Further, the flowcharts of the drawings attached hereto are only a flow chart for explaining the invention, and need not be a flowchart to be completely implemented without a bug on a computer.

Also, the terminal referred to herein may be a general user terminal such as a smart phone, a personal computer, a tablet PC, and the like.

1 is a conceptual diagram illustrating a plant facility management method using a moving object according to an embodiment of the present invention.

1 shows a method of utilizing image information picked up by a video image pickup apparatus implemented in a moving object through setting a moving path of a moving object (for example, a dragon) for picking up a specific object for plant facility management using a moving object And the like.

Referring to FIG. 1, the maximum photographing distance of an object to be imaged by the image capturing apparatus implemented in the moving object is determined (step S100).

The maximum photographing distance may be the maximum distance at which the imaging object can be imaged. That is, the moving object can acquire information about the object to be imaged by performing imaging with respect to the object to be imaged within the maximum imaging distance. The maximum photographing distance of the image capturing apparatus may be determined based on the resolution of the object to be imaged. That is, the maximum photographing distance may be determined based on whether to acquire image information for the object to be captured at a resolution of a certain size. In the present invention, the object to be imaged can be a plant facility.

The moving object acquires three-dimensional modeling information on the object to be imaged (step S110).

The three-dimensional modeling information on the object to be imaged can be obtained by various methods. For example, the moving object can first determine whether or not the imaging object can be imaged at the maximum imaging distance. If imaging is possible with respect to the object to be imaged at the maximum imaging distance, a plurality of images for the object to be imaged can be acquired through imaging with respect to the imaging object at the maximum imaging distance. Thereafter, three-dimensional modeling information on the object to be imaged can be obtained through three-dimensional modeling of the object to be imaged based on the plurality of images.

The moving object can set a non-movable area by extending the three-dimensional model of the object to be imaged (step S120).

The non-movable region may be an area in which collision with the object to be captured obtained based on the three-dimensional modeling information on the object to be captured may occur. For example, when the shape of an object differs depending on the height of the object, the movable range by the moving object may vary depending on the height. According to the embodiment of the present invention, the unmovable area can be set by taking into consideration global positioning system (GPS) error, control error, and the like. GPS error may have a larger vertical error than horizontal error. Or a shaded area in which the GPS signal sensitivity is degraded may be set as an unmovable area.

The moving object can move on the setting movement route and scan the appearance of the object to be imaged (step S130).

The moving object can set the setting moving route on the area excluding the unmovable area among the areas within the maximum photographing distance. The moving object can acquire image information on the object to be imaged by adjusting the distance according to a desired resolution on the setting movement route.

The image capturing apparatus may include an image capturing unit and a distance measuring sensor unit.

The image capturing unit may be implemented to acquire image information on an object on a surface to be examined. The distance measurement sensor unit can be implemented to measure distance information such as the distance to the surface to be surveyed, and the distance to the object on the surveyed surface.

The surface imaged by the imaging unit can be defined as the term field of view. In FIG. 1, it is assumed that the surface to be projected is a flat surface which is perpendicular to a straight line passing vertically through the lens with respect to the center point of the lens of the image pickup unit as a flat surface.

The image pickup device can measure the surface distance L to the surface to be measured based on the distance measurement sensor (step S140).

The distance measurement sensor unit can measure the distance L to the surface perpendicular to the image sensing unit. Specifically, the distance measurement sensor unit can measure a distance L (hereinafter referred to as a surface-to-be-contacted distance L) from a vertically straight line so as to pass through the lens with reference to the center point of the lens included in the image- The surface to be contacted L may be the shortest distance from the center of the lens to the surface to be contacted.

The image capturing apparatus can acquire information on the surface to be projected based on the optical system characteristics of the image capturing unit (step S150).

The image pickup apparatus can obtain information on the horizontal width (H) and the vertical width (V) of the surface to be scanned in consideration of the optical system characteristics of the image pickup unit. For example, when the distance to the subject is L based on the information about the zoom setting, the angle of view setting, the focal length, and the wide angle of the image pickup unit, information about the horizontal width (H) and the vertical width (V) .

The image pickup apparatus can obtain the coordinate information of the vertex of the surface to be scanned based on the information about the surface to be scanned (step S160).

(0, 0) is set at a point where a straight line drawn vertically to penetrate the lens with respect to the center point of the lens of the image pickup unit and the surface to be contacted is set to (0, 0), the distance L to the surface to be imaged, the width of the surface to be imaged is H, V, the coordinate information of each vertex of the surface to be scanned can be determined. Information on the surface to be scanned located within the vertex coordinate can be obtained based on the coordinates of each vertex of the surface to be inspected.

The above procedure is a procedure for obtaining information on the surface to be projected when there is no curvature on the surface to be projected and the plane perpendicular to the straight line is perpendicular to the center of the lens of the image pickup unit.

According to another embodiment of the present invention, the surface to be projected may be a flat surface without bending, or a plane that is not perpendicular to a straight line that is vertically passed through the lens with respect to the center point of the lens of the image sensing unit.

(X, y, z) information of the image capturing apparatus, which is not perpendicular to the surface to be imaged, when the image pickup section of the image capturing apparatus and the surface to be imaged are not perpendicular to each other, calculates coordinate information of the vertex of the surface to be imaged ≪ / RTI > When the photographing axis of the image pickup unit is twisted, the angle of the twisted photographing axis is additionally reflected, and the positions of the four vertexes located on the surface to be scanned on the three-dimensional plane can be determined.

Specifically, when a straight line drawn vertically to penetrate the lens with respect to the center point of the lens of the image pickup unit and the point where the surface to be contacted is set to (0, 0, 0), the positions of the four vertexes located on the surface to be scanned on the three- (L), the angle information of the twisted photographing axis, and the lateral width of the surface to be determined determined in consideration of the optical system characteristics of the image pickup device H) and the vertical width (V).

A case where a moving object such as a drone is not perpendicular to the image pickup unit of the image pickup apparatus due to various external environments (including gimbals) during flight can be issued. In this case, the position of the four vertexes of the surface to be imaged can be determined in the above-described manner in consideration of the angle between the imaging axis of the imaging unit and the surface to be imaged (i.e., the angle of the twisted imaging axis). In the present invention, the surface to be polished can be the surface of the plant facility.

FIG. 2 is a conceptual diagram illustrating the setting of a non-movable area according to an embodiment of the present invention.

In Fig. 2, the maximum photographing distance 200, the unmovable area 220, and the movement route setting area 240 are started.

As described above, the maximum photographing distance 200 may be a distance at which image information of a critical resolution or more with respect to an object to be photographed can be obtained.

The unmovable area 220 may be an area where a moving object may move due to movement of the moving object.

The route setting area 240 can be a region excluding the unmoveable area 220 within the maximum photographing distance 200.

3 is a conceptual diagram illustrating a method of setting a setting travel route on a plant facility management method using a moving object according to an embodiment of the present invention.

FIG. 3 discloses a method for setting a setting movement route on an area excluding a non-movable area out of the areas within a maximum photographing distance of a moving object.

3 (a), the setting movement route 320 can be set to maintain the same distance along the outline line 300 of the object to be imaged. For example, when the outline of the object to be imaged is a straight line, a straight line that maintains a certain distance from the outline may be set as the setting movement route of the moving object. Or if the outline of the object to be imaged is a curve, a curve that maintains a certain distance from the outline may be set as the set travel route of the moving object. In this case, since the moving object moves so that the distance between the object to be imaged and the moving object is constant, the change in the focal length may not be large.

Referring to FIG. 3 (b), the setting movement route 370 may be set as a straight line distance determined based on a point which is the farthest distance from the object to be imaged among the outline 350 of the object to be imaged. As described above, the collision area with the object to be captured can be set as the unmovable area. A setting movement route in a straight line direction may be set in consideration of the farthest point with respect to the position of the object to be captured among the non-movable areas.

The setting movement route 320 of the moving object is set to the same distance along the outline of the object 300 as shown in FIG. 3 (a) or the setting movement route 370 may set a straight line distance based on the farthest point of the object to be picked out of the outline of the object 350 to be picked up. Specifically, when more close-up photographing is required, the setting travel route 320 may be set as shown in Fig. 3 (a), and when only a rough photographing is sufficient than a close-up photographing, The setting movement path 370 can be set as shown in Fig.

4 is a conceptual diagram showing a method of picking up an object to be picked up in a setting travel route according to an embodiment of the present invention.

4, when the moving path of the moving object is set to a straight line distance based on the farthest point of the object to be captured among the outline of the object to be captured as disclosed in FIG. 3 (b), the image capturing apparatus implemented in the moving object It is necessary to adjust the focal distance 400 in order to capture an image of the object to be picked up.

When a curvature exists in the object to be imaged and the moving object moves in a linear direction to take an image of the object to be imaged, the distance to the object to be imaged changes. Therefore, it is necessary to adjust for the focal length 400 of the moving object.

The image capturing apparatus may automatically adjust the focal distance 400 by a distance measuring sensor. However, in the case of imaging the image based on the movement of the moving object, it is necessary to make a quick judgment for adjustment of the focal distance 400. [ Therefore, in the embodiment of the present invention, the moving object can predict the distance between the modeled object to be imaged and the image capturing device, and set the expected focal distance in consideration of the distance between the modeled object and the image capturing device.

For example, when the outline of the object to be imaged is a curve, and the setting movement route of the moving object is a straight line, the distance between the moving object and the object to be imaged due to the movement of the moving object based on the 3D model of the object to be imaged The focal length can be set.

The image capturing apparatus can perform image capturing for the image while moving the focal distance according to the expected focal distance. The image capturing apparatus can additionally utilize the distance measuring sensor based on the estimated focal distance to set the final focal distance. When this method is used, the final focal distance is determined quickly based on the estimated focal distance even when the moving object is moving at a high speed, so that the image information about the object to be captured can be acquired more accurately.

5 is a conceptual diagram illustrating a method of acquiring image information for an object to be imaged using a plurality of moving objects according to an embodiment of the present invention.

FIG. 5 discloses a method in which a plurality of moving objects divides a section to acquire image information on an object to be imaged.

Referring to FIG. 5, when the size of the object 500 is large, a plurality of moving objects can acquire image information about the object 500 to be captured by dividing the image pickup section on the basis of the object to be captured.

At least one moving object among a plurality of moving objects can primarily acquire information about the object 500 to be imaged. A plurality of moving objects may move together with the object 500 to be captured, but one moving object among the plurality of moving objects may preferentially move to divide a region for the search of the object 500, Information can be obtained. An object that performs search for the object 500 to be imaged first may be expressed by the term master moving object. The remaining movable bodies excluding the master moving body among a plurality of moving bodies can be expressed by the term " servant moving body ".

The master moving object can divide the region of the object 500 to be captured into a plurality of sub-image pickup target object regions based on the information about the image pickup object 500 that is acquired primarily. A plurality of divided sub object regions to be imaged may be expressed by the term " partial image region ". The master moving body can allocate the divided imaging region to each of the at least one servant moving body.

At least one servant moving body is allocated a divided imaging region and can generate image information for the divided imaging region through search for the divided imaging region.

The image information obtained by the at least one servant moving body and the master moving body may be combined and a search for one imaging object 500 may be performed.

6 is a conceptual diagram showing a method of setting a divided imaging area according to an embodiment of the present invention.

6, a method in which the master moving body 600 assigns the divided imaging region to the servant moving body 650 is disclosed. In Fig. 6, the number of servant moving bodies 650 is assumed to be plural.

Referring to FIG. 6, the master moving body 600 can determine a divided imaging area by performing area division on the basis of an area excluding a non-movable area (hereinafter referred to as an imaging area) among the areas within a maximum imaging distance.

The master moving body 600 obtains information on the area analysis capability of each of the plurality of servant moving bodies 650 and divides the image pickup enabling area based on the information on the area analysis capability of each of the plurality of servant moving bodies 650 to generate a plurality Can be determined. Thereafter, the master moving body 600 can allocate each of the plurality of divided imaging regions to each of the plurality of servant moving bodies 650.

The master moving body 600 may form a network through a coupling procedure with a plurality of servant moving bodies 650 and may transmit the command to the servant moving body 650 through the formed network. When the master moving body 600 forms a network with each of the plurality of servant moving bodies 650 in advance, the master moving body 600 can request and receive information on the area analyzing capability of each of the plurality of servant moving bodies 650 with a plurality of servant moving bodies 650 have. The information on the area analysis capability may include information on the focal length adjustment capability, the zoom performance, the maximum resolution, the aperture performance, and the movement performance of the moving object.

The master moving body 600 can allocate a wider divided imaging area to the servant moving body 650 that has relatively good area analysis capability in dividing the image capture area.

More specifically, a relatively wider divided imaging region is allocated to the servant moving body 650, which is relatively more excellent in moving performance and relatively superior in image imaging performance, so that image information on the object to be imaged can be acquired more quickly.

The master moving body 600 may also proceed with an imaging procedure for one divided imaging region. Then, the master moving body 600 receives the image information about the plurality of divided imaging regions from each of the plurality of servant moving bodies 650, and acquires the image information about the imaging target object by combining the image information about the plurality of divided imaging regions can do.

5 and 6 illustrate an object to be imaged for the sake of convenience of description, but it can also be applied to an object to be imaged other than an object such as an object to be imaged.

7 is a conceptual diagram illustrating a networking method between a master moving body and a servant moving body according to an embodiment of the present invention.

In Fig. 7, a method is described in which a master moving body 700 and a servant moving body 710 to 740 are set in a plurality of moving bodies.

Referring to FIG. 7, the master moving body 600 and the servant moving body 750 among a plurality of moving bodies can be determined according to the size of the object to be imaged.

In the initial coupling stage, one moving object having the best performance (moving speed, data processing speed, etc.) among a plurality of moving objects is set as the initial master moving object 700, and the remaining moving objects except for the initial master moving object 700 are moved to the servant moving object 750, Lt; / RTI >

The initial master moving body 700 and the servant moving bodies 750 can move to the object to be imaged for imaging of the object to be imaged and the initial master moving body 700 can set the additional master moving body while moving to the object to be imaged Can be determined.

For example, when the imaging object 720 is detected, the initial master moving body 700 predicts the size of the imaging object 720 and adds the imaging object 720 based on the estimated size of the imaging object 720 It can be determined whether or not to set the master moving body.

If the size of the predicted imaging object 720 is smaller than the threshold size, an additional master moving object for detection of the predicted imaging object 720 may not be set. Conversely, when the size of the predicted imaging object 720 is equal to or larger than the threshold size, an additional master mobile body 730 for detection of the predicted imaging object 720 may be set. The additional master moving body 730 may be the moving body having the highest performance among the servant moving bodies 750.

When the additional master moving body 730 is set, the remaining servant moving body can be grouped into the servant moving body for the initial master moving body 700 and the servant moving body for the additional master moving body 730. [

When the initial master moving body 700 and the additional master moving body 730 are adjacent to the object to be imaged, the initial master moving body 700 and the additional master moving body 730 move to the object to be imaged preferentially over the servant moving body, As described above, it is possible to acquire information on the object to be imaged primarily and allocate the divided imaging region to each of the at least one servant moving body.

Specifically, the initial master moving body 700 and the additional master moving body 730 communicate with the initial master moving object detection area and the area corresponding to the imaging object 720 in order to acquire information about the imaging object 720 through mutual networking, It can be divided into additional master moving object detection areas. The initial master moving body 700 can allocate the divided imaging region in which the initial master moving object detection region is divided to the servant moving body connected to the initial master moving body 700. [ The additional master moving body 730 may allocate the divided imaging area in which the additional master moving object detection area is divided to the servant moving body connected to the additional master moving body 730. [

It is also possible to manage large-sized plant facilities quickly by performing an organic mission between the master moving body and the servant moving body.

FIG. 8 is a conceptual view showing a video imaging device and a surface to be imaged according to an embodiment of the present invention.

Referring to Fig. 8A, the surface to be scanned 1200 may be a plane that is perpendicular to a straight line that passes through the lens with respect to the center point of the lens of the image sensing unit as a flat surface without bending.

Referring to FIG. 8 (b), the surface to be inspected 1250 may be a flat surface that is not perpendicular to a line vertically passed through the lens with respect to the center point of the lens of the image sensing unit as a flat surface without bending.

FIG. 9 is a conceptual view showing a video imaging device and a surface to be inspected different from the embodiment of the present invention. FIG.

In Fig. 9, a method for acquiring information on a surface to be subjected to incidence is described.

The distance measurement sensor unit of the image pickup apparatus can measure the distance to a plurality of distance measurement points 1300 instead of one point. For example, the distance measurement sensor unit measures the surface distance (or depth information) of each of the 100 measurement points 1300 on the surface of interest on the basis of the array 10x10 including the plurality of measurement points 1300, Can determine the depth information of the object in the scene.

Various methods can be used to measure depth information for a plurality of measurement points 1300 on the specimen surface. For example, using a range detector (e.g., uni-directional laser, ultrasound, imaging, etc.), a plurality of measurement points on a wavefront corresponding to a dot matrix, The depth information for the image 1300 can be determined.

Or light detection and ranging (LiDAR) is used, depth information for the measurement points 1300 on a plurality of wavefronts can be determined based on multi-line scanning in a cross or lattice form.

Or a time-of-flight camera or a stereo camera may be used to determine the depth information for the measurement points 1300 on a plurality of surfaces to be surveyed.

Depth information on the measurement points 1300 on the plurality of surfaces to be acquired can be obtained through the above method and the depth map for the surface to be surveyed according to the depth information on the measurement points 1300 on the plurality of surfaces to be surveyed ) Can be determined.

Referring to FIG. 9, the image capturing apparatus can arrange virtual arrays on a surface to be inspected to determine depth information on each of a plurality of measurement points 1300 on a surface to be included in the array. Information about the obstacle on the moving path of the moving object such as a drone is obtained through the depth information of each of the plurality of measuring points 1300 on the surface to be scanned, and the moving object can be driven by avoiding the obstacle. In addition to this, information on cracks (information on the position and size of the cracks) on the surface to be projected can also be obtained based on the depth information of each of the plurality of measurement points 1300 on the surface to be inspected.

According to the embodiment of the present invention, the calculation amount of the moving object can be reduced by finding a figure similar to the surface to be acquired and fitting the surface to obtain information about the surface to be surveyed to define the surface to be scanned.

FIG. 10 is a conceptual diagram illustrating a method for acquiring subject surface information based on a similar figure according to an embodiment of the present invention.

Referring to FIG. 10, the distance to a plurality of measurement points on the surface to be measured is measured using a sensor capable of measuring the distance to each of a plurality of measurement points on the surface to be measured at once, as described above with reference to FIG. 9, Depth information for each of a plurality of measurement points can be determined.

Thereafter, by using a combination model for a shape (for example, a plane, a sphere, a cylinder, a staircase, a spiral, etc.) 1400 fitted on the surface to be inspected based on the distance information to each point on the surface to be inspected, The surface can be expressed.

In this way, the calculation amount of the drone can be greatly reduced by finding a figure 1400 similar to the surface to be scanned in a moving object such as a dragon whose hardware performance is limited, and defining the surface to be scanned.

The flight of the drones can be controlled based on the surface of the surface to be inspected determined in this way. If there is a crack on the surface to be inspected through the definition of the surface to be inspected, information on the crack can be obtained through the crack image processing. This can be used to determine if there is a crack or the like on the surface of a complex plant facility.

In addition to being able to avoid obstacles based on information about the slope during flight, robots traveling on the ground can also determine the path of travel based on information about the slope, .

11 is a conceptual diagram illustrating a method for acquiring subject surface information according to an embodiment of the present invention.

In Fig. 11, a method for acquiring information on a surface to be imaged that can not be picked up at an imaging angle of a moving object is disclosed.

Referring to FIG. 11, when bending exists on the surface to be inspected, depth information on a part of the surface to be surveyed may not be obtained due to bending. The portion not acquired in the surface to be inspected is represented by the term " non-imaging region 1500 ".

In this case, the moving object can set a moving path for acquiring information on the non-image sensing area 1500, capture the non-image sensing area 1500 after moving, and combine with the previously captured image information to model the surface to be scanned .

As described above, the moving object can arrange virtual arrays on the surface to be imaged to determine depth information for each of a plurality of measurement points on the surface to be imaged included in the array. A plurality of measurement points on the surface to be inspected can be represented by coordinates (x, y) on a two-dimensional plane. For example, when depth information is measured for a plurality of measurement points based on a 10x10 array, depth information for a plurality of measurement points from (0, 0) to (10, 10) is measured, .

When the moving object performs imaging at the first point, it may be difficult to measure the depth of some of the plurality of measurement points due to the curvature of the surface to be inspected. In this case, the moving object can determine the coordinates of the measurement point corresponding to the non-imaging area 1500. [

For example, when the coordinates corresponding to the non-imaging region 1500 are (a, b), (c, d), (e, f) c, d), (e, f), (g, h).

Specifically, the moving object can photograph the non-image pickup area 1500 and the peripheral area of the non-image pickup area 1500 to determine depth information about the entire measurement point corresponding to the surface to be scanned. In the case where the coordinates corresponding to the non-image sensing area 1500 are (a, b), (c, d), (e, f) (a, b), (c ', d'), (e ', f'), (g ', h) c, d), (e, f), (g, h). The depth information on the surrounding regions of the non-sensing region 1500 and the non-sensing region 1500 as described above may be obtained and 3D modeling of the entire surface to be inspected may be performed.

The moving object can move from the first point to the second point to perform imaging with respect to the non-imaging area 1500 and the surrounding area, and the second point can perform imaging with respect to the non-imaging area 1500 and the surrounding area It can be a possible location. That is, the moving object can determine the second point in consideration of the coordinates corresponding to the non-imaging area 1500 and the coordinates corresponding to the surrounding area. On the other hand, in this case, since the imaging is attempted with respect to the non-imaging region, the moving object may not include all the information about the imaging target. Therefore, in order to prevent the collision between the moving object and the surface to be projected (plant facility), the moving object can be set to move in a line defined by the surface to be scanned, that is, the surface from the surface of the plant.

12 is a conceptual diagram illustrating a method of acquiring target surface information of a moving object according to an embodiment of the present invention.

In Fig. 12, a method for analyzing crack information on a surface to be etched is disclosed.

Referring to FIG. 12, a moving object can primarily determine whether a crack exists by scanning a relatively wide surface to be scanned. For example, the moving object can judge a crack present in the first area of the structure.

As described above, the moving object can arrange a virtual array 1600 on the surface to be measured to determine depth information for each of a plurality of measurement points on the surface to be scanned included in the array 1600. [

To determine the presence of a crack with respect to the first area, the mobile can move the distance between coordinates corresponding to the measurement point. For example, the moving object moves between coordinates corresponding to (x, y), (x + 1, y), (x, y + 1) Can be determined. Here, (x, y) may mean each of a plurality of measurement points. In the case of the 5x5 array 1600, the surface to be sliced is divided into 25 divided surfaces to detect the presence of cracks in each of 25 divided surfaces to be sliced. The moving body moves the distance of 1 in the transverse direction and 1 in the longitudinal direction according to the movement pattern, and at the same time, the presence of cracks can be detected according to the movement pattern for each of the 25 divided surface areas. The movement patterns can be variously set according to the surfaces to be detected, such as X shape, + shape, Z shape, and the like.

That is, when the moving object is moved according to the movement pattern, the array 1600 for determining the depth information is moved on the basis of the movement pattern, and it is determined whether or not there is a crack in each of the divided surfaces to be inspected determined according to the array 1600 .

A plurality of divided surfaces to be sliced can be indexed and the moving body can perform a secondary scan on the divided surface to be sliced where cracks exist among the plurality of divided surfaces to be sliced.

For example, each of the 25 divided surfaces to be pierced can be indexed to the divided surface to be sliced 1 to the divided surface 25. When there is a crack in the divided surface 1050 (1650) among the 25 divided surfaces to be inspected, detailed information on the crack can be detected through the secondary scan of the divided surface 1050 (1650).

In the secondary scan, the array 1600 is focused on the divided surface 1050 (1650) in order to detect the depth information on the divided surface 1050 (1650) to obtain crack-related information such as the size of a specific crack, . For example, when a 10x10 array 1600 is used to measure depth information, the 100 measurement points included in the 10x10 array 1600 are focused on the divided surface 10 (1650) Information on the size of cracks and depths of cracks located in the cracks can be obtained.

At this time, the moving object can fly a plane of a point on the space closest to the object to be imaged (for example, a plant facility) among the image capture possible areas. It is possible to shoot a crack while avoiding collision with an object to be imaged.

The operation of the moving body and the image pickup apparatus described above with reference to Figs. 1 to 12 can be performed based on a processor.

The methods and processes described above may be embodied as instructions for execution by, for example, a processor, controller, or other processing device, or may be encoded or read from a compact disk read only memory (CDROM), magnetic or optical disk, flash memory, (RAM) or read only memory (ROM), erasable programmable read only memory (EPROM), or other machine-readable medium.

Such a medium may be embodied as any device that stores, communicates, propagates, or transports executable instructions for use by or in connection with an instruction executable system, apparatus or device. Alternatively or additionally, as analog or digital logic using one or more integrated circuits, or hardware such as one or more processor execution instructions; Or as application programming interfaces (APIs) or dynamic link libraries (DLLs), software as functions defined in local or remote procedure calls or available in shared memory; Or a combination of hardware and software.

In other implementations, the method may be represented as a signal or a propagation-signal medium. For example, instructions that implement the logic of any given program may take the form of electrical, magnetic, optical, electromagnetic, infrared, or other types of signals. The above-described systems may be configured to receive such signals at a communication interface, such as a fiber optic interface, antenna, or other analog or digital signal interface, to recover instructions from the signal, store them in a machine readable memory, and / And execute them.

Further, the present invention can be implemented in hardware or software. The present invention can also be embodied as computer readable code on a computer readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may be implemented in the form of a carrier wave (for example, transmission via the Internet) . The computer readable recording medium may also be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner. And functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers skilled in the art to which the present invention pertains.

Embodiments of the present invention may include a carrier wave having electronically readable control signals, which may be operated with a programmable computer system in which one of the methods described herein is implemented. Embodiments of the present invention may be implemented as a computer program product having program code, wherein the program code is operated to execute one of the methods when the computer program is run on a computer. The program code may be stored on, for example, a machine readable carrier. One embodiment of the invention may be a computer program having program code for executing one of the methods described herein when the computer program is run on a computer. The present invention may include a computer, or programmable logic device, for performing one of the methods described above. A programmable logic device (e.g., a field programmable gate array, a complementary metal oxide semiconductor based logic circuit) may be used to perform some or all of the functions described above.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit of the invention as set forth in the appended claims. The present invention can be variously modified and changed by those skilled in the art, and it is also within the scope of the present invention.

Claims (7)

A plant facility management method using a moving object is a method
Determining a maximum photographing distance of the object to be imaged by the image capturing device implemented in the moving object;
Acquiring a three-dimensional model of the object to be imaged by the moving object;
Setting a non-movable region of the moving object by extending the three-dimensional model;
The moving object scanning the imaging object on the setting movement route;
Sensing the depth information on the surface to be inspected of the plant facility by the image pickup device; And
Wherein the image pickup device obtains information on a crack in the surface to be scanned based on the depth information,
Wherein the setting movement route is set in an image capture area excluding the unmovable area of the area within the maximum photographing distance,
The setting movement route is set to the first route or the second route depending on the photographing type,
Wherein the first route is set along the outline so as to maintain the same distance as the outline of the object to be imaged,
Wherein the second route is set as a straight line distance based on a point at the farthest distance from the outline of the object to be picked up with reference to the object to be picked up,
When the second route is set, the image pickup device determines an expected focal distance based on the three-dimensional model,
The image capturing apparatus performs image capturing for the object to be captured in consideration of the estimated focal distance in accordance with the movement of the moving object to the second route,
The surface to be imaged by the image pickup unit which meets a straight line drawn vertically so as to pass through the lens with respect to the center point of the lens of the image pickup unit implemented in the image pickup device,
Wherein when the surface to be projected is a plane free from bending and perpendicular to the straight line, the image pickup device is configured to obtain information on a surface to be scanned, which is the shortest distance between the center point of the lens and the surface to be scanned, The position of the four vertexes of the surface to be inspected is determined,
When the surface to be inspected is not the bend-free plane and is not perpendicular to the straight line, the image capturing device is configured to detect, based on the information on the surface to be imaged, the optical system characteristic information, and the angle information distorted by the imaging axis of the image capturing unit, Determining the positions of the four vertexes of the slope,
Wherein the image pickup device measures each of a plurality of depth information for each of a plurality of measurement points located on the surface to be scanned based on the depth measurement array when the surface to be scanned is a plane on which the curvature exists,
Wherein the presence or absence of a crack in the surface to be inspected is determined based on each of the plurality of depth information.
The method according to claim 1,
Wherein the image capturing device performs modeling on the surface to be etched using a combination model for a figure to be fitted to the surface to be etched based on the plurality of depth information,
Wherein the figure to be fitted includes a plane, a spherical surface, a cylinder, a step or a spiral.
3. The method of claim 2,
Wherein the surface to be surveyed is divided into a plurality of divided survey surfaces based on the depth measurement array,
Wherein the presence or absence of the crack is determined for each of the plurality of divided surfaces to be sliced.
The method of claim 3,
When there is a non-imaging area in which the curvature exists on the surface to be imaged and depth information on a part of the surface is not obtained due to bending, the moving object has a first movement path for acquiring information on the non- Capturing an image of the non-imaging region on the second movement path, combining the image information with the previously captured image information, and performing modeling on the surface to be imaged,
Wherein the second movement path is included in a space on the space that is the farthest from the image pickup object among the image pickup area.
5. The method of claim 4,
When a first divided surface to be scanned having a crack is found in the divided surface to be scanned, the moving body sets a third moving path for performing the secondary scanning on the first divided surface to be scanned, Dividing the first divided surface to be re-divided into a plurality of re-divided surfaces to detect information on cracks in the re-divided surface,
Wherein the fourth movement path is included in a space on the space that is the closest distance from the object to be picked up among the image pickup possible areas.
6. The method of claim 5,
Wherein the moving object scans the object to be imaged together with a plurality of other moving objects,
When the moving body is the master movement, dividing the image pickup area into a plurality of divided image pickup areas,
Wherein the moving object allocates each of the plurality of divided imaging areas to each of the other plurality of moving objects.
The method according to claim 6,
Wherein the non-movable area is set in consideration of a global positioning system (GPS) error and a control error with respect to the moving object.

Images