KR101736089B1 - UAV flight control device and method for object shape mapping and real-time guidance using depth map - Google Patents

Abstract

Disclosed is an apparatus and method for controlling a UAV for object mapping and real time guidance using a depth map. The UAV control device for object shape mapping and real-time guidance according to an embodiment of the present invention includes a plurality of cameras for acquiring multiple images, and a depth map generating depth map including depth information through multiple image processing part; An obstacle detection unit for detecting obstacles existing in the multiple images from the depth information acquired based on the depth map; A mapping unit for converting a position in the depth map of the obstacle detected to an absolute position of an inertial frame; And an unmanned launching unit for controlling the position and attitude of the UAV according to the positional relationship between the UAV and the obstacle.

Description

[0001] UAV flight control device and method for object shape mapping and real-time guidance using depth map [

The present invention relates to an apparatus and method for UAV control for object shape mapping and real-time guidance using depth maps.

In general, an aircraft is a machine that can acquire bearing capacity in the atmosphere by the reaction of air to the surface.

Unmanned Aerial Vehicle (UAV) is a non-pilot aircraft that has attracted much attention in the civilian field as well as the military in recent years.

UAVs are used extensively for missions such as surveillance, reconnaissance, monitoring, exploration, mapping, transportation, and disaster relief, as they can perform various tasks effectively and effectively in hazardous environments such as battlefields and disaster areas.

Among these missions, the mapping task can help the efficient work in the planning of the future mission planning by accessing various unknown obstacles including natural structures and artificial structures and grasping the shapes of the obstacles.

As a typical UAV, there is a quadrotor, which is becoming a popular UAV platform due to its ability to carry out vertical takeoff and landing as well as high speed.

The quad rotor is a rotary wing unmanned aircraft constructed by crossing two rods with four rotors. Unlike a typical helicopter, the quad rotor balances using the vertical thrust of a rotor that is reversing. Therefore, a tail rotor is not required, resulting in reduced contact risk. In addition, each rotor of the quad rotor is considerably smaller than the main rotor of the helicopter, thereby reducing the damage effect in the event of a collision.

Path planning is required for UAV to perform autonomous flight.

1 is a diagram for explaining a problem of an online route planning.

As a typical on-line path planning problem, since the pre-planned path 20 from the start point to the target point is designed by the planning algorithm, the UAV will either encounter an unexpected obstacle 10 as shown in FIG. 1, It is necessary to re-plan the route in real time to design the redesigned route 22.

However, if the consideration environment is completely unknown, there is no pre-planned route, and a real-time guidance algorithm is necessary to calculate the path of the UAV.

As a related art, Korean Patent Laid-Open No. 10-2014-0117836 (a traveling path generating device for an unmanned autonomous vehicle and its control method) is applied to a state-based path planning technique defined by a combination of an environmental mode and a traveling mode, A technique capable of generating an effective path of an unmanned autonomous vehicle in an environment is disclosed. However, it is limited to a two-dimensional planar part regarding the vehicle, and there is a limit to establish a route plan for autonomous vehicle running by using a grid map.

The above-described background technology is technical information that the inventor holds for the derivation of the present invention or acquired in the process of deriving the present invention, and can not necessarily be a known technology disclosed to the general public prior to the filing of the present invention.

Korean Patent Publication No. 10-2014-0117836

The object of the present invention is to provide an apparatus and method for controlling object-to-object mapping and real-time guidance using a depth map capable of real-time guidance for mapping an unknown obstacle by using a stereo vision sensor in a situation where there is no pre- .

The present invention relates to object map mapping using a depth map using a state of an unmanned aerial vehicle to generate a path for efficiently mapping obstacles by performing mapping of an obstacle shape by acquiring data points from a depth map, And to provide a flight control device and method.

The present invention makes it possible to acquire obstacle information in all directions and maintain an ideal gap by using the horizontal position state of the UAV, to prevent the collision in the moving direction during the mapping by controlling the heading angle, The object of the present invention is to provide an apparatus and a method for controlling a UAV for real-time guidance and object shape mapping using a depth map capable of mapping a part of an obstacle that does not appear in the image plane.

Other objects of the present invention will become readily apparent from the following description.

According to an aspect of the present invention, there is provided a depth map generation unit for acquiring multiple images through multiple cameras and generating a depth map including depth information through multiple image processing; An obstacle detection unit for detecting obstacles existing in the multiple images from the depth information acquired based on the depth map; A mapping unit for converting a position in the depth map of the obstacle detected to an absolute position of an inertial frame; And an unmanned launching unit for controlling the position and attitude of the UAV according to the positional relationship between the UAV and the obstacle.

The location and shape of the obstacle may be unknown.

The UAV may include a horizontal position controller for controlling the horizontal position of the UAV to generate a path parallel to the xy plane of the inertia frame.

The horizontal position control unit may assign a weight of 0 to the tangent vector for the obstacle and a weight of 1 for the normal vector for the obstacle when the UAV becomes closer to the obstacle from the predetermined range from the ideal distance from the obstacle And when the UAV moves away from the ideal range from the obstacle, the tangent vector is given a weight of 0, and the normal vector is given a weight of -1, Wherein when the UAV is within a predetermined range from the ideal distance, the weight of the tangent vector and the weight of the normal vector are set to a value between -1 and 1, brother This mapping can be performed.

The horizontal position controller may control the horizontal position of the UAV according to the following equation.

는 상기 무인기가 제어될 수평 위치 상태,

은 위치 가중 파라미터,

는 접선 벡터,

는 법선 벡터,

는 상기 무인기의 현재 위치 벡터,

,

은 상기 무인기와 상기 장애물의 최근접 거리,

는 상기 무인기와 상기 장애물의 이상적인 거리,

은 위치 가중 파라미터의 변곡 기준점임.here,

The horizontal position state in which the UAV is controlled,

Is a position weighting parameter,

Is a tangent vector,

Is a normal vector,

Is the current position vector of the UAV,

,

The distance between the UAV and the obstacle,

Is an ideal distance between the UAV and the obstacle,

Is the inflection point of the position weighting parameter.

And a distance measuring sensor for measuring a distance from the obstacle in a moving direction of the UAV, wherein the UAV has a heading angle of the UAV when the part of the obstacle is within a predetermined distance in the moving direction of the UAV, And a heading angle controller for changing the direction of the sensor line of sight of the UAV and the movement direction of the UAV.

The heading angle control unit may control the heading angle of the UAV according to the following equation.

는 상기 무인기가 제어될 요 각도 상태,

는 자세 가중 파라미터,

는 센서 시선 방향의 요 각도,

는 무인기 이동 방향의 요 각도,

는 이동 방향으로의 상기 무인기와 상기 장애물 사이의 간격,

는 허용간격 상한,

는 허용간격 하한임.here,

Is a state in which the UAV is controlled,

A posture weighting parameter,

The yaw angle in the sensor sight direction,

The yaw angle of the UAV movement direction,

The distance between the unmanned aerial vehicle and the obstacle in the moving direction,

Is the upper limit of the allowable interval,

Is the allowable spacing.

The UAV may include a vertical position control unit for activating the UAV when the detected shape of the obstacle is not included in the image plane of the multiple images so that the UAV is placed in the image plane .

If the obstacle is not aligned in the image plane in the up and down directions, the vertical position control unit activates the UAV by selecting one of the up and down directions, and if the obstacle reaches one end of the obstacle, Can be activated.

The mapping unit may perform mapping when the UAV moves in the other direction.

The mapping unit may remove redundant data using a disparity of data points collected for the obstacle.

The mapping unit may compare the time difference with another data point existing within a predetermined range around an arbitrary data point to remove the data points except the data point having a high time difference.

According to another aspect of the present invention, there is provided a UAV flight control method implemented by a UAV that is installed in a UAV and controls the flight of the UAV, and a recording medium on which a program for performing the UAV flight control is recorded.

The method includes the steps of: (a) acquiring multiple images through multiple cameras and generating a depth map including depth information through multiple image processing; (b) detecting an obstacle existing in the multiple image from depth information acquired based on the depth map; (c) controlling the position and posture of the UAV according to the positional relationship between the UAV and the obstacle; And (d) converting the position at the depth map to the absolute position of the inertial frame for the detected obstacle.

The step (c) may include the step (c1) of controlling the horizontal position of the UAV to produce a path parallel to the xy plane of the inertia frame.

In the step (c1), when the UAV becomes closer to the obstacle from the predetermined range from the ideal distance from the obstacle, a weight of 0 is given to the tangent vector to the obstacle and a weight of 1 to the normal vector to the obstacle And when the UAV moves away from the ideal range and away from the obstacle, the tangent vector is given a weight of 0 and the normal vector is given a weight of -1 The weight of the tangent vector and the weight of the normal vector are set to a value between -1 and 1 so that the obstacle is close to the obstacle, Shape mapping of .

The step (c1) may control the horizontal position of the UAV according to the following equation.

는 상기 무인기가 제어될 수평 위치 상태,

은 위치 가중 파라미터,

는 접선 벡터,

는 법선 벡터,

는 상기 무인기의 현재 위치 벡터,

,

은 상기 무인기와 상기 장애물의 최근접 거리,

는 상기 무인기와 상기 장애물의 이상적인 거리,

은 위치 가중 파라미터의 변곡 기준점임.here,

The horizontal position state in which the UAV is controlled,

Is a position weighting parameter,

Is a tangent vector,

Is a normal vector,

Is the current position vector of the UAV,

,

The distance between the UAV and the obstacle,

Is an ideal distance between the UAV and the obstacle,

Is the inflection point of the position weighting parameter.

Further comprising the step of measuring a distance to a part of the obstacle in a moving direction of the UAV, wherein the step (c) further includes a step of, when a part of the obstacle is within a predetermined interval in the moving direction of the UAV, (C2) so that the direction of the sensor line of sight of the UAV and the movement direction are changed.

The step (c2) may control the heading angle of the UAV according to the following equation.

는 상기 무인기가 제어될 요 각도 상태,

는 자세 가중 파라미터,

는 센서 시선 방향의 요 각도,

는 무인기 이동 방향의 요 각도,

는 이동 방향으로의 상기 무인기와 상기 장애물 사이의 간격,

는 허용간격 상한,

는 허용간격 하한임.here,

Is a state in which the UAV is controlled,

A posture weighting parameter,

The yaw angle in the sensor sight direction,

The yaw angle of the UAV movement direction,

The distance between the unmanned aerial vehicle and the obstacle in the moving direction,

Is the upper limit of the allowable interval,

Is the allowable spacing.

Wherein the step (c) includes the step (c3) of causing the obstacle to be placed in the image plane by activating the UAV if the detected shape of the obstacle is not included in the image plane of the multiple images can do.

If the obstacle is not aligned within the image plane in all the up and down directions, the step (c3) may select one of the up and down directions to start the UAV, and if the obstacle reaches one end of the obstacle, .

The step (d) may perform the mapping when the UAV moves in the other direction.

The step (d) may remove redundant data using the parallax of the data points collected for the obstacle.

The step (d) may compare the time difference with another data point existing within a predetermined range around an arbitrary data point to remove the remaining data points except the data point having a high time difference.

Other aspects, features, and advantages will become apparent from the following drawings, claims, and detailed description of the invention.

According to the embodiment of the present invention, there is an effect that the UAV can perform real-time guidance for mapping an unknown obstacle by using the stereo vision sensor in a situation where there is no pre-planned route.

Further, mapping of the obstacle shape is performed by acquiring data points from the depth map, and there is an effect of using the state of the UAV to generate a path for efficiently mapping obstacles.

In addition, it is possible to acquire obstacle information in all directions using the horizontal position state of the UAV, maintain an ideal interval, control the heading angle to prevent collision in the moving direction during mapping, It is possible to map a part of an obstacle that does not appear in the image plane.

1 is a diagram for explaining a problem of an online route planning,
2 is a block diagram illustrating a configuration of a UAV in accordance with an embodiment of the present invention.
Figure 3 shows a quad rotor platform and coordinate system,
4 is a conceptual diagram of multiple images,
5 is a detailed block diagram of the UAV activation section,
6 is a diagram illustrating the direction of a UAV path for an obstacle,
Figure 7 is a graph of a position weighting parameter function,
8 is a graph of the posture weighted parameter function,
9 is a diagram illustrating a process of avoiding collision during mapping,
10 is a view illustrating a state in which a lower portion of an obstacle is explored by controlling an altitude of a UAV;
11 is a view showing an angle used for calculation for placing an obstacle in the field of view of the camera,
12 is a diagram for explaining redundant data processing in the mapping unit,
13 is a diagram showing the locus of the UAV and the mapping result in the simulation 1 case,
14 is a graph showing a time history of mapping-related parameters in the case of simulation 1,
15 is a graph showing a time history of UAV-related parameters in the case of simulation 1,
16 is a diagram showing the locus of the UAV and the mapping result in the case of the simulation 2,
FIG. 17 is a view showing a depth map obtained at each point in FIG. 16;
18 is a graph showing a time history of UAV-related parameters in the case of Simulation 2,
FIG. 19 is a flowchart of a method for controlling a UAV that is performed in a UAV according to an embodiment of the present invention; FIG.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms "comprises" or "having" and the like refer to the presence of stated features, integers, steps, operations, elements, components, or combinations thereof, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

Also, the terms " part, "" module," and the like, which are described in the specification, mean a unit for processing at least one function or operation, and may be implemented by hardware or software or a combination of hardware and software.

It is to be understood that the components of the embodiments described with reference to the drawings are not limited to the embodiments and may be embodied in other embodiments without departing from the spirit of the invention. It is to be understood that although the description is omitted, multiple embodiments may be implemented again in one integrated embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

3 is a diagram illustrating a quadrotor platform and a coordinate system. FIG. 4 is a conceptual diagram of multiple images, and FIG. 5 is a diagram illustrating an example of a multi- FIG. 7 is a graph of a position weighted parameter function, FIG. 8 is a graph of a posture weighted parameter function, and FIG. 9 is a graph showing a collision during mapping FIG. 10 is a view showing a state in which a lower part of an obstacle is controlled by controlling an altitude of an unmanned aerial vehicle, FIG. 11 is a view showing an angle used for calculation for placing an obstacle in the field of view of a camera And FIG. 12 is a diagram for explaining the redundant data processing in the mapping unit.

The UAV is provided in a UAV so that an unknown obstacle is mapped using a depth map obtained using a stereo vision sensor in the absence of a predetermined route, .

The UAV 100 may be applied to a UAV such as a multi-rotor having a plurality of rotors including a propeller and a driving unit for providing driving force to control the UAV.

Hereinafter, for convenience of explanation and understanding of the present invention, it is assumed that the UAV is a quad rotor.

3, the quadrotor includes two rods crossing each other and two pairs of rotors installed at both ends of the two rods, i.e., a front rotor and a rear rotor, and a left rotor and a right rotor. .

The front rotor and the rear rotor rotate in the same direction, and the left rotor and the right rotor rotate in different directions.

By controlling the speed of these rotors, the ideal motion of the quad rotor can be obtained, and translational and rotational dynamic equations of motion are considered, as follows.

[Equation 1]

&Quot; (2) "

와

는 관성 프레임(inertial frame)에서 표현되는 위치 벡터(position vector)와 쿼드로터의 자세각 벡터(attitude angle vector)이고,

과

는 쿼드로터의 질량(mass) 및 관성 행렬(inertial matrix)이며,

와

은 힘과 토크에 대한 기체역학 마찰력 행렬(aerodynamic friction matrices for the force and the torque)이고,

와

는 쿼드로터의 로터 시스템에 의해 생산되는 힘 벡터와 토크 벡터(force and torque vectors)이며,

는 중력가속도 벡터이고,

은 로터 관성이며,

는

이며,

는 i번째 로터의 각속도(angular velocity)이고,

는 두 벡터의 외적(cross product)을 나타내며,

는 몸체 고정 프레임(body-fixed frame)에서 관성 프레임으로의 변환 행렬(transformation matrix)이고,

는 관성 프레임에서 몸체 고정 프레임으로의 회전 속도 행렬(rotation velocity matrix)이다. here,

Wow

Is an attitude angle vector of a quadrotor and a position vector expressed in an inertial frame,

and

Is the mass and inertial matrix of the quad rotor,

Wow

Is the aerodynamic friction matrices for the force and the torque for the force and torque,

Wow

Is the force and torque vectors produced by the rotor system of the quad rotor,

Is a gravitational acceleration vector,

Is rotor inertia,

The

Lt;

Is the angular velocity of the i-th rotor,

Represents the cross product of the two vectors,

Is a transformation matrix from a body-fixed frame to an inertial frame,

Is the rotation velocity matrix from the inertia frame to the body anchorage frame.

The external forces considered in this model are thrust force, aerodynamic force and gravitational force. External torque is torque produced by thrust, aerodynamic force, and rotation effect. A brushless direct current (BLDC) motor is used to model nonlinear actuator dynamics.

&Quot; (3) "

는 단말 펄스폭 변조(PWM) 전압이고,

는 입력 신호(input signal)이며,

은 저항(resistance)이고,

,

,

는 모터 토크 상수(motor torque constant), 부하 토크 상수(load torque constant), 역기전력 상수(back electromotive force constant)이며,

은 모터의 관성이고,

는 포화 함수(saturation function)이며,

는 데드존 함수(deadzone fuction)이고,

와

는 데드존 및 포화의 한계 값(deadzone and saturation limits)이다.
here,

Is the terminal pulse width modulation (PWM) voltage,

Is an input signal,

Is resistance,

,

,

Is a motor torque constant, a load torque constant, and a back electromotive force constant,

Is the inertia of the motor,

Is a saturation function,

Is a deadzone function,

Wow

Are the deadzone and saturation limits.

Referring to FIG. 2, the UAV 100 includes a depth map generator 110, an obstacle detector 120, a mapping unit 130, and a UAV activation unit 140.

The depth map generator 110 acquires multiple images through multiple cameras and generates a depth map including depth information for measuring the distance between the UAV and the object through the multiple image processing.

In the case of a monocular camera, the image processing speed is fast and the structure is simple, but it does not provide the depth information directly. Accordingly, in the present embodiment, multiple cameras capable of acquiring depth information from an image are utilized, and stereo cameras, for example, may be stereoscopic vision cameras.

Unlike monocular cameras, stereovision can calculate depth maps. The depth map is an image in which the pixel contains information about the distance from the view point to the object. When one camera is used, the object in the three-dimensional space is projected on the two-dimensional image plane, and the distance information along the projection direction disappears. However, when two cameras are used, the distance information can be recorded.

축을 향하도록 되어 있을 수 있다. For mapping purposes, the stereovision is mounted on a quad rotor

Axis direction.

The disparity can be obtained by multiplying the actual size of the pixel and the pixel difference between the left and right images of the object. Since the pixel difference value is a positive integer, the depth map does not provide complete information about the actual terrain.

The multiple image processing and depth map generation process using the multiple cameras are obvious to those skilled in the art, and a detailed description thereof will be omitted.

The obstacle detection unit 120 detects obstacles existing in the multiple images from the depth information acquired based on the depth map generated by the depth map generation unit 110. [

The position and shape of the obstacle are assumed to be unknown, so that the UAV does not initially have any information about the obstacle.

보다 작다면, 임의의 물체가 비젼 센서(스테레오 비젼)에 의해 감지된 것으로 볼 수 있다. 즉, 무인기의 전방에

범위 내에 물체가 존재하고 있다는 것이다. 물체를 나타내는 픽셀 범위는 장애물로서 고려될 수 있으며, 맵핑 프로세스가 시작된다. The UAV moves around the pre-allocated area while acquiring the depth map using the depth map generator 110 to find the obstacle. If the pixel depth value in the acquired depth map is less than a predefined threshold

, It can be seen that any object is detected by a vision sensor (stereo vision). That is, in front of the UAV

The object is in the range. The pixel range representing the object can be considered as an obstacle and the mapping process begins.

The mapping unit 130 maps the position of the pixel corresponding to the obstacle (for example, the position on the depth map, that is, the position in the body fixed frame) to the obstacle detected by the obstacle sensing unit 120, The position on the three-dimensional map of the inertia frame).

The mapping process is as follows.

와

를 관성 프레임과 몸체 고정 프레임에 대한 장애물 데이터 중 하나의 위치 벡터로 각각 정의한다. 깊이 지도에서 픽셀 깊이값은

성분에 상응하며, 이미지 평면에서의 위치는

및

성분에 상응한다. first

Wow

Is defined as a position vector of one of the obstacle data for the inertia frame and the body fixed frame, respectively. The pixel depth value in the depth map is

, And the position in the image plane corresponds to

And

≪ / RTI >

)에서 대표되는 이 3가지 성분은 맵핑 프로세스를 통해 관성 프레임 내의 성분으로 평가될 수 있다. 깊이 지도에서 픽셀의 위치 벡터

는 관성 프레임에서의 위치 벡터인

를 계산하기 위해 다음과 같이 사용될 수 있다. Body fixing frame (

) Can be evaluated as components in the inertial frame through the mapping process. The position vector of the pixel in the depth map

Is the position vector in the inertia frame

Can be used as follows.

&Quot; (4) "

All pixels selected in the depth map are transformed and stored in memory to reconstruct the obstacle in a three-dimensional map of the inertial frame.

The UAV activation unit 140 controls the position and posture of the UAV according to the positional relationship between the UAV and the obstacle so that the UAV can efficiently map the obstacle. The position and attitude control of the UAV can be accomplished through the generation and output of the guidance command.

를 사용하여 제어되는

와

의 6가지 상태를 가지고 있기 때문이다. UAVs (quad rotors) are essentially under-actuated systems, which have only four inputs

Controlled

Wow

Because it has six states.

The UAV activation unit 140 may include a position control module and an attitude control module.

과 3개의 출력

이 있으므로, 완전구동 시스템(fully actuated system)에 해당한다. The attitude control module has three inputs

And three outputs

, Which corresponds to a fully actuated system.

중 단지 하나의 제어 입력

(요(yaw) 각도 명령

)이 있는데, 이는 추력이 항상

축 상에서 작용하기 때문이다. 하지만, 제어되어야 하는 3개의 출력

이 있기에, 위치 제어 서브시스템은 부족구동 시스템이 된다. The position control module

Only one control input

(Yaw angle command

), Which means that the thrust is always

Axis. However, there are three outputs to be controlled

, The position control subsystem becomes a deficient drive system.

(롤(roll) 각도)와

(피치(pitch) 각도)가 가상 입력(virtual input)으로 위치 제어 모듈에 도입된다. 가상 입력과 원본 입력을 결합함으로써, 위치 제어 모듈을 설계하기 위해 3개의 입력

가 사용된다. In the present embodiment, in order to solve this problem,

(Roll angle) and

(Pitch angle) is introduced into the position control module as a virtual input. By combining the virtual input and the original input, three inputs

Is used.

A feedback linearization technique may be applied to the UAV 140, which is used to address the non-linearity of quad rotor dynamics. According to this, the nonlinear system is converted into the equivalent linear system. To control the transformed linear system, a linear quadratic tracker is designed. The purpose of the linear secondary tracker is to control the output to follow the specified output.

와

의 2개의 각도가 위치 제어 모듈에 의해 제공된다.
The output specified in the position control module is the position vector of the waypoint. The output specified in the attitude control module is the three attitude angles of the quad rotor,

Wow

Are provided by the position control module.

In the present embodiment, the UAV activation unit 140 can establish a mapping path plan for detecting an obstacle and generating an appropriate path for the UAV to effectively map the obstacle.

)만이 임무 수행을 위해 제어될 수 있다. The UAV must move around the obstacle in order to collect the obstacle information in all directions, and thereby the entire shape of the obstacle can be mapped. The path should be such that there is no collision with obstacles during mapping. However, as described above, only the four components of the target position (x, y, z), yaw angle

) Can be controlled for mission performance.

Referring to FIG. 5, the UAV 140 includes a horizontal position controller 142, a vertical position controller 144, and a heading angle controller 146.

The horizontal position control unit 142 controls the horizontal position (states x and y) of the UAV so as to generate a path parallel to the xy plane of the inertia frame.

와 접선 벡터

를 정의한다. UAV maps obstacles while following a path that travels around the detected obstacle. As shown in Fig. 6,

And tangent vectors

.

를 정의한다. 장애물 데이터 지점의 중심인

은 다음과 같이 계산될 수 있다. It is assumed that n points of the obstacle 10 are viewed in the image plane of the vision sensor. From the inertial frame to the kth data point of the obstacle

. The center of the obstacle data point

Can be calculated as follows.

&Quot; (5) "

를 정의하고, 장애물 중심의 (x, y) 성분 위치 벡터로서

를 정의하면 다음과 같다. Further, as the (x, y) component position vector of the UAV

(X, y) component position vector of the obstacle center

Is defined as follows.

&Quot; (6) "

는

에 수직인 벡터인 것으로 정의할 수 있으며,

의 영 공간(null space)를 계산함으로써 획득될 수 있다. After that

The

And a vector perpendicular to the vector,

And the null space of the first block.

는 다음과 같이 획득될 수 있다. Finally, the vector of the tangential direction with respect to the obstacle 10

Can be obtained as follows.

&Quot; (7) "

는 스케일링 인자(scaling factor)이다. 접선 벡터

는 무인기가 맵핑 동안 장애물(10) 주위를 움직이도록 하기 위해 사용된다.

Is a scaling factor. Tangent vector

Is used to allow the UAV to move around the obstacle 10 during mapping.

는 몸체 고정 프레임에서 장애물의 k번째 데이터 지점으로 정의된다. 그 후 최소

를 가지는 지점을 선택함으로써, 비젼 센서로부터 가장 근접한 장애물 데이터 지점이 획득될 수 있고, 이를

으로 나타낸다. 수학식 4를 사용하면,

은 관성 프레임에서

과 같이 변환될 수 있다. vector

Is defined as the kth data point of the obstacle in the body anchoring frame. After that,

The nearest obstacle data point from the vision sensor can be obtained,

Respectively. Using equation (4)

In the inertia frame

. ≪ / RTI >

&Quot; (8) "

의 (x, y) 성분으로 구성된 벡터로서

를 정의하면,

은 다음과 같이 계산될 수 있다.

As a vector composed of (x, y) components of

By definition,

Can be calculated as follows.

&Quot; (9) "

는 다음과 같이 계산될 수 있다.Vector of normal direction

Can be calculated as follows.

&Quot; (10) "

은 스케일링 인자이다. 법선 벡터

는 장애물(10)과의 간격을 두어 장애물 맵핑 중에 장애물(10)과의 충돌을 피하기 위해 사용된다. 무인기와 가장 근접한 장애물 데이터 지점 사이의 거리(최근접 거리)는 다음과 같이 계산될 수 있다.

Is a scaling factor. Normal vector

Is used to avoid collision with the obstacle 10 during the obstacle mapping by spacing the obstacle 10 from the obstacle 10. The distance between the UAV and the nearest obstacle data point (closest distance) can be calculated as:

&Quot; (11) "

를 도입한다.

이

보다 작아지면, 무인기는 안전거리를 유지하기 위해 장애물(10)로부터 멀어지도록 움직여야 한다.

이

보다 커지면, 무인기는 장애물을 향해 움직여서 무인기가 이상적인 거리에서 장애물(10)을 맵핑할 수 있도록 한다. Now, the ideal distance between the UAV and the obstacle 10

.

this

, The UAV must move away from the obstacle 10 to maintain the safety distance.

this

, The UAV moves toward the obstacle so that the UAV can map the obstacle 10 at an ideal distance.

와

를 결합함으로써 무인기의 바람직한 위치

를 생성하기 위해 위치 가중 파라미터

이 도입된다. In this embodiment,

Wow

The desired position of the UAV

Lt; RTI ID = 0.0 >

.

&Quot; (12) "

&Quot; (13) "

이고,

은 설계 파라미터이다. The difference between the closest distance and the ideal distance

ego,

Is a design parameter.

에 대한

함수 모습이 도시되어 있다. 위치 가중 파라미터

의 변곡 기준점인

이 2인 경우가 예시되어 있으나, 이는 일 실시예에 불과하며, 이상적인 거리의 1/2 혹은 다양한 거리가 선택될 수 있음은 물론이다. Referring to FIG. 7 (a), the equation

For

The function appearance is shown. Position weighting parameter

Quot;

2 is illustrated, but this is merely an example, and it goes without saying that 1/2 of the ideal distance or various distances can be selected.

가 -2m인 경우를 고려해보면, 무인기가 물체에 너무 가까이 있음을 의미한다. 수학식 13 혹은 도 7의 (a)에서

은 1이 되고, 이를 수학식 12에 대입하면 x와 y의 바람직한 상태는 다음과 같이 획득될 수 있다.

Is -2m, it means that the UAV is too close to the object. In Equation 13 or 7 (a)

Becomes 1, and by substituting this into Equation (12), a desirable state of x and y can be obtained as follows.

&Quot; (14) "

Equation (14) means that the UAV is focused on moving only in the direction away from the obstacle in order to prevent collision with the object.

가 0(zero)인 경우를 고려해보면, 무인기가 이상적인 거리에 있음을 의미한다. 이 경우,

은 0(zero)이고, x와 y의 바람직한 상태는 다음과 같이 획득될 수 있다. On the other hand,

Is zero, it means that the UAV is at an ideal distance. in this case,

Is zero, and the desired states of x and y can be obtained as follows.

&Quot; (15) "

Equation (15) implies that the UAV is focused on only moving around the obstacle for mapping.

을 1로 설정함으로써 법선 벡터가 덧셈 연산되어 무인기가 물체와의 충돌을 방지하기 위해 장애물로부터 멀어지는 방향으로만 움직이도록 할 수 있다. That is, when the UAV is approaching the obstacle beyond the designed distance from the ideal distance (range 1), the position weighting parameter

Is set to 1 so that the normal vector is additionally operated so that the UAV moves only in a direction away from the obstacle in order to prevent collision with the object.

을 -1로 설정함으로써 법선 벡터가 뺄셈 연산되어 무인기가 장애물로부터 너무 멀어지는 것을 방지하기 위해 장애물로 가까워지는 방향으로만 움직이도록 할 수 있다. Also, if the UAV is away from the obstacle more than the designed distance from the ideal distance (range 2)

Is set to -1 so that the normal vector is subtracted so as to move only in the direction of approaching the obstacle to prevent the UAV from being too far away from the obstacle.

을 -1과 1 사이의 값으로 설정함으로써 장애물과의 간격을 유지하면서도 장애물과의 충돌을 방지하여 원활할 맵핑이 이루어지도록 할 수 있다. Also, if the UAV is located within a designed distance from the ideal distance (range 3)

Is set to a value between -1 and 1, it is possible to prevent the collision with the obstacle while maintaining the gap with the obstacle, so that the mapping can be smoothly performed.

이 0(zero)로 설정되거나(도 7의 (b) 참조), 완만한 곡선 대신 직선 함수(

)로 설정(도 7의 (c) 참조)될 수도 있다.
If the UAV is located within a designed distance from the ideal distance (range 3), unlike Equation 13, the position weighting parameter

Is set to zero (see Fig. 7 (b)), a straight line function (

(See Fig. 7 (c)).

Referring again to FIG. 5, the heading angle control unit 146 controls the yaw angle of the UAV to change the heading angle of the UAV to prevent the UAV from colliding with a part of the obstacle placed in the moving direction.

축을 따라 이동하는 경우를 고려해 보면, 비젼 센서(스테레오 비젼 카메라)는 맵핑을 위해

축을 바라보고 있다. 이제, 상태

는

방향에서 발생될 수 있는 충돌을 방지하기 위해 사용된다. UAV

Considering the case of moving along an axis, the vision sensor (stereo vision camera)

I am looking at the axis. Now,

The

It is used to prevent collisions that may occur in the direction.

축에 따른 무인기와 장애물 사이의 거리

를 측정할 수 있다. For this, it is assumed that the UAV according to the present embodiment includes a distance measuring sensor 148 such as an ultrasonic sensor. The distance measuring sensor 148 measures the distance

Distance between UAV and obstacle along the axis

Can be measured.

가 미리 정의된 거리

보다 크면, 무인기는

축에 따른 위험이 없는 것으로 감지한다. 이 경우 무인기는

축 상에 최근접 장애물 데이터 지점을 선택하여 맵핑을 위한 요 각도 명령을 계산한다.

Predefined distance

If greater, the UAV is

It is perceived as no risk to the axis. In this case,

Calculate the yaw angle command for the mapping by selecting the nearest obstacle data point on the axis.

&Quot; (16) "

가

보다 작아지면,

축 방향으로의 장애물이 이동 방향을 변경하지 않을 경우 무인기에게 위협이 될 수 있다. 이 경우 무인기는

축으로부터 접근하는 장애물을 바라보고 맵핑하도록 요 각도를 회전해야 한다. 그러므로, 충돌을 회피하기 위한 요 각도 명령은 다음과 같이 주어질 수 있다.

end

If smaller,

If an obstacle in the axial direction does not change the direction of movement, it can be a threat to UAV. In this case,

You must rotate the yaw angle to see and map the obstacles approaching from the axis. Therefore, the yaw angle command for avoiding the collision can be given as follows.

&Quot; (17) "

과 무인기 이동 방향의 요 각도

를 결합함으로써, 바람직한 요(yaw) 상태인

를 생성하기 위해 다음과 같은 자세 가중 파라미터

가 도입될 수 있다. Yaw angle in the direction of the sensor line of sight

And the yaw angle of the direction of movement of the UAV

, A desired yaw state

The following posture weighting parameters

Can be introduced.

&Quot; (18) "

&Quot; (19) "

과 허용간격 하한

은 설계 파라미터이다. Allowable upper limit

And allowable lower limit

Is a design parameter.

에 대한

함수 모습이 도시되어 있다. 여기서,

는 6이고,

는 4인 것으로 가정한다. 8,

For

The function appearance is shown. here,

Is 6,

Is assumed to be four.

가 0인 경우에는 요 각도가 그대로 유지되며, 1인 경우에는 90도 회전을 수행하게 되며, 0과 1 사이에 있는 경우에는 센서 시선 방향의 요 각도

과 무인기 이동 방향의 요 각도

를 적절하게 결합하여 요 각도 명령을 산출하게 된다.

Is 0, the yaw angle is maintained as it is. If it is 1, it is rotated 90 degrees. If it is between 0 and 1, the yaw angle

And the yaw angle of the direction of movement of the UAV

So as to calculate a yaw angle command.

이

보다 클 때 장애물을 맵핑하는 요 각도를 사용한다. As shown in Fig. 9 (a), the UAV

this

Use the yaw angle to map the obstacle when it is bigger.

축을 따라 이동함에 따라, 도 9의 (b)에 도시된 것과 같이

가

보다 작아지게 된다. 이 경우 수학식 19에 의해 자세 가중 파라미터

이 증가함에 따라 수학식 18에서 무인기 이동 방향의 요 각도

부분이 증가한다. UAV

As it moves along the axis, as shown in Figure 9 (b)

end

. In this case, the posture weighting parameter

As a result, the yaw angle of the UAV in the equation (18)

The portion increases.

은 물체의 새로운 측면에서 선택되며, 무인기는 장애물로부터 이상적인 거리를 유지하도록 움직인다. 이는 도 7의 (c)에서 무인기의 이동 방향을 확인하면 알 수 있다.
Then, as shown in FIG. 9 (c), the UAV will rotate the yaw angle in the positive direction, and the UAV will be able to detect a new aspect of the obstacle, the potential object to be mapped and the object to be avoided. in this case

Is selected from a new aspect of the object, and the UAV moves to maintain an ideal distance from the obstacle. This can be seen by checking the moving direction of the UAV in (c) of FIG.

Referring again to FIG. 5, the vertical position controller 144 controls the vertical position of the UAV to elevate the UAV.

The object to be mapped may be very large or high, and the image plane of stereo vision may not cover the object vertically. In this embodiment, to solve this problem, the state z is used to cover the entire obstacle for mapping.

As shown in Figure 10 (a), the rest of the obstacle may be below the image plane. This can be determined by examining the last row of the image plane. If the last row of the image plane contains at least one pixel representing an obstacle, there may be a portion of the obstacle down the image plane. Therefore, the UAV must lower the altitude to explore the lower part of the obstacle for mapping as follows.

&Quot; (20) "

는 설계 파라미터이다. Vertical spacing

Is a design parameter.

중에서 최소

를 가지는 지점(즉, 가장 낮은 위치에 있는 데이터 지점)이 선택될 수 있다. 그 지점의 z 성분을

로 정의한다. The data point of the obstacle

Minimum among

(I.e., the data point at the lowest position) can be selected. The z component of the point

.

와 이미지 평면의 최종 열 사이의 갭이 설정 수직 간격

(설계 파라미터, 픽셀 단위)가 되는 상황을 고려한다(도 10의 (b) 참조). 이 경우 무인기는 아래 방향으로 맵핑될 잔여분이 없는 것으로 판단하고, 고도를 낮추는 것을 중단할 수 있다. While lowering altitude,

And the last column of the image plane is the set vertical spacing

(Design parameter, pixel unit) (refer to Fig. 10 (b)). In this case, the UAV can judge that there is no remaining portion to be mapped downward, and can stop lowering the altitude.

는

중에서 최대

를 가지는

로 대체되고, 이미지 평면의 최종 열은 이미지 평면의 최초 열로 대체된다. 마지막으로 수학식 20은 다음 수학식을 대체된다.Similarly, the logic in the upward direction can be designed. variable

The

Maximum among

Having

And the last column of the image plane is replaced with the first column of the image plane. Finally, equation (20) is replaced by the following equation.

&Quot; (21) "

That is, when the shape of the obstacle detected by the obstacle sensing unit 120 is not included in the image plane, the vertical position control unit 144 may activate the UAV in the up or down direction so that the obstacle is placed in the image plane .

When there is a pixel corresponding to an obstacle in the final column of the image plane, the vertical position control unit 144 activates the UAV in the downward direction. If there is a pixel corresponding to the obstacle in the first row of the image plane, .

If the obstacle is not all aligned in the image plane in the up and down directions, one direction may be selected to explore the unknown portion of the obstacle. The UAV then moves in the other direction to complete the mapping.

Hereinafter, it will be assumed that the upper direction is selected first.

는 다음과 같이 계산될 수 있다. If the obstacles are all aligned in the image plane in the up and down directions, the obstacle must be maintained within the field of view of the camera during the mapping process. In Fig. 11, the angle from the center of the UAV on the xy plane to the center of the obstacle

Can be calculated as follows.

&Quot; (22) "

이

보다 크다면, 이미지 평면의 중간에 장애물의 중심이 놓이도록 무인기의 고도를 증가시킬 수 있다. 반대로,

가

보다 작다면 무인기의 고도를 감소시킬 수 있다. 이러한 기동이 다음과 같이 표현될 수 있다. The vertical position control unit 144 controls the vertical position of the U-

this

The altitude of the UAV can be increased so that the center of the obstacle lies in the middle of the image plane. Contrary,

end

The altitude of the UAV can be reduced. This maneuver can be expressed as:

&Quot; (23) "

The above equations also apply to UAVs that have completed exploration up or down.

Referring again to FIG. 1, the mapping unit 130 maps the shape of the obstacle while the UAV is moving around the obstacle by the induction command as described above.

In the mapping process, the data point information of the obstacle is collected corresponding to the predetermined data collection period. Over time, the amount of collected data will increase. Some of these may be redundant, and some may have relatively inaccurate information.

Thus, in this embodiment, the mapping unit 130 may reduce the number of data points of the collected obstacles using the disparity of the data.

내에 다른 지점을 가지고 있지 않다면, 해당 지점은 상응하는 영역을 대표하는 정보를 포함하는 유일한 지점이 된다. 반면,

범위 내에 몇몇 다른 지점들을 포함하고 있다면, 동일 지역 내에 다수의 지점들이 있는 것이다. 그러므로, 모든 지점을 저장할 필요는 없고, 해당 지역을 대표하는 가장 정확한 정보를 가지고 있는 지점만을 저장할 필요가 있다. 이는 지점들의 시차를 비교함으로써 수행될 수 있다. 시차가 높을수록 거리 오차가 낮음을 이용한다. First, consider the point with obstacle information. If a point has a predefined distance

If there is no other point in the map, the point is the only point containing the information representing the corresponding area. On the other hand,

If you have several other points in the range, then there are a number of points in the same area. Therefore, it is not necessary to store all the points, but only the points having the most accurate information representing the area need to be stored. This can be done by comparing the parallax of the points. The higher the parallax, the lower the distance error.

,

,

,

,

의 5개 지점이 표시되어 있다. This process is performed as follows. 12 shows that the parallax is 16, 20, 19, 13, 14

,

,

,

,

Are displayed.

은

의 범위 내에 다른 지점이 없다. 따라서, 시차에 상관없이

가 남게 된다. 12,

silver

There are no other points within the range of. Therefore, regardless of the time difference

.

는 경계 내에

를 가지고 있으며, 이들 시차인 20와 19를 비교함으로써

가 제거된다.

의 제거로 인해,

와

가 유일하게 중복된 지점들이다. 동일한 과정을 통해,

가 남는다.

Within boundaries

, And comparing these parallaxes 20 and 19

Is removed.

Due to the elimination of <

Wow

Are the only duplicated spots. Through the same process,

Is left.

As a result, two relatively inaccurate and duplicated points can be eliminated as shown in FIG.

In addition, when the UAV is activated by the vertical position controller 144, the mapping unit 130 may start the mapping process after arriving at the upper or lower end of the obstacle. When the UAV is located in the middle part of the obstacle and it is necessary to move downward after moving upward, if the mapping is performed while moving upward, the corresponding data is moved while moving downward, It is highly likely to be duplicated. In this case, since the memory is unnecessarily occupied, data management is required.

Therefore, if the UAV moves in one direction in the vertical direction for obstacle mapping, data is collected through the mapping process after the movement direction is changed at the end of the UAV, thereby preventing the collection of the duplicated data points and efficiently utilizing the memory .

The numerical simulation results on the performance of the UAV flight control method according to the present embodiment are as follows. The UAV parameters and stereo vision and mapping parameters used in the numerical simulation are shown in the following Tables 1 and 2.

[Table 1]

[Table 2]

In the case of obstacles with potential threats (simulation 1), the experimental results are as follows. 14 is a graph showing a time history of the mapping-related parameters in the case of the simulation 1, and Fig. 15 is a graph showing the time history of the UAV-related parameters in the case of the simulation 1 Fig. 13 (a) is an oblique view from above, and Fig. 13 (b) is a plan view.

In Simulation 1, L-shaped obstacles are considered as obstacles with potential threats.

는

보다 작게 되었다. 그 결과 도 14의 (d) 및 도 15(b)에 도시된 것처럼

는 양(positive)이 되고

를 증가시킨다. 도 13의 지점 B(t=9.4s)에서,

는 대략 90도(˚)에 도달한다(도 15의 (b) 참조). 즉, 무인기는 위협이 될 수도 있는 장애물의 측면을 감지하고 재빨리

를 증가시킨 것이다. 그 후 무인기는 도 13에 도시된 것과 같이 장애물을 맵핑한다. 지점 B 이후에는

축 방향으로는 장애물을 감지하지 못하여

는 정의될 수 없다. 따라서,

는 0(zero)으로 남아 있게 된다(도 14의 (d) 참조).
As shown in Fig. 14 (c), from the point A (t = 6.95s) in Fig. 13

The

Respectively. As a result, as shown in Figs. 14 (d) and 15 (b)

Is positive

. At point B (t = 9.4s) in Fig. 13,

Reaches about 90 degrees (see Fig. 15 (b)). In other words, UAV is able to detect aspects of the obstacles that may be a threat,

. Thereafter, the UAV maps the obstacle as shown in FIG. After point B

In the axial direction, the obstacle can not be detected

Can not be defined. therefore,

Remains at 0 (see Fig. 14 (d)).

In the case of high obstacles (simulation 2), the experimental results are as follows. FIG. 16 is a view showing the trajectory of the UAV and the mapping result in the case of the simulation 2, FIG. 17 is a view showing the depth map obtained at each point in FIG. 16, and FIG. 18 is a diagram showing the time history of the UAV- Fig. 16 (a) is an oblique view from above, and Fig. 16 (b) is a plan view.

In Simulation 2, a rectangular parallelepiped having a size of 2m × 2m × 15m and centered on [9 0 2.5] ^T is considered as an obstacle.

Figures 16 (a) and 16 (b) show the results at t = 151.35 s and t = 260 s. It can be confirmed that the UAV is moving up and down to map the entire obstacle.

Figure 17 shows the depth map obtained by stereovision at point A (t = 151.35s) and B (t = 260s).

사이의 갭이

가 되어, 도 16의 (b)의 지점 A에서 상방향으로의 탐사가 완료된다. 이때부터 무인기는 아래로 고도를 낮추면서 장애물의 하부 부분에 대해 탐사를 수행하게 된다. 도 18의 (c) 및 (d)는 지점 A 이후 제어 입력과

가 감소하고 있음을 보여준다. 도 17의 (b)에 도시된 것처럼 이미지 평면의 최종 열과

사이의 갭이

가 되어, 도 16의 (b)의 지점 B에서 하방향으로의 탐사가 완료된다.
When an unmanned aircraft touches an obstacle for the first time, the obstacle will not fit in the image plane in the upward and downward directions. Thus, the UAV first searches the upper part of the obstacle. As shown in Fig. 17 (a), the first column of the image plane

The gap between

And the search in the upward direction at point A in FIG. 16 (b) is completed. From this point, the UAV will perform a survey on the lower part of the obstacle while lowering the altitude downward. 18 (c) and 18 (d) show the control input after point A and

Of the total population. As shown in Fig. 17 (b), the final column of the image plane

The gap between

And the search from the point B to the downward direction in FIG. 16 (b) is completed.

FIG. 19 is a flowchart of a method for controlling a UAV that is performed in an UAV according to an embodiment of the present invention.

Each step of FIG. 19 may be performed by each component of the UAV 100 shown in FIG.

The depth map generation unit 110 acquires a depth map through stereo vision while moving around the pre-allocated area (step 310). Since the location and shape of the obstacle is unknown, there is no pre-planned path, so it moves around the pre-allocated area while acquiring the depth map to find obstacles.

The obstacle detection unit 120 detects obstacles existing in the multiple images from the depth information acquired based on the acquired depth map (step 320).

The UAV activation unit 140 controls the position and attitude of the UAV so that the effective mapping of the detected obstacles is performed (step 330).

Here, the unmanned start step (step 330) may include a horizontal position control step (step 332), a heading angle control step (step 334), and a vertical position control step (step 336).

In the horizontal position control step (step 332), the UAV 140 controls the horizontal position (states x and y) of the UAV to produce a path parallel to the xy plane of the inertia frame.

In this process, if the distance from the ideal range to the obstacle is less than the predetermined range (range 1) so that the UAV does not collide with the obstacle, a weight of 0 is assigned to the tangent vector and a weight of 1 is assigned to the normal vector. (Range 2), the tangent vector is given a weight of 0, and the normal vector is given a weight of -1, so as to approach the obstacle for obstacle mapping. If the distance is within a predetermined range from the ideal distance (range 3), the weight of the tangent vector and the weight of the normal vector may be set to a value between -1 and 1 so that the collision can be prevented while maintaining the distance from the obstacle have.

In the heading angle control step (step 224), the UAV 140 controls the yaw angle of the UAV to change the heading angle of the UAV to prevent the UAV from colliding with a part of the obstacle placed in the moving direction.

If a part of the obstacle is approaching within a predetermined distance in the moving direction of the UAV through the distance measuring sensor, the heading angle of the UAV can be rotated to change the sensor sight direction and the UAV moving direction.

In the vertical position control step (step 336), the UAV 140 may activate the UAV in the up or down direction so that the obstacle falls within the image plane if the shape of the detected obstacle is not all contained within the image plane.

If the obstacle is not all aligned in the image plane in the up and down directions, one direction may be selected to explore the unknown portion of the obstacle. The UAV can then move in the other direction to complete the mapping.

The mapping unit 130 performs obstacle mapping by converting the position of the pixel corresponding to the obstacle (the position on the depth map) to the absolute position (the position on the three-dimensional map of the inertia frame) .

In the mapping step, data management may be performed by removing redundant data using the parallax of the data (step 342).

In the case where the vertical position control is performed, the mapping process may be performed when the UAV starts to move in the other direction after the UAV completes the movement in one direction, thereby preventing duplicate data from being generated in advance.

It is a matter of course that the above-mentioned UAV control method may be performed by an automated procedure in a time series sequence by a built-in or installed program in a digital processing device (UAV). The codes and code segments that make up the program can be easily deduced by a computer programmer in the field. In addition, the program is stored in a computer readable medium readable by the digital processing apparatus, and is read and executed by the digital processing apparatus to implement the method. The information storage medium includes a magnetic recording medium, an optical recording medium, and a carrier wave medium.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention as defined in the following claims And changes may be made without departing from the spirit and scope of the invention.

100: UAV flight control unit 110: Depth map generating unit
120: Obstacle detecting unit 130: Mapping unit
140: UAV activation unit 142: Horizontal position control unit
144: Vertical position control unit 146: Heading angle control unit
148: Distance measuring sensor

Claims (25)

A device installed on a UAV to control a flight,
A depth map generation unit that acquires multiple images through multiple cameras and generates a depth map including depth information through multiple image processing;
An obstacle detection unit for detecting obstacles existing in the multiple images from the depth information acquired based on the depth map;
A mapping unit for converting a position in the depth map of the obstacle detected to an absolute position of an inertial frame; And
And an unmanned launching unit for controlling the position and attitude of the UAV according to the positional relationship between the UAV and the obstacle,
Wherein the position and the shape of the obstacle are in an unknown state.
delete The method according to claim 1,
Wherein the unmanned launch section includes a horizontal position control section for controlling the horizontal position of the UAV so as to generate a path parallel to the xy plane of the inertia frame.
The method of claim 3,
The horizontal position control unit,
When the unmanned aerial vehicle approaches the obstacle from a preset distance from the predetermined distance from the obstacle, a tangent vector for the obstacle is given a weight of 0 and a normal vector for the obstacle is given a weight of 1 to move away from the obstacle Direction of the UAV,
When the unmanned aerial vehicle moves away from the obstacle by a predetermined distance from the predetermined distance, the tangent vector is given a weight of 0, and the normal vector is given a weight of -1 to activate the unmanned aerial vehicle in a direction approaching the obstacle,
And setting the weight of the tangent vector and the weight of the normal vector to a value between -1 and 1 to perform the shape mapping of the obstacle when the UAV is within a preset range from the predetermined distance, UAV flight control system for real - time guidance.
The method of claim 3,
Wherein the horizontal position control unit controls the horizontal position of the UAV according to the following equation: < RTI ID = 0.0 > UAV < / RTI >

here,

The horizontal position state in which the UAV is controlled,

Is a position weighting parameter,

Is a tangent vector,

Is a normal vector,

Is the current position vector of the UAV,

,

The distance between the UAV and the obstacle,

The predetermined distance between the UAV and the obstacle,

Is the inflection point of the position weighting parameter.
The method according to claim 1,
Further comprising a distance measuring sensor for measuring a distance to a part of the obstacle in a moving direction of the UAV,
And a heading angle control unit for rotating the heading angle of the UAV when the part of the obstacle is within a predetermined distance in the movement direction of the UAV so that the direction of the sensor line of sight of the UAV is changed, UAV flight control system for object shape mapping and real - time guidance.
The method according to claim 6,
Wherein the heading angle control unit controls the heading angle of the UAV according to the following equation:

here,

Is a state in which the UAV is controlled,

A posture weighting parameter,

The yaw angle in the sensor sight direction,

The yaw angle of the UAV movement direction,

The distance between the unmanned aerial vehicle and the obstacle in the moving direction,

Is the upper limit of the allowable interval,

Is the allowable spacing.
The method according to claim 1,
Wherein the unmanned launching unit includes a vertical position control unit for activating the UAV when the detected shape of the obstacle is not included in the image plane of the multiple images so that the obstacle is placed in the image plane UAV flight control system for shape mapping and real time guidance.
9. The method of claim 8,
If the obstacle is not aligned in the image plane in the up and down directions, the vertical position control unit activates the UAV by selecting one of the up and down directions, and if the obstacle reaches one end of the obstacle, Wherein the control unit is configured to activate the object-shape mapping and real-time guidance.
10. The method of claim 9,
Wherein the mapping unit performs mapping when the UAV moves in the other direction.
The method according to claim 1,
Wherein the mapping unit removes redundant data by using a disparity of data points collected for the obstacle.
12. The method of claim 11,
Wherein the mapping unit compares a time difference with another data point existing within a predetermined range around an arbitrary data point and removes remaining data points excluding data points having a high time difference, UAV flight control system for.
A UAV flight control method performed by a UAV that is installed in a UAV and controls the flight of the UAV,
(a) acquiring multiple images through multiple cameras and generating a depth map including depth information through multiple image processing;
(b) detecting an obstacle existing in the multiple image from depth information acquired based on the depth map;
(c) controlling the position and posture of the UAV according to the positional relationship between the UAV and the obstacle; And
(d) converting the position in the depth map to the absolute position of the inertial frame for the detected obstacle,
Wherein the position and the shape of the obstacle are in an unknown state.
delete 14. The method of claim 13,
Wherein the step (c) includes the step (c1) of controlling the horizontal position of the UAV to create a path parallel to the xy plane of the inertia frame.
16. The method of claim 15,
The step (c1)
When the unmanned aerial vehicle approaches the obstacle from a preset distance from the predetermined distance from the obstacle, a tangent vector for the obstacle is given a weight of 0 and a normal vector for the obstacle is given a weight of 1 to move away from the obstacle Direction of the UAV,
When the unmanned aerial vehicle moves away from the obstacle by a predetermined distance from the predetermined distance, the tangent vector is given a weight of 0, and the normal vector is given a weight of -1 to activate the unmanned aerial vehicle in a direction approaching the obstacle,
And setting the weight of the tangent vector and the weight of the normal vector to a value between -1 and 1 to perform the shape mapping of the obstacle when the UAV is within a preset range from the predetermined distance, UAV flight control method for real - time guidance.
16. The method of claim 15,
Wherein the step (c1) controls the horizontal position of the UAV according to the following equation: < EMI ID =

here,

The horizontal position state in which the UAV is controlled,

Is a position weighting parameter,

Is a tangent vector,

Is a normal vector,

Is the current position vector of the UAV,

,

The distance between the UAV and the obstacle,

The predetermined distance between the UAV and the obstacle,

Is the inflection point of the position weighting parameter.
14. The method of claim 13,
Further comprising the step of measuring a distance to a portion of the obstacle in a moving direction of the UAV,
Wherein the step (c) includes the step (c2) of rotating the heading angle of the UAV when the part of the obstacle approaches the UAV in the moving direction of the UAV, thereby changing the direction of the sensor line of sight of the UAV and the moving direction A method of UAV control for object shape mapping and real time guidance.
19. The method of claim 18,
Wherein the step (c2) controls the heading angle of the UAV according to the following equation: < EMI ID =

here,

Is a state in which the UAV is controlled,

A posture weighting parameter,

The yaw angle in the sensor sight direction,

The yaw angle of the UAV movement direction,

The distance between the unmanned aerial vehicle and the obstacle in the moving direction,

Is the upper limit of the allowable interval,

Is the allowable spacing.
14. The method of claim 13,
Wherein the step (c) includes the step (c3) of causing the obstacle to be placed in the image plane by activating the UAV if the detected shape of the obstacle is not included in the image plane of the multiple images A method of UAV flight control for object shape mapping and real time guidance.
21. The method of claim 20,
If the obstacle is not aligned within the image plane in all the up and down directions, the step (c3) may select one of the up and down directions to start the UAV, and if the obstacle reaches one end of the obstacle, Wherein the method further comprises the step of activating the object-shape mapping and real-time guidance.
22. The method of claim 21,
Wherein the step (d) performs a mapping when the UAV moves in the other direction.
14. The method of claim 13,
Wherein the step (d) removes redundant data using the disparity of the data points collected for the obstacle.
24. The method of claim 23,
Wherein the step (d) compares the time difference with another data point existing within a predetermined range around an arbitrary data point, and removes the remaining data points except the data point having a high time difference. And UAV flight control method for real - time guidance.
A recording medium on which a program that can be read by a digital processing apparatus for performing object shape mapping and real time guidance for real-time guidance according to any one of claims 13, 15 to 24 is provided.

Images