KR101827363B1 - A tracking control system of the drone for - Google Patents

A tracking control system of the drone for Download PDF

Info

Publication number
KR101827363B1
KR101827363B1 KR1020160014180A KR20160014180A KR101827363B1 KR 101827363 B1 KR101827363 B1 KR 101827363B1 KR 1020160014180 A KR1020160014180 A KR 1020160014180A KR 20160014180 A KR20160014180 A KR 20160014180A KR 101827363 B1 KR101827363 B1 KR 101827363B1
Authority
KR
South Korea
Prior art keywords
constant value
drone
movement
image
control
Prior art date
Application number
KR1020160014180A
Other languages
Korean (ko)
Other versions
KR20170093290A (en
Inventor
최경
조하은
정희선
고희남
이승용
이민우
Original Assignee
강원대학교산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 강원대학교산학협력단 filed Critical 강원대학교산학협력단
Priority to KR1020160014180A priority Critical patent/KR101827363B1/en
Publication of KR20170093290A publication Critical patent/KR20170093290A/en
Application granted granted Critical
Publication of KR101827363B1 publication Critical patent/KR101827363B1/en

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64C—AEROPLANES; HELICOPTERS
    • B64C39/00—Aircraft not otherwise provided for
    • B64C39/02—Aircraft not otherwise provided for characterised by special use
    • B64C39/024—Aircraft not otherwise provided for characterised by special use of the remote controlled vehicle type, i.e. RPV
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64C—AEROPLANES; HELICOPTERS
    • B64C27/00—Rotorcraft; Rotors peculiar thereto
    • B64C27/04—Helicopters
    • B64C27/08—Helicopters with two or more rotors
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64D—EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENTS OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
    • B64D47/00—Equipment not otherwise provided for
    • B64D47/08—Arrangements of cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • B64C2201/127—
    • B64C2201/14—
    • B64C2201/141—

Abstract

The present invention relates to a method and apparatus for photographing a ground object through a camera provided on a drone, recognizing a ground object by dividing the photographed image, and detecting a pitch, a roll, and a yaw of the dron in a recognized direction, Wherein the control constant value for controlling the drone is stored in the control unit. The control constant value includes at least one of a pitch, a roll, a constant value storing unit including a constant value for controlling a roll and yaw; A video input unit for receiving a video shot from the camera; An object detecting unit for dividing the photographed image into a plurality of divided regions and detecting a divided region in which the tracked object is recognized; And a movement controller for controlling the movement of the drone to a control constant value stored by the constant value storage unit according to the detected divided area.
According to the above-described ground object tracking control system of the drone, a ground object is recognized by a camera installed in the dron and the movement of the dron is controlled according to the movement of the recognized ground object, so that the ground object can be automatically traced or photographed .

Description

[0001] The present invention relates to a tracking control system for a drone,

The present invention relates to a method and apparatus for photographing a ground object through a camera provided on a drone, recognizing a ground object by dividing the photographed image, and detecting a pitch, a roll, and a yaw of the dron in a recognized direction, To track a ground object, and to a ground object tracking control system for a drone.

Generally, the principle of flight of the helicopter is that the main rotor rotates to obtain lifting force and floats, and a counter torque is generated in a direction opposite to the rotation direction of the main rotor, A rotor is needed. On the other hand, the drone consists of four main rotors without a tail rotor. There are x-shaped drones and crossed (+) drones, and x-shaped drones are commonly used.

The direction of rotation of each propeller is that the propeller facing diagonally rotates in the same direction (CW) and the propeller on the side is rotating in the other direction (CCW). This is to counteract the backlash torque of the gas like a helicopter. The four propellers rotate at high speed, and the direction of the drone can be changed vertically, horizontally, and backwardly due to the difference in output [Patent Documents 1, 2, and 3].

FIG. 1 shows the principle of adjusting the flight of a doormat.

As shown in FIG. 1, the drone is raised when the four propellers rotate at a high speed at the same speed, and falls when the propeller is rotated slowly.

Also, if the two propellers are faster than the two propellers, the two propellers move forward and the two propellers rotate faster than the two propellers. This is called the advance and reverse of the drone (PITCH). The left-right movement (ROLL) is as follows. In other words, if the two propellers on the left are rotating faster than the two propellers on the right, they will move to the right, and if the two propellers on the right turn faster than the two propellers on the left, they will move to the left. For YAW rotation, the propellers with diagonal lines rotate rapidly. In this case, it rotates clockwise and counterclockwise.

On the other hand, there are many cases where a dron is installed with a camera and shot with a skyview. In other words, you may want to take a picture of yourself and surroundings when you are doing extreme sports such as mountain climbing, skiing, and surfing. The use of drones as controls is often limited.

Therefore, if the drones fly themselves and take pictures of themselves, they will be able to get clean and nice images. Also, the helicop cam, which is widely used in the field of broadcasting in recent years, can be photographed manually by a skilled person. If the drones are in autonomous flight and switching between automatic mode and manual mode is fast, they can be used in the right place in a dynamic shooting environment.

Therefore, it is necessary to trace the moving object by receiving the image data with the camera, and to follow the dron tracing technology to cope with the surroundings.

[Patent Document 1] Korean Patent No. 10-1565979 [Patent Document 2] Korean Patent No. 10-0812756 [Patent Document 3] Korean Patent No. 10-1199536

[Non-Patent Document 1] Gi-hyun and Ji-min Lee "Design of System on Chip for Image Signal Processing," hanbat Journal [Non-Patent Document 2] Lee Hee-suk, "Image Processing Programming" [Non-Patent Document 3] Song Tae Hoon, "FPGA Design Theory and Practice using Verilog_HDL", Hongreung Science, 2006 [Non-Patent Document 4] This bank, "DC motor position control using PID controller ", Dissertation (MS), 1998

SUMMARY OF THE INVENTION The object of the present invention is to solve the above-mentioned problems, and it is an object of the present invention to provide a method for shooting a ground object through a camera provided on a dron, recognizing a ground object by dividing the shot image, ), Roll (roll), and yaw (yaw) to track a ground object.

In particular, the object of the present invention is to provide a camera for shooting in a downward direction of a drone, dividing an image acquired by a camera into N × N lattices, and automatically extracting a dron according to a region where a ground object is found And to provide a ground object tracking control system for a drone.

In order to accomplish the above object, the present invention is directed to a ground object tracking control system for a drone, which is provided at the front and has a camera for photographing a downward direction of a dron and tracks a tracking object moving on the ground, Wherein the control constant value includes a constant value controlling a pitch, a roll, and a yaw according to a divided area, the constant value storing the constant value; A video input unit for receiving a video shot from the camera; An object detecting unit for dividing the photographed image into a plurality of divided regions and detecting a divided region in which the tracked object is recognized; And a movement controller for controlling the movement of the drone to a control constant value stored by the constant value storage unit according to the detected divided area.

According to another aspect of the present invention, there is provided a ground object tracking control system for a drone, wherein the control constant value includes a movement control constant value for the divided region and a movement control time, a roll value, and a yaw value.

Further, the present invention relates to a ground object tracking control system for a drone, wherein the control constant value is a reaction control value obtained by multiplying a movement control constant value for the pitch, roll, and yaw by -1, Constant value and a recoil control time proportional to the movement control time, and the proportional rate to the movement control time is less than 1.

Further, the present invention is a ground object tracking control system for a drone, wherein the system compares a previously detected segment region (hereinafter referred to as a previous detection region) with a currently detected segment region (hereinafter referred to as a current detection region) And a constant adjuster for adjusting the movement control time according to the result.

According to another aspect of the present invention, there is provided a ground object tracking control system for a drone, wherein the constant adjustment unit decreases the movement control time when the current detection area is not adjacent to the previous detection area, The movement control time is decreased, and when the current detection area is lost, the movement control time is decreased.

Further, in the ground object tracking control system of the drone, the photographed image is divided into N × N regions and divided into divided regions.

Further, the present invention is characterized in that in the ground object tracking control system of the drone, N is 3, 4, or 5.

As described above, according to the dirt ground object tracking control system of the present invention, the ground object is recognized by the camera installed in the dron and the movement of the dron is controlled according to the movement of the recognized ground object, The effect of being able to be tracked or photographed can be obtained.

In addition, according to the ground object tracking control system of the present invention, the image captured by the camera is divided into 3 × 3 to recognize the ground object, and the dron is moved to the corresponding divided area, Thus, it is possible to track a ground object in real time even with a low-capacity processor.

BRIEF DESCRIPTION OF THE DRAWINGS FIG.
2 is a block diagram of a configuration of an overall system for implementing a ground object tracking control system of a drone according to an embodiment of the present invention;
3 is an exemplary view showing a camera installation position and a shooting range within a dron according to an embodiment of the present invention;
4 is a block diagram of a configuration of a ground object tracking control system of a drone according to an embodiment of the present invention.
FIG. 5 is a diagram illustrating a divided region in an image taken according to an exemplary embodiment of the present invention; FIG.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the drawings.

In the description of the present invention, the same parts are denoted by the same reference numerals, and repetitive description thereof will be omitted.

First, a configuration of an overall system for implementing a drone ground object tracking control system according to an embodiment of the present invention will be described with reference to FIGS. 1 and 2. FIG.

1, the overall system for carrying out the present invention comprises a dron 10, a camera 20 installed in the dron 10, and a control system 30 for controlling the dron 10 .

In particular, the drone 10 is composed of four propellers 11, four propeller guides 12 for guiding the respective propellers, and a saf bar 13 installed between the propeller guides 12.

As shown in Fig. 2, the camera 20 is installed to photograph the downward direction of the direction of the dron 10. Preferably, the camera 20 is installed between 30 and 60 degrees downward in the advancing direction of the drone 10.

The image photographed from the camera 20 is composed of consecutive frames. For example, if the frame at the current time t is the current frame, the frame at the immediately preceding time t-1 is referred to as the previous frame, and the frame at the time t + 1 is referred to as the next frame. On the other hand, each frame has a color image (or a color image). That is, the color image is an RGB image, and is composed of consecutive frames in time. One frame has one image. In addition, an image may have one frame (or image). That is, the image corresponds to one image.

Detecting the tracked object A in a color image means detection in each color frame (or image) immediately, but the term image will be used below unless there is a need for a particular distinction.

The control system 30 is installed at the center of the dron 10 to control the dron 10 so as to track the object A moving on the ground by controlling the propeller 11 and the like of the dron 10.

A ground object tracking control system of a drone according to an embodiment of the present invention will be described with reference to FIG.

4, the drone ground object tracking control system 30 according to an embodiment of the present invention includes a constant value storage unit 31 for storing constant values for controlling the drone 10, a camera 20, An object recognition unit 33 for recognizing an object from the photographed image and a movement control unit 34 for controlling the drones 10 according to the position at which the object is recognized, . Further, it may include a constant adjusting unit 35 for adjusting the control constant value. In addition, it further includes a storage unit 37 for storing data.

The constant value storage unit 31 stores a control constant value of pitch, roll, and yaw to control the drone 10 according to the position at which the object is recognized in the photographed image.

As described above, the movement of the drone 10 can be controlled by transmitting the values of pitch, roll, and yaw to the drone 10.

The control system 30 according to the present invention divides an image photographed by the camera 20 into N × N pieces and adjusts the pitch, roll, and yaw values according to the divided area in which the tracked object appears. At this time, the control constant values of pitch, roll, and yaw corresponding to the respective divided regions are set in advance. At this time, N is preferably 3 to 5. That is, it is intended to reduce the calculation time for image recognition.

As shown in FIG. 5, the captured image (or input image, captured image) is divided into three equal parts in the vertical and horizontal directions, and divided into nine parts in total. Therefore, the whole is composed of N × N matrices. Preferably, it is constituted by a 3x3 matrix.

For example, an image is divided into nine sub-regions. 3 x 3 matrix. (1, 1) partition is left top, (1, 2) is the middle top, (1, 3) is the right top, (2, 2) is a center, (2, 3) is a middle right, (3, 1) is a left bottom, , (3,2) partition is designated as the middle bottom, and (3,3) partition is designated as the right bottom.

A movement control constant value for each of the divided regions, a movement control time thereof, a reaction control constant value, and a reaction control time thereof are set.

With respect to the center region, the motion control constant value for the center region sets pitch, roll, and yaw to zero. For the upper left region, the yaw is set to -3, roll -1, and pitch +1. The movement control time is set to T 1 .

In addition, the recoil control constant value is set to a value obtained by multiplying the movement control constant value by -1, respectively. That is, the recoil control constant value is a control constant value for controlling in the opposite direction to the movement control. The reaction control time T 2 is set to a time obtained by dividing the movement control time T 1 by 1/2 or 1/3. That is, the reaction control time T 2 is set to be proportional to the movement control time T 1 . The proportional ratio at this time is less than one.

Since the dragon 10 must move by itself, if the movement control constant value is given at once, it can be moved in the moving direction. Therefore, the motion control constant value is given a value which is completely opposite to the motion control constant value. So as to prevent the movement of the drones 10 from leaning.

On the other hand, the movement control time T 1 can be modified during operation according to the image analysis. This will be described below.

Next, the image input section 32 receives the photographed image from the camera 20. Preferably, the captured input image is an RGB color image.

The image photographed from the camera 20 is composed of consecutive frames. For example, if the frame at the current time t is the current frame, the frame at the immediately preceding time t-1 is referred to as the previous frame, and the frame at the time t + 1 is referred to as the next frame. On the other hand, each frame has a color image (or a color image). That is, the color image is an RGB image, and is composed of consecutive frames in time. One frame has one image. In addition, an image may have one frame (or image). That is, the image corresponds to one image.

Detecting the tracked object A in a color image means detection in each color frame (or image) immediately, but the term image will be used below unless there is a need for a particular distinction.

Next, the object detecting section 33 recognizes an object to be tracked in the photographed image. Preferably, the photographed image is divided into segmented regions, and it is determined which segmented region the tracked object is recognized. Preferably, the divided region is divided into N x N photographed images.

A clock of 27 MHz, a vertical sync, a horizontal sync, an odd field reset, and a 720x480 YCbCr422 image data (Video, data) as input to the FPGA.

First, the image is reduced using a clock. 720 x 480 to 180 x 120. This increases the number of frames per second and computation speed. In order to take ¼ of the 720 pixels from the input clock, the clock is generated by dividing the original clock by 4 times. In order to take only ¼ of the 480 lines, we change the period of the horizontal sync and design all the above clocks by ANDing them to match the whole sync. It is possible to reduce the size of the image by 180 × 120 by 16 times by taking only ¼ each of the images.

Next, image data (image data) received by YCbCr is converted into RGB 565 and converted into HSV image. Converting to final HSV creates a basic environment for color recognition. The transformation method uses the existing image processing operation formula. The expression is based on 24 bits. Since input and output are done with 16 bits (bit), the actual operation formula and the operation formula to be developed are inconsistent with each other. To do this, set a new expression. At this time, the area of the hue saturation value (HSV) occupying the color is widened, and the range of the color to be searched becomes large.

That is, the object detecting unit 33 reduces the RGB image to a low resolution RGB image, and converts the reduced RGB image to a hue saturation value (HSV) image. Then, the hue saturation value (HSV) image is used to detect an object based on a color (H) image.

As shown in Fig. 5, the object detecting unit 33 divides the HSV image into NxN, determines whether an object is detected in each divided region, and detects a divided region where the object is detected.

Next, the movement control unit 34 confirms a region (hereinafter referred to as a detection region) in which an object is detected in the divided region, and controls the dragon 10 according to the movement control constant value of the divided region and the movement control time.

That is, the movement control unit 34 controls the pitch, roll, and yaw of the drone 10 by controlling the movement control constant value of the corresponding divided region of the detection region. Also, the control signal is transmitted during the movement control time. After the signal is transmitted, the recoil control constant value is set to the pitch, roll, and yaw of the drones 10 during the recoil control time, respectively, for further control.

When the movement is completed, the movement control unit 34 causes the object detection unit 33 to detect the object again. At this time, the object detecting unit 33 detects the object to be traced again with respect to the image or frame input from the image input unit 32.

Next, the constant adjusting unit 35 detects the traced object A on the photographed image and controls the movement of the drones 10, and then detects the divided region of the traced object A on the next input image. And the current detection area, and adjusts the movement control time according to the comparison result.

Specifically, if the current detection area is not adjacent to the previous detection area, the movement control time is reduced. Further, if the current detection area is the same as the previous detection area, the movement control time is reduced. Further, even when the current detection area is lost, the movement control time is reduced.

It moves the dron after detecting the tracked object in the shot image. If the movement is too large or too small, the movement time is reduced or increased so that the movement control time is adjusted so that the object can be tracked stably.

The invention made by the present inventors has been described concretely with reference to the embodiments. However, it is needless to say that the present invention is not limited to the embodiments, and that various changes can be made without departing from the gist of the present invention.

10: Drone 11: Propeller
12: Propeller guide 13: Safe bar
20: Camera
30: tracking control system 31: constant value storage unit
32: image input unit 33: object detection unit
34: movement control unit 35: constant control unit
37:

Claims (6)

A ground object tracking control system for a drone, which is provided at a front side and has a camera for photographing a downward direction of a dron and tracks a tracking object moving on the ground,
A constant value storage unit for presetting and storing a control constant value for controlling the drone;
A video input unit for receiving a video shot from the camera;
An object detecting unit for dividing the photographed image into N × N and dividing the captured image into a plurality of divided regions and detecting a divided region in which a tracked object is recognized among the plurality of divided regions;
A movement control unit for controlling the movement of the drone to a control constant value stored by the constant value storage unit according to the detected divided area; And
And a constant adjusting unit for adjusting the control constant value,
Wherein the control constant value is set to a movement control constant value and a movement control time for each of the divided regions, and the movement control constant value is a constant value for controlling pitch, roll, and yaw ego,
The constant adjusting unit compares the divided detection region previously detected by the object detection unit (hereinafter, referred to as the previous detection region) with the currently detected divided region (hereinafter referred to as the current detection region), adjusts the movement control time according to the comparison result,
Wherein the N is 3, 4, or 5.
delete The method according to claim 1,
Wherein the control constant value includes a reaction control constant value obtained by multiplying a motion control constant value for the pitch, roll, and yaw by -1 and a reaction control time value proportional to the movement control time Wherein the proportional ratio to the movement control time is less than one.
delete delete delete
KR1020160014180A 2016-02-04 2016-02-04 A tracking control system of the drone for KR101827363B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020160014180A KR101827363B1 (en) 2016-02-04 2016-02-04 A tracking control system of the drone for

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020160014180A KR101827363B1 (en) 2016-02-04 2016-02-04 A tracking control system of the drone for

Publications (2)

Publication Number Publication Date
KR20170093290A KR20170093290A (en) 2017-08-16
KR101827363B1 true KR101827363B1 (en) 2018-02-09

Family

ID=59752439

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020160014180A KR101827363B1 (en) 2016-02-04 2016-02-04 A tracking control system of the drone for

Country Status (1)

Country Link
KR (1) KR101827363B1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20200008810A (en) 2018-07-17 2020-01-29 한림대학교 산학협력단 Method, apparatus and system for controlling a drone
KR20200036195A (en) 2018-09-28 2020-04-07 안정훈 Drone

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102316960B1 (en) * 2019-11-28 2021-10-22 광운대학교 산학협력단 Method and apparatus for realtime object detection in unmanned aerial vehicle image
KR102308700B1 (en) * 2020-04-20 2021-10-05 동국대학교 산학협력단 Tracking drone that can track whether drone is permitted to fly within restricted area and operating method thereof

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100842101B1 (en) * 2007-06-15 2008-06-30 주식회사 대한항공 Automatic recovery method of uav using vision information

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100842101B1 (en) * 2007-06-15 2008-06-30 주식회사 대한항공 Automatic recovery method of uav using vision information

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20200008810A (en) 2018-07-17 2020-01-29 한림대학교 산학협력단 Method, apparatus and system for controlling a drone
KR20200036195A (en) 2018-09-28 2020-04-07 안정훈 Drone

Also Published As

Publication number Publication date
KR20170093290A (en) 2017-08-16

Similar Documents

Publication Publication Date Title
KR101827363B1 (en) A tracking control system of the drone for
US20170236291A1 (en) Drone including a front-view camera with attitude-independent control parameters, in particular auto-exposure control
CN113038016B (en) Unmanned aerial vehicle image acquisition method and unmanned aerial vehicle
CN104126299B (en) Video image stabilisation
CA2513450C (en) Cooperative nesting of mechanical and electronic stabilization for an airborne camera system
US20230087330A1 (en) Object tracking
US20170094184A1 (en) Systems and methods for performing automatic zoom
CN105763790A (en) Video System For Piloting Drone In Immersive Mode
CN106973221B (en) Unmanned aerial vehicle camera shooting method and system based on aesthetic evaluation
CN108141540B (en) Omnidirectional camera with motion detection
US11258949B1 (en) Electronic image stabilization to improve video analytics accuracy
US10084970B2 (en) System and method for automatically generating split screen for a video of a dynamic scene
CN106488115A (en) Control device and the control method of image sensing apparatus
US20160191945A1 (en) Method and system for processing video content
JP2017072986A (en) Autonomous flying device, control method and program of autonomous flying device
US10877365B2 (en) Aerial photography camera system
CN108347577B (en) Imaging system and method
US8023744B2 (en) Pattern matching system and targeted object pursuit system using light quantities in designated areas of images to be compared
US9842260B2 (en) Image processing apparatus and image processing method of performing image segmentation
CN112514366A (en) Image processing method, image processing apparatus, and image processing system
US9904979B2 (en) Method of generating a framed video system
US7990417B2 (en) Targeted object pursuit system
JP6503607B2 (en) Imaging control apparatus, imaging apparatus, imaging system, moving object, imaging control method, and program
EP2978215B1 (en) Method and system for achieving video stitching
KR102564038B1 (en) Method and system for detecting target using time series images

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
E701 Decision to grant or registration of patent right
GRNT Written decision to grant