CN108873933A - A kind of unmanned plane gestural control method - Google Patents
A kind of unmanned plane gestural control method Download PDFInfo
- Publication number
- CN108873933A CN108873933A CN201810688755.3A CN201810688755A CN108873933A CN 108873933 A CN108873933 A CN 108873933A CN 201810688755 A CN201810688755 A CN 201810688755A CN 108873933 A CN108873933 A CN 108873933A
- Authority
- CN
- China
- Prior art keywords
- area
- unmanned plane
- pixel
- image
- control
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
Abstract
The present invention provides a kind of unmanned plane gestural control methods, it initially sets up unmanned plane and controls the connection of host, then the video flowing that unmanned plane camera is passed back is received in control host side, noise is removed to each frame image in video flowing, its color space is transformed into the space YCrCb by rgb space, then determines area of skin color and by image binaryzation；Colour of skin connected domain therein is found to the image after binaryzation, the ratio of its perimeter and area is obtained, determines effective gesture area；The geometric center point for calculating each effective gesture area again determines the moving direction of hand according to moving direction of the central point in image coordinate system, further controls the direction of motion of unmanned plane.Present invention control precision with higher and working efficiency, there is higher intelligence.
Description
Technical field
The invention belongs to unmanned aerial vehicle (UAV) control fields, and in particular to a kind of unmanned plane gestural control method.
Background technique
In recent years, unmanned plane was not only applicable to military field, while being also widely used in civil field.When
Before, the control mode of unmanned plane is mainly artificial remotely controlling.Unmanned plane remotely controls usually by the operation Jing Guo professional training
Person is executed using the remote controler of profession or earth station.Operator will could generally grasp by a large amount of academic program and training
Make control equipment.For example, operator uses remote controler, need to understand in depth the professional knowledges such as steering engine, the throttle delay of unmanned plane,
And carry out simulation and practical flight operation training；About earth station, operator will also be familiar with interface and the relevant operation of earth station,
It is professional very strong.The long-range control of current unmanned plane is not more professional, complicated for operation, intuitive.
Gesture identification is current popular man-machine interaction mode, also has obtained relatively broad application, adopts in control field
Use gesture identification, can greatly simplify the operation of user.Had in the prior art using Gesture Recognition to unmanned plane into
The method that row remotely controls uses the method by neural metwork training, although the method to a certain extent can mostly
The accuracy of identification is improved, but the requirement to hardware processing capability can be increased in this way, so that hardware cost increases.
Summary of the invention
For overcome the deficiencies in the prior art, the present invention provides a kind of unmanned plane gestural control method, which passes through machine
Device vision fast understanding gesture information simultaneously constructs Visual Feedback Control model simultaneously, utilizes the side of vision guide and intelligent control
Method, the autonomous and closed loop ground task that fulfils assignment；Reduce the complexity of control system.Unmanned plane gestural control method of the invention
Control precision with higher and working efficiency, and there is higher intelligence.
In order to achieve the above objectives, a kind of unmanned plane gestural control method provided by the invention, includes the following steps：
Step 1：Establish the long-range connection of control host and unmanned plane；
Step 2：The camera of unmanned generator terminal is imaged, and passes Video stream information back control host；
Step 3：Control posture information and motion information that host receives unmanned plane；The posture information includes inertia measurement
The acceleration information of unit, quaternary number and aircraft towards angle, the motion information include speed under local horizontal coordinates,
Location information；Host publication movement instruction control unmanned plane during flying is controlled simultaneously；
Step 4：Each frame image removal in the Video stream information that control host captures unmanned plane camera is wherein
Grain noise, the color space of image is then transformed into the space YCrCb by rgb space；
Step 5：After image is transformed into the space YCrCb, when the Cr and Cb of pixel meet:133≤Cr≤173,77≤
When Cb≤127, i.e., it is believed that the pixel is area of skin color；The pixel component of area of skin color pixel is adjusted to 255, by non-skin
The pixel component of color area pixel point is adjusted to 0, original image can be converted to binary image；
Step 6：In the binary image obtained in steps of 5, the area of skin color pixel for being 255 by the pixel component of connection
Point is used as a connected domain, and the area for counting each connected domain accounts for the percentage of the area of skin color gross area, filters out area hundred
Divide the connected region than being greater than 10%；
Step 7：For the connected region filtered out in step 6, perimeter and the face of each connected region edge contour are measured
The connected region is determined as effective gesture when the ratio of the perimeter of connected region edge contour and area is greater than 15 by product
Region；
Step 8：With first, each frame image lower left corner by step 4 into the processed Video stream information of step 7
Pixel is that origin establishes coordinate system, and the pixel and ordinate for finding abscissa minimum and maximum in effective gesture area are minimum
With maximum pixel, crossing these point formation, four sides are respectively parallel to the rectangle of coordinate system horizontally and vertically respectively, walk
Effective gesture area obtained in rapid 7 will be fallen into the rectangle frame；
Step 9：Calculate the coordinate of the geometric center point of rectangle obtained in step 8；It finds out a series of in Video stream information
Before the geometric center point coordinate (x1, y1) of first frame image and effective gesture area disappear when effective gesture area occurs in image
The geometric center point coordinate (x2, y2) of last frame image, (x1, y1) and (x2, y2) connection is in alignment, it is straight to calculate this
The moving direction of manpower can be obtained in the angle of line and coordinate system horizontal axis；
Step 10：Control host control unmanned plane moves in the same direction with manpower moving direction.
Further, host is controlled in the step 1 remotely to connect with what unmanned plane was established using ROS robot system,
It is attached by WiFi technology.
Further, the method for each frame image removal grain noise is removed using median filter in the step 4
Grain noise.
The beneficial effects of the invention are as follows：It is anti-due to using machine vision fast understanding gesture information simultaneously while constructing vision
Present Controlling model, and using the method for vision guide and intelligent control, the autonomous and closed loop ground task that fulfils assignment, therefore reduce
The complexity of control system.This method control precision with higher and working efficiency, and there is higher intelligence.
Detailed description of the invention
Fig. 1 is system operational flow diagram of the invention.
Fig. 2 is gestures detection and recognizer flow chart of the invention.
Specific embodiment
Present invention will be further explained below with reference to the attached drawings and examples.
As shown in Figure 1, a kind of unmanned plane gestural control method described in the embodiment of the present invention, is divided into three big threads：Control
The posture information and motion information that unmanned plane is had subscribed in thread, unmanned plane during flying state are controlled by cascade PID algorithm, together
When receive keyboard response thread and new information control unmanned plane during flying that vision-based detection thread is sent；Keyboard response thread
In complete mapping function to keyboard；Vision-based detection thread complete camera information acquisition and gestures detection and
Identification, and result is sent to control thread by treated.
As shown in Fig. 2, the gestures detection of unmanned plane and identification use the following method：
Each frame image in the Video stream information captured to unmanned plane camera, be limited to camera working condition and
Locating environment can generate noise pollution since transmission channel is interfered in image transmitting process, so in first using
Value filtering pre-processes image, removes noise.
After the pretreatment for completing image, its color space is transformed into the space YCrCb by rgb space.Because the space is by bright
Degree influence is smaller, and the influence of brightness Y then can be ignored, and thus the colour of skin, which will generate, preferably birdss of the same feather flock together.In general, when Cr and Cb is full
Foot：When 133≤Cr≤173,77≤Cb≤127, that is, it is regarded as area of skin color；It later can be in the self-regulation threshold being previously set
On the basis of value, empirical adjustment is carried out by Cr the and Cb threshold value to current time, can determine area of skin color more accurately.
The pixel component of area of skin color pixel is adjusted to 255 again, the pixel component of non-area of skin color pixel is adjusted to 0, can be obtained
The binary image of area of skin color.
The area of skin color that each connection is found in the binary image obtained, as connected domain, to each company
Logical domain judges that its area accounts for the percentage of the area of skin color gross area, and judges the size of its percentage to assert whether it is interference
Region.The region that percentage is greater than 10% is recorded, the perimeter and area of these region contours are measured, due to depositing for manpower finger
, therefore the profile perimeter of hand is much larger than the perimeter of general area, is to have when wherein the ratio of perimeter and area is greater than 15
Imitate gesture area；Through actual test, in the case where area of skin color block normally identifies, such method will can accurately reach
The gesture area of finger is distinguished with face area, to effectively determine effective gesture area.
Coordinate system is established using first, each frame image lower left corner pixel as origin, finds abscissa in effective gesture area
The pixel of minimum and maximum and the pixel of ordinate minimum and maximum, excessively these points do four sides and are respectively parallel to
Rectangle horizontally and vertically, the rectangle just frame effective gesture area.
Under original state, gesture flag bit is set to false as, gesture count value is set as 0；When in the image in video flowing
When detecting effective gesture area, setting gesture flag bit is true, while gesture count value adds 1；When calculating gesture count value is 1
The center position (x1, y1) of rectangle in corresponding image, as gesture initial coordinate firstX and firstY；When gesture counts
When value is greater than 1, calculate the center position (x2, y2) of rectangle in corresponding image, as gesture end coordinate lastX and
lastY；The value of firstX and firstY is constant in one cycle, and the value of lastX and lastY is with the increasing of gesture count value
Add and constantly changes.
When can't detect effective gesture area in the image in video flowing, judge whether gesture flag bit is true.If
Be it is true, then the pixel of current record (x1, y1) and (x2, y2) are connected in alignment, calculate the folder of the straight line and horizontal axis
The mobile direction of manpower can be obtained in angle, while gesture flag bit being set to false as, and gesture count value is set as 0；If it is vacation,
Then the image new to a width detects again.
According to the moving direction of hand, controls host and control unmanned plane towards the identical direction movement of manpower moving direction.
A kind of unmanned plane gestural control method provided by the invention combined with Figure 1 and Figure 2, includes the following steps：
Step 1：Establish the long-range connection of control host and unmanned plane；
Step 2：The camera of unmanned generator terminal is imaged, and passes Video stream information back control host；
Step 3：Control posture information and motion information that host receives unmanned plane；The posture information includes inertia measurement
The acceleration information of unit, quaternary number and aircraft towards angle, the motion information include speed under local horizontal coordinates,
Location information；Host publication movement instruction control unmanned plane during flying is controlled simultaneously；
Step 4：Each frame image removal in the Video stream information that control host captures unmanned plane camera is wherein
Grain noise, the color space of image is then transformed into the space YCrCb by rgb space；
Step 5：After image is transformed into the space YCrCb, when the Cr and Cb of pixel meet:133≤Cr≤173,77≤
When Cb≤127, i.e., it is believed that the pixel is area of skin color；The pixel component of area of skin color pixel is adjusted to 255, by non-skin
The pixel component of color area pixel point is adjusted to 0, original image can be converted to binary image；
Step 6：In the binary image obtained in steps of 5, the area of skin color pixel for being 255 by the pixel component of connection
Point is used as a connected domain, and the area for counting each connected domain accounts for the percentage of the area of skin color gross area, filters out area hundred
Divide the connected region than being greater than 10%；
Step 7：For the connected region filtered out in step 6, perimeter and the face of each connected region edge contour are measured
The connected region is determined as effective gesture when the ratio of the perimeter of connected region edge contour and area is greater than 15 by product
Region；
Step 8：With first, each frame image lower left corner by step 4 into the processed Video stream information of step 7
Pixel is that origin establishes coordinate system, and the pixel and ordinate for finding abscissa minimum and maximum in effective gesture area are minimum
With maximum pixel, crossing these point formation, four sides are respectively parallel to the rectangle of coordinate system horizontally and vertically respectively, walk
Effective gesture area obtained in rapid 7 will be fallen into the rectangle frame；
Step 9：Calculate the coordinate of the geometric center point of rectangle obtained in step 8；It finds out a series of in Video stream information
Before the geometric center point coordinate (x1, y1) of first frame image and effective gesture area disappear when effective gesture area occurs in image
The geometric center point coordinate (x2, y2) of last frame image, (x1, y1) and (x2, y2) connection is in alignment, it is straight to calculate this
The moving direction of manpower can be obtained in the angle of line and coordinate system horizontal axis；
Step 10：Control host control unmanned plane moves in the same direction with manpower moving direction.
The method remotely connected that control host and unmanned plane are established in the step 1 is led to using ROS robot system
WiFi technology is crossed to be attached.
The method of each frame image removal grain noise is to remove grain noise using median filter in the step 4.
Claims (3)
1. a kind of unmanned plane gestural control method, which is characterized in that include the following steps：
Step 1：Establish the long-range connection of control host and unmanned plane；
Step 2：The camera of unmanned generator terminal is imaged, and passes Video stream information back control host；
Step 3：Control posture information and motion information that host receives unmanned plane；The posture information includes Inertial Measurement Unit
Acceleration information, quaternary number and aircraft towards angle, the motion information includes speed under local horizontal coordinates, position
Information；Host publication movement instruction control unmanned plane during flying is controlled simultaneously；
Step 4：Each frame image in Video stream information that control host captures unmanned plane camera removes therein
Grain noise, is then transformed into the space YCrCb by rgb space for the color space of image；
Step 5：After image is transformed into the space YCrCb, when the Cr and Cb of pixel meet:133≤Cr≤173,77≤Cb≤
When 127, i.e., it is believed that the pixel is area of skin color；The pixel component of area of skin color pixel is adjusted to 255, by non-colour of skin area
The pixel component of domain pixel is adjusted to 0, original image can be converted to binary image；
Step 6：In the binary image obtained in steps of 5, the area of skin color pixel that the pixel component of connection is 255 is made
For a connected domain, the area for counting each connected domain accounts for the percentage of the area of skin color gross area, filters out area percentage
Connected region greater than 10%；
Step 7：For the connected region filtered out in step 6, the perimeter and area of each connected region edge contour are measured, when
When the perimeter of connected region edge contour and the ratio of area are greater than 15, i.e., the connected region is determined as effective gesture area；
Step 8：With first, each frame image lower left corner pixel by step 4 into the processed Video stream information of step 7
Coordinate system is established for origin, finds in effective gesture area the pixel of abscissa minimum and maximum and ordinate minimum and most
Big pixel, cross these points formed one respectively four sides be respectively parallel to the rectangle of coordinate system horizontally and vertically, in step 7
Obtained effective gesture area will be fallen into the rectangle frame；
Step 9：Calculate the coordinate of the geometric center point of rectangle obtained in step 8；Find out a series of images in Video stream information
In effectively gesture area before the geometric center point coordinate (x1, y1) of first frame image and effective gesture area disappear when occurring finally
The geometric center point coordinate (x2, y2) of one frame image, by (x1, y1) and (x2, y2) connect it is in alignment, calculate the straight line with
The moving direction of manpower can be obtained in the angle of coordinate system horizontal axis；
Step 10：Control host control unmanned plane moves in the same direction with manpower moving direction.
2. a kind of unmanned plane gestural control method according to claim 1, which is characterized in that establish control in the step 1
The method of host and unmanned plane processed remotely connected is to be attached using ROS robot system by WiFi technology.
3. a kind of unmanned plane gestural control method according to claim 1, which is characterized in that each frame in the step 4
The method of image removal grain noise is to remove grain noise using median filter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810688755.3A CN108873933A (en) | 2018-06-28 | 2018-06-28 | A kind of unmanned plane gestural control method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810688755.3A CN108873933A (en) | 2018-06-28 | 2018-06-28 | A kind of unmanned plane gestural control method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108873933A true CN108873933A (en) | 2018-11-23 |
Family
ID=64296587
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810688755.3A Pending CN108873933A (en) | 2018-06-28 | 2018-06-28 | A kind of unmanned plane gestural control method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108873933A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112528802A (en) * | 2020-12-04 | 2021-03-19 | 杭州电子科技大学 | Unmanned ship automatic docking method based on machine vision |
| CN112732083A (en) * | 2021-01-05 | 2021-04-30 | 西安交通大学 | Unmanned aerial vehicle intelligent control method based on gesture recognition |
| WO2022269595A1 (en) * | 2021-06-21 | 2022-12-29 | Camhera Ltd | Unmanned autonomous vehicle camerabot |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103679145A (en) * | 2013-12-06 | 2014-03-26 | 河海大学 | Automatic gesture recognition method |
| CN104796611A (en) * | 2015-04-20 | 2015-07-22 | 零度智控（北京）智能科技有限公司 | Method and system for remotely controlling unmanned aerial vehicle to implement intelligent flight shooting through mobile terminal |
| CN106127146A (en) * | 2016-06-22 | 2016-11-16 | 电子科技大学 | A kind of unmanned aerial vehicle flight path guidance method based on gesture identification |
| CN107589850A (en) * | 2017-09-26 | 2018-01-16 | 深圳睛灵科技有限公司 | A kind of recognition methods of gesture moving direction and system |
-
2018
- 2018-06-28 CN CN201810688755.3A patent/CN108873933A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103679145A (en) * | 2013-12-06 | 2014-03-26 | 河海大学 | Automatic gesture recognition method |
| CN104796611A (en) * | 2015-04-20 | 2015-07-22 | 零度智控（北京）智能科技有限公司 | Method and system for remotely controlling unmanned aerial vehicle to implement intelligent flight shooting through mobile terminal |
| CN106127146A (en) * | 2016-06-22 | 2016-11-16 | 电子科技大学 | A kind of unmanned aerial vehicle flight path guidance method based on gesture identification |
| CN107589850A (en) * | 2017-09-26 | 2018-01-16 | 深圳睛灵科技有限公司 | A kind of recognition methods of gesture moving direction and system |
Non-Patent Citations (1)
| Title |
|---|
| 韩炜等: "基于ROS的AR.Drone的图像跟踪飞行设计", 《机械研究与应用》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112528802A (en) * | 2020-12-04 | 2021-03-19 | 杭州电子科技大学 | Unmanned ship automatic docking method based on machine vision |
| CN112528802B (en) * | 2020-12-04 | 2024-02-02 | 杭州电子科技大学 | Automatic docking method for unmanned ship based on machine vision |
| CN112732083A (en) * | 2021-01-05 | 2021-04-30 | 西安交通大学 | Unmanned aerial vehicle intelligent control method based on gesture recognition |
| WO2022269595A1 (en) * | 2021-06-21 | 2022-12-29 | Camhera Ltd | Unmanned autonomous vehicle camerabot |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104778690B (en) | A kind of multi-target orientation method based on camera network | |
| CN107168527B (en) | The first visual angle gesture identification and exchange method based on region convolutional neural networks | |
| CN110858295B (en) | Traffic police gesture recognition method and device, vehicle control unit and storage medium | |
| CN106127145B (en) | Pupil diameter and tracking | |
| CN106598226A (en) | UAV (Unmanned Aerial Vehicle) man-machine interaction method based on binocular vision and deep learning | |
| CN108873933A (en) | A kind of unmanned plane gestural control method | |
| CN103984928A (en) | Finger gesture recognition method based on field depth image | |
| CN104463117A (en) | Sample collection method and system used for face recognition and based on video | |
| CN100393486C (en) | Method and apparatus for quick tracing based on object surface color | |
| CN107589850A (en) | A kind of recognition methods of gesture moving direction and system | |
| CN103353935A (en) | 3D dynamic gesture identification method for intelligent home system | |
| CN109086724A (en) | A kind of method for detecting human face and storage medium of acceleration | |
| CN106446862A (en) | Face detection method and system | |
| US10282601B2 (en) | Electronic device and gesture recognition method applied therein | |
| AU2020300067B2 (en) | Layered motion representation and extraction in monocular still camera videos | |
| CN104392210A (en) | Gesture recognition method | |
| CN107392941A (en) | A kind of takeoff and landing tracking system and method | |
| CN105159452A (en) | Control method and system based on estimation of human face posture | |
| Guo | Research of hand positioning and gesture recognition based on binocular vision | |
| CN108089695A (en) | A kind of method and apparatus for controlling movable equipment | |
| CN106814853A (en) | A kind of eye control tracking based on machine learning | |
| CN109271840A (en) | A kind of video gesture classification method | |
| CN109118516A (en) | A kind of target is from moving to static tracking and device | |
| CN110245600A (en) | Adaptively originate quick stroke width unmanned plane Approach for road detection | |
| CN104268551B (en) | The controlling steering angle method of view-based access control model characteristic point |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20181123 |