CN102219051B - Method for controlling four-rotor aircraft system based on human-computer interaction technology - Google Patents

Abstract

A method for controlling a four-rotor aircraft system based on the human-computer interaction technology, which belongs to the intelligent flying robot field, is characterized in that a manipulator can control the four-rotor aircraft by gestures. The flight attitude control is accomplished by the cooperative running of the four rotors which are arranged at geometrical vertexes of the four-rotor aircraft with three degrees of freedom being yaw angle, pitch angle and roll angle. The human-computer interaction technology mainly utilizes OpenCV and OpenGL. The system captures depth images of the manipulator hands through a depth camera; the depth images are analyzed and processed by the computer to obtain gesture information and generate control signals corresponding to the gesture information; and then the control signals are sent through a radio communication device to the aircraft for execution, so as to accomplish the mapping from the motion state of the manipulator hands to the motion state of the aircraft and complete the gesture control. With the help of far controlling distance and more visual gesture corresponding relation, the gesture control can be applied to danger experiments and industrial production processes with high execution difficulty.

Description

Control method of quadrotor aircraft system based on human-computer interaction technology

technical field

The invention belongs to the field of intelligent flying robots, and in particular relates to a method for controlling a quadrotor aircraft by using a depth camera to capture gestures of a human body.

technical background

The quadrotor aircraft is the operation object of the present invention. It is an aircraft with four propellers, and the four propellers are in a cross-shaped cross structure, and the opposite two sets of quadrotors have the same rotation direction. In China, quadrotor aircraft is still in its infancy, and few relatively mature results have been released; however, quadrotor aircraft is more popular in the world. Many famous universities in the West have taken the lead in the control of quadrotor aircraft. Enthusiasts devote themselves to the production and research of quadrotor aircraft. The quadrotor aircraft has won the favor of the majority of aircraft enthusiasts with its advantages of small size, light weight and high flexibility.

The control means of the present invention is the gesture recognition of the depth camera, and what utilizes is the Natal depth camera. There are already some well-formed solutions for spatial image processing, but because of their complex algorithms and huge computing libraries, researchers in many surrounding fields have been shut out. The Natal depth camera used in the present invention cleverly uses a visible light camera, an infrared camera and an infrared dot matrix generator to complete the depth processing of spatial images, which provides convenient conditions for gesture recognition. At present, this technology has begun to be applied in the field of machine vision and has been widely acclaimed.

Contents of the invention

The purpose of the present invention is to provide a method for controlling the attitude of an aircraft by using a depth camera to capture gestures of the human body, which fully reflects the flexibility and convenience of the operator's gesture control, reduces the general difficulty of aircraft control, and expands the application range of small aircraft.

Based on the above purpose, the concept of the present invention is: a control system is composed of a quadrotor aircraft, a plane position capture device and a gesture recognition device. The plane position capture device includes an environmental infrared camera and a group of brackets placed in parallel with adjustable spacing. The environmental infrared camera communicates with the hub computer through a wireless video transmission module, which is used to feed back the real-time horizontal plane flight status of the quadrotor aircraft to the computer. The gesture recognition device is composed of a Natal depth camera and a computer. The depth camera is used to capture human gestures, and the gesture information is transmitted to the computer through wired transmission; the quadrotor aircraft communicates with the computer through a ZigBee wireless data transmission module.

The control method of the four-rotor aircraft system based on human-computer interaction technology provided by the present invention, the specific steps are as follows:

Step 1: Obtain the altitude information of the quadrotor aircraft 1 through the ultrasonic ranging module 13 onboard the quadrotor aircraft 1, and the altitude information is processed by the onboard microcontroller 8 of the quadrotor aircraft 1 into decimal numbers expressed in ASCII codes. ZigBee wireless communication module 14 transmits to computer 7;

Step 2: Capture the plane position information and angle information of the two infrared reflective cloths 11 on the upper surface of the quadrotor aircraft 1 through the wireless environment infrared camera 5, thereby obtaining the plane position information and yaw angle of the quadrotor aircraft 1 and transmitting them to the computer 7 ;

Step 3: The computer 7 synthesizes the obtained height information and plane position information into the x, y, z coordinates in the coverage space of the wireless environment infrared camera 5, so far the current position space coordinates of the quadrotor aircraft 1 are obtained;

Step 4: Take the depth image of the manipulator's hand through the Natal depth camera 6 and transmit it to the computer 7;

Step 5: The computer 7 analyzes the obtained depth image, and separates the x, y, z coordinates of the operator's hand within the coverage of the Natal depth camera 6 and the angle information between the operator's thumb and index finger;

Step 6: The computer 7 maps the current position space coordinates of the operator's hand obtained in step 5 to the target position space coordinates of the space range where the quadrotor aircraft 1 is located through space coordinate transformation;

Step 7: The computer 7 calculates the PID control for the spatial position of the quadrotor aircraft 1 according to the space coordinates of the current position of the quadrotor aircraft 1 obtained in step 3 and the space coordinates of the target position of the quadrotor aircraft 1 obtained in step 6 amount, including the forward and backward control amount, the left and right translation control amount, and the rise and fall control amount. According to the angle information between the manipulator's thumb and index finger obtained in step 5, the control amount for controlling the kneading angle of the manipulator 12 is generated. According to the step 2 The yaw angle obtained calculates the yaw angle control amount for suppressing the four-rotor aircraft 1 spin, and transmits to the on-board microcontroller 8 of the four-rotor aircraft by the ZigBee wireless communication module 14;

Step 8: The onboard microcontroller 8 of the quadrotor aircraft transmits the obtained forward and backward, left and right translation, elevation reduction and yaw angle control quantities to the flight attitude controller 9 in the form of PWM signals, and the obtained manipulator The control quantity is transmitted to the manipulator 12 in the form of PWM signal;

Step 9: The flight attitude controller 9 parses the obtained control information into four-way motor drive signals for performing corresponding action tasks, and after processing and power amplification by the electronic governor 15, finally drives four brushless motors 10 to control the quadrotor aircraft 1. Move to the specified position, and the manipulator will knead according to the specified angle to complete the grabbing action.

Compared with other aircraft controls in the corresponding field, the present invention has the following beneficial effects:

First, due to the unique structural design of the quadrotor aircraft itself, it is more suitable for flexible operation in a narrow space; in addition to using the space positioning system of the present invention, its flight positioning operation is more accurate. The above two points enable this quadrotor aircraft to complete difficult fine movements, such as auxiliary tasks such as object grasping and text writing.

Second, since the human-computer interaction of the quadrotor aircraft control system is based on the depth camera, the upper computer can accurately acquire and effectively process the gesture information, thereby realizing the real-time gesture control of the operator. The application of this technology makes the control of the quadrotor aircraft more flexible and convenient, and can complete complex aircraft space control through simple gesture space correspondence.

The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

Description of drawings

Fig. 1 is a schematic structural diagram of a four-rotor aircraft system based on human-computer interaction technology used in the present invention;

Figure 2 is a top view of the quadrotor aircraft

Figure 3 is a bottom view of the quadrotor aircraft

Fig. 4 is a system block diagram of the present invention

In the figure: 1-quadrotor aircraft, 2-plane position capture device, 3-gesture recognition device, 4-two parallel adjustable brackets, 5-wireless environment infrared camera, 6-Natal depth camera, 7-computer , 8-Onboard microcontroller, 9-Flight attitude controller, 10-Brushless motor, 11-Infrared reflective cloth, 12-Manipulator, 13-Ultrasonic ranging module, 14-ZigBee wireless communication module, 15-Electronic controller accelerator.

Detailed ways

The first part realizes the method of controlling the spatial position of the quadrotor aircraft by moving the position of the hand

Step 1: By triggering the onboard ultrasonic ranging module 13 of the quadrotor aircraft, a pulse signal positively correlated with the time difference of the trigger signal and the height of the quadrotor aircraft is obtained, and is captured and measured by the onboard microcontroller MC9S12XS128 of the quadrotor aircraft This time difference obtains the altitude information of the quadrotor aircraft, and then is processed as a decimal number represented by ASCII code and transmitted to the computer through the ZigBee wireless communication module 14;

Step 2: Use the ambient infrared camera to capture the infrared light reflected by the two infrared reflective cloths 11 symmetrically arranged on the upper surface of the quadrotor aircraft (other wave bands are filtered by infrared filters), and convert this optical signal into an infrared image by the CMOS sensor Video stream, this video stream is transmitted to the computer through the wireless video transmission module;

Step 3: The computer parses the obtained video stream through OpenCV:

1. Grayscale the infrared image;

2. Find the ROI area;

3. Perform Gaussian filtering for binarization;

4. Detect the Contours contour and calculate the number of contours and number each independent contour to calculate the x, y coordinates and yaw angle of the geometric center of the quadrotor aircraft in the field of view of the environmental infrared camera;

5. Using the perspective principle of the camera, combined with the height information from the ultrasonic ranging module sent back by the onboard microcontroller of the quadrotor aircraft, the z coordinate of the aircraft is calculated;

Step 4: Take the depth image of the operator's hand through the Natal depth camera and transmit it to the computer;

Step 5: The computer analyzes the obtained depth image, separates the x, y, and z coordinates of the operator's hand within the coverage of the Natal depth camera, and thus obtains the current position space coordinates of the operator's hand;

Step 6: The computer maps the current position space coordinates of the operator's hand obtained in step 5 to the target position space coordinates of the space range where the quadrotor aircraft is located through space coordinate transformation;

Step 7: The computer calculates the PID control amount according to the space coordinates of the current position of the quadrotor aircraft obtained in step 3 and the target position space coordinates of the quadrotor aircraft obtained in step 5, including forward and backward control amount, left and right translation The control amount, the control amount of raising and lowering, and transmit to the on-board microcontroller of the quadrotor aircraft through the ZigBee wireless communication module;

Step 8: The onboard microcontroller of the quadrotor aircraft transmits the obtained forward and backward, left and right translation, and elevation and reduction control quantities to the flight attitude controller 9 in the form of PWM signals;

Step 9: The flight attitude controller analyzes the obtained control information and executes the four-way motor drive signals for corresponding action tasks. After processing and power amplification by the electronic governor 15, the four brushless motors are finally driven to control the quadrotor aircraft to move to the specified position. Location.

The second part is the implementation method of controlling the quadrotor aircraft on-board manipulator to grab objects by grabbing gestures

Step 1: Take the depth image of the operator's hand through the Natal depth camera and transmit it to the computer;

Step 2: The computer analyzes the obtained depth image:

1. The three-dimensional image of the operator's hand is restored by OpenGL;

2. Using the Bayesian framework proposed by Youding Zhu and researcher Kikuo Fujimura of the Honda Institute of Technology to track the human body posture in the depth image, extract the angle information between the operator's thumb and index finger from the three-dimensional image and pass it through the ZigBee wireless communication module Transmission to the on-board microcontroller of the quadrotor;

Step 3: The onboard microcontroller of the quadrotor aircraft processes the obtained angle information into corresponding steering gear control signals to control the pinching distance of the manipulator onboard the quadrotor aircraft, and finally completes the grasping action.

Claims (1)

1. The control method of quadrotor aircraft system based on human-computer interaction technology, the specific steps are as follows: Step 1: Obtain the height information of the quadrotor aircraft (1) through the ultrasonic ranging module (13) onboard the quadrotor aircraft (1), and the height information is passed through the onboard microcontroller (8) of the quadrotor aircraft (1) Be processed as the decimal number represented by ASCII code and transmit to computer (7) by ZigBee wireless communication module (14); Step 2: Capture the plane position information and angle information of the two infrared reflective cloths (11) on the upper surface of the quadrotor aircraft (1) through the wireless environment infrared camera (5), thereby obtaining the plane position information and angle information of the quadrotor aircraft (1) The yaw angle is also transmitted to the computer (7); Step 3: The computer (7) synthesizes the obtained height information and plane position information into the x, y, z coordinates in the coverage space of the wireless environment infrared camera (5), so far the current position space coordinates of the quadrotor aircraft (1) are obtained ; Step 4: take the depth image of the operator's hand by Natal depth camera (6) and transmit to computer (7); Step 5: The computer (7) analyzes the obtained depth image, and separates the x, y, z coordinates of the operator's hand within the coverage of the Natal depth camera (6) and the angle information between the operator's thumb and index finger ; Step 6: The computer (7) maps the current position space coordinates of the operator's hand obtained in step 5 to the target position space coordinates of the space range where the quadrotor aircraft (1) is located through space coordinate transformation; Step 7: The computer (7) calculates the target position space coordinates of the quadrotor aircraft (1) obtained in step 3 according to the space coordinates of the current position of the quadrotor aircraft (1) obtained in step 6. (1) The PID control amount of the spatial position, including the forward and backward control amount, the left and right translation control amount, and the rise and fall control amount, according to the angle information between the operator's thumb and index finger obtained in step 5 to generate the control manipulator (12) Knead the control amount of angle, calculate the yaw angle control amount for suppressing the spin of quadrotor aircraft (1) according to the yaw angle obtained in step 2, transmit to the machine of quadrotor aircraft by ZigBee wireless communication module (14) onboard microcontroller (8); Step 8: The on-board microcontroller (8) of the quadrotor aircraft transmits the obtained forward and backward, left and right translation, elevation and reduction, and yaw angle control quantities to the flight attitude controller (9) in the form of PWM signals, and the The obtained manipulator control quantity is transmitted to the manipulator (12) in the form of PWM signal; Step 9: The flight attitude controller (9) parses the obtained control information into four motor drive signals for performing corresponding action tasks, and finally drives four brushless motors ( 10) Control the quadrotor aircraft (1) to move to the specified position, and the manipulator performs kneading according to the specified angle to complete the grasping action.

Images