CN104700403B - A virtual teaching method of gesture control hydraulic support based on kinect - Google Patents

Abstract

The invention discloses a virtual teaching system and method for a kinect-based gesture control hydraulic support, comprising a kinect module, a depth of field data acquisition module, a joint point information processing module, a gesture feature matching module, a control module and a display module, and the kinect module in turn It is connected with the depth of field data acquisition module, joint point information processing module, gesture feature matching module, control module and display module. As long as the operator makes a specific gesture, the virtual hydraulic support can be observed through the display module to make corresponding actions, so that people can better understand the working process of the hydraulic support.

Description

A virtual teaching method of gesture control hydraulic support based on kinect

technical field

The invention relates to a virtual teaching system and method for a kinect-based gesture control hydraulic support and belongs to the technical field of automatic control.

Background technique

Human-computer interaction technology refers to the communication between human and computer through the input and output devices of the computer. A common way of human-computer interaction is to use a system that tracks the position and posture of the human hand in real time. This type of equipment includes electromagnetic tracking equipment, inertial sensors, and data gloves. These are contact sensors that require wearing additional equipment. The interaction process is not natural enough. Microsoft's somatosensory interactive device kinect is a non-contact sensor, which can control the computer through the operator's hand movements, without the need for people to wear additional equipment, so that people can interact with the computer in a natural way.

The hydraulic support is a structure used to control the mine pressure in the coal mining face. As a necessary equipment for comprehensive mechanized coal mining, it provides a safe working space for the shearer, scraper conveyor and staff in the fully mechanized mining face. But it is difficult for people to vividly learn and understand the working process of the hydraulic support. In order to improve people's interactive experience of the working process of the hydraulic support, it is necessary to design a human-hydraulic support virtual three-dimensional environment interactive control system, so that people can better understand the working process of the hydraulic support.

Contents of the invention

Aiming at the problems existing in the above-mentioned prior art, the present invention provides a kinect-based virtual teaching system and method for gesture-controlled hydraulic supports. Only the operator needs to make specific gestures to observe the virtual hydraulic supports through the display module and make corresponding actions. The action is convenient for people to better understand the working process of the hydraulic support.

In order to achieve the above object, the technical solution adopted by the present invention is: the virtual teaching system of the kinect-based gesture control hydraulic support includes a kinect module, a depth of field data acquisition module, a joint point information processing module, a gesture feature matching module, and a control module And the display module, the kinect module is sequentially connected with the depth of field data acquisition module, the joint point information processing module, the gesture feature matching module, the control module and the display module.

The specific steps of the virtual teaching method of the kinect-based gesture control hydraulic support are:

(1) preset eight different standard gestures, and store the eight gestures in the control module;

(2) The control module corresponds the eight standard gestures to the control signals of the hydraulic support's eight actions of lowering the column, moving the frame, raising the column, pushing, lowering the side board, raising the side board, retracting the front beam and raising the front beam;

(3) Initialize the kinect module, set the elevation angle of the camera, and the operator stands between 1.2 meters and 3.5 meters away from the camera, and makes one of the eight standard gestures set;

(4) The kinect module detects the actions of the operator, and obtains the human skeleton information data frame from the color flow, depth flow and skeleton flow;

(5) The depth of field data acquisition module analyzes and processes the human skeleton information data frame sent by the kinect module, obtains the image depth of field data, extracts the human skeleton information, and obtains the coordinates of the human arm joint nodes by establishing the 3D coordinate system of the human arm joints to identify different parts of the human body;

(6) The joint point information processing module calculates the rotation angle on the three degrees of freedom directions of the human shoulder joint and the rotation angle on the two degrees of freedom directions of the elbow joint through the obtained 3D coordinates of the human arm joint; Rotation angle information of joints and elbow joints, identifying the rotation information of human arm bone nodes, and performing data processing by capturing changes in the angles of different bone joint points;

(7) The gesture feature matching module carries out feature matching with each of the above-mentioned different joint angle information and each joint angle information of eight standard gestures stored in the control module; if the matching is not successful, then return to step (3), if the matching is successful , the gesture feature matching module transmits the data to the control module;

(8) The control module receives the data of successful matching from the gesture feature matching module, and after analyzing and processing, sends out the action control signal of the hydraulic support corresponding to the gesture, and then controls the action of the three-dimensional virtual hydraulic support, and displays it through the display module .

Compared with the prior art, the present invention uses the kinect module to collect gesture information, uses the three-dimensional information that can extract bone points to track the position of each joint of the arm, and controls the virtual hydraulic support to make corresponding actions by identifying specific gestures, which is convenient People better understand the working process of hydraulic supports.

Description of drawings

Fig. 1 is an electrical principle block diagram of the present invention;

Fig. 2 is a schematic diagram of human arm movement when gestures are made in the present invention;

Fig. 3 is the D-H coordinate system of arm among the present invention;

Fig. 4 is a flowchart of the present invention.

detailed description

The present invention will be further described below in conjunction with accompanying drawing.

As shown in Fig. 1 and Fig. 2, the virtual teaching system of the kinect-based gesture control hydraulic support includes a kinect module, a depth of field data acquisition module, a joint point information processing module, a gesture feature matching module, a control module and a display module, The kinect module is sequentially connected with the depth of field data acquisition module, joint point information processing module, gesture feature matching module, control module and display module.

The specific steps of the virtual teaching method of the kinect-based gesture control hydraulic support are:

(1) preset eight different standard gestures, and store the eight gestures in the control module;

(2) The control module corresponds the eight standard gestures to the control signals of the hydraulic support's eight actions of lowering the column, moving the frame, raising the column, pushing, lowering the side board, raising the side board, retracting the front beam and raising the front beam;

(3) Initialize the kinect module, set the elevation angle of the camera, and the operator stands between 1.2 meters and 3.5 meters away from the camera, and makes one of the eight standard gestures set;

(4) The kinect module detects the actions of the operator, and obtains the human skeleton information data frame from the color flow, depth flow and skeleton flow;

(5) The depth of field data acquisition module analyzes and processes the human skeleton information data frame sent by the kinect module, obtains the image depth of field data, extracts the human skeleton information, and obtains the coordinates of the human arm joint nodes by establishing the 3D coordinate system of the human arm joints to identify different parts of the human body;

(6) The joint point information processing module calculates the rotation angle on the three degrees of freedom directions of the human shoulder joint and the rotation angle on the two degrees of freedom directions of the elbow joint through the obtained 3D coordinates of the human arm joint; Joint and elbow rotation angle information, identify the rotation information of human arm bone nodes, and perform data processing by capturing changes in the angles of different bone joint points. The specific process is:

As shown in FIG. 3 , the movement of the shoulder joint of the right arm of the human body is divided into flexion and extension movement, extension and retraction movement, and rotation movement, corresponding to angles α ₁ , β ₁ , and γ ₁ . The movement of the human elbow joint is flexion and extension, and the corresponding angle is δ ₁ . Establish a reference coordinate system o ₀ -x ₀ y ₀ z ₀ at the middle point of the human shoulder joint, set the 3D coordinates of the shoulder joint bone node of the right arm of the human body as (x ₁ , y ₁ , z ₁ ), and the elbow joint bone node of the right arm The 3D coordinates of is (x ₂ , y ₂ , z ₂ ), the 3D coordinates of the right arm wrist joint bone node are (x ₃ , y ₃ , z ₃ );

right arm shoulder width $L_0=\sqrt{x_1^2+{\mathrm{the\; y}}_1^2+z_1^2}$

right arm length $L_1=\sqrt{{(x_2-x_1)}^2+{({\mathrm{the\; y}}_2-{\mathrm{the\; y}}_1)}^2+{(z_2-z_1)}^2}$

Right arm forearm length $L_2=\sqrt{{(x_3-x_2)}^2+{({\mathrm{the\; y}}_3-{\mathrm{the\; y}}_2)}^2+{(z_3-z_2)}^2}$

For shoulder flexion and extension, -40°<α ₁ <90°,

${\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; arctan</span>}<span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; )</span>$

For shoulder extension and retraction, -90°<β ₁ <20°,

${\mathrm{<span\; class="notranslate">\; \&beta;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; arctan</span>}<span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; )</span>$

For elbow flexion and extension, 0°<δ ₁ <130°,

${\mathrm{<span\; class="notranslate">\; \&delta;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; arccos</span>}\frac{<span\; class="notranslate">\; [</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>}{{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}$

For the shoulder joint rotation motion, it cannot be calculated directly from the known coordinates. Here, the DH coordinate system of the human body joint is established, and the coordinate expression of the wrist joint node is obtained according to the coordinate transformation principle, and then γ ₁ is deduced according to the known coordinates of the wrist joint node.

According to the D-H parameters in Table 1, write out the affine transformation matrix corresponding to each joint.

${\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{cccc}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& {\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$

${\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{cccc}\mathrm{<span\; class="notranslate">\; cos</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; sin</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; sin</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; cos</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$

${\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{cccc}\mathrm{<span\; class="notranslate">\; cos</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; sin</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; sin</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; cos</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$

${\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{cccc}\mathrm{<span\; class="notranslate">\; cos</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; sin</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; sin</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; cos</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$

${\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{cccc}\mathrm{<span\; class="notranslate">\; cos</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; sin</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; cos</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}\\ \mathrm{<span\; class="notranslate">\; sin</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}& \mathrm{<span\; class="notranslate">\; cos</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; sin</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$

The total transformation matrix is obtained as

$\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{cccc}{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}& {\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}& {\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}\\ {\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}& {\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}& {\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}\\ {\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}}& {\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}}& {\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$

in

x ₃ ＝p _x ＝L ₁ cosθ ₂ sinθ ₃ +L ₂ cosθ ₅ (sinθ ₂ sinθ ₄ +cosθ ₂ cosθ ₃ cosθ ₄ )+L ₂ cosθ ₂ sinθ ₃ sinθ ₅

y ₃ ＝p _x ＝L ₁ sinθ ₂ sinθ ₃ -L ₂ cosθ ₅ (cosθ ₂ sinθ ₄ +cosθ ₃ cosθ ₄ sinθ ₂ )+L ₂ sinθ ₂ sinθ ₃ sinθ ₅

z ₃ ＝L ₀ -L ₁ cosθ ₃ -L ₂ cosθ ₃ sinθ ₅ +L ₂ cosθ ₄ cosθ ₅ sinθ ₃

in

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&pi;</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}\\ {\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&beta;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&pi;</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}\\ {\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&gamma;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&pi;</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}\\ {\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; \&delta;</span>}\mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right.$

According to the known angle and length, substituting into the above formula, the rotation angle γ ₁ of the shoulder joint can be obtained (-90°<γ ₁ <45°);

(7) The gesture feature matching module carries out feature matching with each of the above-mentioned different joint angle information and each joint angle information of eight standard gestures stored in the control module; if the matching is not successful, then return to step (3), if the matching is successful And continue for 5 seconds, then the gesture feature matching module transmits the data to the control module;

(8) The control module receives the data of successful matching from the gesture feature matching module, and after analyzing and processing, sends out the action control signal of the hydraulic support corresponding to the gesture, and then controls the action of the three-dimensional virtual hydraulic support, and displays it through the display module ; Table 2 is the hydraulic direct action corresponding to each different gesture feature.

Table 1:

i alpha a d θ 1 0 0 L ₀ 0 2 90 0 0 θ ₂ 3 90 0 0 θ ₃ 4 90 0 L ₁ θ ₄ 5 0 L ₂ 0 θ ₅

Table 2:

alpha ₁ beta ₁ | ₁ δ ₁ action 1 70 -70 0 90 drop column 2 30 0 -90 90 Move frame 3 0 -90 0 90 rising column 4 0 -90 -90 90 push back 5 0 -90 45 0 Lower guard 6 90 0 45 90 Lift guard 7 90 0 45 0 front beam 8 90 -90 0 0 Shengqian beam

Claims (1)

1. the Virtual Demonstration method of gesture based on a kinect control hydraulic support, it is characterised in that its concrete steps It is:

(1) preset eight different standard gestures, and eight gestures are stored in control module；

(2) control module is by the fall post of the most corresponding for eight standard gestures hydraulic support, shifting frame, setting prop, pushing and sliding, fall wall supporting Plate, liter face guard, receipts front-axle beam and the control signal of liter eight actions of front-axle beam；

(3) initializing kinect module, set the video camera elevation angle, standing stands in distance photographic head 1.2 meters to 3.5 Between rice scope, one of eight standard gestures making setting；

(4) action of kinect module detection operator, and from color stream, deep stream and skeleton stream, obtain skeleton Information data frame；

(5) the skeleton information data frame that depth of field data acquisition module analyzing and processing kinect module transmits, it is thus achieved that image Depth of field data, extracts skeleton information, by setting up the 3D coordinate system of human hand and arm joint, obtains human hand and arm joint Node coordinate, the position different to identify human body；

(6) articulare message processing module is by the 3D coordinate of obtained human hand and arm joint, calculates human body shoulder joint Rotational angle on joint three degree of freedom direction, the rotational angle on two degree of freedom directions of elbow joint；According to human body shoulder joint With elbow joint rotational angle information, identify the rotation information of human arm skeleton node, by catching different skeletal joint point The change of angle carries out data process；

(7) gesture feature matching module is by each above-mentioned different joint angles information and eight marks of storage in control module Each joint angles information of quasi-gesture carries out characteristic matching；If mating unsuccessful, then return step (3), if the match is successful, Then gesture feature matching module passes data to control module；

(8) control module receives gesture feature matching module and transmits the data that the match is successful, after analyzing and processing, send with The action control signal of the hydraulic support that this gesture is corresponding, and then control three-dimensional hydraulic support action, and by display Device module shows.