CN102368290B - Hand gesture identification method based on finger advanced characteristic - Google Patents

Abstract

The invention provides a hand gesture identification method based on a finger advanced characteristic. The method comprises the following steps: (1) crude segmentation of gestures; (2) extraction of a gesture edge; (3) extraction of a finger center area; (4) extraction of a gesture advanced characteristic; (5) gesture identification: utilizing the gesture advanced characteristic formed by a palm center position, a gesture direction, a wrist position, positioning points of five fingers, an angle characteristic of the five fingers, and a stretching state of the five fingers which are obtained in step (4), constructing a gesture model, and obtaining a gesture identification result. According to the invention, by utilizing a stretching/retracting state of the five fingers, 2<5>=32 different hand gestures can be identified independent of a rotation angle of a hand. Considering an application scene of a hand gesture direction, an obtained hand gesture direction can be utilized to identify more user gestures.

Description

A kind of gesture identification method based on the finger advanced features

Technical field

The present invention relates to a kind of gesture identification method, specifically is a kind of gesture identification method based on the finger advanced features, belongs to field of human-computer interaction.

Background technology

The continuous development of Along with computer technology, the interacting activity of people and computing machine becomes the important component part of people's daily life gradually.In this case, more natural, meet the human-computer interaction technology that the people exchanges custom more and obtained bigger development.These technology comprise gesture identification, speech recognition and eye tracking or the like.

Gesture Recognition based on computer vision provides a kind of natural Man Machine Interface.The image characteristics extraction of gesture is the important component part of gesture identification, with the speed and the robustness of directly influence identification.Current the most frequently used gesture feature has:

(1) colour of skin, human body skin have the cluster characteristic of height in some color spaces, utilize the colour of skin to distinguish staff and background is a kind of rough dividing gesture method commonly used.

(2) edge, edge feature receive illumination effect less, can reflect the structure of gesture, often are used as images match, and then location identification gesture.

(3) profile refers to the gesture outline that obtains after the dividing gesture.Contour feature both can directly be used for images match with the identification gesture, also can obtain advanced features such as fingertip location through local characteristicses such as analysis curvature.

(4) finger tip, the i.e. position of finger tip in image.The finger tip characteristic can obtain through the finger tip template matches or to the curvature analysis of gesture profile.Fingertip location and quantity can directly reflect gesture.

(5) gesture center, i.e. the positional information of gesture in the space, moving continuously of center can be used for defining dynamic gesture.It is a kind of method commonly used that the barycenter of barycenter or point of sketch figure of directly using gesture is located the gesture center.But this method receives the influence of different gesture shapes bigger.

(6) direction of gesture, promptly the directional information of gesture in the space can be used for defining tell-tale gesture.The direction of gesture can be confirmed by arm, detect arm surely but differ in the image sequence; The direction of gesture also can the transverse direction through match gesture outline confirm, but and is not suitable for all gestures.

Because the shape more complicated of hand, even same gesture, the slight change along with the different of position and angle and finger angle also can present bigger difference in image.Existing gesture identification method exists following not enough:

(1) the real-time gesture method of estimation requires the user to wear the color mark thing usually more accurately, and is very inconvenient in practical application;

(2) a large amount of templates need be prepared usually based on the method for template matches, the identification of limited quantity gesture can only be applied to;

(3) aspect advanced features, existing finger detection method generally can only detect the finger that stretches out, and can't detect the finger of packing up, and is unfavorable for the estimation of whole gesture;

(4) recognition result can receive exposed arm regions interference;

(5) be difficult to estimate gesture direction accurately;

(6) recognition result can be pointed variable angle, the hand-screw gyration influences.

Summary of the invention

The objective of the invention is to overcome the above-mentioned defective that prior art exists, a kind of gesture identification method based on the finger advanced features is provided.The user needn't wear any label; Also needn't training in advance be used to the template of mating in a large number; The gesture advanced features such as position of position, the centre of the palm that can the extract real-time hand, wrist location, gesture direction, all fingers (comprise stretch out and pack up finger), and then discern gesture.This method has rotational invariance and good real-time, and not influenced by the bare arm area of skin color.

The present invention realizes through following technical scheme:

A kind of gesture identification method based on the finger advanced features comprises the steps: that (1) gesture rough segmentation cuts; (2) extraction at gesture edge; (3) extract the finger central area; (4) the gesture advanced features extracts; (5) gesture identification; It is characterized in that,

Extract said finger central area: according to the approximate parallel characteristic of finger both sides of the edge, utilize the central area of the operator extraction finger of design; Operator is isotropic annular operator, can be expressed as:

N=0.25D wherein _Finger, D _FingerBe the finger mean breadth, d representes that operator puts the distance at operator center, and K is a positive count; Adopt this operator to carry out convolution to the gesture edge, and carry out threshold binarization, the span of threshold value does

The binaryzation result is carried out dilation operation, obtain pointing central area figure;

Said gesture advanced features extracts: central area figure calculates barycenter to finger, as the finger central point; Utilize the finger central point to cut apart the hand zone again, remove arm segment, obtain hand zone more accurately; Erosion operation is carried out in hand zone to being partitioned into, and the barycenter of Corrosion results is as the palm center; Can confirm the gesture direction by finger center and palm center; According to palm center, gesture direction and palm radius, the location wrist location; With the wrist location is limit, and it directly is the weighting radially projecting conversion of weighting coefficient that the finger central area figure that extracts is carried out with the utmost point, obtains one dimension gesture feature sequence; This sequence is carried out LPF and normalization processing, the gesture feature sequence after obtaining handling; Extract the local maximum of 5 maximums of this gesture feature sequence, detection range wrist location point farthest in the angular range that local maximum is corresponding on the gesture outline is as the anchor point of 5 fingers; Wrist location arrives the vectorial angle of every finger locating point as the finger angle character; According to the distance of wrist location to every finger locating point, the differentiation finger stretches out or packs up;

Said gesture identification: the gesture advanced features that directly utilizes the flexible state of palm center that step (4) obtains, gesture direction, wrist location, the anchor point of 5 fingers, the angle character of 5 fingers, 5 fingers to constitute; Construct gesture model, obtain the result of gesture identification.

The said gesture rough segmentation of step (1) is cut: adopt the skin color segmentation based on the YCbCr color space, be used for the initial gross separation background; Adopt morphologic filtering, cut apart the zone of selling, and obtain the gesture outline.

The extraction at the said gesture of step (2) edge: tentatively extract the gesture edge by the Canny edge detection algorithm; Through gray values of pixel points on the Canny edge is carried out threshold decision; And according to the lower characteristics of both sides of the edge gray-scale value of packing up finger; From the Canny edge, extract the both sides of the edge of packing up finger, carry out and set operation with outline again, obtain the gesture edge.

For the discrete picture signal, said annular operator is the annular operator of (6N+1) * (6N+1).

Said method can realize through software on computer system, also can on embedded system, digital signal processor and FPGA, realize through firmware.

Compared with prior art, the present invention has advantage and effect:

(1) user need not wear any label;

(2) do not need a large amount of matching templates of training in advance just can effectively discern multiple gesture;

(3) can extract the characteristic of all fingers, comprise the finger of packing up;

(4) can effectively avoid the interference of exposed arm regions;

(5) can accurately extract the gesture direction character;

(6) influence that does not receive the hand-screw gyration and point variable angle;

(7) algorithm is simple, can real-time implementation.

Description of drawings

Fig. 1 is the process flow diagram of gesture identification method of the present invention.

Fig. 2 is that parallel lines central area annular detects the operator synoptic diagram, and Fig. 2 (a) is the detection operator of continuous signal, and Fig. 2 (b) is the detection operator (size is 7 * 7) of discrete signal, and Fig. 2 (c) is the detection operator (size is 13 * 13) of discrete signal.

Fig. 3 is that annular detects the parallel lines central area synoptic diagram that operator detects.

Fig. 4 is the method synoptic diagram of removing arm regions and confirming characteristics such as finger central point, gesture direction, wrist location.

Fig. 5 is a finger locating method synoptic diagram.

Embodiment

Below in conjunction with accompanying drawing the specific embodiment of the invention is described further, but need to prove, embodiment does not constitute the restriction that the present invention is required protection domain.

Practical implementation method flow of the present invention is as shown in Figure 1, and key step is following:

Step 1: the gesture rough segmentation is cut

This embodiment adopts the skin color segmentation method based on the YCbCr color space.To the coloured image of the RGB color space that obtains through monocular cam, adopt formula (1), it is transformed into the YCbCr color space:

$\left\{\begin{array}{c}Y=0.299R+0.587G+0.114B\\ \mathrm{Cb}=0.564(B-Y)+128\\ \mathrm{Cr}=0.713(R-Y)+128\end{array}\right.---\left(1\right)$

Fixed threshold boundary model according to formula (2) carries out colour of skin differentiation:

$R_{\mathrm{st}}=\left\{\begin{array}{c}84<\mathrm{Cb}<127\\ 137<\mathrm{Cr}<177\\ 190<(\mathrm{Cb}+0.6\mathrm{Cr})<215\end{array}\right.---\left(2\right)$

If the YCbCr value of pixel is at R _StIn the scope, then differentiate for colour of skin point, this point is made as 1, non-colour of skin point is made as 0, thereby obtains broca scale I _SkinTo broca scale I _Skin, adopting morphologic filtering, the noise spot that filtering is less extracts the connected domain that area satisfies threshold condition, gets region mask I in one's hands _{Hand_mask}Convert the RGB coloured image of input into gray-scale map I, utilize hand region mask I _{Hand_mask}From gray-scale map I, extract the gray-scale map I in hand zone _Hand

Step 2: the extraction at gesture edge

Adopt Canny edge detection algorithm adversary area grayscale figure I earlier _HandCarry out rim detection, obtain Canny outline map I _CannyBecause the finger both sides of the edge point gray-scale value of packing up is less, and the gray-scale value that skinfold edge etc. disturb is bigger, adversary's area grayscale figure I _Hand, get threshold value T ₁(T ₁Desirable 50) carry out binaryzation, get the low gray-scale value figure I in the zone in one's hands _Dark:

$I_{\mathrm{dark}}=\left\{\begin{array}{cc}1& ,I_{\mathrm{hand}}<T_1\\ 0& ,I_{\mathrm{hand}}\&GreaterEqual;T_1\end{array}\right.---\left(3\right)$

Outline by the hand zone obtains the wide I of handwheel _Contour, gesture edge I then _Edge=I _Canny∩ I _Dark∪ I _Contour, wherein ∩ representes intersection operation, ∪ representes and set operation.

Step 3: the extraction of finger central area

Fig. 2 is the detection operator G that extracts the finger central area in this implementation method.For the discrete picture signal, G is the annular operator of (6N+1) * (6N+1), wherein N=0.25D _Finger, D _FingerBe the finger mean breadth.Fig. 2 (b), Fig. 2 (c) represent N=1 respectively, 2 o'clock operator G.Note d puts the distance at operator center for certain, and then annular operator can be expressed as:

Wherein K is a positive count, and calculating K desirable 1 for ease.

The step that extract the finger central area is: at first adopt annular operator G to I _EdgeCarry out convolution and obtain I _w=I _Edge* G; Then with threshold value T ₂(T ₂Can get

Numerical value in the scope) to I _wCarry out binaryzation and obtain I ' _Finger(x y), sees formula (5);

${I^{\&prime;}}_{\mathrm{finger}}(x,y)=\left\{\begin{array}{cc}1& I_w(x,y)>T_2\\ 0& I_w(x,y)\&le;T_2\end{array}\right.---\left(5\right)$

Fig. 3 is the parallel lines central area synoptic diagram through the detection operator extraction.Get I ' at last _Finger(x, y) and I _SkinCommon factor, and carry out dilation operation, obtain pointing central area figure I _Finger(x, y).

Step 4: the gesture advanced features extracts

In this embodiment, the gesture advanced features extracts sees Fig. 4, Fig. 5.

Wherein, Fig. 4 representes to point the method for distilling of center, gesture direction and wrist location, is used to calculate characteristics such as finger center and gesture direction, and step is following: calculate finger central area figure I _FingerBarycenter, be designated as the finger center P _FcUtilize the center to be P _Fc, radius is 2.5R _PalmCircle cut apart the hand zone again, remove arm regions, obtain revised gesture outline I _{Hand_f}, R wherein _PalmBe the palm mean radius; To I _{Hand_f}Carry out erosion operation and calculate the barycenter of Corrosion results, obtain palm center P _HcGesture direction vector of unit length

Try to achieve by formula (6):

${\stackrel{\&RightArrow;}{N}}_p=\frac{\stackrel{\&RightArrow;}{P_{\mathrm{hc}}{P}_{\mathrm{fc}}}}{\left|\stackrel{\&RightArrow;}{P_{\mathrm{hc}}{P}_{\mathrm{fc}}}\right|}---\left(6\right)$

Wrist location P _wTry to achieve by formula (7):

$P_w=P_{\mathrm{hc}}-R_{\mathrm{palm}}{\stackrel{\&RightArrow;}{N}}_p---\left(7\right)$

Fig. 5 representes the method for finger locating.Purpose is to calculate respectively to point angle and fingertip location.To point central area figure I _Finger(x y) converts into wrist location P _wPolar coordinate representation I for limit _f(r θ), carries out equal angles cutting at interval, calculates the weighting radially projecting in each angle intervals:

$p_n={\&Integral;}_{r1}^{r2}{\&Integral;}_{\mathrm{\&theta;n}}^{\mathrm{\&theta;n}+1}{\mathrm{rI}}_f(r,\mathrm{\&theta;})\mathrm{d\&theta;dr},$ n=0,1,2，...,M-1 (8)

Wherein, p _nCorresponding n polar angle scope [θ _n, θ _N+1) eigenwert; R representes utmost point footpath, as weighting coefficient; R1, r2 are the scope of integration variable r, desirable r1=0.5R _Palm, r2=2.5R _Palm, M is the space-number (desirable 36 ~ 360) of angle cutting.With p _nArrange from small to large by angle, obtain pointing characteristic sequence { p _n; To { p _nTo carry out LPF level and smooth, obtain after level and smooth the result p ' _n; To at last p ' _nNormalization obtains { p _{N_norm}}:

$p_{n\_\mathrm{norm}}=\frac{p_n^{\&prime;}}{\underset{i=0}{\overset{M-1}{\mathrm{\&Sigma;}}}{p}_n^{\&prime;}},$ n＝0，1，2，...，M-1 (9)

Obtain normalization characteristic sequence { p successively _{N_norm}Five local maximum points, its corresponding polar angle scope is 5 angular ranges that finger is corresponding.The angle of finger can be taken as (θ _n+ θ _N+1)/2.In the angular range of every finger correspondence, detection range wrist location P on the gesture outline _wPoint P farthest _TiAnd maximum distance D _TiWork as D _Ti>α R _PalmThe time, P _TiDifferentiation is the fingertip location of a finger that stretches out; Work as D _Ti≤α R _PalmThe time, P _TiArticulation point corresponding to a finger of packing up.To middle three fingers, threshold alpha is desirable 3, to the finger (thumb and little finger of toe) of both sides, threshold alpha desirable 2.2.Wrist location P _wTo a P _TiVectorial angle as the finger angle character.

Step 5: gesture identification

The gesture advanced features that directly utilizes the flexible state of palm center that above-mentioned steps obtains, gesture direction, wrist location, the anchor point of 5 fingers, the angle character of 5 fingers, 5 fingers to constitute; Construct gesture model, obtain the result of gesture identification.

In this implementation method, directly utilize the stretching out of 5 fingers/collapsed state can discern 2 ⁵=32 kinds of different gestures, and irrelevant with the anglec of rotation of hand.Consider to utilize the gesture direction of acquisition in the application scenarios of gesture direction identification more users gesture at needs.

Claims (4)

1. the gesture identification method based on the finger advanced features comprises the steps: that (1) gesture rough segmentation cuts; (2) extraction at gesture edge; (3) extract the finger central area; (4) the gesture advanced features extracts; (5) gesture identification; It is characterized in that,

Extract said finger central area: according to the approximate parallel characteristic of finger both sides of the edge, utilize the central area of the operator extraction finger of design; Operator is isotropic annular operator, can be expressed as:

N=0.25D wherein _Finger, D _FingerBe the finger mean breadth, d representes that operator puts the distance at operator center, and K is a positive count; Adopt this operator to carry out convolution to the gesture edge, and carry out threshold binarization, the span of threshold value does

The binaryzation result is carried out dilation operation, obtain pointing central area figure;

Said gesture advanced features extracts: central area figure calculates barycenter to finger, as the finger central point; Utilize the finger central point to cut apart the hand zone again, remove arm segment, obtain hand zone more accurately; Erosion operation is carried out in hand zone to being partitioned into, and the barycenter of Corrosion results is as the palm center; Can confirm the gesture direction by finger center and palm center; According to palm center, gesture direction and palm radius, the location wrist location; With the wrist location is limit, and it directly is the weighting radially projecting conversion of weighting coefficient that the finger central area figure that extracts is carried out with the utmost point, obtains one dimension gesture feature sequence; This sequence is carried out LPF and normalization processing, the gesture feature sequence after obtaining handling; Extract the local maximum of 5 maximums of this gesture feature sequence, detection range wrist location point farthest in the angular range that local maximum is corresponding on the gesture outline is as the anchor point of 5 fingers; Wrist location arrives the vectorial angle of every finger locating point as the finger angle character; According to the distance of wrist location to every finger locating point, the differentiation finger stretches out or packs up;

Said gesture identification: the gesture advanced features that directly utilizes the flexible state of palm center that step (4) obtains, gesture direction, wrist location, the anchor point of 5 fingers, the angle character of 5 fingers, 5 fingers to constitute; Construct gesture model, obtain the result of gesture identification.

2. recognition methods according to claim 1 is characterized in that, the said gesture rough segmentation of step (1) is cut: adopt the skin color segmentation based on the YCbCr color space, be used for the initial gross separation background; Adopt morphologic filtering, cut apart the zone of selling, and obtain the gesture outline.

3. recognition methods according to claim 1; It is characterized in that the extraction at the said gesture of step (2) edge: tentatively extract the gesture edge by the Canny edge detection algorithm, through gray values of pixel points on the Canny edge is carried out threshold decision; And according to the lower characteristics of both sides of the edge gray-scale value of packing up finger; From the Canny edge, extract the both sides of the edge of packing up finger, carry out and set operation with outline again, obtain the gesture edge.

4. recognition methods according to claim 1 is characterized in that, for the discrete picture signal, said annular operator is the annular operator of (6N+1) * (6N+1) in the step (3).

Images