CN101901350B - Characteristic vector-based static gesture recognition method - Google Patents

Abstract

The invention discloses a characteristic vector-based static gesture recognition method, which comprises the following steps: carrying out segmentation processing on a gesture by using color characteristics in an YCbCr color space; selecting a characteristic vector to describe the gesture, and carrying out length and angle normalization processing; determining the position of a fingertip in the gesture according to the characteristic vector; and finally recognizing the gesture by using the relative positions of the fingertip and the gesture center. The method has the advantages of rotational invariance, scale invariance and good real-time, and can meet the requirement for human-computer interaction application.

Description

A kind of static gesture identification method based on proper vector

(1) technical field:

The present invention relates to a kind of static gesture identification method, especially a kind of static gesture identification method based on proper vector.Belong to field of human-computer interaction.

(2) background technology:

Along with the development of human-computer interaction technology, the man-machine interaction mode of natural harmony receives people's attention day by day.Based on the Gesture Recognition of vision, with its naturality, terseness and direct characteristics, the Man Machine Interface of a nature is provided, received researcher's attention.Because the characteristic of the non-rigid body of staff; Traditional recognition methods with rotational invariance and convergent-divergent unchangeability; For example fourier descriptors, Image Moment Invariants etc. have run into variety of problems in the application of gesture identification, can not satisfy the requirement of man-machine interaction to the gesture identification real-time.

Through the existing literature source investigation is found; In the paper " based on the gesture identification method of fourier descriptors-BP neural network " that Qin Wenjun etc. 2009 delivered in " Northeastern University's journal (natural science edition) " the 30th the 9th phase of volume a kind of gesture identification method based on fourier descriptors and BP neural network has been proposed; At first the gesture dividing method through many Feature Fusion extracts the gesture zone; Combine then fourier descriptors preferably profile describe ability and BP neural network self-learning capability preferably, with fourier descriptors and BP neural network method gesture is discerned.The method of this paper has obtained high recognition, but the algorithm computation amount is big, and the identification piece image needs 4.5 second time, far can not reach the requirement of real-time application; Because fourier descriptors is applicable to the profile description to rigid body, therefore be applied in the gesture identification simultaneously, can not well describe the same gesture under the different angles, the poor robustness of its algorithm.

(3) summary of the invention:

The objective of the invention is to propose a kind of static gesture identification method based on proper vector, prior art calculated amount in gesture identification is used is big to solve, recognition result receives the big problem of gesture aspect effect, overcomes defective of the prior art.

Technical scheme of the present invention is summarised as: at first in the YCbCr color space, with the colour of skin characteristic gesture is carried out dividing processing; Selected characteristic vector description gesture then; Carrying out length and angle normalization handles; Confirm the position of finger tip in the gesture finally gesture to be discerned according to proper vector with the relative position at finger tip and gesture center.

A kind of static gesture identification method of the present invention based on proper vector, its concrete steps are following:

Step 1: gesture is cut apart

Adopt Logitech QuickCam Pro9000 camera collection image, obtaining picture size is 320 * 240, and picture format is JPEG, and shooting background does not have the zone of the similar colour of skin, and the user who is taken wears non-colour of skin caftan.To the image that obtains, at first be transformed into the YCbCr color space from the RGB color space, select the central value and the threshold value of Cb, Cr component afterwards, wherein Cb component central value Cb _Mid=115, Cr component central value Cr _Mid=145, threshold value l _Thres=15, each pixel of input picture is calculated the Euclidean distance dist of its Cb, Cr value and central value:

$\mathrm{dist}=\sqrt{{\left({\mathrm{Cb}-\mathrm{Cb}}_{\mathrm{mid}}\right)}^2+{\left({\mathrm{Cr}-\mathrm{Cr}}_{\mathrm{mid}}\right)}^2}---\left(1\right)$

If dist is less than corresponding threshold value l _Thres, then the value with this pixel is made as 1, otherwise its value is made as 0, and input picture is converted into bianry image.Bianry image to obtaining once corrodes expansion process, and largest connected zone is the gesture region in the supposition image.

Step 2: the center, direction and the edge that obtain gesture

After images of gestures is converted into bianry image, through calculating center, the direction that obtains gesture, and extract the edge of gesture.Concrete grammar is following:

Calculate gesture center point coordinate x and y by formula (2) and formula (3):

$x=\frac{{\mathrm{\Σ}}_{i=1}^n{\mathrm{\Σ}}_{j=1}^m\mathrm{jB}[i,j]}{{\mathrm{\Σ}}_{i=1}^n{\mathrm{\Σ}}_{j=1}^{m}B[i,j]}---\left(2\right)$

$y=\frac{{\mathrm{\Σ}}_{i=1}^n{\mathrm{\Σ}}_{j=1}^m\mathrm{iB}[i,j]}{{\mathrm{\Σ}}_{i=1}^n{\mathrm{\Σ}}_{j=1}^{m}B[i,j]}---\left(3\right)$

In the formula, the value of the capable j row of i in B [i, the j] presentation video, the size of image is n * m.

The calculating formula of gesture direction θ is following:

$\mathrm{\θ}=\frac{1}{2}{\tan }^{-1}\left[\frac{{2\mathrm{\μ}}_{11}}{{\mathrm{\μ}}_{20}-{\mathrm{\μ}}_{02}}\right]---\left(4\right)$

In the formula, μ ₁₁, μ ₂₀, μ ₀₂Second-order moment around mean for image.

From bianry image, extract the edge of gesture with the Canny operator.

Step 3: selected characteristic vector

Calculating the angle of gesture central point and edge each point line, find out the immediate point of angle and gesture deflection, is starting point with this point, and the n five equilibrium is carried out on the gesture border, gets borderline n point.With the gesture central point is starting point, and one of borderline n point characterizes this gesture for terminating point structure vector with n vector.Be the influence of size of eliminating gesture in the input picture to discerning, the longest vector of length in the selected characteristic vector, its length is set to 100, and the length normalization method that other is vectorial is to (0,100).Through experiment, adopt n=60.

Step 4: location fingertip location

The proper vector that step 3 is obtained is regarded as an end to end Vector Groups, eliminates starting point and selects the influence to Vector Groups, guarantees that proper vector is insensitive to rotating.From Vector Groups, obtain local maximum and local minimum, the point that wherein local maximum is corresponding is a finger tip point candidate point, and the differential seat angle of its two adjacent local minizing points is the angle of finger region.Selected finger length threshold value l _ThWith angle threshold θ _Th, use finger tip to compare with setting threshold with the angle θ that points the region and judge whether the finger tip candidate point is finger tip to the length l at gesture center.As l＞l _ThAnd θ＜θ _ThThe time, then the terminal point position of this proper vector is a fingertip location, otherwise this is non-finger tip.Through experiment, adopt l _Th=53, θ _Th=1.87.

Step 5: gesture identification

Obtain gesture central point and fingertip location according to step 2 and step 4, constructing one group is starting point with the gesture central point, and fingertip location is the vector of terminal point.Maximum angle between finding out in these vectors in twos; With one in two vectors of angle maximum is start vector; Its angle is set to 0; Guaranteeing under the constant prerequisite of each vectorial relative angle, making other vector minimum, obtaining unique normalization result with the counterclockwise angle and the sum of start vector.Obtain confirming the finger number that gesture comprises after the normalized Vector Groups of angle; According to the relative position and the length information of finger, the different gestures that comprise the same hand index are distinguished.

Advantage of the present invention and effect are: the identification to gesture has rotational invariance and convergent-divergent unchangeability, calculates simply, satisfies real-time processing requirements, is applicable to the identification of the different gestures of different staff.

(4) description of drawings:

Fig. 1 is the algorithm block diagram of the inventive method.

Fig. 2 is the center and the direction synoptic diagram of gesture, and rhombus is represented the center of gesture among the figure, and arrow is represented the direction of gesture among the figure.

Fig. 3 is the proper vector synoptic diagram of gesture.

Wherein, Fig. 3 (a) is 2 finger gesture proper vector synoptic diagram;

Fig. 3 (b) is 4 finger gesture proper vector synoptic diagram.

Fig. 4 is fingertip location figure, and rhombus is the fingertip location of the inventive method location among the figure.

Wherein, Fig. 4 (a) is 3 finger locating figure as a result;

Fig. 4 (b) is 5 finger locating figure as a result;

(5) practical implementation method:

Technical scheme of the present invention is summarised as: at first in the YCbCr color space, with the colour of skin characteristic gesture is carried out dividing processing; Selected characteristic vector description gesture then; Carrying out length and angle normalization handles; Confirm the position of finger tip in the gesture finally gesture to be discerned according to proper vector with the relative position at finger tip and gesture center.

Below in conjunction with accompanying drawing technical scheme of the present invention is done further to describe in detail.Key step is following:

Step 1: gesture is cut apart

The present invention adopts Logitech QuickCam Pro9000 camera collection image, obtains coloured image and is of a size of 320 * 240, and picture format is JPEG, and shooting background does not have the zone of the similar colour of skin, and the user who is taken wears non-colour of skin caftan.To the images of gestures that obtains, at first be transformed into the YCbCr color space from the RGB color space, select the central value and the threshold value of Cb, Cr component afterwards, wherein Cb component central value Cb _Mid=115, Cr component central value Cr _Mid=145, threshold value l _Thres=15, each pixel of input picture is calculated the Euclidean distance dist of its Cb, Cr value and central value:

$\mathrm{dist}=\sqrt{{\left({\mathrm{Cb}-\mathrm{Cb}}_{\mathrm{mid}}\right)}^2+{\left({\mathrm{Cr}-\mathrm{Cr}}_{\mathrm{mid}}\right)}^2}---\left(1\right)$

If dist is less than corresponding threshold value l _Thres, then this pixel value is made as 1, otherwise its value is made as 0, thereby input picture is converted into bianry image.Bianry image to obtaining once corrodes expansion process, and largest connected zone is the gesture region in the supposition image.

Step 2: the center, direction and the edge that obtain gesture

After images of gestures is converted into bianry image, through calculating center, the direction that obtains gesture, and extract the edge of gesture.

Calculate gesture center point coordinate x and y by formula (2) and formula (3):

$x=\frac{{\mathrm{\Σ}}_{i=1}^n{\mathrm{\Σ}}_{j=1}^m\mathrm{jB}[i,j]}{{\mathrm{\Σ}}_{i=1}^n{\mathrm{\Σ}}_{j=1}^{m}B[i,j]}---\left(2\right)$

$y=\frac{{\mathrm{\Σ}}_{i=1}^n{\mathrm{\Σ}}_{j=1}^m\mathrm{iB}[i,j]}{{\mathrm{\Σ}}_{i=1}^n{\mathrm{\Σ}}_{j=1}^{m}B[i,j]}---\left(3\right)$

In the formula, the value of the capable j row of i in B [i, the j] presentation video, the size of image is n * m.

The calculating formula of gesture direction θ is following:

$\mathrm{\θ}=\frac{1}{2}{\tan }^{-1}\left[\frac{{2\mathrm{\μ}}_{11}}{{\mathrm{\μ}}_{20}-{\mathrm{\μ}}_{02}}\right]---\left(4\right)$

In the formula, μ ₁₁, μ ₂₀, μ ₀₂Second-order moment around mean for image.

From bianry image, extract the edge of gesture with the Canny operator.

Fig. 2 is the center and the direction synoptic diagram of gesture, and rhombus is represented the center of gesture among the figure, and arrow is represented the direction of gesture among the figure.

Step 3: selected characteristic vector

Calculating the angle of gesture central point and edge each point line, find out the immediate point of angle and gesture deflection, is starting point with this point, and the n five equilibrium is carried out on the gesture border, gets borderline n point.With the gesture central point is starting point, and one of borderline n point characterizes this gesture for terminating point structure vector with n vector.Be the influence of size of eliminating gesture in the input picture to discerning, the longest vector of length in the selected characteristic vector, its length is set to 100, and the length normalization method that other is vectorial is to [0,100].Through experiment, adopt n=60.

Fig. 3 (a) is to 2 finger gesture selected characteristic vector result figure, and Fig. 3 (b) is to 4 finger gesture selected characteristic vector result figure,

Step 4: location fingertip location

The proper vector that step 3 is obtained is regarded as an end to end Vector Groups, eliminates starting point and selects the influence to Vector Groups, guarantees that proper vector is insensitive to rotating.From Vector Groups, obtain local maximum and local minimum, the point that wherein local maximum is corresponding is a finger tip point candidate point, and the differential seat angle of its two adjacent local minizing points is the angle of finger region.Selected finger length threshold value l _ThWith angle threshold θ _Th, use finger tip to compare with setting threshold with the angle θ that points the region and judge whether the finger tip candidate point is finger tip to the length l at gesture center.As l＞l _ThAnd θ＜θ _ThThe time, then the terminal point position of this proper vector is a fingertip location, otherwise this is non-finger tip.Through experiment, adopt l _Th=53, θ _Th=1.87.

Fig. 4 (a) is the finger tip positioning result figure to 3 finger gestures, and Fig. 4 (b) is the finger tip positioning result figure to 5 finger gestures.

Step 5: gesture identification

Obtain gesture central point and fingertip location according to step 2 and step 4, constructing one group is starting point with the gesture central point, and fingertip location is the vector of terminal point.Maximum angle between finding out in these vectors in twos; With one in two vectors of angle maximum is start vector; Its angle is set to 0; Guaranteeing under the constant prerequisite of each vectorial relative angle, making other vector minimum, obtaining unique normalization result with the counterclockwise angle and the sum of start vector.Obtain having confirmed the finger number that gesture comprises after the normalized Vector Groups of angle; According to the relative position and the length information of finger, the different gestures that comprise the same hand index are distinguished.The inventive method was discerned a width of cloth images of gestures about 0.4 second consuming time, and that the method for Qin Wenjun etc. is discerned a width of cloth images of gestures is consuming time about 4.5 seconds.Be in advantage of the present invention and effect: the identification to gesture has rotational invariance and convergent-divergent unchangeability, calculates simply, satisfies real-time processing requirements, and good discrimination is arranged.

The English symbol that occurs in the Figure of description, its implication is following:

L: with the length of finger tip to the gesture center;

θ: the angle of finger region;

l _Th: the finger length threshold value is made as 53;

θ _Th: angle threshold is made as 1.87.

Claims (1)

1. the static gesture identification method based on proper vector is characterized in that, comprises following several steps:

Step 1: gesture is cut apart

Adopt Logitech QuickCam Pro9000 camera collection image, obtaining picture size is 320 * 240, and picture format is JPEG, and shooting background does not have the zone of the similar colour of skin, and the user who is taken wears non-colour of skin caftan; To the image that obtains, at first be transformed into the YCbCr color space from the RGB color space, select the central value and the threshold value of Cb, Cr component afterwards, wherein Cb component central value Cb _Mid=115, Cr component central value Cr _Mid=145, threshold value l _Thres=15, each pixel of input picture is calculated the Euclidean distance dist of its Cb, Cr value and central value:

$\mathrm{dist}=\sqrt{{\left({\mathrm{Cb}-\mathrm{Cb}}_{\mathrm{mid}}\right)}^2+{\left({\mathrm{Cr}-\mathrm{Cr}}_{\mathrm{mid}}\right)}^2}---\left(1\right)$

If dist is less than corresponding threshold value l _Thres, then the value with this pixel is made as 1, otherwise its value is made as 0, thereby input picture is converted into bianry image; Bianry image to obtaining once corrodes expansion process, and largest connected zone is the gesture region in the supposition image;

Step 2: the center, direction and the edge that obtain gesture

After images of gestures is converted into bianry image, through calculating center, the direction that obtains gesture, and extract the edge of gesture;

Calculate gesture center point coordinate x and y by formula (2) and formula (3):

$x=\frac{{\mathrm{\Σ}}_{i=1}^n{\mathrm{\Σ}}_{j=1}^m\mathrm{jB}[i,j]}{{\mathrm{\Σ}}_{i=1}^n{\mathrm{\Σ}}_{j=1}^{m}B[i,j]}---\left(2\right)$

$y=\frac{{\mathrm{\Σ}}_{i=1}^n{\mathrm{\Σ}}_{j=1}^m\mathrm{iB}[i,j]}{{\mathrm{\Σ}}_{i=1}^n{\mathrm{\Σ}}_{j=1}^{m}B[i,j]}---\left(3\right)$

In the formula, the value of the capable j row of i in B [i, the j] presentation video, the size of image is n * m;

The calculating formula of gesture direction θ is following:

$\mathrm{\θ}=\frac{1}{2}{\tan }^{-1}\left[\frac{{2\mathrm{\μ}}_{11}}{{\mathrm{\μ}}_{20}-{\mathrm{\μ}}_{02}}\right]---\left(4\right)$

In the formula, μ ₁₁, μ ₂₀, μ ₀₂Second-order moment around mean for image;

From bianry image, extract the edge of gesture with the Canny operator;

Step 3: selected characteristic vector

Calculating the angle of gesture central point and edge each point line, find out the immediate point of angle and gesture deflection, is starting point with this point, and the n five equilibrium is carried out on the gesture border, obtains borderline n point; With the gesture central point is starting point, and one of borderline n point characterizes this gesture for terminating point structure vector with this n vector; Be the influence of size of eliminating gesture in the input picture to discerning, the longest vector of length in the selected characteristic vector, its length is set to 100, and the length normalization method that other is vectorial is to (0,100); Adopt n=60;

Step 4: location fingertip location

The proper vector that step 3 is obtained is regarded as an end to end Vector Groups, eliminates starting point and selects the influence to Vector Groups, guarantees that proper vector is insensitive to rotating; From Vector Groups, obtain local maximum and local minimum, the point that wherein local maximum is corresponding is a finger tip point candidate point, and the differential seat angle of its two adjacent local minizing points is the angle of finger region; Selected finger length threshold value l _ThWith angle threshold θ _Th, use finger tip to compare with setting threshold with the angle θ that points the region and judge whether the finger tip candidate point is finger tip to the length l at gesture center; As l＞l _ThAnd θ＜θ _ThThe time, then the terminal point position of this proper vector is a fingertip location, otherwise this is non-finger tip; Adopt l _Th=53, θ _Th=1.87;

Step 5: gesture identification

Obtain gesture central point and fingertip location according to step 2 and step 4, constructing one group is starting point with the gesture central point, and fingertip location is the vector of terminal point; Maximum angle between finding out in these vectors in twos; With one in two vectors of angle maximum is start vector; Its angle is set to 0; Guaranteeing under the constant prerequisite of each vectorial relative angle, making other vector minimum, obtaining unique normalization result with the counterclockwise angle and the sum of start vector; After obtaining the normalized Vector Groups of angle, then confirmed the finger number that gesture comprises; According to the relative position and the length information of finger, the different gestures that comprise the same hand index are distinguished.

Images