CN103745228A - Dynamic gesture identification method on basis of Frechet distance - Google Patents

Abstract

The invention discloses a dynamic gesture identification method on the basis of a Frechet distance, which at least comprises the following steps of: acquiring gesture position information of a dynamic gesture fragment to be identified in an input video; carrying out matching on an acquired gesture state variation characteristic sequence and a characteristic sequence in a preset model according to the Frechet distance; and according to Frechet distance matching information, acquiring and outputting a similarity result. According to the invention, the extracted characteristic sequence and the pre-obtained model are subjected to similarity measurement in a certain mode and according to the similarity degree, a corresponding classification of the gesture to be identified is determined; and by utilizing the characteristic that the Frechet distance keeps stretching transformation of a time sequence curve along the time dimension unchanged, the dynamic gesture identification method can be well suitable for the condition of nonuniform distribution of the change speed of the dynamic gesture along the time dimension.

Description

Based on the dynamic gesture identification method of Fr é chet distance

Technical field

The present invention relates to a kind of dynamic gesture identification method based on Fr é chet distance.

Background technology

Gesture is that people are daily to express heart activity, to carry out the important method of communication exchange with other people.We are the situation of change on time dimension according to gesture conventionally, and gesture is divided into static gesture and dynamic gesture.

The particular space state of the hand that static gesture refers to the position of at a time relatively-stationary finger, palm, form towards features such as, attitudes.Specific static gesture can with in feature space with it corresponding specified point represent.What answer in contrast is dynamic gesture, and it is consisted of motion, the attitude sequence of continually varying hand in a period of time region.For dynamic gesture, if we choose suitable feature, the gesture information of particular point in time is described, so along with time t, t=1,2 ..., the continuous variation of T, we have just obtained one section of characteristic sequence (F corresponding to this dynamic gesture ₁, F ₂..., F _t).If we make the each element in characteristic sequence successively in feature space, dynamic gesture always can be described with a corresponding curve in feature space so.

And after dynamic gesture describes out, problem is below gesture identification.Generally speaking, the basic task of gesture identification matches corresponding gesture feature and existing model exactly, thereby the curve in corresponding gesture characteristic parameter space or point is divided into the process of different sets or classification.

At present, the flow process framework of gesture recognition system generally consists of three parts such as gesture modeling, gesture analysis and gesture couplings.

Gesture modeling is normally for given gesture.Under normal circumstances, choosing of gesture model need to be determined in conjunction with actual application background.As for dynamic gesture, choosing of gesture model need to reflect its feature on time, Spatial Dimension, the process being closely connected before and after simultaneously the motion of gesture will being considered as.

Once gesture model is decided, we will pass through the relevant feature parameters of gesture analysis process computation gesture, and in fact these features have represented for certain gestures attitude and certain description of movement locus.The process of gesture analysis is comparatively complicated, and it is the process such as gesture location, feature selecting, feature extraction, model parameter estimation comprehensive normally.By these a series of processes, we separate the part relevant to gesture from image or video, and according to the location to gesture, from relevant range or according to the relation of front and back frame video, extract characteristic in order to the observer state of gesture is carried out to suitable description, according to training sample feature, model parameter is estimated simultaneously.The implementation method of above steps need to design according to the concrete application background of problem equally.

The basic task of gesture coupling is that feature and model are matched, thereby the curve in corresponding gesture characteristic parameter space or point is divided into the process of different subsets.At present conventional thinking is the similarity measurement that the feature of extraction is carried out to certain form with the model obtaining in advance, according to similarity degree, determines the classification that gesture to be identified is corresponding.

Summary of the invention

For the problems referred to above, the object of the present invention is to provide a kind of dynamic gesture identification method based on Fr é chet distance.

For achieving the above object, a kind of dynamic gesture identification method based on Fr é chet distance of the present invention, at least comprises the following steps:

Obtain the hand gesture location information of dynamic gesture fragment to be identified in input video;

Obtain the described hand gesture location place gesture state variation characteristic sequence of front and back frame continuously;

The gesture state variation characteristic sequence obtaining is mated according to Fr é chet distance with the characteristic sequence in preset model;

According to Fr é chet, apart from match information, obtain similarity result output.

Preferably, the concrete steps of obtaining the hand gesture location information of dynamic gesture fragment to be identified in input video described in are:

According to the rgb value of pixel in video, obtain the probable value that in any two field picture of video, area of skin color occurs;

According to described probable value judgement, obtain all area of skin color that distribute in the each two field picture of video;

Obtain the light stream value of each area of skin color two field picture before and after continuously;

According to the light stream value of all area of skin color that distribute, obtain the region of average light flow valuve maximum, i.e. hand gesture location region.

Preferably, the gesture state variation characteristic sequence that obtains continuous front and back, described hand gesture location place frame described in comprises movement locus variation characteristic sequence and attitude variation characteristic sequence.

Preferably, the coupling step of described movement locus variation characteristic sequence is:

Any frame of hand gesture location region inner video image is set and the average light stream of former frame is direction vector, that is: F=(x, y), wherein x, y represents that respectively average light stream is at the component laterally, on longitudinally;

Obtain the corresponding motion feature sequence of the each frame of inputted video image for (F ₁, F ₂..., F _t);

Obtaining the motion feature sequence arranging in model is (M ₁, M ₂..., M _t);

Choose motion feature sequence (F ₁, F ₂..., F _t) in any one section of sequence fragment f=(F _i, F _i+1..., F _j), obtain any vector (x in this fragment ₁, y ₁) and motion feature sequence (M ₁, M ₂..., M _t) in any vector (x ₂, y ₂) distance, be:

$d_1((x_1,y_1),(x_2,y_2))=1-\frac{x_1\·x_2+y_1\·y_2}{\sqrt{x_1^2+y_1^2}\·\sqrt{x_2^2+y_2^2}};$

From the multiple d that obtain ₁((x ₁, y ₁), (x ₂, y ₂)) maximizing, ask for boundary under maximal value, just can obtain f=(F _i, F _i+1..., F _j) and (M ₁, M ₂..., M _t) Fr é chet apart from δ _f;

One threshold epsilon is set ₁, judge this threshold value and δ _fsize:

If δ _f≤ ε ₁, judge that in video to be identified, corresponding gesture motion shown in i frame to the j frame is the gesture of model coupling;

Otherwise, be not.

Preferably, the coupling step of described attitude variation characteristic sequence is:

Finger fingertip and middle finger in any frame of hand gesture location region inner video image are set and refer to that the coordinate of the corresponding finger of root center is attitude vector, be Z, wherein Z is the proper vector of 12 dimensions;

Obtain the corresponding posture feature sequence of the each frame of inputted video image for (Z ₁, Z ₂..., Z _t);

Obtaining the posture feature sequence arranging in model is (N ₁, N ₂..., N _t);

Choose attitude motion characteristic sequence (Z ₁, Z ₂..., Z _t) in any one section of sequence fragment f=(Z _i, Z _i+1..., Z _j), obtain any vector Z and attitude motion characteristic sequence (N in this fragment ₁, N ₂..., N _t) in the distance of any vectorial N, be:

d ₂(Z,N)=||Z-N|| ₂；

From the multiple d that obtain ₂(Z, N) maximizing, asks for boundary under maximal value, just can obtain f=(Z _i, Z _i+1..., Z _j) and (N ₁, N ₂..., N _t) Fr é chet apart from δ _f;

One threshold epsilon is set ₂, judge this threshold value and δ _fsize:

If δ _f≤ ε ₂, judge that in video to be identified, corresponding gesture motion shown in i frame to the j frame is the gesture of model coupling;

Otherwise, be not.

Beneficial effect of the present invention is:

The present invention is by carrying out the characteristic sequence of extraction the similarity measurement of certain form with the model obtaining in advance, according to similarity degree, determine the classification that gesture to be identified is corresponding, and utilize Fr é chet distance to there is the characteristic that time-serial position is remained unchanged along time dimension stretching, can adapt to well dynamic gesture pace of change along time dimension situation pockety.

Accompanying drawing explanation

Fig. 1 is the framework schematic diagram of the dynamic gesture identification method based on Fr é chet distance described in the embodiment of the present invention;

Fig. 2 is embodiment of the present invention certain gestures orbiting motion schematic diagram;

Fig. 3 is embodiment of the present invention certain gestures attitude change procedure figure;

Fig. 4 is that embodiment of the present invention gesture posture feature is chosen schematic diagram;

Fig. 5 is the framework schematic diagram that embodiment of the present invention posture feature is chosen.

Embodiment

Below in conjunction with Figure of description, the present invention will be further described.

As shown in Figure 1, a kind of dynamic gesture identification method based on Fr é chet distance described in the embodiment of the present invention, at least comprises the following steps:

Obtain the hand gesture location information of dynamic gesture fragment to be identified in input video;

Obtain the described hand gesture location place gesture state variation characteristic sequence of front and back frame continuously;

The gesture state variation characteristic sequence obtaining is mated according to Fr é chet distance with the characteristic sequence in preset model;

According to Fr é chet, apart from match information, obtain similarity result output.

Wherein, it should be noted that: definition Fr é chet distance is: any one belongs to

the parametric curve in space can be expressed as Continuous Mappings

wherein a<b.For curve

with

fr é chet distance is

${\mathrm{\&delta;}}_F(f,g):=\underset{\underset{\mathrm{\&beta;}:\left[\mathrm{0,1}\right]\&RightArrow;[b,b^{\&prime;}]}{\mathrm{\&alpha;}:\left[\mathrm{0,1}\right]\&RightArrow;[a,a^{\&prime;}]}}{\inf }\underset{t\&Element;\left[\mathrm{0,1}\right]}{\max }d\left(f(\mathrm{\&alpha;}\left(t\right)\right),g\left(\mathrm{\&beta;}\left(t\right)\right))$

α in this formula, β is for being defined in the continuous monotonically increasing function on [0,1], and meets α (0)=a, α (1)=a', β (0)=b, β (0)=b'.

Said method step is further explained:

First to determine the place, position of gesture in video image, namely want the hand gesture location of positioning image.Its concrete steps of determining hand gesture location information are:

According to the rgb value of pixel in video, obtain the probable value that in any two field picture of video, area of skin color occurs;

According to described probable value judgement, obtain all area of skin color that distribute in the each two field picture of video;

Obtain the light stream value of each area of skin color two field picture before and after continuously;

According to the light stream value of all area of skin color that distribute, obtain the region of average light flow valuve maximum, i.e. hand gesture location region.

The specific explanations of above-mentioned steps is such: in the identification of gesture track, specific gesture implication shows by hand movement locus as a whole.For each two field picture, can to the hand moving in video, position by color and kinematic constraint.

On the one hand, in video, each two field picture is comprised of R, G, B triple channel, need to be by the rgb value of area of skin color and non-area of skin color in some pictures be added up, obtain one large little be 256 × 256 × 256 form, in form, each represents the probability that its corresponding rgb value occurs with colour of skin form.Like this, for each pixel of the each two field picture in video, just obtained its approximation probability as skin, and then the distribution of the region that can roughly estimate the approximate colour of skin in the each two field picture of video.

On the other hand, because hand in gesture is expressed is kept in motion as a whole all the time, for a certain two field picture in video, need the light stream by itself and former frame image, the region of choosing average light stream maximum in the region of the approximate colour of skin, has obtained the location to motion hand.

Behind hand gesture location location, then to extract characteristic sequence.Because the described gesture state variation characteristic sequence that obtains continuous front and back, described hand gesture location place frame comprises movement locus variation characteristic sequence and attitude variation characteristic sequence.Therefore, need respectively the feature of movement locus aspect and attitude aspect to be extracted.

The coupling step of described movement locus variation characteristic sequence is:

Any frame of hand gesture location region inner video image is set and the average light stream of former frame is direction vector, that is: F=(x, y), wherein x, y represents that respectively average light stream is at the component laterally, on longitudinally;

Obtain the corresponding motion feature sequence of the each frame of inputted video image for (F ₁, F ₂..., F _t);

Obtaining the motion feature sequence arranging in model is (M ₁, M ₂..., M _t);

Choose motion feature sequence (F ₁, F ₂..., F _t) in any one section of sequence fragment f=(F _i, F _i+1..., F _j), obtain any vector (x in this fragment ₁, y ₁) and motion feature sequence (M ₁, M ₂..., M _t) in any vector (x ₂, y ₂) distance, be:

$d_1((x_1,y_1),(x_2,y_2))=1-\frac{x_1\&CenterDot;x_2+y_1\&CenterDot;y_2}{\sqrt{x_1^2+y_1^2}\&CenterDot;\sqrt{x_2^2+y_2^2}};$

From the multiple d that obtain ₁((x ₁, y ₁), (x ₂, y ₂)) maximizing, ask for boundary under maximal value, just can obtain f=(F _i, F _i+1..., F _j) and (M ₁, M ₂..., M _t) Fr é chet apart from δ _f;

One threshold epsilon is set ₁, judge this threshold value and δ _fsize:

If δ _f≤ ε ₁, judge that in video to be identified, corresponding gesture motion shown in i frame to the j frame is the gesture of model coupling;

Otherwise, be not.

Specific explanations is:

1, model is set up

Certain gestures track is to be combined in a fixed order by some the most basic strokes, and therefore, we choose the sequence specifically consisting of direction vector (x, y) as the model that represents gesture track.As shown in Figure 2, each point corresponding each element of seasonal effect in time series of representative model successively in figure, the corresponding direction vector of each element (x, y) is marked by arrow in the drawings.

2, feature selecting and extraction

The sequence that gesture track consists of specific direction vector represents, and the direction of motion of gesture can be by asking for the calculating of light stream (optical flow).According to the location estimation to motion hand, can be by the average light stream F=(x, y) of the image calculation gesture region of this frame in video and adjacent former frame, wherein x, y represents that respectively average light stream is along horizontal, longitudinal component, and sets it as this two field picture character pair.

Like this for the each frame (V in video ₁, V ₂..., V _t), just obtained corresponding characteristic sequence (F ₁, F ₂..., F _t).

3, Fr é chet is apart from coupling

Choose motion feature sequence (F ₁, F ₂..., F _t) in any one section of sequence fragment f=(F _i, F _i+1..., F _j), obtain any vector (x in this fragment ₁, y ₁) and motion feature sequence (M ₁, M ₂..., M _t) in any vector (x ₂, y ₂) distance, be:

$d_1((x_1,y_1),(x_2,y_2))=1-\frac{x_1\&CenterDot;x_2+y_1\&CenterDot;y_2}{\sqrt{x_1^2+y_1^2}\&CenterDot;\sqrt{x_2^2+y_2^2}};$

From the multiple d that obtain ₁((x ₁, y ₁), (x ₂, y ₂)) maximizing, ask for boundary under maximal value, just can obtain f=(F _i, F _i+1..., F _j) and (M ₁, M ₂..., M _t) Fr é chet apart from δ _f;

One threshold epsilon is set ₁, judge this threshold value and δ _fsize:

If δ _f≤ ε ₁, judge that in video to be identified, corresponding gesture motion shown in i frame to the j frame is the gesture of model coupling;

Otherwise, be not.

The coupling step of described attitude variation characteristic sequence is:

Finger fingertip and middle finger in any frame of hand gesture location region inner video image are set and refer to that the coordinate of the corresponding finger of root center is attitude vector, be Z, wherein Z is the proper vector of 12 dimensions;

Obtain the corresponding posture feature sequence of the each frame of inputted video image for (Z ₁, Z ₂..., Z _t);

Obtaining the posture feature sequence arranging in model is (N ₁, N ₂..., N _t);

Choose attitude motion characteristic sequence (Z ₁, Z ₂..., Z _t) in any one section of sequence fragment f=(Z _i, Z _i+1..., Z _j), obtain any vector Z and attitude motion characteristic sequence (N in this fragment ₁, N ₂..., N _t) in the distance of any vectorial N, be:

d ₂(Z,N)=||Z-N|| ₂；

From the multiple d that obtain ₂(Z, N) maximizing, asks for boundary under maximal value, just can obtain f=(Z _i, Z _i+1..., Z _j) and (N ₁, N ₂..., N _t) Fr é chet apart from δ _f;

One threshold epsilon is set ₂, judge this threshold value and δ _fsize:

If δ _f≤ ε ₂, judge that in video to be identified, corresponding gesture motion shown in i frame to the j frame is the gesture of model coupling;

Otherwise, be not.

Specific explanations is:

1, model is set up

For specific gesture attitude change procedure, choose its representative gesture stationary posture sequence as the model in order to describe gesture attitude change procedure.Each element in attitude sequence is all taken from the stationary posture database of hand, and they are corresponding one by one with feature cited below.Be illustrated in figure 3 attitude variation diagram.

2, character selection and abstraction

Specific gesture attitude is determined towards information such as, finger and palm relative positions by hand.Therefore choose each finger fingertip and middle finger and refer to that the position coordinates at relative wrist center, root joint is as the feature of description certain gestures attitude.As shown in Figure 4.

Be illustrated in figure 5 the framework schematic diagram that posture feature is chosen, actual carry out feature estimate time, need in the stationary posture database of hand, retrieve by the characteristics of image of this two field picture, obtain the stationary posture model being similar to the most with it, estimation as it in this moment attitude, and by each finger fingertip of model and middle finger refer to root joint with respect to wrist center laterally, lengthwise position coordinate information, totally 12 dimension real numbers are as to this moment gesture attitude real features.

3, Fr é chet is apart from coupling

Choose attitude motion characteristic sequence (Z ₁, Z ₂..., Z _t) in any one section of sequence fragment f=(Z _i, Z _i+1..., Z _j), obtain any vector Z and attitude motion characteristic sequence (N in this fragment ₁, N ₂..., N _t) in the distance of any vectorial N, be:

d ₂(Z,N)=||Z-N|| ₂；

From the multiple d that obtain ₂(Z, N) maximizing, asks for boundary under maximal value, just can obtain f=(Z _i, Z _i+1..., Z _j) and (N ₁, N ₂..., N _t) Fr é chet apart from δ _f;

One threshold epsilon is set ₂, judge this threshold value and δ _fsize:

If δ _f≤ ε ₂, judge that in video to be identified, corresponding gesture motion shown in i frame to the j frame is the gesture of model coupling;

Otherwise, be not.

Above; be only preferred embodiment of the present invention, but protection scope of the present invention is not limited to this, any be familiar with those skilled in the art the present invention disclose technical scope in; the variation that can expect easily or replacement, within all should being encompassed in protection scope of the present invention.Therefore, protection scope of the present invention should be as the criterion with the protection domain that claim was defined.

Claims (5)

1. the dynamic gesture identification method based on Fr é chet distance, is characterized in that, at least comprises the following steps:

Obtain the hand gesture location information of dynamic gesture fragment to be identified in input video;

Obtain the described hand gesture location place gesture state variation characteristic sequence of front and back frame continuously;

The gesture state variation characteristic sequence obtaining is mated according to Fr é chet distance with the characteristic sequence in preset model;

According to Fr é chet, apart from match information, obtain similarity result output.

2. the dynamic gesture identification method based on Fr é chet distance according to claim 1, is characterized in that, described in obtain the hand gesture location information of dynamic gesture fragment to be identified in input video concrete steps be:

According to the rgb value of pixel in video, obtain the probable value that in any two field picture of video, area of skin color occurs;

According to described probable value judgement, obtain all area of skin color that distribute in the each two field picture of video;

Obtain the light stream value of each area of skin color two field picture before and after continuously;

According to the light stream value of all area of skin color that distribute, obtain the region of average light flow valuve maximum, i.e. hand gesture location region.

3. the dynamic gesture identification method based on Fr é chet distance according to claim 1, it is characterized in that, described in obtain described hand gesture location place continuously before and after the gesture state variation characteristic sequence of frame comprise movement locus variation characteristic sequence and attitude variation characteristic sequence.

4. the dynamic gesture identification method based on Fr é chet distance according to claim 3, is characterized in that, the coupling step of described movement locus variation characteristic sequence is:

Any frame of hand gesture location region inner video image is set and the average light stream of former frame is direction vector, that is: F=(x, y), wherein x, y represents that respectively average light stream is at the component laterally, on longitudinally;

Obtain the corresponding motion feature sequence of the each frame of inputted video image for (F ₁, F ₂..., F _t);

Obtaining the motion feature sequence arranging in model is (M ₁, M ₂..., M _t);

Choose motion feature sequence (F ₁, F ₂..., F _t) in any one section of sequence fragment f=(F _i, F _i+1..., F _j), obtain any vector (x in this fragment ₁, y ₁) and motion feature sequence (M ₁, M ₂..., M _t) in any vector (x ₂, y ₂) distance, be:

$d_1((x_1,y_1),(x_2,y_2))=1-\frac{x_1\&CenterDot;x_2+y_1\&CenterDot;y_2}{\sqrt{x_1^2+y_1^2}\&CenterDot;\sqrt{x_2^2+y_2^2}};$

From the multiple d that obtain ₁((x ₁, y ₁), (x ₂, y ₂)) maximizing, ask for boundary under maximal value, just can obtain f=(F _i, F _i+1..., F _j) and (M ₁, M ₂..., M _t) Fr é chet apart from δ _f;

One threshold epsilon is set ₁, judge this threshold value and δ _fsize:

If δ _f≤ ε ₁, judge that in video to be identified, corresponding gesture motion shown in i frame to the j frame is the gesture of model coupling;

Otherwise, be not.

5. the dynamic gesture identification method based on Fr é chet distance according to claim 3, is characterized in that, the coupling step of described attitude variation characteristic sequence is:

Finger fingertip and middle finger in any frame of hand gesture location region inner video image are set and refer to that the coordinate of the corresponding finger of root center is attitude vector, be Z, wherein Z is the proper vector of 12 dimensions;

Obtain the corresponding posture feature sequence of the each frame of inputted video image for (Z ₁, Z ₂..., Z _t);

Obtaining the posture feature sequence arranging in model is (N ₁, N ₂..., N _t);

Choose attitude motion characteristic sequence (Z ₁, Z ₂..., Z _t) in any one section of sequence fragment f=(Z _i, Z _i+1..., Z _j), obtain any vector Z and attitude motion characteristic sequence (N in this fragment ₁, N ₂..., N _t) in the distance of any vectorial N, be:

d ₂(Z,N)=||Z-N|| ₂；

From the multiple d2 that obtain ₍z, N) maximizing, ask for boundary under maximal value, just can obtain f=(Z _i, Z _i+1..., Z _j) and (N ₁, N ₂..., N _t) Fr é chet apart from δ _f;

One threshold epsilon is set ₂, judge this threshold value and δ _fsize:

If δ _f≤ ε ₂, judge that in video to be identified, corresponding gesture motion shown in i frame to the j frame is the gesture of model coupling;

Otherwise, be not.

Images