CN104463099A - Multi-angle gait recognizing method based on semi-supervised coupling measurement of picture - Google Patents

Abstract

The invention belongs to the field of pattern recognizing, and particularly relates to a multi-angle gait recognizing method based on semi-supervised coupling measurement of a picture. The multi-angle gait recognizing method comprises the steps that a target outline sequence is obtained from video streaming through a codebook detecting method; the overall feature of a gait is extracted from a cycle through a gait energy picture; an off-line training stage of a multi-angle gait recognizing system is established, and a semi-supervised coupling projection matrix pair based on the picture is obtained through training; target outline extracting is carried out on a test video, the gait cycle is detected for the size-normalized outline sequence, the gait energy picture feature of the single cycle is generated, and the selected semi-supervised coupling projection matrix pair based on the picture is estimated through the viewing angle. The multi-angle gait recognizing method solves the problem that according to a traditional gait recognizing method, gait features of all viewing angles need to be stored, and the storage volume is high; the identity recognizing for gaits of walking of any angle is valid.

Description

A kind of multi-angle gait recognition method of the semi-supervised Coupling Metric based on figure

Technical field

The invention belongs to area of pattern recognition, be specifically related to a kind of multi-angle gait recognition method of the semi-supervised Coupling Metric based on figure.

Background technology

Gait Recognition is the research direction received much concern in computer vision and living things feature recognition field in recent years, and the posture that it is intended to walk according to people carries out identification ^[1,2].Compared with other biological feature identification technique, Gait Recognition uniquely can know method for distinguishing at a distance in living things feature recognition.Further, gait untouchable, not easily pretend, the advantage such as remote, in intelligent video monitoring, have very large application prospect.

But, Gait Recognition also faces many difficult points in actual applications, be mainly manifested in the impact that pedestrian can be subject to external environment and oneself factor in the process of walking, such as different rows walks the factors such as road surface, different resolution, different visual angles, different dress ornament, different belongings.In above-mentioned influence factor, visual angle change is one of main factor affecting Gait Recognition system performance.Visual angle change problem is the unavoidable problem of Gait Recognition, and the direction of travel because of people is completely freely random, and the camera location difference of zones of different.At present, traditional Gait Recognition technology can obtain good performance under fixed viewpoint, and when visual angle acute variation or existence are blocked, and cannot be suitable for or recognition performance obviously declines.

In order to solve the impact of visual angle change on Gait Recognition system, propose a lot of method both at home and abroad.The people such as Kale [3] adopt the method for Angles Projections model and light stream structure to obtain gait silhouette at any angle.The people such as Jean [4] propose the track obtaining body parts standard angle from monocular video sequence, but the method is only effective within the scope of certain angle.The people such as Han [5] obtain angle invariant features from gait energygram GEI, only have selected a part and be used for building multi-angle gait feature in the gait sequence of different angles superposition.These methods are only effective to the angle of several direction of travel above, the interference and characteristic extraction procedure is easily blocked.In addition, also can with solving multi-angle problem from the mode of multiple video structure three-dimensional model.The vision hull (IBVH) based on image that the people [6] such as such as Shakhnarovich propose characterizes the visual angle of Gait Recognition, the method obtains a series of monocular angle calculation IBVH from the shooting of multiple standard, estimate the camera position of standard, the image then obtained from these visual angles is for the standard of angle.The people such as Bodor [7] rebuild from all directions gait feature automatically with 3D vision structural model.The people such as Zhang [8] propose the gait recognition method irrelevant with visual angle based on three-dimensional line pattern type and Bayes rule.Gait recognition method above based on multi-cam needs complicated standardization multi-camera system, and computation complexity is higher, is not suitable for real-time Gait Recognition system.Also having a kind of is the projection relation of gait feature finding different angles.These class methods will by the gait feature standardization from different angles before gait similarity mode, contrast to gait method compare, being a little of the optimum projector space of this searching (1) only need one need not standardization picture pick-up device, do not need to test gait and each frame synchronization of registration gait trained; (2) test process saves time, and is applicable to real-time gait system.Such as the frequency domain gait pattern of different angles is transformed to same angle by Makihara [9] angular transformation model (VTM), by svd matrix, in training set, gait feature matrix can be decomposed into the matrix had nothing to do with object and angle.Be used to build VTMs with the matrix that object is irrelevant.The people such as Kusakunniran [8] instead of Fourier feature by the gait energygram GEI feature that line style discriminatory analysis (LDA) is optimized.Above method is all that supposition gait feature matrix can be decomposed into two independent matrix, does not have the element repeated.But mathematically this hypothesis unclear is proved, therefore just can not ensure the VTM obtaining optimum.Canonical correlation analysis (CCA) is applied on the correlation modeling of different angles gait sequence by Bashir et al. [11], first by CCA, the gait feature in two different directions is projected to the maximum space of correlativity, the similarity of gait is judged again according to relevant intensity, compared with VTM, CCA can well process the characteristic matching problem of different angles, and the noise for feature has good robustness.But CCA is a kind of unsupervised dimension reduction method, does not consider classification information.

The open report relevant to invention comprises:

[1] beautifully adorned Xian is firelight or sunlight, Xu Sen, Wang Kejun. the feature representation of pedestrian's gait and identification summary [J]. and pattern-recognition and artificial intelligence, 2012,25 (1): 71-81.

[2] Wang Kejun, Hou Benbo. Gait Recognition summary [J]. Journal of Image and Graphics, 2007,12 (7): 1152-1160.

[3]A.Kale,A.K.R.Chowdhury,and R.Chellappa:Towards a view invariant gait recognition algorithm.In Advanced Video and Signal Based Surveillance,2003.Proceedings.IEEE Conference on,pp.143-150(2003).

[4]F.Jean,R.Bergevin,and A.B.Albu:Computing and evaluating view-normalized body part trajectories.Image and Vision Computing,vol.27,pp.1272-1284,(2009).

[5]J.Han,B.Bhanu,and A.Roy-Chowdhury:A study on view-insensitive gait recognition.In Image Processing,2005.ICIP 2005.IEEE International Conference on,pp.III-297-300(2005).

[6]G.Shakhnarovich,L.Lee,and T.Darrell:Integrated face and gait recognition from multiple views.In Computer Vision and Pattern Recognition,2001.CVPR 2001.Proceedings of the 2001IEEE Computer Society Conference on,pp.I-439-I-446vol.1(2001).

[7]R.Bodor,A.Drenner,D.Fehr,O.Masoud,and N.Papanikolopoulos.View-independent human motion classification using image-based reconstruction.Image and Vision Computing,vol.27,pp.1194-1206,(2009).

[8]Z.Zhang and N.F.Troje:View-independent person identification from human gait.Neurocomputing,vol.69,pp.250-256,(2005).

[9]Y.Makihara,R.Sagawa,Y.Mukaigawa,T.Echigo,and Y.Yagi:Gait recognition using a view transformation model in the frequency domain.In Computer Vision–ECCV 2006,ed:Springer,pp.151-163(2006).

[10]W.Kusakunniran,Q.Wu,H.Li,and J.Zhang:Multiple views gait recognition using view transformation model based on optimized gait energy image.In Computer Vision Workshops(ICCV Workshops),2009IEEE 12th International Conference on,pp.1058-1064(2009).

[11]K.Bashir,T.Xiang,and S.Gong:Cross View Gait Recognition Using Correlation Strength.In BMVC,pp.1-11(2010).

Summary of the invention

The object of the present invention is to provide a kind of multi-angle gait recognition method improving a kind of semi-supervised Coupling Metric based on figure of recognition performance.

The object of the present invention is achieved like this:

(1) adopt code book detection method from video flowing, obtain objective contour sequence, and make human body between two parties unified size be 64*64;

(2) according to after two field picture center for standard every in gait video sequence, the cycle of people's periodic statistics gait of the degree that two legs are separated in walking, and adopt gait energygram to extract the global feature of gait in one cycle;

(3) build the off-line training step of various visual angles Gait Recognition system, train the semi-supervised coupling projection matrix pair obtained based on figure; Test video is carried out to the extraction of objective contour, to the profile Sequence Detection gait cycle of size normalization, generate monocyclic gait energygram feature, the semi-supervised coupling projection matrix pair based on figure chosen is estimated by visual angle, project in coupled in common gait feature space, and adopt nearest neighbor classifier to carry out identity differentiation in coupled in common gait feature space.

Step (2) according in figure after two field picture center for standard every in gait video flowing, the cyclical variation situation of periodic statistics gait of people's degree that two legs are separated in walking, the cycle is:

$W=\frac{1}{(h_2-h_1+1)}\underset{i=h_1}{\overset{h_2}{\mathrm{\Σ}}}(R_i-L_i)$

H ₁and h ₂be respectively the ankle of people's (prospect) and the anthropometry height of knee in certain two field picture, R _iand L _iin the i-th row, belong to the Far Left of prospect and rightmost location of pixels respectively.

Feature extraction adopts gait energygram to extract the global feature of gait:

$G(x,y)=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{B}_i(x,y)$

N is the frame number of the gait sequence of one-period, and i represents the time, and (x, y) represents two dimensional image plane coordinate.

The standard viewing angle of gait energygram chooses 90 ° of visual angles, and the gait feature at all the other visual angles is carried out joint training with standard viewing angle gait feature respectively.

Obtain semi-supervised coupling projection matrix based on figure to P _xand P _ymethod be: carry out joint training with the gait feature under the gait feature under 90 ° of standard viewing angle and 0 ° of visual angle, the gait feature under the gait feature under 0 ° of visual angle and 90 ° of visual angles projected to respectively in common coupling gait feature space:

$\min J(P_x,P_y)=\min \mathrm{Tr}\left({\left[\begin{array}{c}{P}_x\\ P_y\end{array}\right]}^T\left[\begin{array}{cc}X& \\ & Y\end{array}\right]\left[\begin{array}{cc}{C}_x& -C\\ {-C}^T& C_y\end{array}\right]{\left[\begin{array}{cc}X& \\ & Y\end{array}\right]}^T\left[\begin{array}{c}{P}_x\\ P_y\end{array}\right]\right)$

In formula, X represents 90 ° of visual angle training sample set, and Y represents 0 ° of visual angle training sample set, and C is the correlationship matrix between set X and Y, and size is N _x× N _y, N _x, N _ybe respectively X and Y and gather sample number, Matrix C is obtained by neighbour figure, C _xand C _yfor relation diagonal matrix in the class in single space, its diagonal entry is respectively the cumulative sum of the corresponding row of Matrix C, and the cumulative sum of respective column;

Order $P=\left[\begin{array}{c}{P}_x\\ P_y\end{array}\right],$ $Z=\left[\begin{array}{cc}X& \\ & Y\end{array}\right],$ $\mathrm{\Ω}=\left[\begin{array}{cc}{C}_x& -C\\ {-C}^T& C_y\end{array}\right],$ Optimum solution is obtained by the proper vector solving generalized eigenvalue Ep=λ Fp, wherein E=Z Ω Z ^t, F=ZZ ^t, p is the proper vector corresponding to eigenvalue λ, and P is corresponding D _cthe proper vector of individual minimal eigenvalue, P=[P _xp _y] ^tdefinition, obtain corresponding to the transformation matrix p of data acquisition X _x, size is D _x× D _c, corresponding to the transformation matrix P of data acquisition Y _y, size is D _y× D _c.

Correlation matrix C is:

$C_{\mathrm{ij}}=\left\{\begin{array}{cc}\exp -{\hat{D}}_{\mathrm{ij}},& (i,j)\∈(\mathrm{\Ξ}\∪{\mathrm{\Ξ}}_x\∪{\mathrm{\Ξ}}_y)\\ 0,& \mathrm{otherwise}\end{array}\right.$

In formula, calculate new set spacing matrix with Dijkstra shortest path first

${\hat{D}}_{\mathrm{ij}}=\underset{p,q}{\min }(D_{\mathrm{ip}}^x+D_{\mathrm{pq}}+D_{\mathrm{qj}}^y),x_p\∈X,y_q\∈Y$

In formula, $D_{\mathrm{ij}}^x=\left\{\begin{array}{cc}\frac{{\left|\right|x_i-x_j\left|\right|}^2}{{\mathrm{\σ}}_x^2},& (i,j)\∈{\mathrm{\Ξ}}_x\\ \∞,& \mathrm{otherwise}\end{array}\right.,$ $D_{\mathrm{ij}}^y=\left\{\begin{array}{cc}\frac{{\left|\right|x_i-x_j\left|\right|}^2}{{\mathrm{\σ}}_y^2},& (i,j)\∈{\mathrm{\Ξ}}_y\\ \∞,& \mathrm{otherwise}\end{array}\right.,$ $D_{\mathrm{ij}}=\left\{\begin{array}{c}0,(i,j)\∈\mathrm{\Ξ}\\ \∞,\mathrm{otherwise}\end{array}\right.,$ and D _ijrespectively in the set of 0 ° of visual angle in the distance of two samples, 90 ° of visual angles set under the distance of two samples, 0 ° of visual angle and under 90 ° of visual angles the distance of same person be 0, Ξ _x, Ξ _yand Ξ represent respectively X set, Y set and these two set between neighborhood relationships set.

View angle theta belonging to gait feature is:

θ＝arg min d(g ^test,μ _i)

θ _i,1≤i≤N _f

Wherein g ^testrepresent the gait feature vector in test set, μ _ifor the gait feature vector under visual angle i in training set; N _ffor look-out angle number in training set; D () is metric function, chooses Euclidean distance function.

Nearest neighbor classifier, refers to find the gait feature of registered set minimum with test gait characteristic distance in coupled room:

$k=\underset{1\≤j\≤N_g}{\arg \min d}(z_j^{\mathrm{gallary}},z^{\mathrm{test}})=\left|\right|P_x^{\mathrm{\θ}}{G}_j^{\mathrm{gallary}}-P_y^{\mathrm{\θ}}{g}^{\mathrm{test}}\left|\right|$

In formula, represent a jth gait feature of registered set, N _grepresent the number of samples of registered set, with for training the optimum of standard viewing angle and the view angle theta obtained to differentiate coupling projection matrix pair, making the classification indicator function of function c () belonging to proper vector, determining that the category label of cycle tests is:

Beneficial effect of the present invention is:

The gait feature of the present invention only under storage standards visual angle and the semi-supervised coupling projection matrix pair based on figure of gait feature between all the other multiple look-out angle and standard viewing angle, solve the high storage demand problem that traditional gait recognition method needs to store gait feature under all visual angles, effective to the identification of the gait of arbitrarily angled walking.

Accompanying drawing explanation

Fig. 1 multi-angle Gait Recognition process flow diagram;

The body gait profile that Fig. 2 extracts;

Fig. 3 anthropometry height map;

Fig. 4 gait cycle testing result;

The gait image of Fig. 5 one-period;

Fig. 6 is with the gait energygram under a group traveling together's different visual angles;

The training process that Fig. 7 nearest-neighbor coupled projection matrix is right;

The connection layout of Fig. 8 two Set-dissection elements;

The test process of Fig. 9 identification.

Embodiment

The invention solves existing Gait Recognition technology in visual angle change, when namely test gait visual angle is not mated with registered set gait visual angle, the problem that recognition performance obviously declines, and the problem of preserving various visual angles gait information at substantial storage resources in registered set.

This method divides three steps: the first step, adopts code book detection method from video flowing, obtain objective contour sequence, and make human body between two parties unified size be 64*64; Second step, after two field picture center for standard every in gait video sequence, the cyclical variation situation of the periodicity observation gait of people's degree that two legs are separated in walking, and adopt gait energygram (GEI) to extract the global feature of gait in one cycle; 3rd step, the off-line training step of the various visual angles Gait Recognition system of structure, obtains the semi-supervised coupling projection matrix pair based on figure; In the ONLINE RECOGNITION stage, first test video is carried out to the extraction of objective contour, then to the profile Sequence Detection gait cycle of size normalization, generate monocyclic GEI feature.Finally, estimated the semi-supervised coupling projection matrix pair based on figure chosen by visual angle, project in the coupled in common gait feature space of similar arest neighbors, and adopt nearest neighbor classifier to carry out identity differentiation within this space.

The present invention can also comprise:

The method of the acquisition of 1, described pedestrian target profile is: code book detection method obtains the gait sequence of binaryzation from gait video flowing, and making the human body in this sequence unify size is between two parties 64*64 pixel.

The cycle detection of 2, described gait is according in figure after two field picture center for standard every in gait video flowing, the cyclical variation situation of the periodicity observation gait of people's degree that two legs are separated in walking, and cycle detection formula is:

$W=\frac{1}{(h_2-h_1+1)}\underset{i=h_1}{\overset{h_2}{\mathrm{\Σ}}}(R_i-L_i)---\left(1\right)$

H ₁and h ₂be respectively the ankle of people's (prospect) and the anthropometry height of knee in certain two field picture, R _iand L _iin the i-th row, belong to the Far Left of prospect and rightmost location of pixels respectively.

3, described feature extraction adopts gait energygram (GEI) to extract the global feature of gait, and formula is:

$G(x,y)=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{B}_i(x,y)---\left(2\right)$

In formula, N is the frame number of the gait sequence of one-period, and i represents the time, and (x, y) represents two dimensional image plane coordinate.

4, described off-line training process: the GEI feature choosing a standard viewing angle, (this patent chooses 90 ° of visual angles as standard viewing angle), gait feature under all the other multiple visual angles (0 °, 18 °, 36 °, 54 °, 72 °, 90 °, 108 °, 126 °, 144 °, 162 °) is carried out joint training with standard viewing angle gait feature respectively, make the relevant gait feature of same a group traveling together under different visual angles (same visual angle) as far as possible close in coupled room, obtain the coupling projection matrix pair of corresponding arest neighbors.

5, the described coupling projection matrix obtaining similar arest neighbors is to P _xand P _ymethod be: carry out joint training for the gait feature under the gait feature under 90 ° of standard viewing angle and 0 ° of visual angle, gait feature under gait feature under 0 ° of visual angle and 90 ° of visual angles is projected in common coupling gait feature space respectively, makes same a group traveling together relevant gait feature at (same visual angle) under different visual angles as far as possible close in coupled room:

$\min J(P_x,P_y)=\min \mathrm{Tr}\left({\left[\begin{array}{c}{P}_x\\ P_y\end{array}\right]}^T\left[\begin{array}{cc}X& \\ & Y\end{array}\right]\left[\begin{array}{cc}{C}_x& -C\\ {-C}^T& C_y\end{array}\right]{\left[\begin{array}{cc}X& \\ & Y\end{array}\right]}^T\left[\begin{array}{c}{P}_x\\ P_y\end{array}\right]\right)---\left(3\right)$

In formula, X represents 90 ° of visual angle training sample set, and Y represents 0 ° of visual angle training sample set, and C is the correlationship matrix between set X and Y, and size is N _x× N _y(N _x, N _ybe respectively X and Y and gather sample number), this patent Matrix C is obtained by neighbour figure.C _xand C _yfor relation diagonal matrix in the class in single space, its diagonal entry is respectively the cumulative sum of the corresponding row of Matrix C, and the cumulative sum of respective column.

Order $P=\left[\begin{array}{c}{P}_x\\ P_y\end{array}\right],$ $Z=\left[\begin{array}{cc}X& \\ & Y\end{array}\right],$ $\mathrm{\Ω}=\left[\begin{array}{cc}{C}_x& -C\\ {-C}^T& C_y\end{array}\right],$ Optimum solution can be obtained by the proper vector solving generalized eigenvalue Ep=λ Fp, wherein E=Z Ω Z ^t, F=ZZ ^t, p is the proper vector corresponding to eigenvalue λ, and P is corresponding D _cthe proper vector of individual minimal eigenvalue.According to P=[P _xp _y] ^tdefinition, can obtain corresponding to the transformation matrix p of data acquisition X _x, its size is D _x× D _c, corresponding to the transformation matrix P of data acquisition Y _y, its size is D _y× D _c.

6, described semi-supervised information is obtained by neighbour figure, and the correlation matrix C formula of semi-supervised information is:

$C_{\mathrm{ij}}=\left\{\begin{array}{cc}\exp -{\hat{D}}_{\mathrm{ij}},& (i,j)\∈(\mathrm{\Ξ}\∪{\mathrm{\Ξ}}_x\∪{\mathrm{\Ξ}}_y)\\ 0,& \mathrm{otherwise}\end{array}\right.---\left(4\right)$

In formula, calculate new set spacing matrix with Dijkstra shortest path first formula is:

${\hat{D}}_{\mathrm{ij}}=\underset{p,q}{\min }(D_{\mathrm{ip}}^x+D_{\mathrm{pq}}+D_{\mathrm{qj}}^y),x_p\∈X,y_q\∈Y---\left(5\right)$

In formula, $D_{\mathrm{ij}}^x=\left\{\begin{array}{cc}\frac{{\left|\right|x_i-x_j\left|\right|}^2}{{\mathrm{\σ}}_x^2},& (i,j)\∈{\mathrm{\Ξ}}_x\\ \∞,& \mathrm{otherwise}\end{array}\right.,$ $D_{\mathrm{ij}}^y=\left\{\begin{array}{cc}\frac{{\left|\right|x_i-x_j\left|\right|}^2}{{\mathrm{\σ}}_y^2},& (i,j)\∈{\mathrm{\Ξ}}_y\\ \∞,& \mathrm{otherwise}\end{array}\right.,$ $D_{\mathrm{ij}}=\left\{\begin{array}{c}0,(i,j)\∈\mathrm{\Ξ}\\ \∞,\mathrm{otherwise}\end{array}\right.,$ For 0 ° of visual angle and 90 ° of visual angles, and D _ijrespectively in the set of 0 ° of visual angle in the distance of two samples, 90 ° of visual angles set under the distance of two samples, 0 ° of visual angle and under 90 ° of visual angles the distance of same person be 0, Ξ _x, Ξ _yand Ξ represent respectively X set, Y set and these two set between neighborhood relationships set.

7, the test gait view angle theta in described identification is obtained by following formulae discovery:

θ＝arg min d(g ^test,μ _i) (6)

θ _i,1≤i≤N _f

Wherein g ^testrepresent the gait feature vector in test set, μ _ifor visual angle in training set ⁱunder gait feature vector; N _ffor look-out angle number in training set; D () is metric function, chooses Euclidean distance function here.

8, the nearest neighbor classifier in described identification is to individual sample classification, and find the gait feature of registered set minimum with test gait characteristic distance in coupled room exactly, formula is:

$k=\underset{1\≤j\≤N_g}{\arg \min d}(z_j^{\mathrm{gallary}},z^{\mathrm{test}})=\left|\right|P_x^{\mathrm{\θ}}{G}_j^{\mathrm{gallary}}-P_y^{\mathrm{\θ}}{g}^{\mathrm{test}}\left|\right|---\left(7\right)$

In formula, represent a jth gait feature of registered set, N _grepresent the number of samples of registered set, with coupling projection matrix pair is differentiated for training the optimum of standard viewing angle and the view angle theta obtained.Make the classification indicator function of function c () belonging to proper vector, finally determine that the category label of cycle tests is:

Embodiment

The semi-supervised coupling projection of the present invention based on figure realizes various visual angles Gait Recognition.Method divides three steps: the first step, adopts code book detection method from video flowing, obtain objective contour sequence, and make human body between two parties unified size be 64*64; Second step, after two field picture center for standard every in gait video sequence, the cyclical variation situation of the periodicity observation gait of people's degree that two legs are separated in walking, and adopt gait energygram (GEI) to extract the global feature of gait in one cycle; 3rd step, the off-line training step of the various visual angles Gait Recognition system built, choose a standard viewing angle (registration visual angle) gait feature, gait feature under all the other multiple visual angles is carried out joint training with standard viewing angle gait feature respectively, stores accordingly based on the semi-supervised coupling projection matrix pair of figure; In the ONLINE RECOGNITION stage, first test video is carried out to the extraction of objective contour, then to the profile Sequence Detection gait cycle of size normalization, generate monocyclic GEI feature.Finally, estimated the semi-supervised coupling projection matrix pair based on figure chosen by visual angle, project in the coupled in common gait feature space of similar arest neighbors, and adopt nearest neighbor classifier to carry out identity differentiation within this space.Said process as shown in Figure 1.Below in conjunction with accompanying drawing citing, the present invention is described in more detail:

1. everyone walking video under different visual angles in pair training set, by setting up code book model to foreground area cluster, draw foreground area, as shown in Figure 2, and normalized is for making human body contour outline placed in the middle, is 64 × 64 pixels by the unification of the size of image;

2. in pair training set, sample carries out cycle detection, according in figure after two field picture center for standard every in gait video flowing, and the cyclical variation situation of the periodicity observation gait of people's degree that two legs are separated in walking, cycle detection formula is:

$W=\frac{1}{(h_2-h_1+1)}\underset{i=h_1}{\overset{h_2}{\mathrm{\Σ}}}(R_i-L_i)---\left(1\right)$

H ₁and h ₂be respectively the ankle of people's (prospect) and the anthropometry height of knee in certain two field picture, Fig. 3 is anthropometry aspect ratio illustration.R _iand L _ibe belong to the Far Left of prospect and rightmost location of pixels in the i-th row respectively, Fig. 4 is cycle detection result curve, and Fig. 5 is the gait image sequence of one-period.

3. adopt gait energygram (GEI) to extract the global feature of gait, formula is:

$G(x,y)=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{B}_i(x,y)---\left(2\right)$

In formula, N is the frame number of the gait sequence of one-period, and i represents the time, and (x, y) represents two dimensional image plane coordinate, and Fig. 6 is the GEI under 0 °, 18 °, 36 °, 54 °, 72 °, 90 °, 108 °, 126 °, 144 °, 162 ° visual angles.

4. off-line training process: the GEI feature choosing a standard viewing angle, (this patent chooses 90 ° of visual angles as standard viewing angle), gait feature under all the other multiple visual angles (0 °, 18 °, 36 °, 54 °, 72 °, 90 °, 108 °, 126 °, 144 °, 162 °) is carried out joint training with standard viewing angle gait feature respectively, make the relevant gait feature of same a group traveling together under different visual angles (same visual angle) as far as possible close in coupled room, obtain the coupling projection matrix pair of corresponding arest neighbors.

The coupling projection matrix obtaining similar arest neighbors described in 4.1 is to P _xand P _ymethod be: carry out joint training for the gait feature under the gait feature under 90 ° of standard viewing angle and 0 ° of visual angle, gait feature under gait feature under 0 ° of visual angle and 90 ° of visual angles is projected in common coupling gait feature space respectively, makes same a group traveling together relevant gait feature at (same visual angle) under different visual angles as far as possible close in coupled room:

$\min J(P_x,P_y)=\min \mathrm{Tr}\left({\left[\begin{array}{c}{P}_x\\ P_y\end{array}\right]}^T\left[\begin{array}{cc}X& \\ & Y\end{array}\right]\left[\begin{array}{cc}{C}_x& -C\\ {-C}^T& C_y\end{array}\right]{\left[\begin{array}{cc}X& \\ & Y\end{array}\right]}^T\left[\begin{array}{c}{P}_x\\ P_y\end{array}\right]\right)---\left(3\right)$

In formula, X represents 90 ° of visual angle training sample set, and Y represents 0 ° of visual angle training sample set, and C is the correlationship matrix between set X and Y, and size is N _x× N _y(N _x, N _ybe respectively X and Y and gather sample number), this patent Matrix C is obtained by neighbour figure.C _xand C _yfor relation diagonal matrix in the class in single space, its diagonal entry is respectively the cumulative sum of the corresponding row of Matrix C, and the cumulative sum of respective column.

Order $P=\left[\begin{array}{c}{P}_x\\ P_y\end{array}\right],$ $Z=\left[\begin{array}{cc}X& \\ & Y\end{array}\right],$ $\mathrm{\Ω}=\left[\begin{array}{cc}{C}_x& -C\\ {-C}^T& C_y\end{array}\right],$ Optimum solution can be obtained by the proper vector solving generalized eigenvalue Ep=λ Fp, wherein E=Z Ω Z ^t, F=ZZ ^t, p is the proper vector corresponding to eigenvalue λ, and P is corresponding D _cthe proper vector of individual minimal eigenvalue.According to P=[P _xp _y] ^tdefinition, can obtain corresponding to the transformation matrix p of data acquisition X _x, its size is D _x× D _c, corresponding to the transformation matrix P of data acquisition Y _y, its size is D _y× D _c.

4.2 in semi-supervised learning, can think that the sample that there is neighbor relationships at sample space has identical category attribute.Traditional neighbour figure defines in singleton, therefore can not directly use, and between different sets, neighbour's expansion of correlation matrix can be completed by several step below:

Step 1: set up neighbour figure.Respectively neighborhood relationships is set up to the element of each set, builds the connection layout of X set and Y set, then build corresponding neighborhood relationships set Ξ with k neighbour _xand Ξ _y, to gather X, the define method of k neighbour is: given positive integer parameter k, for x _iif x _jits k neighbour, Ξ _x=Ξ _x∪ (i, j).Building the connection layout between two set according to existing supervision message, just connecting between the sample that there is restriction relation as only having in figure.Fig. 9 is the connection layout between two Set-dissection elements, and wherein, represents set X, and zero represents set Y.

According to Fig. 9, x in set X ₁with x ₂and x ₃there is neighbor relationships, therefore, have limit to connect, and and x ₄without neighbor relationships, then boundlessly to connect.Y in set Y ₁with y ₂and y ₃there is neighbor relationships, therefore, have limit to connect, and and y ₄without neighbor relationships, then boundlessly to connect.Between two set, x ₁with y ₁there is restriction relation, then there is limit between these two elements.

Step 2: the weights calculating limit.Define the Distance matrix D in each set ^xand D ^y, in order to describe the relation between two set, also needing the Distance matrix D between definition set, being defined as follows:

$D_{\mathrm{ij}}^x=\left\{\begin{array}{cc}\frac{{\left|\right|x_i-x_j\left|\right|}^2}{{\mathrm{\σ}}_x^2},& (i,j)\∈{\mathrm{\Ξ}}_x\\ \∞,& \mathrm{otherwise}\end{array}\right.,$ $D_{\mathrm{ij}}^y=\left\{\begin{array}{cc}\frac{{\left|\right|x_i-x_j\left|\right|}^2}{{\mathrm{\σ}}_y^2},& (i,j)\∈{\mathrm{\Ξ}}_y\\ \∞,& \mathrm{otherwise}\end{array}\right.,$ $D_{\mathrm{ij}}=\left\{\begin{array}{c}0,(i,j)\∈\mathrm{\Ξ}\\ \∞,\mathrm{otherwise}\end{array}\right.---\left(4\right)$

Then asked for the distance of any two elements between two set by " neighbour's transmission " rule of shortest path, adopt Dijkstra shortest path first to calculate new set spacing matrix here namely for x _ito y _jthe length of shortest path.

${\hat{D}}_{\mathrm{ij}}=\underset{p,q}{\min }(D_{\mathrm{ip}}^x+D_{\mathrm{pq}}+D_{\mathrm{qj}}^y),x_p\∈X,y_q\∈Y---\left(5\right)$

Such as from x ₂to y ₂a paths be

Correlation matrix C formula based on the semi-supervised information of figure is:

$C_{\mathrm{ij}}=\left\{\begin{array}{cc}\exp -{\hat{D}}_{\mathrm{ij}},& (i,j)\∈(\mathrm{\Ξ}\∪{\mathrm{\Ξ}}_x\∪{\mathrm{\Ξ}}_y)\\ 0,& \mathrm{otherwise}\end{array}\right.---\left(6\right)$

5. identification comprises the test determination at visual angle and the classification of test sample book, and process as shown in Figure 1.

5.1 test gait view angle theta are obtained by following formulae discovery:

θ＝arg min d(g ^test,μ _i) (7)

θ _i,1≤i≤N _f

Wherein g ^testrepresent the gait feature vector in test set, μ _ifor the gait feature vector under visual angle i in training set; N _ffor look-out angle number in training set; D () is metric function, chooses Euclidean distance function here.

Nearest neighbor classifier in 5.2 identifications is to individual sample classification, and find the gait feature of registered set minimum with test gait characteristic distance in coupled room exactly, formula is:

$k=\underset{1\≤j\≤N_g}{\arg \min d}(z_j^{\mathrm{gallary}},z^{\mathrm{test}})=\left|\right|P_x^{\mathrm{\θ}}{G}_j^{\mathrm{gallary}}-P_y^{\mathrm{\θ}}{g}^{\mathrm{test}}\left|\right|---\left(8\right)$

In formula, represent a jth gait feature of registered set, N _grepresent the number of samples of registered set, with coupling projection matrix pair is differentiated for training the optimum of standard viewing angle and the view angle theta obtained.Make the classification indicator function of function c () belonging to proper vector, finally determine that the category label of cycle tests is:

Claims (8)

1., based on a multi-angle gait recognition method for the semi-supervised Coupling Metric of figure, it is characterized in that:

(1) adopt code book detection method from video flowing, obtain objective contour sequence, and make human body between two parties unified size be 64*64;

(2) according to after two field picture center for standard every in gait video sequence, the cycle of people's periodic statistics gait of the degree that two legs are separated in walking, and adopt gait energygram to extract the global feature of gait in one cycle;

(3) build the off-line training step of various visual angles Gait Recognition system, train the semi-supervised coupling projection matrix pair obtained based on figure; Test video is carried out to the extraction of objective contour, to the profile Sequence Detection gait cycle of size normalization, generate monocyclic gait energygram feature, the semi-supervised coupling projection matrix pair based on figure chosen is estimated by visual angle, project in coupled in common gait feature space, and adopt nearest neighbor classifier to carry out identity differentiation in coupled in common gait feature space.

2. the multi-angle gait recognition method of a kind of semi-supervised Coupling Metric based on figure according to claim 1, it is characterized in that: described step (2) is according in figure after two field picture center for standard every in gait video flowing, the cyclical variation situation of people's periodic statistics gait of the degree that two legs are separated in walking, the cycle is:

$W=\frac{1}{(h_2-h_1+1)}\underset{i=h_1}{\overset{h_2}{\mathrm{\Σ}}}(R_i-L_i)$

H ₁and h ₂be respectively the ankle of people's (prospect) and the anthropometry height of knee in certain two field picture, R _iand L _iin the i-th row, belong to the Far Left of prospect and rightmost location of pixels respectively.

3. the multi-angle gait recognition method of the semi-supervised Coupling Metric based on figure according to claim 1, is characterized in that: feature extraction adopts gait energygram to extract the global feature of gait:

$G(x,y)=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{B}_i(x,y)$

N is the frame number of the gait sequence of one-period, and i represents the time, and (x, y) represents two dimensional image plane coordinate.

4. the multi-angle gait recognition method of a kind of semi-supervised Coupling Metric based on figure according to claim 3, it is characterized in that: the standard viewing angle of described gait energygram chooses 90 ° of visual angles, and the gait feature at all the other visual angles is carried out joint training with standard viewing angle gait feature respectively.

5. the multi-angle gait recognition method of a kind of semi-supervised Coupling Metric based on figure according to claim 1, is characterized in that: described obtain semi-supervised coupling projection matrix based on figure to P _xand P _ymethod be: carry out joint training with the gait feature under the gait feature under 90 ° of standard viewing angle and 0 ° of visual angle, the gait feature under the gait feature under 0 ° of visual angle and 90 ° of visual angles projected to respectively in common coupling gait feature space:

$\min J(P_x,P_y)=\min \mathrm{Tr}\left({\left[\begin{array}{c}{P}_x\\ P_y\end{array}\right]}^T\left[\begin{array}{cc}X& \\ & Y\end{array}\right]\left[\begin{array}{cc}{C}_x& -C\\ -C^T& C_y\end{array}\right]{\left[\begin{array}{cc}X& \\ & Y\end{array}\right]}^T\left[\begin{array}{c}{P}_x\\ P_y\end{array}\right]\right)$

In formula, X represents 90 ° of visual angle training sample set, and Y represents 0 ° of visual angle training sample set, and C is the correlationship matrix between set X and Y, and size is N _x× N _y, N _x, N _ybe respectively X and Y and gather sample number, Matrix C is obtained by neighbour figure, C _xand C _yfor relation diagonal matrix in the class in single space, its diagonal entry is respectively the cumulative sum of the corresponding row of Matrix C, and the cumulative sum of respective column;

Order $P=\left[\begin{array}{c}{P}_x\\ P_y\end{array}\right],Z=\left[\begin{array}{cc}X& \\ & Y\end{array}\right],\mathrm{\Ω}=\left[\begin{array}{cc}{C}_x& -C\\ -C^T& C_y\end{array}\right],$ Optimum solution is obtained by the proper vector solving generalized eigenvalue Ep=λ Fp, wherein E=Z Ω Z ^t, F=ZZ ^t, p is the proper vector corresponding to eigenvalue λ, and P is corresponding D _cthe proper vector of individual minimal eigenvalue, P=[P _xp _y] ^tdefinition, obtain corresponding to the transformation matrix p of data acquisition X _x, size is D _x× D _c, corresponding to the transformation matrix P of data acquisition Y _y, size is D _y× D _c.

6. the multi-angle gait recognition method of a kind of semi-supervised Coupling Metric based on figure according to claim 5, is characterized in that: described correlation matrix C is:

$C_{\mathrm{ij}}=\left\{\begin{array}{cc}\exp -{\hat{D}}_{\mathrm{ij}},& (i,j)\∈(\mathrm{\Ξ}\∪{\mathrm{\Ξ}}_x\∪{\mathrm{\Ξ}}_y)\\ 0,& \mathrm{otherwise}\end{array}\right.$

In formula, calculate new set spacing matrix with Dijkstra shortest path first

${\hat{D}}_{\mathrm{ij}}=\underset{p,q}{\min }(D_{\mathrm{ip}}^x+D_{\mathrm{pq}}+D_{\mathrm{qj}}^y),x_p\∈X,y_q\∈Y$

In formula, $D_{\mathrm{ij}}^x=\left\{\begin{array}{cc}\frac{{\left|\right|x_i-x_j\left|\right|}^2}{{\mathrm{\σ}}_x^2},& (i,j)\∈{\mathrm{\Ξ}}_x\\ \∞,& \mathrm{otherwise}\end{array}\right.,$ $D_{\mathrm{ij}}^y=\left\{\begin{array}{cc}\frac{{\left|\right|y_i-y_j\left|\right|}^2}{{\mathrm{\σ}}_y^2},& (i,j)\∈{\mathrm{\Ξ}}_y\\ \∞,& \mathrm{otherwise}\end{array}\right.,$ $D_{\mathrm{ij}}\left\{\begin{array}{c}0,(i,j)\∈\mathrm{\Ξ}\\ \∞,\mathrm{otherwise}\end{array}\right.,$ and D _ijrespectively in the set of 0 ° of visual angle in the distance of two samples, 90 ° of visual angles set under the distance of two samples, 0 ° of visual angle and under 90 ° of visual angles the distance of same person be 0, Ξ _x, Ξ _yand Ξ represent respectively X set, Y set and these two set between neighborhood relationships set.

7. the multi-angle gait recognition method of a kind of semi-supervised Coupling Metric based on figure according to claim 1, is characterized in that: belonging to described gait feature, view angle theta is:

$\mathrm{\θ}=\underset{{\mathrm{\θ}}_i,1\≤i\≤N_f}{\arg \min d}(g^{\mathrm{test}},{\mathrm{\μ}}_i)$

Wherein g ^testrepresent the gait feature vector in test set, μ _ifor the gait feature vector under visual angle i in training set; N _ffor look-out angle number in training set; D () is metric function, chooses Euclidean distance function.

8. the multi-angle gait recognition method of a kind of semi-supervised Coupling Metric based on figure according to claim 1, is characterized in that: described nearest neighbor classifier, refers to find the gait feature of registered set minimum with test gait characteristic distance in coupled room:

$k=\underset{1\≤j{\≤N}_g}{\arg \min d}(z_j^{\mathrm{gallary}},z^{\mathrm{test}})=\left|\right|P_x^{\mathrm{\θ}}{G}_j^{\mathrm{gallary}}-P_y^{\mathrm{\θ}}{g}^{\mathrm{test}}\left|\right|$

In formula, represent a jth gait feature of registered set, N _grepresent the number of samples of registered set, with for training the optimum of standard viewing angle and the view angle theta obtained to differentiate coupling projection matrix pair, making the classification indicator function of function c () belonging to proper vector, determining that the category label of cycle tests is: