CN107085716A - Across the visual angle gait recognition method of confrontation network is generated based on multitask - Google Patents
Across the visual angle gait recognition method of confrontation network is generated based on multitask Download PDFInfo
- Publication number
- CN107085716A CN107085716A CN201710373017.5A CN201710373017A CN107085716A CN 107085716 A CN107085716 A CN 107085716A CN 201710373017 A CN201710373017 A CN 201710373017A CN 107085716 A CN107085716 A CN 107085716A
- Authority
- CN
- China
- Prior art keywords
- msub
- mrow
- gait
- visual angle
- exemplary feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/49—Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/285—Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Abstract
The invention belongs to computer vision and machine learning field, specially a kind of across visual angle gait recognition method that confrontation network is generated based on multitask.Present invention mainly solves Gait Recognition the problem of big visual angle change drag Generalization Capability is reduced.For original pedestrian's sequence of frames of video, every two field picture is pre-processed first, and extracts gait exemplary feature；Then the hidden expression of gait is encoded to by neutral net, and carries out in latent space angular transformation；The gait exemplary feature that confrontation network reconfiguration goes out other visual angles is generated by multitask again；Finally it is identified using the hidden expression of gait.Compared to based on classification or other methods based on reconstruct, the present invention has stronger interpretation, and can lift recognition performance.
Description
Technical field
The invention belongs to computer vision, machine learning techniques field, and in particular to across the visual angle gait based on video is known
Other method.
Background technology
One of the problem of across visual angle Gait Recognition problem based on video is computer vision and machine learning area research.
Gait sequence of frames of video under given different visual angles, it is desirable to which gait frame sequence is judged according to computer vision or machine learning algorithm
Whether the main body of row is same object.There have been many previous works in the current field, and it is big that its main method can be divided into three
Class：Method based on reconstruct, the method based on subspace and the method based on deep learning.The following is some of this three classes method
Bibliography：
[1]W.Kusakunniran,Q.Wu,J.Zhang,and H.Li,“Support vector regression
for multi-view gait recognition based on local motion feature selection,”in
Conference on Computer Vision and Pattern Recognition,pp.974–981,2010.
[2]M.Hu,Y.Wang,Z.Zhang,J.J.Little,and D.Huang,“View-invariant
discriminative projection for multi-view gait-based human identification,”
IEEE Transactions on Information Forensics and Security,vol.8,no.12,pp.2034–
2045,2013.
[3]W.Kusakunniran,Q.Wu,J.Zhang,H.Li,and L.Wang,“Recognizing gaits
across views through correlated motion co-clustering,”IEEE Transactions on
Image Processing,vol.23,no.2,pp.696–709,2014.
[4]S.Yu,H.Chen,Q.Wang,L.Shen,and Y.Huang,“Invariant feature
extraction for gait recognition using only one uniform model,”Neurocomputing,
vol.239,pp.81–93,2017.
[5]Z.Wu,Y.Huang,L.Wang,X.Wang,and T.Tan,“A comprehensive study on
cross-view gait based human identification with deep CNNs,”IEEE Transactions
on Pattern Analysis and Machine Intelligence,vol.39,no.2,pp.209–226,2017.
[6]Y.Feng,Y.Li,and J.Luo,“Learning effective gait features using
LSTM,”in International Conference on Pattern Recognition,pp.320–325,2016.
[7]H.Iwama,M.Okumura,Y.Makihara,and Y.Yagi,“The OU-ISIR gait database
comprising the large population dataset and performance evaluation of gait
recognition,”IEEE Transactions on Information Forensics and Security,vol.7,
no.5,pp.1511–1521,2012.
[8]Y.Makihara,R.Sagawa,Y.Mukaigawa,T.Echigo,and Y.Yagi,“Gait
recognition using a view transformation model in the frequency domain,”in
European Conference on Computer Vision,pp.151–163,Springer,2006.
[9]W.Kusakunniran,Q.Wu,H.Li,and J.Zhang,“Multiple views gait
recognition using view transformation model based on optimized gait energy
image,”in International Conference on Computer Vision,pp.1058–1064,2009.
[10]X.Xing,K.Wang,T.Yan,and Z.Lv,“Complete canonical correlation
analysis with application to multi-view gait recognition,”Pattern
Recognition,vol.50,pp.107–117,2016.。
The first method based on reconstruct is main based on reconstructing gait masterplate, such as the VTM moulds in [1,3,8,9]
Type, this method predicts the value of each pixel in exemplary feature under aspect, this method by training multiple models, respectively
Big, the low problem of recognition accuracy with computing cost.In order to reduce computing cost, the method [4] based on self-encoding encoder is come same
When reconstruct each pixel of exemplary feature, and extract the feature representation of unchanged view angle.
The gait feature of different visual angles is projected to a public subspace by the method [2,10] based on subspace, and in public affairs
Altogether similarity is calculated in subspace.But subspace method is typically only capable to, the linear dependence modeling feature, have ignored non-
Linear dependence, discrimination is relatively low.
Recently, the model based on deep neural network has obtained larger application in computer vision field.[5,6] will be deep
Spending network is used for Gait Recognition, and the feature representation of unchanged view angle has been extracted automatically.Although compared with previous methods on recognition accuracy
Larger lifting has been obtained, but depth model lacks interpretation.
The content of the invention
It is an object of the invention to provide across the visual angle Gait Recognition based on video that a kind of discrimination height, computing cost are saved
Method.
The relevant substance of the present invention is introduced first.
First, periodical energy figure
(a) in order to set up more effective gait exemplary feature, it is necessary to be pre-processed to original pedestrian's video sequence.It is right
Each frame in video sequence, the separation of background, extracts gait profile diagram, afterwards occupies gait profile diagram before carrying out first
In and be aligned to picture center, the gait profile diagram standardized；
(b) the gait profile graphic sequence based on standardization, is obtained often first by the cycle detection technology in Gait Recognition
The period position of the standardization of one frame.A gait cycle is defined between two adjacent Local Minimums standardization period positions
Time-domain.Afterwards, profile graphic sequence is divided into one or more gait cycles, and discarding can not constitute a complete gait week
The frame of phase.According to the gait profile diagram sequence B of centralizationtWith the period position r of corresponding standardizationt, construction is with ncIndividual passage
Periodical energy figure (PEI) exemplary feature.Value in periodical energy figure PEI at (x, y) coordinate of k-th of passage is counted as the following formula
Calculate:
Wherein：
M represents the time-domain window size that each passage is covered, and T (k) determines the time-domain of each passage covering
Scope.By this method, the spatial information of profile graphic sequence can be divided to by n according to the period position of each framecThe individual time
On domain, to encode time and the spatial information of gait simultaneously with this.
2nd, multitask generation confrontation network
The multitask generation confrontation network for Gait Recognition of the present invention is as shown in figure 1, main become by encoder, visual angle
Change layer, maker and the part of arbiter four composition.Wherein：
(a) encoder E：In order to obtain the hidden expression of certain viewing angles, using convolutional neural networks as model in coding
Device.The input x of encoderuIt is the PEI masterplates that visual angle proposed by the present invention is u.The scale size of input is 64 × 64 × nc。PEI
Each passage of masterplate use time pond and is extracted effectively by independent feeding encoder, afterwards after full articulamentum
The hidden expression z of featureuIt is used as the output of encoder.For time pond, the mode used is average pond；
(b) view transformation layer V：It is assumed that gait data is as the change profile at visual angle is in the manifold of a higher-dimension.Manifold
In sample the change at visual angle can be realized under conditions of identity information is kept towards specific direction movement.Given latent space
Hidden expression zu, view transformation can be described as：
Wherein, hiRepresent the conversion vector by visual angle i-1 to visual angle i.zvIt is the hidden expression under the v of visual angle；
(c) maker G：By hidden expression zvAs input, the gait masterplate under the v of visual angle can be generated.Maker
It is made up of five warp laminations, and uses ReLU as excitation function.Because directly reconstruct PEI masterplates have certain difficulty,
Therefore, a passage of random selection PEI masterplates is reconstructed, and ensures hidden to represent that the letter of all passages can be preserved with this
Breath.The input for defining maker is [zv, c], c represents for the one-hot coding of passage；
(d) arbiter D：Dimension and encoder of the framework of arbiter except not using the operation of time pondization and output layer
Different outer, remaining structure is identical with encoder.The output dimension for defining arbiter is nv+nc+nd, wherein nvIt is the quantity at visual angle,
ncIt is the quantity of PEI passages, ndIt is the quantity of different identity in training set.The output correspondence n of arbiterv+nc+ndHeight differentiates
The parameter in addition to last layer is shared between device, sub- arbiter.Also, each arbiter is responsible for different differentiation tasks
Judge whether that the sample of generation belongs to specific distribution.For example, in a model, preceding nv+ncIndividual sub- arbiter enables maker
Generation meets the distribution of certain viewing angles and passage, then ndIndividual sub- arbiter can make maker generation belong to dividing for specific identity
Sample corresponding to cloth.
3rd, loss function
The present invention trains multitask generation confrontation network using two kinds of loss functions.
(a) lose pixel-by-pixel：In order to strengthen the hidden ability for representing to keep identity information, generation template is minimized first and true
Loss pixel-by-pixel between real masterplate.According to exemplary feature xvWith pseudo- exemplary featureCalculating loss pixel-by-pixel is：
(b) multitask confrontation loss：According to exemplary feature xvWith pseudo- exemplary featureCalculate multitask confrontation loss：
Wherein, E represents the expectation for correspondence sample set, | | | |1Represent L1 norms, vectorial s be identity, passage with
The one-hot coding of angle information.D () represents the output of arbiter, and the dimension of arbiter output is identical with vector s dimension, to
Nonzero element in amount s determines the distribution that pseudo- exemplary feature should belong to；
Defining final loss function is：
L=Lp+αLa (6)
α is used for weighing loss pixel-by-pixel as hyper parameter and lost with multitask confrontation, defines after final loss function,
Use the parameter of Back Propagation Algorithm alternating more new encoder, view transformation layer, maker and arbiter.
Across the visual angle gait recognition method proposed by the present invention that confrontation network is generated based on multitask, is concretely comprised the following steps：
(1) pedestrian's sequence of frames of video under input different visual angles, constructs gait exemplary feature：
Vector xiIt is the gait exemplary feature under the i of visual angle, nvIt is the quantity at all visual angles；
(2) for any different visual angle u, v, corresponding gait exemplary feature x is encoded using convolutional neural networksuIt is extremely hidden
Space, and obtain in latent space hidden be expressed as zu；
(3) to hidden expression z in latent spaceuView transformation is carried out, angle v is converted into, obtains hidden expression zv；
(4) by hidden expression zvAnd the one-hot coding of passage generates the life in confrontation network as input by multitask
Pseudo- exemplary feature after into network under output angle v
(5) according to exemplary feature xuWith pseudo- exemplary featureL is lost in calculating pixel-by-pixelp；
(6) the differentiation network in confrontation network is generated using multitask, according to exemplary feature xuWith pseudo- exemplary featureMeter
Calculate multitask confrontation loss La；
(7) loss resists loss with multitask pixel-by-pixel for weighting, according to total losses L=Lp+αLaTrain multitask generation pair
Anti- network, α is used for weighing loss pixel-by-pixel as hyper parameter and lost with multitask confrontation.
In the present invention, the construction step of described gait masterplate is：
(1) preceding background separation is carried out to each frame of original video frame sequence, extracts gait profile diagram；And by gait wheel
Wide figure translation zooms to picture centre, obtains the gait profile graphic sequence { B of centralization1,B2,B3,…,Bn}；
(2) to each frame, the period position r of normalized in the gait profile graphic sequence of centralizationt, represent centralization
Gait profile diagram t frames standardization period position；
(3) according to the gait profile diagram sequence B of centralizationtWith the period position r of corresponding standardizationt, construction is with ncIt is individual
The periodical energy figure exemplary feature of passage；Value in periodical energy figure PEI at (x, y) coordinate of k-th of passage is calculated as follows:
Wherein：
M represents the time-domain window size that each passage is covered, and T (k) determines the time-domain of each passage covering
Scope.
In the present invention, described view transformation step is：
Hidden expression z under given visual angle uu, transforming to visual angle v calculating process can be described as：
Wherein, hiRepresent the conversion vector by visual angle i-1 to visual angle i.
In the present invention, described loss LpWith LaCalculation procedure be：
(1) according to exemplary feature xvWith pseudo- exemplary featureCalculating is lost pixel-by-pixel：
(2) according to exemplary feature xvWith pseudo- exemplary featureCalculate multitask confrontation loss：
Wherein, E represents the expectation for correspondence sample set, | | | |1Represent L1 norms, vectorial s be identity, passage with
The one-hot coding of angle information.D () represents the output of arbiter.
The inventive method uses the Nonlinear Modeling ability of convolutional neural networks, is extracted the specific hidden expression in visual angle.It is logical
Cross and view transformation is carried out in latent space, reduce computing cost；And the ability of distribution modeling is carried based on generation confrontation network
The feature with more expressiveness has been taken out, recognition efficiency is greatly improved.
Brief description of the drawings
Fig. 1：The detailed model flow chart of the present invention.
Fig. 2：OU-ISIR, CASIA-B, USF data set sample are shown.
Fig. 3:Average Accuracy under different walking states on CASIA-B data sets.
Embodiment
After the specific steps and model that describe the present invention, test of the invention in several gait data collection is shown below
Effect.
Experiment employs three data sets, including OU-ISIR data sets, CASIA-B data sets and USF data sets.Fig. 2
Illustrate some samples of these three data sets.
OU-ISIR data sets one have 4007 different people, wherein male 2135, women 1872, age distribution
Scope was from 1 years old to 94.The data have 4 different angles.Respectively 55 °, 65 °, 75 °, 85 °.Carried using the method for invention
PEI masterplates are taken out, and set port number to be 3, after the picture of PEI interpolation to 64 × 64 pixel sizes, multitask generation are put into
It is trained in confrontation network.
CASIA-B data sets one have 124 different people, 11 different visual angles.Wherein, everyone is at each visual angle
On have 6 groups of gait sequences normally walked, 2 groups are carried the gait sequence that packet rows are walked, 2 groups of gait sequences of overcoat for wearing.
CASIA-B has bigger angular field of view compared to OU-ISIR, but number is relatively fewer.We set PEI port number to be 10,
The Gait Recognition accuracy rate tested respectively under different walking states.
USF is another conventional gait data collection, and one has 122 different people, everyone have 5 kinds it is various forms of
Gait sequence, close to real scene.In an experiment, we are only tested using the gait sequence under different visual angles.I
Set PEI port number be 10.
Experiment uses Rank-1 recognition accuracies as performance indications.By being used most in the latent space of view transformation layer
Nearest Neighbor Classifier is identified.
Experimental example 1：The recognition performance of multitask generation confrontation network
This part Experiment illustrates different models, across recognition accuracy under visual angle.Method as a comparison, we select
Self-encoding encoder, canonical correlation analysis, linear discriminant analysis, convolutional neural networks and local tensors discrimination model.Table 1 is illustrated
Method of the invention and other method compare on three data sets.It can be seen that, the present invention has very big compared to other method
Lifting.
Experimental example 2：Influence of the different loss functions to model performance
When table 2 is illustrated using different loss functions, performance change of the model on CASIA-B data sets.It can be seen that,
Lost with reference to multitask confrontation and loss can be with the recognition performance of lift scheme pixel-by-pixel；And different loss functions are used alone
When, the performance of model can be reduced.
Experimental example 3：Influence of the different walking states to model performance
Fig. 3 shows across the visual angle recognition accuracy on CASIA-B under different walking states, and one has three kinds of different rows
Walk state：Normal walking, carries packet rows and walks, and wears overcoat walking.From the figure, it can be seen that accuracy rate during normal walking is most
Height, the gait sequence for wearing overcoat is more notable compared to reduction of the gait sequence of bag to model performance is carried.
Experimental example 4：Influence of different masterplate and the PEI port numbers to recognition accuracy
Table 3 illustrates the influence to recognition accuracy using different masterplates and PEI port numbers.We use gait energy
Scheme (GEI) and time step morphotype version (CGI), and analyze average recognition accuracy when PEI port numbers distinguish 3,5,10.Can be with
See, the PEI masterplates that we invent compare 6EI and C6I, there is higher recognition accuracy, and with the increasing of PEI masterplate numbers
Plus, recognition accuracy can be lifted further.
Table 1：Recognition accuracy (%) under distinct methods
Table 2：Model Identification accuracy rate (%) under different loss functions
| 54° | 90° | 126° | |
| Loss+multitask confrontation loss pixel-by-pixel | 82.4 | 73.1 | 81.3 |
| Lose pixel-by-pixel | 81.5 | 71.7 | 83.5 |
| Multitask confrontation loss | 74.6 | 68.6 | 75.4 |
The influence of table 3 different masterplates and PEI port numbers to recognition accuracy
Claims (5)
1. across the visual angle gait recognition method of confrontation network is generated based on multitask, it is characterised in that concretely comprise the following steps：
(1) pedestrian's sequence of frames of video under input different visual angles, constructs gait exemplary feature：
<mrow>
<mi>X</mi>
<mo>=</mo>
<mo>{</mo>
<msub>
<mi>x</mi>
<mn>1</mn>
</msub>
<mo>,</mo>
<msub>
<mi>x</mi>
<mn>2</mn>
</msub>
<mo>,</mo>
<msub>
<mi>x</mi>
<mn>3</mn>
</msub>
<mo>,</mo>
<mo>...</mo>
<mo>,</mo>
<msub>
<mi>x</mi>
<msub>
<mi>n</mi>
<mi>v</mi>
</msub>
</msub>
<mo>}</mo>
</mrow>
Vector xiIt is the gait exemplary feature under the i of visual angle, nvIt is the quantity at all visual angles；
(2) for any different visual angle u, v, corresponding gait exemplary feature x is encoded using convolutional neural networksuTo latent space,
And obtain in latent space hidden be expressed as zu；
(3) to hidden expression z in latent spaceuView transformation is carried out, angle v is converted into, obtains hidden expression zv；
(4) by hidden expression zvAnd the one-hot coding of passage generates the generation network in confrontation network as input by multitask
Pseudo- exemplary feature under output angle v afterwards
(5) according to exemplary feature xuWith pseudo- exemplary featureL is lost in calculating pixel-by-pixelp；
(6) the differentiation network in confrontation network is generated using multitask, according to exemplary feature xuWith pseudo- exemplary featureCalculate many
Task confrontation loss La；
(7) loss resists loss with multitask pixel-by-pixel for weighting, according to total losses L=Lp+αLaTrain multitask generation confrontation net
Network.
2. gait recognition method according to claim 1, it is characterised in that the construction step of described gait masterplate is：
(1) preceding background separation is carried out to each frame of original video frame sequence, extracts gait profile diagram；And by gait profile diagram
Translation zooms to picture centre, obtains the gait profile graphic sequence { B of centralization1,B2,B3,…,Bn}；
(2) to each frame, the period position r of normalized in the gait profile graphic sequence of centralizationt, represent the step of centralization
The period position of the standardization of the t frames of state profile diagram；
(3) according to the gait profile diagram sequence B of centralizationtWith the period position r of corresponding standardizationt, construction is with ncIndividual passage
Periodical energy figure (PEI) exemplary feature；Value in periodical energy figure at (x, y) coordinate of k-th of passage is calculated as follows:
<mrow>
<msub>
<mi>PEI</mi>
<mi>k</mi>
</msub>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<mfrac>
<mn>1</mn>
<mi>n</mi>
</mfrac>
<munder>
<mo>&Sigma;</mo>
<mrow>
<msub>
<mi>r</mi>
<mi>t</mi>
</msub>
<mo>&Element;</mo>
<mi>T</mi>
<mrow>
<mo>(</mo>
<mi>k</mi>
<mo>)</mo>
</mrow>
</mrow>
</munder>
<msub>
<mi>B</mi>
<mi>t</mi>
</msub>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
</mrow>
Wherein：
<mrow>
<mi>T</mi>
<mrow>
<mo>(</mo>
<mi>k</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<mo>&lsqb;</mo>
<mfrac>
<mrow>
<mi>k</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
<mrow>
<msub>
<mi>n</mi>
<mi>c</mi>
</msub>
<mo>+</mo>
<mn>1</mn>
</mrow>
</mfrac>
<mfrac>
<mrow>
<mi>k</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
<mrow>
<msub>
<mi>n</mi>
<mi>c</mi>
</msub>
<mo>+</mo>
<mn>1</mn>
</mrow>
</mfrac>
<mo>+</mo>
<mi>m</mi>
<mo>&rsqb;</mo>
</mrow>
M represents the time-domain window size that each passage is covered, and T (k) determines the scope of the time-domain of each passage covering.
3. gait recognition method according to claim 1, it is characterised in that described view transformation step is：
Hidden expression z under given visual angle uu, transform to the hidden expression z under the v of visual anglevCalculating process be：
<mrow>
<msub>
<mi>z</mi>
<mi>v</mi>
</msub>
<mo>=</mo>
<msub>
<mi>z</mi>
<mi>u</mi>
</msub>
<mo>+</mo>
<munderover>
<mo>&Sigma;</mo>
<mrow>
<mi>i</mi>
<mo>=</mo>
<mi>u</mi>
</mrow>
<mrow>
<mi>v</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
</munderover>
<msub>
<mi>h</mi>
<mi>i</mi>
</msub>
</mrow>
Wherein, hiRepresent the conversion vector by visual angle i-1 to visual angle i.
4. gait recognition method according to claim 1, it is characterised in that described loss LpWith LaCalculation procedure be：
(1) according to exemplary feature xvWith pseudo- exemplary featureCalculating is lost pixel-by-pixel：
<mrow>
<msub>
<mi>L</mi>
<mi>p</mi>
</msub>
<mo>=</mo>
<mi>E</mi>
<mo>|</mo>
<mo>|</mo>
<msub>
<mi>x</mi>
<mi>u</mi>
</msub>
<mo>-</mo>
<msub>
<mover>
<mi>x</mi>
<mo>^</mo>
</mover>
<mi>v</mi>
</msub>
<mo>|</mo>
<msub>
<mo>|</mo>
<mn>1</mn>
</msub>
</mrow>
(2) according to exemplary feature xvWith pseudo- exemplary featureCalculate multitask confrontation loss：
<mrow>
<msub>
<mi>L</mi>
<mi>a</mi>
</msub>
<mo>=</mo>
<mi>E</mi>
<mo>&lsqb;</mo>
<msup>
<mi>s</mi>
<mi>T</mi>
</msup>
<mi>log</mi>
<mi> </mi>
<mi>D</mi>
<mrow>
<mo>(</mo>
<msub>
<mi>x</mi>
<mi>v</mi>
</msub>
<mo>)</mo>
</mrow>
<mo>&rsqb;</mo>
<mo>+</mo>
<mi>E</mi>
<mo>&lsqb;</mo>
<msup>
<mi>s</mi>
<mi>T</mi>
</msup>
<mi>l</mi>
<mi>o</mi>
<mi>g</mi>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>-</mo>
<mi>D</mi>
<mo>(</mo>
<msub>
<mover>
<mi>x</mi>
<mo>^</mo>
</mover>
<mi>v</mi>
</msub>
<mo>)</mo>
<mo>)</mo>
</mrow>
<mo>&rsqb;</mo>
</mrow>
Wherein, E represents the expectation for correspondence sample set, | | | |1L1 norms are represented, vectorial s is identity, passage and angle
The one-hot coding of information.D () represents the output of arbiter.
5. gait recognition method according to claim 1, it is characterised in that the multitask generation confrontation network it is main by
Encoder, view transformation layer, maker and the part of arbiter four composition；Wherein：
(a) encoder：Encoder in using convolutional neural networks as model, the input x of encoderuIt is the PEI that visual angle is u
Masterplate；The scale size of input is 64 × 64 × nc；Each passage of PEI masterplates is made afterwards by independent feeding encoder
Extract that effective feature is hidden to represent z with time pond and after full articulamentumuIt is used as the output of encoder；
(b) view transformation layer：It is assumed that gait data is as the change profile at visual angle is in the manifold of a higher-dimension, the sample in manifold
This is towards specific direction movement to keep realizing under conditions of identity information the change at visual angle；The hidden expression of given latent space
zu, view transformation can be described as：
<mrow>
<msub>
<mi>z</mi>
<mi>v</mi>
</msub>
<mo>=</mo>
<msub>
<mi>z</mi>
<mi>u</mi>
</msub>
<mo>+</mo>
<msubsup>
<mi>&Sigma;</mi>
<mrow>
<mi>i</mi>
<mo>=</mo>
<mi>u</mi>
</mrow>
<mrow>
<mi>v</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
</msubsup>
<msub>
<mi>h</mi>
<mi>i</mi>
</msub>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>3</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, hiRepresent the conversion vector by visual angle i-1 to visual angle i, zvIt is the hidden expression under the v of visual angle；
(c) maker：By hidden expression zvAs input, the gait masterplate under the v of visual angle is generated；Maker is by five warps
Lamination is constituted, and uses ReLU as excitation function；One passage of random selection PEI masterplates is reconstructed, and is ensured with this
Hidden expression can preserve the information of all passages；The input for defining maker is [zv, c], c represents for the one-hot coding of passage；
(d) arbiter：Except not using, time pondization is operated the framework of arbiter and the dimension of output layer is different with encoder
Outside, remaining structure is identical with encoder；The output dimension for defining arbiter is nv+nc+nd, wherein nvIt is the quantity at visual angle, ncIt is
The quantity of PEI passages, ndIt is the quantity of different identity in training set；The output correspondence n of arbiterv+nc+ndIndividual sub- arbiter, son
The parameter in addition to last layer is shared between arbiter；Also, each arbiter is responsible for different differentiation tasks to judge
The sample whether generated belongs to specific distribution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710373017.5A CN107085716B (en) | 2017-05-24 | 2017-05-24 | Cross-view gait recognition method based on multi-task generation countermeasure network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710373017.5A CN107085716B (en) | 2017-05-24 | 2017-05-24 | Cross-view gait recognition method based on multi-task generation countermeasure network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107085716A true CN107085716A (en) | 2017-08-22 |
| CN107085716B CN107085716B (en) | 2021-06-04 |
Family
ID=59607349
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710373017.5A Active CN107085716B (en) | 2017-05-24 | 2017-05-24 | Cross-view gait recognition method based on multi-task generation countermeasure network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107085716B (en) |
Cited By (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107563428A (en) * | 2017-08-25 | 2018-01-09 | 西安电子科技大学 | Classification of Polarimetric SAR Image method based on generation confrontation network |
| CN107609587A (en) * | 2017-09-11 | 2018-01-19 | 浙江工业大学 | A kind of multi-class multi views data creation method that confrontation network is generated based on depth convolution |
| CN107993210A (en) * | 2017-11-30 | 2018-05-04 | 北京小米移动软件有限公司 | Image repair method, device and computer-readable recording medium |
| CN108009568A (en) * | 2017-11-14 | 2018-05-08 | 华南理工大学 | A kind of pedestrian detection method based on WGAN models |
| CN108171701A (en) * | 2018-01-15 | 2018-06-15 | 复旦大学 | Conspicuousness detection method based on U networks and confrontation study |
| CN108256627A (en) * | 2017-12-29 | 2018-07-06 | 中国科学院自动化研究所 | The mutual generating apparatus of audio-visual information and its training system that generation network is fought based on cycle |
| CN108491380A (en) * | 2018-03-12 | 2018-09-04 | 苏州思必驰信息科技有限公司 | Confrontation multitask training method for speech understanding |
| CN108596026A (en) * | 2018-03-16 | 2018-09-28 | 中国科学院自动化研究所 | Across the visual angle Gait Recognition device and training method of confrontation network are generated based on double fluid |
| CN108629823A (en) * | 2018-04-10 | 2018-10-09 | 北京京东尚科信息技术有限公司 | The generation method and device of multi-view image |
| CN108846355A (en) * | 2018-06-11 | 2018-11-20 | 腾讯科技（深圳）有限公司 | Image processing method, face identification method, device and computer equipment |
| CN109543546A (en) * | 2018-10-26 | 2019-03-29 | 复旦大学 | The gait age estimation method returned based on the distribution of depth sequence |
| CN109583298A (en) * | 2018-10-26 | 2019-04-05 | 复旦大学 | Across visual angle gait recognition method based on set |
| CN109697389A (en) * | 2017-10-23 | 2019-04-30 | 北京京东尚科信息技术有限公司 | Personal identification method and device |
| CN109726654A (en) * | 2018-12-19 | 2019-05-07 | 河海大学 | A kind of gait recognition method based on generation confrontation network |
| CN110119780A (en) * | 2019-05-10 | 2019-08-13 | 西北工业大学 | Based on the hyperspectral image super-resolution reconstruction method for generating confrontation network |
| WO2019196626A1 (en) * | 2018-04-12 | 2019-10-17 | 腾讯科技（深圳）有限公司 | Media processing method and related apparatus |
| CN110381268A (en) * | 2019-06-25 | 2019-10-25 | 深圳前海达闼云端智能科技有限公司 | method, device, storage medium and electronic equipment for generating video |
| CN110516525A (en) * | 2019-07-01 | 2019-11-29 | 杭州电子科技大学 | SAR image target recognition method based on GAN and SVM |
| CN110659586A (en) * | 2019-08-31 | 2020-01-07 | 电子科技大学 | Cross-view gait recognition method based on identity maintenance cyclic generation type countermeasure network |
| CN111144170A (en) * | 2018-11-02 | 2020-05-12 | 银河水滴科技（北京）有限公司 | Gait information registration method, system and storage medium |
| CN111144165A (en) * | 2018-11-02 | 2020-05-12 | 银河水滴科技（北京）有限公司 | Gait information identification method, system and storage medium |
| CN111144167A (en) * | 2018-11-02 | 2020-05-12 | 银河水滴科技（北京）有限公司 | Gait information identification optimization method, system and storage medium |
| CN111149104A (en) * | 2017-09-21 | 2020-05-12 | 诺基亚技术有限公司 | Apparatus, method and computer program product for biometric identification |
| CN111144171A (en) * | 2018-11-02 | 2020-05-12 | 银河水滴科技（北京）有限公司 | Abnormal crowd information identification method, system and storage medium |
| CN111639580A (en) * | 2020-05-25 | 2020-09-08 | 浙江工商大学 | Gait recognition method combining feature separation model and visual angle conversion model |
| CN112001254A (en) * | 2020-07-23 | 2020-11-27 | 浙江大华技术股份有限公司 | Pedestrian identification method and related device |
| CN112488984A (en) * | 2019-09-11 | 2021-03-12 | 中信戴卡股份有限公司 | Method and device for acquiring defect picture generation network and defect picture generation method |
| CN112733704A (en) * | 2021-01-07 | 2021-04-30 | 浙江大学 | Image processing method, electronic device, and computer-readable storage medium |
| CN112975968A (en) * | 2021-02-26 | 2021-06-18 | 同济大学 | Mechanical arm simulation learning method based on third visual angle variable main body demonstration video |
| CN113112572A (en) * | 2021-04-13 | 2021-07-13 | 复旦大学 | Hidden space search-based image editing method guided by hand-drawn sketch |
| CN113111797A (en) * | 2021-04-19 | 2021-07-13 | 杭州电子科技大学 | Cross-view gait recognition method combining self-encoder and view transformation model |
| CN113657463A (en) * | 2021-07-28 | 2021-11-16 | 浙江大华技术股份有限公司 | Gait recognition model training method, gait recognition method and related device |
| JP2022514935A (en) * | 2019-01-04 | 2022-02-16 | ソニー コーポレイション オブ アメリカ | Multiple prediction network |
| CN114120076A (en) * | 2022-01-24 | 2022-03-01 | 武汉大学 | Cross-view video gait recognition method based on gait motion estimation |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102426645A (en) * | 2011-08-30 | 2012-04-25 | 北京航空航天大学 | Multi-view and multi-state gait recognition method |
| CN104299012A (en) * | 2014-10-28 | 2015-01-21 | 中国科学院自动化研究所 | Gait recognition method based on deep learning |
| CN105574510A (en) * | 2015-12-18 | 2016-05-11 | 北京邮电大学 | Gait identification method and device |
| CN106096532A (en) * | 2016-06-03 | 2016-11-09 | 山东大学 | A kind of based on tensor simultaneous discriminant analysis across visual angle gait recognition method |
-
2017
- 2017-05-24 CN CN201710373017.5A patent/CN107085716B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102426645A (en) * | 2011-08-30 | 2012-04-25 | 北京航空航天大学 | Multi-view and multi-state gait recognition method |
| CN104299012A (en) * | 2014-10-28 | 2015-01-21 | 中国科学院自动化研究所 | Gait recognition method based on deep learning |
| CN105574510A (en) * | 2015-12-18 | 2016-05-11 | 北京邮电大学 | Gait identification method and device |
| CN106096532A (en) * | 2016-06-03 | 2016-11-09 | 山东大学 | A kind of based on tensor simultaneous discriminant analysis across visual angle gait recognition method |
Non-Patent Citations (2)
| Title |
|---|
| ZIFENG WU ET AL: "A Comprehensive Study on Cross-View Gait Based Human Identification with Deep CNNs", 《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》 * |
| 汪青: "基于深度学习的多视角步态识别算法设计与研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (53)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107563428A (en) * | 2017-08-25 | 2018-01-09 | 西安电子科技大学 | Classification of Polarimetric SAR Image method based on generation confrontation network |
| CN107609587A (en) * | 2017-09-11 | 2018-01-19 | 浙江工业大学 | A kind of multi-class multi views data creation method that confrontation network is generated based on depth convolution |
| CN107609587B (en) * | 2017-09-11 | 2020-08-18 | 浙江工业大学 | Multi-class multi-view data generation method for generating countermeasure network based on deep convolution |
| CN111149104A (en) * | 2017-09-21 | 2020-05-12 | 诺基亚技术有限公司 | Apparatus, method and computer program product for biometric identification |
| CN111149104B (en) * | 2017-09-21 | 2023-10-27 | 诺基亚技术有限公司 | Apparatus, method and computer readable storage medium for biometric identification |
| CN109697389A (en) * | 2017-10-23 | 2019-04-30 | 北京京东尚科信息技术有限公司 | Personal identification method and device |
| CN109697389B (en) * | 2017-10-23 | 2021-10-01 | 北京京东尚科信息技术有限公司 | Identity recognition method and device |
| CN108009568A (en) * | 2017-11-14 | 2018-05-08 | 华南理工大学 | A kind of pedestrian detection method based on WGAN models |
| CN107993210A (en) * | 2017-11-30 | 2018-05-04 | 北京小米移动软件有限公司 | Image repair method, device and computer-readable recording medium |
| CN108256627A (en) * | 2017-12-29 | 2018-07-06 | 中国科学院自动化研究所 | The mutual generating apparatus of audio-visual information and its training system that generation network is fought based on cycle |
| CN108171701A (en) * | 2018-01-15 | 2018-06-15 | 复旦大学 | Conspicuousness detection method based on U networks and confrontation study |
| CN108171701B (en) * | 2018-01-15 | 2021-06-22 | 复旦大学 | Significance detection method based on U network and counterstudy |
| CN108491380A (en) * | 2018-03-12 | 2018-09-04 | 苏州思必驰信息科技有限公司 | Confrontation multitask training method for speech understanding |
| CN108596026B (en) * | 2018-03-16 | 2020-06-30 | 中国科学院自动化研究所 | Cross-view gait recognition device and training method based on double-flow generation countermeasure network |
| CN108596026A (en) * | 2018-03-16 | 2018-09-28 | 中国科学院自动化研究所 | Across the visual angle Gait Recognition device and training method of confrontation network are generated based on double fluid |
| CN108629823A (en) * | 2018-04-10 | 2018-10-09 | 北京京东尚科信息技术有限公司 | The generation method and device of multi-view image |
| CN108629823B (en) * | 2018-04-10 | 2022-09-06 | 北京京东尚科信息技术有限公司 | Method and device for generating multi-view image |
| WO2019196626A1 (en) * | 2018-04-12 | 2019-10-17 | 腾讯科技（深圳）有限公司 | Media processing method and related apparatus |
| US11335127B2 (en) | 2018-04-12 | 2022-05-17 | Tencent Technology (Shenzhen) Company Ltd | Media processing method, related apparatus, and storage medium |
| US11403876B2 (en) * | 2018-06-11 | 2022-08-02 | Tencent Technology (Shenzhen) Company Limited | Image processing method and apparatus, facial recognition method and apparatus, and computer device |
| WO2019237846A1 (en) * | 2018-06-11 | 2019-12-19 | 腾讯科技（深圳）有限公司 | Image processing method and apparatus, face recognition method and apparatus, and computer device |
| CN108846355A (en) * | 2018-06-11 | 2018-11-20 | 腾讯科技（深圳）有限公司 | Image processing method, face identification method, device and computer equipment |
| CN109543546B (en) * | 2018-10-26 | 2022-12-20 | 复旦大学 | Gait age estimation method based on depth sequence distribution regression |
| CN109583298A (en) * | 2018-10-26 | 2019-04-05 | 复旦大学 | Across visual angle gait recognition method based on set |
| CN109583298B (en) * | 2018-10-26 | 2023-05-02 | 复旦大学 | Cross-view gait recognition method based on set |
| CN109543546A (en) * | 2018-10-26 | 2019-03-29 | 复旦大学 | The gait age estimation method returned based on the distribution of depth sequence |
| CN111144165B (en) * | 2018-11-02 | 2024-04-12 | 银河水滴科技(宁波)有限公司 | Gait information identification method, system and storage medium |
| CN111144171A (en) * | 2018-11-02 | 2020-05-12 | 银河水滴科技（北京）有限公司 | Abnormal crowd information identification method, system and storage medium |
| CN111144167A (en) * | 2018-11-02 | 2020-05-12 | 银河水滴科技（北京）有限公司 | Gait information identification optimization method, system and storage medium |
| CN111144165A (en) * | 2018-11-02 | 2020-05-12 | 银河水滴科技（北京）有限公司 | Gait information identification method, system and storage medium |
| CN111144170A (en) * | 2018-11-02 | 2020-05-12 | 银河水滴科技（北京）有限公司 | Gait information registration method, system and storage medium |
| CN109726654A (en) * | 2018-12-19 | 2019-05-07 | 河海大学 | A kind of gait recognition method based on generation confrontation network |
| JP2022514935A (en) * | 2019-01-04 | 2022-02-16 | ソニー コーポレイション オブ アメリカ | Multiple prediction network |
| JP7379494B2 (en) | 2019-01-04 | 2023-11-14 | ソニー コーポレイション オブ アメリカ | multiple prediction network |
| CN110119780A (en) * | 2019-05-10 | 2019-08-13 | 西北工业大学 | Based on the hyperspectral image super-resolution reconstruction method for generating confrontation network |
| CN110381268B (en) * | 2019-06-25 | 2021-10-01 | 达闼机器人有限公司 | Method, device, storage medium and electronic equipment for generating video |
| CN110381268A (en) * | 2019-06-25 | 2019-10-25 | 深圳前海达闼云端智能科技有限公司 | method, device, storage medium and electronic equipment for generating video |
| CN110516525B (en) * | 2019-07-01 | 2021-10-08 | 杭州电子科技大学 | SAR image target recognition method based on GAN and SVM |
| CN110516525A (en) * | 2019-07-01 | 2019-11-29 | 杭州电子科技大学 | SAR image target recognition method based on GAN and SVM |
| CN110659586A (en) * | 2019-08-31 | 2020-01-07 | 电子科技大学 | Cross-view gait recognition method based on identity maintenance cyclic generation type countermeasure network |
| CN110659586B (en) * | 2019-08-31 | 2022-03-15 | 电子科技大学 | Gait recognition method based on identity-preserving cyclic generation type confrontation network |
| CN112488984A (en) * | 2019-09-11 | 2021-03-12 | 中信戴卡股份有限公司 | Method and device for acquiring defect picture generation network and defect picture generation method |
| CN111639580A (en) * | 2020-05-25 | 2020-09-08 | 浙江工商大学 | Gait recognition method combining feature separation model and visual angle conversion model |
| CN112001254A (en) * | 2020-07-23 | 2020-11-27 | 浙江大华技术股份有限公司 | Pedestrian identification method and related device |
| CN112733704A (en) * | 2021-01-07 | 2021-04-30 | 浙江大学 | Image processing method, electronic device, and computer-readable storage medium |
| CN112975968B (en) * | 2021-02-26 | 2022-06-28 | 同济大学 | Mechanical arm imitation learning method based on third visual angle variable main body demonstration video |
| CN112975968A (en) * | 2021-02-26 | 2021-06-18 | 同济大学 | Mechanical arm simulation learning method based on third visual angle variable main body demonstration video |
| CN113112572A (en) * | 2021-04-13 | 2021-07-13 | 复旦大学 | Hidden space search-based image editing method guided by hand-drawn sketch |
| CN113111797A (en) * | 2021-04-19 | 2021-07-13 | 杭州电子科技大学 | Cross-view gait recognition method combining self-encoder and view transformation model |
| CN113111797B (en) * | 2021-04-19 | 2024-02-13 | 杭州电子科技大学 | Cross-view gait recognition method combining self-encoder and view transformation model |
| CN113657463A (en) * | 2021-07-28 | 2021-11-16 | 浙江大华技术股份有限公司 | Gait recognition model training method, gait recognition method and related device |
| CN114120076B (en) * | 2022-01-24 | 2022-04-29 | 武汉大学 | Cross-view video gait recognition method based on gait motion estimation |
| CN114120076A (en) * | 2022-01-24 | 2022-03-01 | 武汉大学 | Cross-view video gait recognition method based on gait motion estimation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107085716B (en) | 2021-06-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107085716A (en) | Across the visual angle gait recognition method of confrontation network is generated based on multitask | |
| CN109344736B (en) | Static image crowd counting method based on joint learning | |
| CN108537743B (en) | Face image enhancement method based on generation countermeasure network | |
| Liu et al. | Attribute-aware face aging with wavelet-based generative adversarial networks | |
| CN105469034B (en) | Face identification method based on Weighting type distinctive sparse constraint Non-negative Matrix Factorization | |
| CN104268593B (en) | The face identification method of many rarefaction representations under a kind of Small Sample Size | |
| CN104616316B (en) | Personage's Activity recognition method based on threshold matrix and Fusion Features vision word | |
| CN105825183B (en) | Facial expression recognizing method based on partial occlusion image | |
| CN102938070B (en) | A kind of behavior recognition methods based on action subspace and weight behavior model of cognition | |
| Zhu et al. | Deep learning multi-view representation for face recognition | |
| CN111090764B (en) | Image classification method and device based on multitask learning and graph convolution neural network | |
| CN107145836A (en) | Hyperspectral image classification method based on stack boundary discrimination self-encoding encoder | |
| CN104915658B (en) | A kind of emotion component analyzing method and its system based on emotion Distributed learning | |
| CN106897669A (en) | A kind of pedestrian based on consistent iteration various visual angles transfer learning discrimination method again | |
| CN109389045A (en) | Micro- expression recognition method and device based on mixing space-time convolution model | |
| CN104298974A (en) | Human body behavior recognition method based on depth video sequence | |
| CN105868711B (en) | Sparse low-rank-based human behavior identification method | |
| CN104408731B (en) | Region graph and statistic similarity coding-based SAR (synthetic aperture radar) image segmentation method | |
| CN102592150B (en) | Gait identification method of bidirectional two-dimensional principal component analysis based on fuzzy decision theory | |
| CN107590427A (en) | Monitor video accident detection method based on space-time interest points noise reduction | |
| CN108460400A (en) | A kind of hyperspectral image classification method of combination various features information | |
| CN102184384A (en) | Face identification method based on multiscale local phase quantization characteristics | |
| CN102867171B (en) | Label propagation and neighborhood preserving embedding-based facial expression recognition method | |
| CN108985161A (en) | A kind of low-rank sparse characterization image feature learning method based on Laplace regularization | |
| Lim et al. | Pose transforming network: Learning to disentangle human posture in variational auto-encoded latent space |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |