CN114140880A

CN114140880A - Gait recognition method and device

Info

Publication number: CN114140880A
Application number: CN202111464046.5A
Authority: CN
Inventors: 刘鑫辰; 刘武; 梅涛; 周伯文
Original assignee: Beijing Jingdong Shangke Information Technology Co Ltd
Current assignee: Beijing Jingdong Shangke Information Technology Co Ltd
Priority date: 2021-12-03
Filing date: 2021-12-03
Publication date: 2022-03-04

Abstract

The embodiment of the disclosure discloses a gait recognition method and a device. One embodiment of the method comprises: acquiring a first step image sequence and a second step image sequence; three-dimensional modeling is carried out on the human body in each gait image in the first step image sequence and the second step image sequence to obtain a first step model sequence and a second step model sequence; aligning the visual angles of the first step model sequence and the second step model sequence; and performing gait recognition according to the aligned first step model sequence and the second step model sequence. This embodiment helps to improve the accuracy of the gait recognition result.

Description

Gait recognition method and device

Technical Field

The embodiment of the disclosure relates to the technical field of computers, in particular to a gait recognition method and device.

Background

Pedestrian re-identification (Person re-identification), also known as pedestrian re-identification, is a technique that uses computer vision techniques to determine whether a particular pedestrian is present in an image or video sequence. Given an image or video of a specified pedestrian, the specified pedestrian is searched in the image or video under the cross-device, aiming at remedying the visual limitation of the fixed camera. The pedestrian re-identification technology can be combined with pedestrian detection or pedestrian tracking technology, and can be applied to the fields of intelligent video monitoring, intelligent security and the like. However, differences between different image pickup apparatuses, appearance wearing, size, occlusion, posture, viewing angle, and the like of pedestrians have a great influence on the accuracy of the result of pedestrian re-recognition.

Gait recognition is a new biological feature recognition technology, and aims to identify the identity of a pedestrian through the walking posture. Compared with other biological recognition technologies, gait recognition has the advantages of non-contact remote distance and difficulty in camouflage. In the field of intelligent video monitoring, the method has more advantages than image recognition. The existing gait recognition method mainly comprises two types of gait recognition based on a model and gait recognition based on a shape. The gait recognition based on the model is mainly to describe gait parameters (such as a motion trail, a limb length, a limb bending angle and the like) by modeling a human limb motion mode, and then to distinguish different pedestrians by using the difference of the motion modes. The gait recognition based on the shape mainly extracts the space characteristic and the time sequence characteristic of the gait from the human body gait contour sequence, and then completes the gait matching and recognition by calculating the similarity between the characteristics. However, the human body is a three-dimensional target, the gait contour sequences of the human body shot under different visual angles have larger difference, and the intra-class and inter-class characteristic difference caused by the visual angle difference has larger influence on the accuracy of the gait recognition result.

Disclosure of Invention

The embodiment of the disclosure provides a gait recognition method and device.

In a first aspect, an embodiment of the present disclosure provides a gait recognition method, including: acquiring a first step image sequence and a second step image sequence; three-dimensional modeling is carried out on the human body in each gait image in the first step image sequence and the second step image sequence to obtain a first step model sequence and a second step model sequence; aligning the visual angles of the first step model sequence and the second step model sequence; and performing gait recognition according to the aligned first step model sequence and the second step model sequence.

In a second aspect, embodiments of the present disclosure provide a gait recognition device, which includes: an acquisition unit configured to acquire a first step-state image sequence and a second step-state image sequence; the modeling unit is configured to perform three-dimensional modeling on the human body in each gait image in the first step image sequence and the second step image sequence to obtain a first step model sequence and a second step model sequence; an alignment unit configured to perform view alignment on the first step-wise model sequence and the second step-wise model sequence; and the identification unit is configured to perform gait identification according to the aligned first step model sequence and second step model sequence.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including: one or more processors; storage means for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method as described in any implementation of the first aspect.

In a fourth aspect, embodiments of the present disclosure provide a computer-readable medium on which a computer program is stored, which computer program, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

According to the gait recognition method and device provided by the embodiment of the disclosure, the two gait image sequences with matching are respectively subjected to three-dimensional modeling to obtain the two corresponding gait model sequences, then the two gait model sequences are subjected to visual angle alignment, and gait recognition is carried out based on the aligned gait model sequences, so that gait matching and recognition at the same visual angle can be realized, interference caused by visual angle difference on gait matching and recognition is avoided, and the accuracy of gait recognition results is improved.

Drawings

Other features, objects and advantages of the disclosure will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:

FIG. 1 is an exemplary system architecture diagram in which one embodiment of the present disclosure may be applied;

fig. 2 is a flow chart of one embodiment of a gait recognition method according to the present disclosure;

fig. 3a is a schematic view of a gait image according to the present disclosure;

fig. 3b is a schematic view of a gait model according to the present disclosure;

fig. 4 is a flow chart of yet another embodiment of a gait recognition method according to the present disclosure;

fig. 5 is a flow chart of yet another embodiment of a gait recognition method according to the present disclosure;

fig. 6 is a flow chart of another embodiment of a gait recognition method according to the disclosure;

fig. 7 is a schematic diagram of an application scenario of a gait recognition method according to an embodiment of the disclosure;

fig. 8 is a schematic structural diagram of one embodiment of a gait recognition device according to the disclosure;

FIG. 9 is a schematic structural diagram of an electronic device suitable for use in implementing embodiments of the present disclosure.

Detailed Description

The present disclosure is described in further detail below with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.

It should be noted that the data acquisition (such as the image or video of the user, etc.) referred to in this disclosure is performed on the basis of the authorization acquired to the relevant subject, and all the data acquisition complies with the regulations of the relevant laws and regulations.

It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

Fig. 1 illustrates an exemplary architecture 100 to which embodiments of the gait recognition method or gait recognition apparatus of the present disclosure may be applied.

As shown in fig. 1, the system architecture 100 may include

terminal devices

101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the

terminal devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The

terminal devices

101, 102, 103 interact with a server 105 via a network 104 to receive or send messages or the like. Various client applications may be installed on the

terminal devices

101, 102, 103. Such as browser-type applications, search-type applications, instant messaging tools, shopping-type applications, multimedia processing-type applications, and so forth.

The

terminal apparatuses

101, 102, and 103 may be hardware or software. When the

terminal devices

101, 102, 103 are hardware, they may be various electronic devices, including but not limited to smart phones, tablet computers, e-book readers, smart camera devices (such as cameras, video recorders, etc.), laptop portable computers, desktop computers, and the like. When the

terminal apparatuses

101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented as multiple pieces of software or software modules (e.g., multiple pieces of software or software modules to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.

The server 105 may be a server that provides various services, such as a backend server that provides various services for the

terminal devices

101, 102, 103. The server 105 may process the first step image sequence and the second step image sequence according to a request of the

terminal devices

101, 102, 103 to obtain a gait recognition result.

It should be noted that the gait recognition method provided by the embodiment of the present disclosure is generally executed by the server 105, and accordingly, the gait recognition device is generally disposed in the server 105.

It should be further noted that the

terminal devices

101, 102, and 103 may also have a multimedia processing application installed therein, and the

terminal devices

101, 102, and 103 may also process the first step image sequence and the second step image sequence based on the multimedia processing application to obtain a gait recognition result. In this case, the gait recognition method may be executed by the

terminal devices

101, 102, 103, and accordingly, the gait recognition apparatus may be provided in the

terminal devices

101, 102, 103. At this point, the exemplary system architecture 100 may not have the server 105 and the network 104.

The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster composed of a plurality of servers, or may be implemented as a single server. When the server 105 is software, it may be implemented as multiple pieces of software or software modules (e.g., multiple pieces of software or software modules used to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.

It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

With continued reference to fig. 2, a flow 200 of one embodiment of a gait recognition method according to the disclosure is shown. The gait recognition method comprises the following steps:

step 201, a first step image sequence and a second step image sequence are obtained.

In the present embodiment, the gait image may refer to an image showing a walking posture of a person. The gait image sequence may consist of several gait images. Generally, each gait image in a sequence of gait images is a gait image of the same person.

It should be noted that the gait image may be various types of images. For example, a video frame may be selected from a walking video of a specified person as a gait image, or an image obtained by processing (such as sharpening, binarization processing, and the like) the selected video frame may be used as the gait image.

The first step image sequence and the second step image sequence may be arbitrary gait image sequences. It should be noted that, for convenience of describing the two gait image sequences to be matched, the two gait image sequences are named as a first step image sequence and a second step image sequence respectively, and those skilled in the art should understand that the first and the second of the two gait image sequences do not constitute a specific limitation to the gait image sequences.

An executing subject of the gait recognition method (e.g., the server 105 shown in fig. 1, etc.) can acquire the first step image sequence and the second step image sequence from a local or other electronic device (e.g., the

terminal devices

101, 102, 103, etc. shown in fig. 1).

Step 202, performing three-dimensional modeling on the human body in each gait image in the first step image sequence and the second step image sequence to obtain a first step model sequence and a second step model sequence.

In this embodiment, three-dimensional modeling may be performed on the human body in each gait image in the first step image sequence, so as to obtain a gait model corresponding to each gait image, and further obtain a first step model sequence composed of the gait models corresponding to each gait image in the first step image sequence.

Similarly, the human body in each gait image in the second step image sequence can be three-dimensionally modeled to obtain a gait model corresponding to each gait image, and then a second step model sequence composed of the gait models corresponding to each gait image in the second step image sequence is obtained.

Referring now to fig. 3a and 3b, fig. 3a is a schematic diagram of a gait image in the present embodiment, and fig. 3b is a schematic diagram of a gait model in the present embodiment.

Specifically, for a gait image, the human body in the gait image can be correspondingly three-dimensionally modeled by using various existing three-dimensional modeling methods. For example, three-dimensional modeling methods include, but are not limited to: polygon Modeling (Polygon Modeling), Parametric Modeling (Parametric Modeling), Reverse Modeling (Reverse Modeling), surface Modeling (NURBS Modeling), and the like. As an example, an existing three-dimensional mesh reconstruction method may be employed for three-dimensional modeling.

The process of three-dimensional modeling can be described using the following formula:

R(x_k)＝ROMP(x_k)

wherein, X_kRepresenting the kth gait image in the gait image sequence, ROMP () representing the three-dimensional modeling process, and R () representing the gait model obtained after three-dimensional modeling.

And step 203, aligning the view angles of the first step model sequence and the second step model sequence.

In this embodiment, the view angle of the gait model sequence may refer to a shooting view angle corresponding to each gait model in the gait model sequence. The view angle of the gait model sequence can be flexibly determined according to the view angle corresponding to each gait model.

For example, an average value of the view angles corresponding to the respective gait models in the sequence of gait models may be determined as the view angle of the sequence of gait models. For another example, the view angle with the largest number of occurrences among the view angles corresponding to the respective gait models in the sequence of gait models may be determined as the view angle of the sequence of gait models. For another example, a view angle sequence composed of view angles corresponding to respective gait models in the gait model sequence may be determined as the view angle of the gait model sequence.

And aligning the visual angle of the first step model sequence with the visual angle of the second step model sequence, so that the aligned visual angles corresponding to the first step model sequence and the second step model sequence are respectively matched.

For example, the perspective corresponding to the first step-mode model sequence is a front shooting perspective, and the second step-mode model sequence is a side shooting perspective, so that the side shooting perspective corresponding to the second step-mode model sequence can be adjusted to the front shooting perspective to complete the alignment of the first step-mode model sequence and the second step-mode model sequence, the front shooting perspective corresponding to the first step-mode model sequence can also be adjusted to the side shooting perspective to complete the alignment of the first step-mode model sequence and the second step-mode model sequence, and the front shooting perspective corresponding to the first step-mode model sequence and the side shooting perspective corresponding to the second step-mode sequence can also be adjusted to the same designated shooting perspective (such as a back shooting perspective) to complete the alignment of the first step-mode sequence and the second step-mode sequence.

If the view angles respectively corresponding to the first step-mode model sequence and the second step-mode model sequence comprise view angle sequences, various methods can be flexibly adopted to align the view angles respectively corresponding to the first step-mode model sequence and the second step-mode model sequence.

For example, the view angle corresponding to the second step-mode model sequence is a view angle sequence formed by view angles corresponding to the gait models respectively, and then the view angles of the gait models corresponding to the gait models in the second gait model can be respectively adjusted to the view angles corresponding to the first step-mode model sequence, so that the view angle alignment of the first step-mode model sequence and the second step-mode model sequence is completed.

For another example, the view angle corresponding to the first step model sequence is a view angle sequence composed of view angles corresponding to the gait models respectively, and the view angle corresponding to the second step model sequence is a view angle sequence composed of view angles corresponding to the gait models respectively, so that the view angles of the gait models in the first step model sequence and the view angles of the gait models in the second step model sequence can be adjusted to the same specified view angle respectively, and the view angle alignment of the first step model sequence and the second step model sequence is completed.

Specifically, various existing view angle alignment methods may be employed to align the view angles of the first-step mode sequence and the second-step mode sequence. For example, the view angles of the first step model sequence and the second step model sequence may be aligned by rotating the gait model.

And 204, performing gait recognition according to the aligned first step model sequence and the aligned second step model sequence.

In this embodiment, after the viewing angles are aligned, the obtained aligned first step model sequence and the aligned second step model sequence correspond to the same viewing angle. At this time, gait recognition may be performed based on the aligned first step model sequence and the aligned second step model sequence at the same view angle, so as to determine whether the person corresponding to the first step model sequence and the person corresponding to the second step model sequence are the same person as a gait recognition result.

Specifically, after obtaining the aligned first step model sequence and the aligned second step model sequence, gait recognition can be performed by using various existing gait recognition methods to obtain a gait recognition result. The gait recognition result can be used to indicate whether the person in each gait image in the first step image sequence is the same as the person in each gait image in the second step image sequence.

In some optional implementation manners of this embodiment, after obtaining the aligned first step model sequence and the aligned second step model sequence, the gait features of the aligned first step model sequence may be extracted to obtain the first step features, and the gait features of the aligned second step model sequence may be extracted to obtain the second step features. Then, the similarity between the first step state feature and the second step state feature can be determined, and the gait recognition result is determined according to the determined similarity.

The gait features can be obtained by flexibly adopting the existing various gait feature extraction methods to process the gait model sequence. For example, the gait features may be extracted based on the time sequence of the gait model sequence, or the gait features may be extracted based on the physiological features of the human body in the gait model.

Generally, if the similarity between the first step state feature and the second step state feature is greater than a preset similarity threshold, the first step state image sequence and the second step state image sequence may be considered to correspond to the same person. Correspondingly, if the similarity between the first step state feature and the second step state feature is not greater than the preset similarity threshold, the first step state image sequence and the second step state image sequence may be considered to correspond to different people respectively.

According to the method provided by the embodiment of the disclosure, the three-dimensional modeling is carried out on the two gait image sequences to be recognized, and then the two gait model sequences obtained correspondingly are subjected to view angle alignment, so that the gait recognition at different view angles is converted into the gait recognition at the same view angle, the influence of view angle difference on the gait recognition is avoided, and the accuracy of the gait recognition result at the cross-view angle is improved.

With further reference to fig. 4, a flow 400 of yet another embodiment of a gait recognition method is shown. The process 400 of the gait recognition method includes the following steps:

step 401, a first step image sequence and a second step image sequence are obtained.

And 402, performing three-dimensional modeling on the human body in each gait image in the first step image sequence and the second step image sequence to obtain a first step model sequence and a second step model sequence.

And 403, aligning the view angles of the first step model sequence and the second step model sequence.

And 404, performing two-dimensional operation on each gait model in the aligned first step model sequence to obtain a third step image sequence.

In the present embodiment, the two-dimensionalization may refer to a process of converting an object higher than two dimensions into two dimensions. Specifically, the gait model can be subjected to two-dimensional transformation by using various existing two-dimensional transformation methods (such as a two-dimensional transformation function provided by various image processing tools, binarization processing and the like).

For each gait model in the first step model sequence after the visual angle alignment, the gait model can be subjected to two-dimensional processing to obtain a two-dimensional gait image corresponding to the gait model, and a third step image sequence consisting of the two-dimensional gait images respectively corresponding to the gait models in the aligned first step model sequence is obtained on the basis of the two-dimensional gait images.

And 405, performing two-dimensional operation on each gait model in the aligned second step model sequence to obtain a fourth gait image sequence.

In this embodiment, for each gait model in the second step model sequence after the perspective alignment, the gait model may be subjected to two-dimensional processing to obtain a two-dimensional gait image corresponding to the gait model, and based on this, a fourth gait image sequence composed of two-dimensional gait images respectively corresponding to each gait model in the aligned second step model sequence is obtained.

And step 406, performing gait recognition according to the third step image sequence and the fourth gait image sequence.

In this embodiment, various gait recognition methods can be adopted to perform gait recognition according to the third step image sequence and the fourth gait image sequence obtained based on two-dimensionality to obtain a gait recognition result.

In some optional implementation manners of this embodiment, a pre-trained gait feature extraction network may be used to extract gait features corresponding to the third step image sequence and the fourth step image sequence, then calculate a similarity between the gait features of the third step image sequence and the gait features of the fourth step image sequence, and determine a gait recognition result according to the calculated similarity.

The gait feature extraction network can be obtained by training the initial model by using preset training data and a loss function. The training data may include sequences of gait images, and the label of each sequence of gait images may indicate its corresponding person. The initial model can be flexibly built by technicians according to actual requirements, and can also be built based on various existing neural network models.

For example, the initial model may be constructed based on a deep learning model such as GaitSet. At this time, the input gait image sequence of the gait feature extraction network may be a human body contour image sequence. The human body contour image sequence can be obtained by performing binarization and other processing on a human body image.

The loss function can be preset by the skilled person depending on the actual application. For example, the loss function L may be set as follows:

wherein, N represents the number of input training samples, and i corresponds to the number of input training samples. And F () represents the characteristic extraction process of the characteristic extraction network, namely gait characteristics output by the characteristic extraction network. X^aGait image sequence X representing any pedestrian^pIs represented by the formula X^aOther gait image sequences, X, corresponding to the same pedestrianⁿIs represented by the formula X^aCorresponding to the gait image sequence of different pedestrians. M denotes a margin parameter. []₊Indicates a non-negative value, i.e., if]If the result of the calculation is negative, L is set to 0.

The similarity between gait features may be determined based on various similarity calculation methods (e.g., euclidean distance, cosine similarity, etc.). If the calculated similarity is greater than the preset similarity threshold, the third step image sequence and the fourth step image sequence may be considered to correspond to the same person, that is, the first step image sequence and the second step image sequence correspond to the same person. Correspondingly, if the calculated similarity is not greater than the preset similarity threshold, the third step image sequence and the fourth step image sequence may be considered to correspond to different persons, that is, the first step image sequence and the second step image sequence correspond to different persons, respectively.

The content that is not described in detail in this embodiment may refer to the related description in the embodiment corresponding to fig. 2, and is not described herein again.

In the method provided by the above embodiment of the present disclosure, after two gait model sequences are obtained by using three-dimensional modeling, the gait model sequence is converted into a two-dimensional gait image sequence through two-dimensional modeling, and then gait recognition is performed based on the two-dimensional gait image sequence. Compared with two-dimensional visual angle transformation based on Space Transformer Networks (STNs) and the like, gait information can be kept as much as possible by carrying out visual angle transformation based on a three-dimensional model and then carrying out two-dimensional transformation, so that the gait recognition accuracy is improved.

With further reference to fig. 5, a flow 500 of yet another embodiment of a gait recognition method is shown. The process 500 of the gait recognition method includes the following steps:

step 501, a first step image sequence and a second step image sequence are obtained.

And 502, performing three-dimensional modeling on the human body in each gait image in the first step image sequence and the second step image sequence to obtain a first step model sequence and a second step model sequence.

Step 503, determining an average view angle corresponding to each gait model in the first step model sequence.

In this embodiment, an average value of the view angles corresponding to the gait models in the first step model sequence may be determined as an average view angle.

And step 504, respectively adjusting the view angle of each gait model in the second step model sequence to be aligned with the average view angle.

In this embodiment, for each gait model in the second step-wise model sequence, the perspective of the gait model may be adjusted to align with the average perspective corresponding to the first step-wise model sequence.

And 505, performing gait recognition according to the aligned first step model sequence and second step model sequence.

The content that is not described in detail in this embodiment may refer to the related description in the corresponding embodiment of fig. 2 and fig. 4, and is not described herein again.

According to the method provided by the embodiment of the disclosure, when the view angles of the gait model sequences are aligned, the average view angle of each gait model in one gait model sequence can be determined, and then the view angle of each gait model in the other gait model sequence is adjusted to the average view angle, so that the number of the gait models with the view angles needing to be adjusted can be reduced, the calculated amount is reduced, the gait information of a person corresponding to the gait model sequences without the need of adjusting the view angles can be reserved as far as possible, and the accuracy of subsequent gait matching and identification is improved.

With further reference to fig. 6, a flow 600 of another embodiment of a gait recognition method is shown. The process 600 of the gait recognition method includes the following steps:

step 601, acquiring a first step image sequence of the target person.

In this embodiment, the target person may be any person, and may be specifically specified in advance by a technician according to an actual application scenario. The first step image sequence of the target person may be composed of gait images that exhibit a walking posture of the target person.

Step 602, obtaining a target video, and extracting gait image sequences of each person in the target video respectively to obtain at least one second step image sequence.

In the present embodiment, the target video may be an arbitrary video. The target video can be flexibly set according to the actual application scene or application requirements.

One or more individuals may be included in the target video. At this time, for each person in the target video, the gait image sequence of the person can be extracted to form a second step image sequence. Based on the above, the second-step image sequences corresponding to the persons in the target video can be obtained.

And 603, performing three-dimensional modeling on the human body in each gait image in the first step image sequence to obtain a first step model sequence.

And step 604, for each second-step image sequence in at least one second-step image sequence, performing three-dimensional modeling on the human body in each gait image in the second-step image sequence to obtain a second-step model sequence corresponding to the second-step image sequence.

And 605, respectively aligning the obtained at least one second step mode sequence with the first step mode sequence in view angle.

As an example, the process of view alignment can be described using the following formula:

where P () represents a view alignment operation of two objects. g denotes a second step model sequence, and q denotes a first step model sequence. X^g _i,kRepresenting the kth gait model in the ith second step model sequence. Theta^g _i,kDenotes the ithA perspective of a kth gait model in the second step model sequence. Theta^q _kRepresenting the view of the kth gait model in the first step model sequence. K1 represents the length of the first-step model sequence. R () represents a rotation operation on the view angle. R^|() Representing the second step model sequence after the view angle is aligned.

The lengths of the first step-mode model sequences and the lengths of the second step-mode model sequences may be the same or different.

And 606, extracting the gait features of the aligned first step model sequences, and respectively extracting at least one gait feature corresponding to the aligned second step model sequences.

In this embodiment, the aligned second step-mode model sequences may be first subjected to two-dimensional transformation to obtain corresponding two-dimensional gait image sequences, and then corresponding gait features may be extracted based on the obtained two-dimensional gait image sequences.

As an example, the processing procedure for performing the two-dimensional rendering by the binarization and rendering operations can be described using the following formula:

wherein, X^g _i,kRepresenting the kth gait model in the ith second step model sequence. R^|() Representing the second step model sequence after the view angle is aligned. X () represents a renderer. B () represents image binarization processing. G^|() And representing a two-dimensional image obtained after the two-dimensional processing.

Step 607, for each aligned second step model sequence in the at least one aligned second step model sequence, determining the similarity between the gait feature of the aligned second step model sequence and the gait feature of the aligned first step model sequence, and obtaining the similarity corresponding to the aligned second step model sequence.

And 608, determining a gait recognition result of the target video according to the similarity corresponding to each aligned second step model sequence in the at least one aligned second step model sequence.

In this embodiment, after obtaining the similarity between the gait features of each aligned second-step model sequence and the gait features of the aligned first-step model sequence, various methods can be flexibly adopted to determine the gait recognition result of the target video. The gait recognition result of the target video can be used for indicating whether the target video comprises the target person or not.

For example, the maximum similarity value may be selected first, and then it is determined whether the maximum similarity value is greater than a preset similarity threshold, and if so, the person corresponding to the aligned second-step model sequence corresponding to the maximum similarity value may be considered as the target person.

The content that is not described in detail in this embodiment may refer to the related description in the corresponding embodiments of fig. 2, fig. 4, and fig. 5, and is not described herein again.

With continued reference to fig. 7, fig. 7 is an exemplary application scenario 700 of the gait recognition method according to the embodiment. In the application scenario of fig. 7, the executing subject may acquire a video 702 captured by the camera 701, and extract a gait image sequence 7031 of the pedestrian "a" and a gait image sequence 7032 of the pedestrian "B" from the video 702.

Then, three-dimensional modeling can be performed on each gait image in the gait image sequence 7031 of the pedestrian "a", so as to obtain a corresponding gait model sequence 7041. Meanwhile, three-dimensional modeling can be performed on each gait image of the gait image sequence 7032 of the pedestrian 'B' to obtain a corresponding gait model sequence 7042.

In addition, the executing body may acquire a gait image sequence 705 of the target person "C" in advance, and perform three-dimensional modeling on each gait image therein to obtain a corresponding gait model sequence 706. The perspective of the gait model sequence 7041 may then be adjusted to the perspective of the gait model sequence 706 to obtain an aligned gait model sequence 7071, and the perspective of the gait model sequence 7042 may be adjusted to the perspective of the gait model sequence 706 to obtain an aligned gait model sequence 7072, thereby completing the perspective alignment.

Then, each gait model in the aligned gait model sequence 7071 can be subjected to two-dimensional transformation to obtain a corresponding two-dimensional gait image sequence 7081, and the gait features 7001 of the two-dimensional gait image sequence 7081 are extracted by using the pre-trained gait feature extraction network 709. Meanwhile, each gait model in the aligned gait model sequence 7072 can be subjected to two-dimensional transformation to obtain a corresponding two-dimensional gait image sequence 7082, and the gait feature extraction network 709 is used for extracting the gait features 7002 of the two-dimensional gait image sequence 7082.

Meanwhile, the gait feature 710 of the gait image sequence 705 is extracted by using the gait feature extraction network 709, then the similarity 711 between the gait feature 7001 of the pedestrian "a" and the gait feature of the target person "C" is calculated to be 95%, the similarity 712 between the gait feature 7002 of the pedestrian "B" and the gait feature of the target person "C" is calculated to be 60%, and a gait recognition result 713 can be obtained based on the similarity, so as to represent that the pedestrian "a" in the video 702 is the target person "C".

The method provided by the above embodiment of the disclosure specifies the gait image sequence of the target person, extracts the gait image sequence of each pedestrian from the target video, aligns the gait model sequence corresponding to the gait image sequence of each pedestrian in the target video with the gait model sequence corresponding to the gait image sequence of the target person respectively, and performs gait matching, thereby determining whether the target video includes the specified target person, and realizing accurate pedestrian recognition under a cross-view angle.

With further reference to fig. 8, as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a gait recognition device, which corresponds to the embodiment of the method shown in fig. 2, and which can be applied in various electronic devices.

As shown in fig. 8, the gait recognition apparatus 800 provided by the present embodiment includes an acquisition unit 801, a modeling unit 802, an alignment unit 803, and a recognition unit 804. Wherein the obtaining unit 801 is configured to obtain a first step image sequence and a second step image sequence; the modeling unit 802 is configured to perform three-dimensional modeling on a human body in each gait image in the first step image sequence and the second step image sequence to obtain a first step model sequence and a second step model sequence; the alignment unit 803 is configured to align the view angles of the first step-wise model sequence and the second step-wise model sequence; the identifying unit 804 is configured to perform gait identification according to the aligned first step model sequence and second step model sequence.

In the present embodiment, the gait recognition apparatus 800 includes: the specific processing of the obtaining unit 801, the modeling unit 802, the aligning unit 803, and the identifying unit 804 and the technical effects thereof can refer to the related descriptions of step 201, step 202, step 203, and step 204 in the corresponding embodiment of fig. 2, which are not described herein again.

In some optional implementations of the present embodiment, the identifying unit 804 is further configured to: respectively extracting gait features corresponding to the aligned first-step mode sequence and second-step mode sequence to obtain a first-step feature and a second-step feature; determining the similarity between the first step state characteristic and the second step state characteristic; and determining a gait recognition result according to the similarity.

In some optional implementations of the present embodiment, the identifying unit 804 is further configured to: performing two-dimensionalization on each gait model in the aligned first step model sequence to obtain a third step image sequence; performing two-dimensionalization on each gait model in the aligned second step model sequence to obtain a fourth gait image sequence; and performing gait recognition according to the third step image sequence and the fourth gait image sequence.

In some optional implementations of the present embodiment, the identifying unit 804 is further configured to: respectively extracting gait features corresponding to the third step image sequence and the fourth step image sequence by utilizing a pre-trained gait feature extraction network; and determining the similarity of the gait features respectively corresponding to the third step image sequence and the fourth step image sequence, and determining the gait recognition result according to the determined similarity.

In some optional implementations of the present embodiment, the alignment unit 803 is further configured to: determining an average visual angle corresponding to each gait model in the first step model sequence; and respectively adjusting the visual angle of each gait model in the second step model sequence to be aligned with the average visual angle.

In some optional implementations of the present embodiment, the obtaining unit 801 is further configured to: acquiring a first step image sequence of a target person; acquiring a target video, and respectively extracting gait image sequences of all people in the target video to obtain at least one second-step image sequence; and the modeling unit 802 described above is further configured to: carrying out three-dimensional modeling on the human body in each gait image in the first step image sequence to obtain a first step model sequence; and for each second-step image sequence in at least one second-step image sequence, performing three-dimensional modeling on the human body in each gait image in the second-step image sequence to obtain a second-step model sequence.

In some optional implementations of the present embodiment, the alignment unit 803 is further configured to: and respectively aligning the obtained at least one second step model sequence with the first step model sequence in view angle.

The apparatus provided by the foregoing embodiment of the present disclosure acquires, by an acquiring unit, a first step image sequence and a second step image sequence; the modeling unit carries out three-dimensional modeling on the human body in each gait image in the first step image sequence and the second step image sequence to obtain a first step model sequence and a second step model sequence; the alignment unit aligns the visual angles of the first step model sequence and the second step model sequence; the identification unit identifies the gaits according to the aligned first-step model sequence and the aligned second-step model sequence, so that the gaits can be matched and identified at the same visual angle, the interference of visual angle difference on the gaits matching and identification is avoided, and the accuracy of the gaits identification result is improved.

Referring now to FIG. 9, shown is a schematic diagram of an electronic device (e.g., the server of FIG. 1) 900 suitable for use in implementing embodiments of the present disclosure. The server shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 9, the electronic device 900 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 901 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)902 or a program loaded from a storage means 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data necessary for the operation of the electronic apparatus 900 are also stored. The processing apparatus 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. An input/output (I/O) interface 905 is also connected to bus 904.

Generally, the following devices may be connected to the I/O interface 905: input devices 906 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 907 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 908 including, for example, magnetic tape, hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic apparatus 900 to perform wireless or wired communication with other apparatuses to exchange data. While fig. 9 illustrates an electronic device 900 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 9 may represent one device or may represent multiple devices as desired.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device 909, or installed from the storage device 908, or installed from the ROM 902. The computer program, when executed by the processing apparatus 901, performs the above-described functions defined in the methods of the embodiments of the present disclosure.

It should be noted that the computer readable medium described in the embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In embodiments of the present disclosure, however, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring a first step image sequence and a second step image sequence; three-dimensional modeling is carried out on the human body in each gait image in the first step image sequence and the second step image sequence to obtain a first step model sequence and a second step model sequence; aligning the visual angles of the first step model sequence and the second step model sequence; and performing gait recognition according to the aligned first step model sequence and the second step model sequence.

Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present disclosure may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes an acquisition unit, a modeling unit, an alignment unit, and an identification unit. The names of the units do not form a limitation to the units themselves in some cases, and for example, the acquiring unit may also be described as a "unit that acquires a first-step image sequence and a second-step image sequence".

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept as defined above. For example, the above features and (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.

Claims

1. A gait recognition method comprising:

acquiring a first step image sequence and a second step image sequence;

carrying out three-dimensional modeling on the human body in each gait image in the first step image sequence and the second step image sequence to obtain a first step model sequence and a second step model sequence;

aligning the visual angles of the first step model sequence and the second step model sequence;

and performing gait recognition according to the aligned first step model sequence and the second step model sequence.

2. The method according to claim 1, wherein the performing gait recognition according to the aligned first-step model sequence and second-step model sequence comprises:

respectively extracting gait features corresponding to the aligned first-step mode sequence and second-step mode sequence to obtain a first-step feature and a second-step feature;

determining the similarity between the first step state characteristic and the second step state characteristic;

and determining a gait recognition result according to the similarity.

3. The method according to claim 1, wherein the performing gait recognition according to the aligned first-step model sequence and second-step model sequence comprises:

performing two-dimensionalization on each gait model in the aligned first step model sequence to obtain a third step image sequence;

performing two-dimensionalization on each gait model in the aligned second step model sequence to obtain a fourth gait image sequence;

and carrying out gait recognition according to the third step image sequence and the fourth gait image sequence.

4. The method of claim 3, wherein said performing gait recognition from said third and fourth gait image sequences comprises:

respectively extracting gait features corresponding to the third step image sequence and the fourth step image sequence by utilizing a pre-trained gait feature extraction network;

and determining the similarity of the gait features respectively corresponding to the third step image sequence and the fourth step image sequence, and determining a gait recognition result according to the determined similarity.

5. The method according to one of claims 1 to 4, wherein the view-angle aligning the first and second stepped model sequences comprises:

determining an average view angle corresponding to each gait model in the first step model sequence;

and respectively adjusting the view angle of each gait model in the second step-state model sequence to be aligned with the average view angle.

6. The method of one of claims 1 to 4, wherein the acquiring the first and second step image sequences comprises:

acquiring a first step image sequence of a target person;

acquiring a target video, and respectively extracting gait image sequences of all people in the target video to obtain at least one second-step image sequence; and

the three-dimensional modeling of the human body in each gait image in the first step image sequence and the second step image sequence to obtain a first step model sequence and a second step model sequence comprises:

carrying out three-dimensional modeling on the human body in each gait image in the first step image sequence to obtain a first step model sequence;

and for each second-step image sequence in the at least one second-step image sequence, performing three-dimensional modeling on the human body in each gait image in the second-step image sequence to obtain a second-step model sequence.

7. The method of claim 6, wherein the view-aligning the first and second sequences of step-wise models comprises:

and respectively aligning the obtained at least one second step model sequence with the first step model sequence in view angle.

8. A gait recognition device comprising:

an acquisition unit configured to acquire a first step-state image sequence and a second step-state image sequence;

the modeling unit is configured to perform three-dimensional modeling on the human body in each gait image in the first step image sequence and the second step image sequence to obtain a first step model sequence and a second step model sequence;

an alignment unit configured to perform view alignment on the first step-mode model sequence and the second step-mode model sequence;

and the identification unit is configured to perform gait identification according to the aligned first step model sequence and second step model sequence.

9. An electronic device, comprising:

one or more processors;

a storage device having one or more programs stored thereon;

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.

10. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-7.