CN114494732A - Gait recognition method and device - Google Patents
Gait recognition method and device Download PDFInfo
- Publication number
- CN114494732A CN114494732A CN202111635252.8A CN202111635252A CN114494732A CN 114494732 A CN114494732 A CN 114494732A CN 202111635252 A CN202111635252 A CN 202111635252A CN 114494732 A CN114494732 A CN 114494732A
- Authority
- CN
- China
- Prior art keywords
- key frame
- sequence
- gait
- preliminary
- silhouette
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The present disclosure provides a gait recognition method and apparatus. The method comprises the following steps: acquiring a gait sequence of a target to be detected; screening out a preliminary key frame sequence according to a key frame template for restricting the information of key points of limbs based on the gait sequence, and generating a preliminary silhouette sequence based on the preliminary key frame sequence; judging whether the preliminary key frame sequence has missing key frames or not; if the missing key frame exists, reconstructing the missing key frame according to the key frame template and the preliminary key frame sequence to form a reconstructed attitude image, generating a reconstructed silhouette based on the reconstructed attitude image, and combining the preliminary silhouette sequence and the reconstructed silhouette to generate a phase set; if the missing key frame does not exist, taking the preliminary silhouette sequence as a phase set; and extracting the gait feature vector based on the phase set, and carrying out gait recognition according to the gait feature vector.
Description
Technical Field
The disclosure relates to the field of gait recognition, and particularly provides a missing gait recognition method based on phase reconstruction and a device using the same.
Background
With the rapid development of multi-modal biometric identification technology, pedestrian gait characteristics are receiving wide attention as a new biometric characteristic. In the prior art, gait can be used as an identifier to distinguish different pedestrians. The gait characteristics are mainly used for expressing posture change information in the walking process of the human body. Compared with other physiological characteristics such as fingerprints, human faces, irises and the like, the gait has the advantages of being difficult to disguise, easy to collect, free of pedestrian cooperation and the like. The pedestrian identity can be accurately identified in the remote low-video-quality environment based on the human gait characteristics.
However, the gait recognition method in the prior art needs to input a complete gait sequence to extract and recognize the gait features. In practical application, the situation that the input data quality is not ideal enough and can not contain a complete gait sequence is often encountered. For example: firstly, the whole body cannot be shot due to the visual field problem of the camera, and only a part of continuous gaits can be obtained; secondly, when the walking is shielded for one time or multiple times, the middle part of the frame is lost, and continuous and complete gait cannot be obtained; thirdly, some frames are seriously distorted due to network problems in the transmission process; fourthly, abnormal actions or direction conversion are caused during walking, and therefore, the number of useless frames is excessive.
Disclosure of Invention
The present disclosure provides a gait recognition method and a device using the same, and more particularly, to a missing gait recognition method based on phase reconstruction and a device using the same.
An aspect of the present disclosure provides a gait recognition method, the method including: acquiring a gait sequence of a target to be detected; screening out a preliminary key frame sequence according to a key frame template for restricting the information of key points of limbs based on the gait sequence, and generating a preliminary silhouette sequence based on the preliminary key frame sequence; judging whether the preliminary key frame sequence has missing key frames or not; if the missing key frame exists, reconstructing the missing key frame according to the key frame template to form a reconstructed attitude image, generating a reconstructed silhouette based on the reconstructed attitude image, and combining the preliminary silhouette sequence and the reconstructed silhouette to generate a phase set; if the missing key frame does not exist, taking the preliminary silhouette sequence as a phase set; and extracting gait feature vectors based on the phase set, and carrying out gait recognition according to the gait feature vectors.
In an example embodiment, the key frame template may include: local constraints for constraining a range of angles formed by bones sharing a key point; the spatial constraint condition is used for constraining the range of the spatial relation of the bones which do not share the key points; and the time sequence constraint condition is a time sequence relation matrix of the same key point which is constructed according to the time sequence relation of the key frame based on the local constraint condition and the space constraint condition.
In an example embodiment, if there is a missing key frame, reconstructing the missing key frame from the key frame template and the preliminary key frame sequence to form a reconstructed pose image, the step of generating a reconstructed silhouette based on the reconstructed pose image may comprise: constructing an end-to-end posture reconstruction network based on a convolutional neural network, taking a key frame template and a preliminary key frame sequence as input, and reconstructing a missing key frame based on the posture reconstruction network to form a reconstructed posture image; and generating a reconstructed silhouette according to the reconstructed attitude image and the preliminary silhouette sequence through a mapping function based on the generated confrontation network.
In an example embodiment, the key frame template includes a template group corresponding to a key frame in one gait cycle, wherein the gait cycle is composed of four gait phases, and the number of key frame templates of each gait phase is the same as each other.
In an example embodiment, the preliminary silhouette sequence and reconstructed silhouettes in the phase set correspond one-to-one to keyframe templates located in the same gait phase.
Another aspect of the present disclosure discloses a gait recognition apparatus, the apparatus including: the image acquisition unit is configured to acquire a gait sequence of a target to be detected; a preliminary silhouette generating unit configured to screen out a preliminary key frame sequence according to a key frame template for restricting information of key points of a limb based on a gait sequence, and generate a preliminary silhouette sequence based on the preliminary key frame sequence; the missing key frame reconstruction unit is configured to judge whether the preliminary key frame sequence has missing key frames or not, if so, reconstruct the missing key frames according to a key frame template to form a reconstructed attitude image, and generate a reconstructed silhouette based on the reconstructed attitude image; a phase set generating unit configured to take the preliminary silhouette sequence as a phase set when there is no missing key frame, or combine the preliminary silhouette sequence and the reconstructed silhouette to generate a phase set when there is a missing key frame; and the identification unit is configured to extract the gait feature vector based on the phase set and carry out gait identification according to the gait feature vector.
In an example embodiment, the missing key frame reconstruction unit is configured to perform the steps of: constructing an end-to-end posture reconstruction network based on the convolutional neural network, and reconstructing the missing key frame according to the key frame template to form a reconstructed posture image; and generating a reconstructed silhouette according to the reconstructed attitude image and the preliminary silhouette sequence through a mapping function based on the generated confrontation network.
Another aspect of the present disclosure provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements a gait recognition method as described above.
Another aspect of the present disclosure provides a computer device, comprising: a processor; a memory storing a computer program which, when executed by the processor, implements the gait recognition method as described above.
In accordance with one or more aspects of the present disclosure, a gait recognition method and apparatus are presented. The gait sequence of the input missing part key frame is reconstructed by taking a video containing gait information shot by any camera (such as a monocular camera) as input, the gait cycle is decomposed into four phases according to the gait recognition method disclosed by the invention, a gait phase set is constructed by extracting the key frame, and a phase gait feature extraction network is input to extract feature vectors for gait recognition. Further, gait recognition can be performed by using only a single phase gait phase set as an input, in which case efficient recognition of a missing gait can be achieved even in the case of an excessive number of missing frames.
Drawings
The above and other aspects, features and advantages of the present disclosure will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which:
fig. 1 is a diagram illustrating a scenario in which a gait recognition method according to an embodiment of the present disclosure is implemented;
fig. 2 is a flow chart illustrating a gait recognition method according to an embodiment of the present disclosure;
figure 3 is a schematic diagram illustrating gait phases according to an embodiment of the disclosure;
FIG. 4 is a schematic diagram illustrating a key frame template according to an embodiment of the present disclosure;
FIG. 5 is a diagram illustrating an example of reconstructing a missing key frame according to an embodiment of the present disclosure;
fig. 6 is a block diagram illustrating a gait recognition apparatus according to an embodiment of the present disclosure; and
fig. 7 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.
Detailed Description
The following detailed description is provided to assist the reader in obtaining a thorough understanding of the methods, devices, and/or systems described herein. Various changes, modifications, and equivalents of the methods, apparatus, and/or systems described herein will, however, be apparent to those of ordinary skill in the art. For example, the order of operations described herein is merely an example and is not limited to the order set forth herein, but rather, variations may be made which will be apparent to those of ordinary skill in the art in addition to operations which must be performed in a particular order. Furthermore, descriptions of features and structures that will be well known to those of ordinary skill in the art may be omitted for the sake of clarity and conciseness. The features described herein may be embodied in different forms and should not be construed as limited to the examples described herein. Rather, the examples described herein have been provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. Examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. The embodiments are described below in order to explain the present disclosure by referring to the figures.
Fig. 1 is a schematic diagram illustrating a usage scenario implementing a gait recognition method according to an embodiment of the disclosure.
The gait recognition technology is one of video biological characteristic recognition technologies with wide development prospects at present. As shown in fig. 1, according to the embodiment of the present disclosure, a monocular camera and a computer vision technology are combined to realize a remote capture and acquisition of a walking process of a human body to identify the identity of the human body, without any behavioral coordination of an object to be identified.
In order to solve the above problems, according to the gait recognition method of the embodiment of the disclosure, the gait sequence of the target to be detected is divided into four phases, and the gait sequence located in the same phase is matched with the corresponding key frame template, and whether the missing key frame exists is determined. And if the missing key frame exists, reconstructing the missing key frame according to the key frame template so as to obtain a phase set capable of representing the complete phase. And extracting gait feature vectors based on the phase set, and identifying the gait according to the gait feature vectors. In this case, even if the gait sequence of the target to be detected does not have a complete gait cycle due to the absence of the key frame, the sufficiently accurate feature vector can be obtained through the matching of the key frame template to realize the gait recognition. In addition, missing key frames positioned in the same phase can be reconstructed according to the key frame template and the gait sequence of the target to be detected, so that the missing gait can be accurately identified.
However, the scenario shown in fig. 1 is merely an example and should not be construed as a limitation of the present disclosure, and any scenario applicable to gait recognition (including but not limited to outdoor, indoor, pre-set gait recognition/recording scenarios) may be included within the scope of the present disclosure.
Compared with the prior art, the method has better usability due to the fact that the missing gait is identified in the practical application scene, and the use cost is reduced due to the fact that only one monocular visible light camera is adopted.
Fig. 2 is a flowchart illustrating a gait recognition method according to an embodiment of the present disclosure.
In step S10, a gait sequence of the target to be detected is acquired. For example, an RGB monocular camera is used to capture a walking video of an object to be detected, and the walking video is processed into a gait sequence (e.g., a frame picture sequence).
In step S20, a preliminary key frame sequence is screened out from the key frame template of the information for constraining the key points of the limb based on the gait sequence, and a preliminary silhouette sequence is generated based on the preliminary key frame sequence.
In step S30, it is determined whether there is a missing key frame in the preliminary key frame sequence.
According to the embodiment of the present disclosure, the order of steps S20 and S30 is not particularly limited as long as the matching of the key frame sequence and the key frame template can be achieved, the key frames and/or the missing key frames are screened out, and the missing key frames are reconstructed.
In another embodiment, in step S20, the gait sequence as the input data may be matched and selected by a preset key frame template, so as to remove redundant parts (including still, abnormal motion, non-key frame, etc.) of the data. And secondly, positioning and marking the missing key frames so as to determine whether the missing key frames exist and obtain the sequence numbers and the time sequence information of the key frames available in the preliminary key frame sequence and the sequence numbers and the time sequence information of the missing key frames. And finally, converting the available key frames in the preliminary key frame sequence into a binary silhouette image through a segmentation algorithm. The segmentation algorithm includes, but is not limited to, background subtraction, frame difference method, deep learning segmentation, mask rcnn, etc. In a specific embodiment, through a preset key frame template, attitude estimation (attitude estimation includes but is not limited to openpos, alpha-pos, etc.) is performed on all frame pictures, then matching is performed on the images with the key frame template, redundant frame data which are not matched with the key frame template are removed, instance segmentation processing is performed on the selected key frames (i.e., available key frames), the attitude images of the key frames are retained, and missing frame sequence numbers and time sequence information are counted. Wherein the pose image may represent an abstract image composed of key points and bones. In step S30, it is determined whether there is a missing key frame based on the missing frame number and timing information counted in step S20. The step of matching and selecting the gait sequence by the preset key frame template will be further described with reference to fig. 3 and 4.
In step S40, if there is a missing key frame, reconstructing the missing key frame according to the key frame template and the preliminary key frame sequence to form a reconstructed pose image, generating a reconstructed silhouette based on the reconstructed pose image, and merging the preliminary silhouette sequence and the reconstructed silhouette to generate a phase set; if there are no missing key frames, the preliminary silhouette sequence is taken as the phase set. For example, in the embodiment, if a missing key frame exists, an end-to-end posture reconstruction network is built based on a convolutional neural network, a key frame template and the preliminary key frame sequence are used as input, and the missing key frame is reconstructed based on the posture reconstruction network to form a reconstructed posture image; and generating a reconstructed silhouette according to the reconstructed attitude image and the preliminary silhouette sequence through a mapping function based on the generated confrontation network. The preliminary silhouette sequence and the reconstructed silhouette in the phase set correspond to key frames of the same gait phase one to one. That is, the silhouette sequences in the phase set (preliminary silhouette sequence and reconstructed silhouette in the presence of missing frames, and preliminary silhouette sequence in the absence of missing frames) correspond to key frames of a single phase. The step of reconstructing the missing key frames will be further described with reference to fig. 5.
In step S50, a gait feature vector is extracted based on the phase set, and gait recognition is performed based on the gait feature vector. In the embodiment, the phase set is input into a gait recognition network, the gait features of the phases are extracted through multilayer convolution operation, and the corresponding person labels are recognized through a classifier.
In a specific exemplary embodiment, the phase set includes the preliminary silhouette sequence filtered by the key frame template, or includes the preliminary silhouette sequence filtered by the key frame template and a reconstructed silhouette corresponding to the missing key frame obtained by reconstruction. Firstly, a phase gait recognition network is built based on a convolutional neural network, and phase gait feature extraction is carried out. Secondly, the phase set is used as input, gait feature vectors are extracted, the similarity is calculated through the Euclidean distance, and gait recognition is carried out. In the prior art, if a single phase in a gait cycle is used as an input, gait recognition cannot be performed due to insufficient input data. However, according to the embodiments of the present disclosure, since the determination of the missing key frame is performed and the missing key frame is reconstructed to form a complete phase set, gait recognition can be achieved in the presence of the missing frame.
Fig. 3 is a schematic diagram illustrating gait phases according to an embodiment of the disclosure.
As shown in FIG. 3, the gait cycle means that a person walks two steps to form a cycle, and for example, the walking cycle is considered to be walking from the left foot to the next time when the left foot starts to walk behind.
According to an embodiment of the present disclosure, one gait cycle is divided into four gait phases. For example, a first phase GP01 transitioning from right foot to right foot support left foot lift at the front left foot rear (right front left rear to right branch left lift); a second phase GP02 (right branch left-right front-right back) for lifting and transition from the right foot supporting the left foot to the left foot behind the front right foot; a third phase GP03 (left front right back-left right leg right lift) transitioning from the left foot front right foot back to the left foot supporting right foot lift; the fourth phase GP04 transitions from left foot supporting the right foot rising to right foot behind the front left foot (left foot rising right front left back).
Each gait cycle has different numbers of frames under the conditions of different cameras, different people and the like, and because the time interval of some frames is too short and no obvious action change exists, only the most representative walking action in one gait cycle is taken as a key frame, and one gait phase can be composed of N key frames and is used for representing the key action in each phase. As shown in FIG. 3, each phase contains 7 key frames (GP01-1 through GP 01-7). Too many frames may cause overfitting, and too few frames may result in insufficient feature extraction.
Fig. 4 is a schematic diagram illustrating a key frame template according to an embodiment of the present disclosure. Fig. 4 shows an example of key frame template matching for the last gait image (GP04-7) in the fourth phase GP04 shown in fig. 3.
As shown in fig. 4, the key frame template further includes a template group corresponding to the gait cycle shown in fig. 3, wherein the gait cycle is composed of four gait phases as shown in fig. 3, the number of key frame templates of each gait phase is the same as each other, for example, each gait phase includes 7 key frame templates. The key frame template includes: local constraints, spatial constraints, and timing constraints.
Local constraints are used to constrain the range of angles formed by bones that share keypoints (e.g., bones can be abstracted as represented by keypoint connecting lines). For example, the angular range between bones having a common key point can be paired by defining the corresponding angular range for the elbow joint, shoulder tilt, knee joint, crotch joint, ankle joint, etc. of the human body for each key frame.
The spatial constraint is used to constrain the range of spatial relationships of bones that do not share a keypoint. For example, the spatial relationship of the two-legged joints, the two-arm joints, the arms to the legs, the shoulders, and the like is restricted. Here, the spatial relationship means a positional relationship between bones, an angle formed by extension lines thereof, and the like.
The time sequence constraint condition represents a time sequence relation matrix of the same key point which is constructed according to the time sequence relation of the key frame based on the local constraint condition and the space constraint condition. For example, the timing constraint may represent a spatial location matrix that varies with timing in the case where the key point and the bone satisfy the local constraint and the spatial constraint described above. That is, given local constraints and spatial constraints, if a key frame is matched within the same phase, the time-position relationship of the key point can be deduced from the key frame and the time-sequence constraints.
Still taking the example of each phase comprising seven key frame templates, the left diagram of fig. 4 shows the pose image constructed by the seventh key frame template of the fourth phase. Wherein the pose image may represent an abstract image composed of key points and bones. That is, the pose image corresponds to a visualized image of the template. In an embodiment, taking the lower leg in a keyframe template normalized to 128 × 128 pixels as an example, the local constraint of the lower leg can be expressed as: the angle range of the left shank and the left thigh sharing a key point with the left shank is 175 degrees +/-2 degrees, and the angle range of the right shank and the right thigh sharing a key point with the right shank is 178 degrees +/-2 degrees; the spatial constraint of the lower leg can be expressed as: the angle formed by the extension lines of the left lower leg and the right lower leg is 25 degrees +/-5 degrees, the distance between the upper end points of the left lower leg and the right lower leg is 8 +/-4 pixels, and the distance between the lower end points is 19 +/-4 pixels. The temporal constraints of the lower leg may represent the temporal-positional relationship of the upper and lower end points of the left lower leg with respect to the previous key frame template in the form of a matrix. It should be noted that, although specific examples of the above numerical ranges are given in describing examples of the key frame template, the above numerical values are merely for convenience and description and understanding, and should not be considered as limiting the concepts of the present disclosure.
The right diagram of fig. 4 is an example of the last gait image GP04-7 in the fourth phase GP04 shown in fig. 3 matched with the seventh keyframe template of the fourth phase.
In the embodiment, the preset constraint conditions of the key frame template are obtained through experimental tests. For each frame in the gait time sequence as input, acquiring a key point position of each frame based on a human body key point detection algorithm, constructing a posture image according to the key points, calculating the matching degree of the key points and bones of each frame in the gait time sequence with the key points and bones in each key frame template through Euclidean distance, and if a frame with the matching degree larger than a preset matching degree (for example, the variance of the Euclidean distance is smaller than a preset value) exists in one key frame template, considering the frame to be matched with the key frame template, namely, the frame is an available key frame; if there is no frame greater than a predetermined matching degree (e.g., the variance of the euclidean distance is less than a predetermined value) for a key frame template, the key frame corresponding to the key frame template is considered to be missing, and the missing key frame is located and labeled. Although not shown, the key frame template may be set so that it can be matched one-to-one with the frames shown in fig. 3.
As shown in fig. 3 and 4, key points (e.g., key points representing movable joints) may be first extracted from the gait image, and the skeleton may be represented by key point lines. And matching the gait image according to the key frame template so as to determine the sequence number and the time sequence information of the key frame template to which the gait image belongs. Therefore, redundant information can be eliminated, and the gait time sequence capable of representing gait characteristics can be obtained. Here, a single gait phase (for example, any one of the first phase GP-01 to the fourth phase GP-04) may be used as a basic input unit of the gait recognition network, and after being divided, accurate gait recognition may be achieved even if the gait cycle is not complete due to frame loss or the like.
Fig. 5 is a block diagram illustrating an example of reconstructing a missing key frame according to an embodiment of the present disclosure.
Referring to fig. 5, after going through the methods described with reference to fig. 3 and 4, it can be confirmed that there are three missing key frames. Firstly, an end-to-end pose reconstruction network can be built based on a convolutional neural network, the key frame template and the preliminary key frame sequence are used as input, and the missing key frame is reconstructed based on the pose reconstruction network to form a reconstructed pose image. And secondly, generating a reconstructed silhouette according to the reconstructed attitude image and the preliminary silhouette sequence through a mapping function based on a generated confrontation network.
In a specific embodiment, an end-to-end gait image reconstruction network can be built based on a convolutional neural network, and comprises two parts of posture generation and silhouette mapping. Firstly, generating a posture image of a missing gait part according to a local constraint condition, a space constraint condition and a time sequence constraint condition defined by a key frame template. And secondly, based on the generation countermeasure network, constructing a silhouette generation network, extracting a parameter relation matrix according to the attitude image and the silhouette image, and inputting the parameter relation matrix into a mapping function to generate a reconstructed silhouette corresponding to the missing key frame. Finally, the reconstructed silhouette may be optimized. For example, the edges of the reconstructed silhouette are subjected to a filtering process to remove abnormal noise. For example, the filtering process performed on the reconstructed silhouette includes, but is not limited to, gaussian filtering, mean filtering, median filtering, and the like.
According to the gait recognition method of the embodiment of the disclosure, firstly, the gait sequence of the target to be detected is divided into four phases, and the gait sequence in the same phase is matched with the corresponding key frame template, and whether the missing key frame exists is judged. If there are phases out of the four that do not miss a key frame, the phases can be taken as a set of phases that can characterize a complete single phase. If the four phases are the phases with missing key frames, the phases with relatively less missing key frames can be selected, and the missing key frames are reconstructed according to the key frame template, so that a complete phase set capable of representing a single phase is obtained. And extracting gait feature vectors based on the phase set, and identifying the gait according to the gait feature vectors. In this case, even if the gait sequence of the target to be detected does not have a complete gait cycle due to the absence of the key frame, the sufficiently accurate feature vector can be obtained through the matching of the key frame template to realize the gait recognition. In addition, missing key frames positioned in the same phase can be reconstructed according to the key frame template and the gait sequence of the target to be detected, so that the missing gait can be accurately identified.
Fig. 6 is a block diagram illustrating a gait recognition apparatus 100 according to an embodiment of the present disclosure.
Referring to fig. 6, the gait recognition apparatus 100 includes an image acquisition unit 110, a preliminary silhouette generation unit 120, a missing key frame judgment unit 130, a phase set generation unit 140, and a recognition unit 150.
The image acquisition unit 110 is configured to acquire a gait sequence of the object to be detected, and is configured to perform the method described with reference to step S10 of fig. 2.
The preliminary silhouette generating unit 120 is configured to screen out a preliminary key frame sequence from a key frame template of information for constraining key points of the limb based on the gait sequence, and generate a preliminary silhouette sequence based on the preliminary key frame sequence. The preliminary silhouette generating unit 120 is configured to perform the method described with reference to step S20 of fig. 2.
The missing key frame reconstruction unit 130 is configured to determine whether there is a missing key frame in the preliminary key frame sequence. The missing key frame reconstruction unit 130 is configured to perform the method described with reference to step S30 of fig. 2.
The phase set generation unit 140 is configured to reconstruct missing key frames according to the key frame template and the preliminary key frame sequence to form a reconstructed pose image, generate a reconstructed silhouette based on the reconstructed pose image, and combine the preliminary silhouette sequence and the reconstructed silhouette to generate a phase set, if the missing key frames exist; if there are no missing key frames, the preliminary silhouette sequence is taken as the phase set. The phase set generation unit 140 is configured to perform the method described with reference to step S40 of fig. 2.
The identification unit 150 is configured to extract a gait feature vector based on the phase set and perform gait identification according to the gait feature vector. The identification unit 150 is configured to perform the method described with reference to step S50 of fig. 2.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module/unit performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated herein.
Fig. 7 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.
The electronic device 700 shown in fig. 7 comprises at least one memory 701 and at least one processor 702, the memory 701 storing a computer program which, when executed by the processor 702, causes the processor to implement the gait recognition method as described above.
The various elements of the gait recognition apparatus shown in fig. 7 can be configured as software, hardware, firmware or any combination thereof that performs specific functions. For example, each unit may correspond to an application-specific integrated circuit, to pure software code, or to a module combining software and hardware. Furthermore, one or more functions implemented by the respective units may also be uniformly executed by components in a physical entity device (e.g., a processor, a client, a server, or the like).
Further, the gait recognition method described with reference to fig. 2 to 5 may be implemented by a program (or instructions) recorded on a computer-readable storage medium. For example, according to an exemplary embodiment of the present disclosure, a computer-readable storage medium storing instructions may be provided, wherein the instructions, when executed by at least one computing device, cause the at least one computing device to perform a gait recognition method according to the present disclosure.
The computer program in the computer-readable storage medium may be executed in an environment deployed in a computer device such as a client, a host, a proxy device, a server, and the like, and it should be noted that the computer program may also be used to perform additional steps other than the above steps or perform more specific processing when the above steps are performed, and the content of the additional steps and the further processing is already mentioned in the description of the related method with reference to fig. 2 to 5, and therefore will not be described again in order to avoid repetition.
It should be noted that each unit in the gait recognition apparatus according to the exemplary embodiment of the present disclosure may completely depend on the running of the computer program to realize the corresponding function, that is, each unit corresponds to each step in the functional architecture of the computer program, so that the whole system is called by a special software package (for example, a lib library) to realize the corresponding function.
Alternatively, the various elements shown in FIG. 6 may be implemented by hardware, software, firmware, middleware, microcode, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments to perform the corresponding operations may be stored in a computer-readable medium such as a storage medium, so that a processor may perform the corresponding operations by reading and executing the corresponding program code or code segments.
For example, exemplary embodiments of the present disclosure may also be implemented as a computing device including a storage component having a set of computer-executable instructions stored therein that, when executed by a processor, perform a gait recognition method according to exemplary embodiments of the present disclosure.
In particular, computing devices may be deployed in servers or clients, as well as on node devices in a distributed network environment. Further, the computing device may be a PC computer, tablet device, personal digital assistant, smart phone, web application, or other device capable of executing the set of instructions.
The computing device need not be a single computing device, but can be any device or collection of circuits capable of executing the instructions (or sets of instructions) described above, individually or in combination. The computing device may also be part of an integrated control system or system manager, or may be configured as a portable electronic device that interfaces with local or remote (e.g., via wireless transmission).
In a computing device, a processor may include a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a programmable logic device, a special purpose processor system, a microcontroller, or a microprocessor. By way of example, and not limitation, processors may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, and the like.
Some of the operations described in the gait recognition method according to the exemplary embodiment of the present disclosure may be implemented by software, some of the operations may be implemented by hardware, and furthermore, the operations may be implemented by a combination of hardware and software.
The processor may execute instructions or code stored in one of the memory components, which may also store data. The instructions and data may also be transmitted or received over a network via a network interface device, which may employ any known transmission protocol.
The memory component may be integral to the processor, e.g., having RAM or flash memory disposed within an integrated circuit microprocessor or the like. Further, the storage component may comprise a stand-alone device, such as an external disk drive, storage array, or any other storage device usable by a database system. The storage component and the processor may be operatively coupled or may communicate with each other, such as through an I/O port, a network connection, etc., so that the processor can read files stored in the storage component.
In addition, the computing device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the computing device may be connected to each other via a bus and/or a network.
The method of gait recognition according to an exemplary embodiment of the present disclosure may be described as various interconnected or coupled functional blocks or functional diagrams. However, these functional blocks or functional diagrams may be equally integrated into a single logic device or operated on by non-exact boundaries.
Accordingly, the gait recognition method described with reference to fig. 2 to 5 may be implemented by a system comprising at least one computing device and at least one storage device storing instructions.
According to an exemplary embodiment of the present disclosure, the at least one computing device is a computing device for performing a gait recognition method according to an exemplary embodiment of the present disclosure, the storage device having stored therein a set of computer-executable instructions that, when executed by the at least one computing device, performs the gait recognition method described with reference to fig. 2 to 5.
While various exemplary embodiments of the present disclosure have been described above, it should be understood that the above description is exemplary only, and not exhaustive, and that the present disclosure is not limited to the disclosed exemplary embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the disclosure. Therefore, the protection scope of the present disclosure should be subject to the scope of the claims.
Claims (10)
1. A gait recognition method, characterized in that the method comprises:
acquiring a gait sequence of a target to be detected;
screening out a preliminary key frame sequence according to a key frame template for restricting the key point information of the limb based on the gait sequence, and generating a preliminary silhouette sequence based on the preliminary key frame sequence;
judging whether the preliminary key frame sequence has missing key frames or not;
if the missing key frame exists, reconstructing the missing key frame according to the key frame template and the preliminary key frame sequence to form a reconstructed posture image, generating a reconstructed silhouette based on the reconstructed posture image, and combining the preliminary silhouette sequence and the reconstructed silhouette to generate a phase set; if the missing key frame does not exist, taking the preliminary silhouette sequence as a phase set; and
and extracting gait feature vectors based on the phase set, and carrying out gait recognition according to the gait feature vectors.
2. The method of claim 1, wherein the key frame template comprises:
local constraints for constraining a range of angles formed by bones sharing a key point;
the spatial constraint condition is used for constraining the range of the spatial relation of the bones which do not share the key points; and
and the time sequence constraint condition is a time sequence relation matrix of the same key point which is constructed according to the time sequence relation of the key frame based on the local constraint condition and the space constraint condition.
3. The method of claim 1 or 2, wherein the reconstructing the missing key frame, if any, from the key frame template and the preliminary key frame sequence to form a reconstructed pose image, generating a reconstructed silhouette based on the reconstructed pose image, comprises:
constructing an end-to-end posture reconstruction network based on a convolutional neural network, taking the key frame template and the preliminary key frame sequence as input, and reconstructing the missing key frame based on the posture reconstruction network to form a reconstructed posture image;
and generating a reconstructed silhouette according to the reconstructed attitude image and the preliminary silhouette sequence through a mapping function based on a generated confrontation network.
4. The method of claim 1, wherein the key frame templates comprise a set of templates corresponding to key frames in a gait cycle, wherein the gait cycle is comprised of four gait phases, and the number of key frame templates for each gait phase is the same as each other.
5. The method of claim 4, wherein the preliminary silhouette sequence and the reconstructed silhouettes in the phase set correspond one-to-one to a keyframe template of the same gait phase.
6. A gait recognition apparatus, characterized in that the apparatus comprises:
the image acquisition unit is configured to acquire a gait sequence of a target to be detected;
a preliminary silhouette generating unit configured to screen out a preliminary key frame sequence according to a key frame template for constraining key points of a limb based on the gait sequence, and generate a preliminary silhouette sequence based on the preliminary key frame sequence;
a missing key frame judging unit, configured to judge whether the preliminary key frame sequence has a missing key frame;
a phase set generation unit configured to reconstruct the missing key frame from the key frame template and the preliminary key frame sequence to form a reconstructed pose image, generate a reconstructed silhouette based on the reconstructed pose image, and combine the preliminary silhouette sequence and the reconstructed silhouette to generate a phase set, if the missing key frame exists; if the missing key frame does not exist, taking the preliminary silhouette sequence as a phase set; and
and the identification unit is configured to extract a gait feature vector based on the phase set and carry out gait identification according to the gait feature vector.
7. The apparatus of claim 6, wherein the key frame template comprises:
local constraints for constraining a range of angles formed by bones sharing a key point;
the spatial constraint condition is used for constraining the range of the spatial relation of the bones which do not share the key points;
and the time sequence constraint condition is a time sequence relation matrix of the same key point which is constructed according to the time sequence relation of the key frame based on the local constraint condition and the space constraint condition.
8. The apparatus according to claim 6 or 7, wherein the phase set generation unit is configured to perform the steps of:
constructing an end-to-end posture reconstruction network based on a convolutional neural network, taking the key frame template and the preliminary key frame sequence as input, and reconstructing the missing key frame based on the posture reconstruction network to form a reconstructed posture image;
and generating a reconstructed silhouette according to the reconstructed attitude image and the preliminary silhouette sequence through a mapping function based on a generated confrontation network.
9. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements a gait recognition method according to any one of claims 1 to 5.
10. A computer device, characterized in that the computer device comprises:
a processor;
a memory storing a computer program which, when executed by the processor, implements the gait recognition method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111635252.8A CN114494732A (en) | 2021-12-29 | 2021-12-29 | Gait recognition method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111635252.8A CN114494732A (en) | 2021-12-29 | 2021-12-29 | Gait recognition method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114494732A true CN114494732A (en) | 2022-05-13 |
Family
ID=81508097
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111635252.8A Pending CN114494732A (en) | 2021-12-29 | 2021-12-29 | Gait recognition method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114494732A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114863567A (en) * | 2022-05-19 | 2022-08-05 | 北京中科睿医信息科技有限公司 | Method and device for determining gait information |
-
2021
- 2021-12-29 CN CN202111635252.8A patent/CN114494732A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114863567A (en) * | 2022-05-19 | 2022-08-05 | 北京中科睿医信息科技有限公司 | Method and device for determining gait information |
| CN114863567B (en) * | 2022-05-19 | 2023-03-10 | 北京中科睿医信息科技有限公司 | Method and device for determining gait information |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zheng et al. | Deep learning-based human pose estimation: A survey | |
| Zhou et al. | Towards 3d human pose estimation in the wild: a weakly-supervised approach | |
| Sun et al. | Compositional human pose regression | |
| Sridhar et al. | Interactive markerless articulated hand motion tracking using RGB and depth data | |
| CN111414797B (en) | System and method for estimating pose and pose information of an object | |
| CN114049681A (en) | Monitoring method, identification method, related device and system | |
| CN111783506B (en) | Method, apparatus and computer readable storage medium for determining target characteristics | |
| CN113658211B (en) | User gesture evaluation method and device and processing equipment | |
| Zhou et al. | Hemlets posh: Learning part-centric heatmap triplets for 3d human pose and shape estimation | |
| WO2022120843A1 (en) | Three-dimensional human body reconstruction method and apparatus, and computer device and storage medium | |
| CN111462209A (en) | Action migration method, device, equipment and storage medium | |
| Chen et al. | Markerless monocular motion capture using image features and physical constraints | |
| Cha et al. | Learning 3D skeletal representation from transformer for action recognition | |
| Balakrishnan et al. | Visual deprojection: Probabilistic recovery of collapsed dimensions | |
| CN114494732A (en) | Gait recognition method and device | |
| Sarker et al. | Enhanced visible–infrared person re-identification based on cross-attention multiscale residual vision transformer | |
| Meng et al. | Gait recognition based on 3D human body reconstruction and multi-granular feature fusion | |
| CN116580169B (en) | Digital man driving method and device, electronic equipment and storage medium | |
| CN114445910A (en) | Gait visual angle detection method and device and gait recognition method and device | |
| Wang et al. | A short survey on deep learning for skeleton-based action recognition | |
| Adeli-Mosabbeb et al. | Model-based human gait tracking, 3D reconstruction and recognition in uncalibrated monocular video | |
| Tiwari et al. | Deep learning approach for generating 2D pose estimation from video for motion capture animation | |
| Lu et al. | Intelligent recognition method of athlete wrong movement based on image vision | |
| Sharma et al. | Analysis of Machine Learning for Recognizing Hand Gestures | |
| CN114140883A (en) | Gait recognition method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |