CN111476198B - Gait recognition method, device, system, storage medium and server based on artificial intelligence - Google Patents
Gait recognition method, device, system, storage medium and server based on artificial intelligence Download PDFInfo
- Publication number
- CN111476198B CN111476198B CN202010334982.3A CN202010334982A CN111476198B CN 111476198 B CN111476198 B CN 111476198B CN 202010334982 A CN202010334982 A CN 202010334982A CN 111476198 B CN111476198 B CN 111476198B
- Authority
- CN
- China
- Prior art keywords
- gait
- video
- module
- human body
- recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a gait recognition method, device, system, storage medium and server based on artificial intelligence, wherein the gait recognition method comprises the following steps: A. identifying a human body in the video; B. identifying and tracking main joints of the human body, and storing the position information of the main joints in a video frame to form a video sequence; C. learning and extracting gait features in the video sequence; D. and comparing the gait characteristics with gait characteristics preset in a storage module to obtain an identification result. The gait recognition device comprises a moving human body recognition module, a human body joint tracking module, a self-coding learning module, a gait feature recognition module and a storage module. The gait recognition system comprises a video acquisition terminal, a using terminal and a server. The invention is based on artificial intelligence technology, avoids ineffective search of optimal gait features in high-dimensional space caused by artificial selection of gait features, and realizes robust and complete extraction of the optimal gait features to the greatest extent.
Description
Technical Field
The invention belongs to the technical field of biological feature recognition, and particularly relates to a gait recognition method, device, system, storage medium and server based on artificial intelligence.
Background
Gait refers to the posture of a human body as well as all the actions of walking, and gait information includes, but is not limited to, information such as walking frequency, step length, lower limb swing angle, acceleration and the like when walking. In general, a human gait can be composed of 7 time-sequential phases, respectively: heel strike, ball strike, heel lift, toe off, early swing, mid swing and tail swing (heel strike), these 7 phases make up a basic gait cycle. The gait cycle of each human foot represents unique biological characteristics, so that the gait is the same as the characteristics of fingerprints, irises, faces and the like, and can be used as a biological characteristic of a human body for identifying and locking a natural human body. At present, gait recognition can be matched with face recognition, iris recognition and the like, and is applied to an access control recognition, public security space network monitoring system and the like, and the gait of a person with unknown identity is searched in a database for the gait of the person matched with the person with unknown identity, so that the identity of the person is confirmed.
At present, the characteristics of gait cycle and gait sequence are mostly selected manually by manpower, the characteristics of gait to be extracted can only be selected manually, the gait characteristics are extracted from the video through a corresponding program or algorithm, for example, the walking frequency characteristics, the span characteristics, the swing duration characteristics and the like of walking are selected, and the special gait characteristics of human body are extracted from the video through the data of the target characteristics through a tracking algorithm. The feature space of the gait features is generally a high-dimensional space, and the features of the gait features include a space dimension and a time dimension, and the features of the gait are manually determined, so that the feature vector which can represent the human gait features most accurately cannot be judged, and the probability of missing the features which are most valuable and can represent the human gait most accurately is very high and the probability of obtaining the optimal gait features is relatively low when the gait features are selected from the feature space of the gait sequence. Meanwhile, these shortcomings are one of the important reasons that the current gait recognition technology cannot be utilized and deployed on a large scale.
Accordingly, the prior art is subject to improvement and development.
Disclosure of Invention
The invention provides a gait recognition method, device, system, storage medium and server based on artificial intelligence, which automatically extracts and recognizes gait features through an intelligent module, avoids ineffective search of optimal gait features in a high-dimensional space caused by extracting the gait features by the artificial method, and maximally realizes robust and complete extraction of the optimal gait features.
In order to solve the technical problems, the gait recognition method based on artificial intelligence provided by the invention comprises the following steps:
A. identifying a human body in the video;
B. identifying and tracking main joints of the human body, and storing the position information of the main joints in a video frame to form a video sequence;
C. learning and extracting gait features in the video sequence;
the gait feature comprises position information of the primary joint in the video frame, and motion information of the primary joint in the video sequence;
D. and comparing the gait characteristics with gait characteristics preset in a storage module to obtain an identification result.
Further, the step D is to compare the gait characteristics with the gait characteristics preset in the storage module, and the recognition result is: calculating the distance between the gait feature in a feature space and the gait feature preset in the storage module through a measurement function, wherein if the distance is in a range [ a,1], the gait feature is the gait feature of different angles of a human body to which the gait feature preset in the storage module belongs, and if the distance is in a range [0, a), the gait feature is not matched with the gait feature preset in the storage module; the gait characteristics of the same human body preset in the storage module are distributed in a multi-element Gaussian mode in a characteristic space.
Further, the step B further includes a step Bn. of interpolating the video to generate gait video frames with a frame number greater than or equal to 20; before executing the step Bn, executing the step B1. Judging the time length of the human body gait in the video, and if the time length of the human body gait is less than 0.5s, executing the step Bn.
Further, the above main joints include ankle joint, hip joint, knee joint, wrist joint, elbow joint and shoulder joint.
Further, the step A identifies the human body in the video, and the corresponding ID numbers are generated from the human body identified in the video.
The invention provides an artificial intelligence-based gait recognition device, which comprises a moving human body recognition module, a human body joint tracking module, a self-coding learning module, a gait feature recognition module and a storage module, wherein the moving human body recognition module is used for recognizing the gait feature of a human body;
the moving human body identification module is used for identifying a human body from the video;
the human body joint tracking module is connected with the moving human body identification module and is used for identifying and tracking the main joint of the human body and storing the position information of the main joint in a video frame to form a video sequence;
the self-coding learning module is connected with the human joint tracking module and is used for learning and extracting gait characteristics in the video sequence; the gait feature comprises position information of the primary joint in a video frame, and motion information of the primary joint in a video sequence;
the gait feature recognition module is connected with the self-coding learning module and is used for comparing the gait features with gait features preset in the storage module to obtain a recognition result;
the storage module and the gait feature recognition module are used for storing preset gait features.
Further, the gait recognition device based on artificial intelligence further comprises a gait video frame generation module, wherein the gait video frame generation module is connected with the moving human body recognition module and is used for interpolating and generating gait video frames with the frame number of more than or equal to 20 in the video when the gait time of the human body in the video is less than 0.5 s.
The gait recognition system based on artificial intelligence provided by the invention comprises a video acquisition terminal, a using terminal and a server;
the video acquisition terminal is used for acquiring videos of human body movement;
the server is used for extracting gait characteristics in the video, comparing the gait characteristics with gait characteristics preset in the storage module and obtaining an identification result;
the terminal is used for monitoring the gait recognition process and outputting a recognition result.
The storage medium provided by the invention is provided with a computer program stored therein, and when the computer program runs on a computer, the computer is caused to execute the gait recognition method based on artificial intelligence.
The server provided by the invention comprises a processor and a memory, wherein a computer program is stored in the memory, and the processor is used for executing the gait recognition method based on artificial intelligence by calling the computer program stored in the memory.
According to the artificial intelligence-based gait recognition method, the human body is converted into the virtual human body formed by the mutual connection of the joint points, the gait characteristics of the human body are represented by the motion data of the joints of the human body, the discernability is improved for the extraction of the subsequent gait characteristics, and the missing of the characteristics which are most valuable and can represent the gait of the human body is avoided. According to the gait recognition device based on artificial intelligence, the human body and the main joints thereof are accurately recognized through the motion human body recognition module and the human body joint tracking module, and the motion of the main joints is accurately obtained through tracking; the self-coding learning module replaces manual design and extraction of gait features, so that optimal gait features can be accurately retrieved and extracted from a high-dimensional feature space, complete and robust extraction of the gait features is realized, and meanwhile, high-dimensional time and space features are converted into low-dimensional feature vectors, so that the calculation burden in gait feature retrieval is greatly reduced; and finally, realizing identification through a gait feature identification module, and improving identification precision.
Drawings
FIG. 1 is a flow chart of an artificial intelligence based gait recognition method of the invention.
FIG. 2 is another flow chart of the artificial intelligence based gait recognition method shown in FIG. 1.
Fig. 3 is a schematic structural diagram of an artificial intelligence-based gait recognition device of the invention.
Fig. 4 is a schematic diagram of another construction of the artificial intelligence based gait recognition device shown in fig. 3.
Fig. 5 is a schematic diagram of another construction of the artificial intelligence based gait recognition device shown in fig. 4.
FIG. 6 is a schematic diagram of an artificial intelligence based gait recognition system of the invention.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for explaining the present invention and are not to be construed as limiting the present invention.
In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more of the described features. In the description of the present invention, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
In the description of the present invention, it should be noted that, unless explicitly specified and limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be either fixedly connected, detachably connected, or integrally connected, for example; can be mechanically connected, electrically connected or can be communicated with each other; can be directly connected or indirectly connected through an intermediate medium, and can be communicated with the inside of two elements or the interaction relationship of the two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.
In the present invention, unless expressly stated or limited otherwise, a first feature "above" or "below" a second feature may include both the first and second features being in direct contact, as well as the first and second features not being in direct contact but being in contact with each other through additional features therebetween. Moreover, a first feature being "above," "over" and "on" a second feature includes the first feature being directly above and obliquely above the second feature, or simply indicating that the first feature is higher in level than the second feature. The first feature being "under", "below" and "beneath" the second feature includes the first feature being directly under and obliquely below the second feature, or simply means that the first feature is less level than the second feature.
The following disclosure provides many different embodiments, or examples, for implementing different features of the invention. In order to simplify the present disclosure, components and arrangements of specific examples are described below. They are, of course, merely examples and are not intended to limit the invention. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples, which are for the purpose of brevity and clarity, and which do not themselves indicate the relationship between the various embodiments and/or arrangements discussed. In addition, the present invention provides examples of various specific processes and materials, but one of ordinary skill in the art will recognize the application of other processes and/or the use of other materials.
As shown in fig. 1, the gait recognition method based on artificial intelligence of the invention comprises the following steps:
A. a human body in the video is identified.
In the step, the method can be executed by adopting a trained moving human body identification module, so that human bodies in the video are distinguished from other animals or objects, and when one or more human bodies appear in the video, different human bodies are accurately identified in real time, and identification and tracking are performed.
Specifically, the identification may use an identification frame to frame the human body, and the identification frame may use a rectangular frame. Therefore, the user can conveniently and quickly see the human body from the video.
In some embodiments, the human body identified from the video is each generated with a corresponding ID number. Therefore, the gait features of different human bodies are marked and distinguished, and subsequent storage, retrieval and identification work is facilitated.
B. And identifying and tracking the main joints of the human body, and storing the position information of the main joints in video frames to form a video sequence.
In this step, a trained human joint tracking module may be employed for execution. When a human body appears in the video, the human body joint tracking module identifies the main joint of the human body in real time and stores the position of the main joint in the video frame, and as the human body joint tracking module identifies and stores the position of the main joint in each frame of video frame, the real-time tracking of the main joint of the human body is realized, and the record of the position information of the stored main joint forms an important feature of the human body posture. In particular, the position information of the primary joint may be coordinate information of the primary joint in the video frame.
When a human body moves, the movement of the human body joint is represented by the change of the position of each joint point along the time axis, the gait characteristics of the human body are represented as the movement characteristics of the human body joint, and the movement characteristics (the gait characteristics of the human body in the embodiment) of the whole are obtained by tracking the movement characteristics of the midpoint (the main human body joint in the embodiment) of the whole (the human body in the embodiment), so that the identification degree is improved for the extraction of the subsequent gait characteristics, and the missing of the characteristics which are most valuable and can represent the gait of the human body accurately is avoided.
In some embodiments, the primary joints include ankle, hip, knee, wrist, elbow, and shoulder joints. Therefore, the gait characteristics of the human body are expanded to the motion characteristics of the joints of the whole body of the human body, the characteristic space of the gait characteristics is increased, meanwhile, the human body joints outside the joints of the lower limbs are identified and tracked, the key characteristics for identification are effectively increased, the identification degree of the gait characteristics is further increased, and the error rate of the gait characteristic identification can be remarkably reduced.
C. Learning and extracting gait features in the video sequence; the gait feature comprises position information of the primary joint in a video frame, and motion information of the primary joint in a video sequence. Specifically, the position information may be coordinate information of the main joint in the video frame, and the motion information may be vector information of the main joint in the two connected video frames.
In particular, this step may be performed using a trained self-encoding learning module. The self-coding learning module extracts gait features from the video sequence according to the motion data of the main joint, codes the extracted gait features and converts the high-dimensional feature space of the gait features into low-dimensional feature vectors, thereby effectively reducing the calculation burden of searching the gait features.
D. And comparing the gait characteristics with gait characteristics preset in a storage module to obtain an identification result.
This step may be performed using a trained gait feature recognition module. If the gait characteristics of the same human body are identified in the storage module, the result is displayed to be successfully identified, and the ID number of the corresponding human body is displayed; if the gait features of the same human body are not recognized in the database, and the gait features of the human body are proved to be not recorded, the result shows that the gait features are not matched.
The human body has multi-angle shooting in the video, so that the gait characteristics of the same human body have multi-angle problems, namely, the codes of the gait characteristics are deviated due to different shooting angles, so that the subsequent gait characteristic recognition failure is caused, and the method is also an important factor affecting the gait characteristic recognition precision.
In some embodiments, the step d. Compares the gait feature with gait features preset in the storage module to obtain an identification result of:
calculating the distance between the gait feature in a feature space and the gait feature preset in the storage module through a measurement function, wherein if the distance is in a range [ a,1], the gait feature is the gait feature of different angles of a human body to which the gait feature preset in the storage module belongs, and if the distance is in a range [0, a), the gait feature is not matched with the gait feature preset in the storage module; the gait characteristics of the same human body preset in the storage module are in multi-Gaussian distribution in the characteristic space, the measurement function is a probability density function corresponding to the multi-Gaussian distribution, and the specific calculation method can be applied to the calculation method of the probability density function in the prior art. Specifically, the critical value a can be adjusted in practical application, and the user can make adjustment according to the error condition in the application. Thereby, accuracy and uniqueness of gait feature recognition are ensured.
Specifically, the method for establishing the storage module includes the steps of:
A. identifying a human body in the video;
B. identifying and tracking main joints of the human body, and storing the position information of the main joints in a video frame to form a video sequence;
C. learning and extracting gait features in the video sequence; the method comprises the following steps:
step E, storing the gait characteristics.
The extraction of gait features depends on the acquired video sequence, which is acquired from the motion data of the main joint in the video sequence. If the human body only has extremely short movement in the video acquisition range, namely the gait time of the human body in the video is extremely short, the obtained video sequence has too few frames, and gait characteristics are difficult to extract. For example, a human body only rubs on the edge of the video acquisition range and does not completely pass through the video acquisition range, the gait time of the human body in the video is less than 0.5s, for example, only 0.1s, the obtained video sequence is only 2-3 frames, and a gait cycle cannot be displayed, so that the human body is insufficient to extract gait characteristics.
As shown in fig. 2, in some embodiments, the method further includes the following steps after the step B:
bn. interpolates the acquired video to generate gait video frames having a frame number of 20 or more.
Before executing the step Bn, the following steps are also executed:
B1. judging the gait time of the human body in the acquired video, and executing the step Bn if the gait time is less than 0.5 s.
The step Bn is necessary supplement and reinforcement of the step B, and overcomes the technical defect that the follow-up gait feature extraction is difficult due to the fact that the number of frames of the video sequence is small.
In particular, steps Bn and B1 described above may be performed using a trained gait video frame generation module, the operation of which is based on artificial intelligence techniques. The gait video frame generation module recognizes the motion curve trend according to the incomplete gait cycle obtained from the video, calculates the incomplete motion curve part by utilizing nonlinear regression, and generates a gait video frame.
Specifically, before the step a is performed to identify the human body in the video, the method further includes a step A0. of acquiring the video. In the step, the video can be acquired by using a camera in real time, or the video can be imported by the electronic equipment and an external storage.
Specifically, the motion human body recognition module, the human body joint tracking module, the self-coding learning module, the gait feature recognition module and the gait video frame generation module are obtained through training and learning by artificial intelligence technology. More specifically, the motion human body recognition module and the human body joint tracking module perform supervised learning training through the gait video frame generation module, and the self-coding learning module and the gait feature recognition module perform unsupervised learning training. The training of the above modules may employ artificial intelligence module training techniques in the art and will not be described in detail herein.
In some embodiments, the training of the gait feature recognition module includes: and E, extracting gait characteristics of the same human body at different angles from the gait characteristics stored in the step E, and randomly extracting the gait characteristics of any other human body for training. The end of the training process is determined that all human gait characteristics stored in step E have been used and the error function falls to a reasonable threshold. Therefore, in the training process, a large amount of gait video data do not need to be manually marked and checked, and an accurate gait feature recognition module can be obtained on the premise of greatly saving manpower and material resource investment in the training process.
According to the artificial intelligence-based gait recognition method, the human body is converted into the virtual human body formed by the mutual connection of the joint points, the gait characteristics of the human body are represented by the motion data of the joints of the human body, the recognition degree is improved for the extraction of the subsequent gait characteristics, the characteristics which are most valuable and can represent the human gait accurately can be obtained accurately, and the optimal gait characteristics can be extracted.
As shown in fig. 3, the gait recognition device based on artificial intelligence of the invention comprises a moving human body recognition module 102, a human body joint tracking module 103, a self-coding learning module 104, a gait feature recognition module 106 and a storage module 105.
The moving body recognition module 102 is used to recognize a body in a video. Specifically, the moving body recognition module 102 distinguishes the bodies in the video from other animals or items, accurately recognizes different bodies in real time as one or more bodies appear in the video, and performs identification and tracking. The above-mentioned sign can use the sign frame to frame human body out, and this sign frame can use the rectangle frame. Therefore, the user can conveniently and quickly see the human body from the video.
In some embodiments, the human body identified from the video is each generated with a corresponding ID number. Therefore, the gait features of different human bodies are marked and distinguished, and subsequent storage, retrieval and identification work is facilitated.
The human joint tracking module 103 is connected to the moving human body identifying module 102, and is used for identifying and tracking the main joint of the human body, and storing the position information of the main joint in the video frame to form a video sequence. Specifically, when a human body appears in the video, the human body joint tracking module 103 identifies the main joint of the human body in real time and stores the position of the main joint in the video frame, and as the human body joint tracking module 103 identifies and stores the position of the main joint in each video frame, the real-time tracking of the main joint of the human body is realized, and the recorded and stored position information of the main joint forms an important feature of the human body posture. In particular, the position information of the primary joint may be coordinate information of the primary joint in the video frame.
In some embodiments, the primary joints include ankle, hip, knee, wrist, elbow, and shoulder joints. Therefore, the gait characteristics of the human body are expanded to the motion characteristics of the joints of the whole body of the human body, the characteristic space of the gait characteristics is increased, meanwhile, the human body joints outside the joints of the lower limbs are identified and tracked, the key characteristics for identification are effectively increased, the identification degree of the gait characteristics is further increased, and the error rate of the gait characteristic identification can be remarkably reduced.
The self-coding learning module 104 is connected with the human joint tracking module 103 and is used for learning and extracting gait characteristics in the video sequence; the gait feature comprises position information of the primary joint in a video frame, and motion information of the primary joint in a video sequence. Specifically, the position information may be coordinate information of the main joint in the video frame, and the motion information may be vector information of the main joint in the two connected video frames.
The gait feature recognition module 106 is connected to the self-coding learning module 104, and is configured to compare the gait feature with the gait feature preset in the storage module 105, so as to obtain a recognition result.
In some embodiments, the gait feature recognition module 106 includes a metric function by which the distance of the gait feature in the feature space relative to the gait feature preset in the storage module is calculated, if the distance is in the interval [ a,1], the gait feature is a gait feature of a different angle of the human body to which the gait feature preset in the storage module belongs, if the distance is in the interval [0, a ], the gait feature does not match the gait feature preset in the storage module; the gait characteristics of the same human body preset in the storage module are in multi-Gaussian distribution in the characteristic space, the measurement function is a probability density function corresponding to the multi-Gaussian distribution, and the specific calculation method can be applied to the calculation method of the probability density function in the prior art. Specifically, the critical value a can be adjusted in practical application, and the user can make adjustment according to the error condition in the application. Thereby, accuracy and uniqueness of gait feature recognition are ensured.
The storage module 105 is connected with the gait feature recognition module 106 and is used for storing preset gait features; specifically, the storage module may utilize a database technology, and the database may identify the human body in the video through the above step a; B. identifying and tracking main joints of the human body, and storing the position information of the main joints in a video frame to form a video sequence; C. learning and extracting gait features in the video sequence; and E, storing gait characteristics establishment and acquisition. The self-coding learning module accurately retrieves and extracts the optimal gait characteristics from the high-dimensional characteristic space, and converts the high-dimensional time and space characteristics into low-dimensional characteristic vectors, so that the gait characteristics are effectively stored.
As shown in fig. 4, in some embodiments, the gait recognition device based on artificial intelligence further includes a gait video frame generation module 107, where the gait video frame generation module 107 is configured to interpolate a gait video frame with a frame number greater than or equal to 20 in the video when a duration of occurrence of gait of the human body in the video is less than 0.5 s. The gait video frame generation module 107 is a necessary supplement and reinforcement of the human joint tracking module 103, and overcomes the technical defect that the subsequent gait feature extraction is difficult due to the fact that the number of frames of the video sequence is small.
Specifically, the gait video frame generation module recognizes the curve trend of the motion of the gait video frame according to the incomplete gait cycle obtained from the video, and calculates the incomplete part of the motion curve by using nonlinear regression to generate the gait video frame.
In some embodiments, the gait recognition device based on artificial intelligence further comprises a video acquisition module 101, wherein the video acquisition module 101 is used for acquiring video; specifically, the video acquisition 101 may use a camera, and take a photograph of a situation of a use site through the camera and generate a video, for example, when the door access identification application is performed, the camera may use a plurality of cameras according to specific situations, so as to take a photograph of a human body at a plurality of angles on the use site; the video acquisition 101 may employ a data access port through which video is acquired from an electronic device or external storage, such as a certificate in a public security office.
In some embodiments, the artificial intelligence based gait recognition device further comprises a result output module 108, the result output module 108 for outputting a recognition result to inform a user. Specifically, the result output module 108 may use a computer, a mobile phone or a player to inform the user of the identification result in the form of text display or voice broadcast.
In some embodiments, the artificial intelligence based gait recognition device further comprises a retrieval module 109, the retrieval module 109 being coupled to the storage module 105 for retrieval of gait features when training and deploying the working modules, including the moving body recognition module 102, the body joint tracking module 103, the self-encoding learning module 104, the gait feature recognition module 106 and the gait video frame generation module 107. Thus, the retrieval speed of the gait features is increased by the retrieval module.
In some embodiments, as shown in fig. 5, the functions of the above-described modules may be implemented by providing a multi-function database 100 coupled to the gait feature recognition module 106 and the self-encoding learning module 104 using database technology. For example, the multi-function database 100 may include the storage module 105 and the retrieval module 109 to implement a function of storing gait features and a function of retrieving stored gait features.
In specific operation, the video acquisition module 101 acquires video and sends the video to the moving body recognition module 102; the motion human body recognition module 102 recognizes a human body from a video, and meanwhile, the human body joint tracking module 103 recognizes and tracks the position of a main joint of the human body, stores the position of the main joint in a video frame to form a video sequence, and sends the video sequence to the self-coding learning module 104 for extracting gait characteristics; after being extracted, the gait features are sent to the gait feature recognition module 106 for recognition, the gait feature recognition module 106 recognizes from the storage module 105, searches whether the gait features of the same human body are matched with the gait features to be recognized, and sends the recognition result to the result output module 108. In some embodiments, the extracted gait features may also be sent to the storage module 105 for storage.
According to the gait recognition device based on artificial intelligence, the human body and the main joints thereof are accurately recognized through the motion human body recognition module and the human body joint tracking module, and the motion of the main joints is accurately obtained through tracking; the self-coding learning module replaces manual design and extraction of gait features, so that optimal gait features can be accurately retrieved and extracted from a high-dimensional feature space, complete and robust extraction of the gait features is realized, and meanwhile, high-dimensional time and space features are converted into low-dimensional feature vectors, so that the calculation burden in gait feature retrieval is greatly reduced; and finally, realizing identification through a gait feature identification module, and improving identification precision. The motion human body recognition module, the human body joint tracking module, the self-coding learning module, the gait feature recognition module and the gait video frame generation module are obtained through training and learning by artificial intelligence technology.
When the method is specifically applied, the method can be applied to an entrance guard identification scene of a district, a camera is arranged at a gate of the district to acquire videos of visiting persons, a player can be further arranged to broadcast identification results, and gait characteristics of persons allowed to enter the district are stored in a storage module in advance; when a person walks to a gate of a district, a camera at the gate acquires a video, the video identifies a human body and extracts gait characteristics thereof through a motion human body identification module, a human body joint tracking module and a self-coding learning module, the gait characteristics of the person are identified through a gait characteristic identification module, if the gait characteristics are matched with the gait characteristics prestored in a storage module, the identification is successful, a player broadcasts a prompt of 'identification success, please enter', and meanwhile, the district gate is opened, and the person can enter the district; if the gait feature does not identify the matched gait feature in the storage module, the identification fails, the player broadcasts a prompt of 'identification failure', the cell door is still closed, the personnel cannot directly enter the cell, the personnel can be registered by the cell security and then put in the cell, or the personnel can enter the cell after other manual verification modes, and the safety of the cell is improved.
As shown in fig. 6, the gait recognition system based on artificial intelligence provided by the invention comprises a video acquisition terminal 201, a using terminal 203 and a server 202.
Wherein, the video acquisition terminal 201 is used for acquiring video; specifically, the video acquisition terminal 201 may be a camera, or a data access port, or the like.
The server 202 is configured to extract and identify gait features in the video, and compare the gait features with gait features preset in the storage module to obtain an identification result.
Specifically, the steps of extracting and identifying the gait features in the video, comparing the gait features with the gait features preset in the storage module, and obtaining the identification result include the following steps:
A. identifying a human body in the video;
B. identifying and tracking main joints of the human body, and storing the position information of the main joints in a video frame to form a video sequence;
C. learning and extracting gait features in the video sequence;
the gait feature comprises position information of the primary joint in the video frame, and motion information of the primary joint in the video sequence;
D. and comparing the gait characteristics with gait characteristics preset in a storage module to obtain an identification result.
Specifically, the position information may be coordinate information of the main joint in the video frame, and the motion information may be vector information of the main joint in the two connected video frames.
In some embodiments, the primary joints include ankle, hip, knee, wrist, elbow, and shoulder joints. Therefore, the gait characteristics of the human body are expanded to the motion characteristics of the joints of the whole body of the human body, the characteristic space of the gait characteristics is increased, meanwhile, the human body joints outside the joints of the lower limbs are identified and tracked, the key characteristics for identification are effectively increased, the identification degree of the gait characteristics is further increased, and the error rate of the gait characteristic identification can be remarkably reduced.
In some embodiments, the extracting and identifying gait features in the video, comparing the gait features with gait features preset in the storage module, and obtaining the identification result further includes the following steps:
bn. interpolates the acquired video to generate gait video frames having a frame number of 20 or more.
The following steps are also executed before the step Bn:
B1. judging the gait time of the human body in the acquired video, and executing the step Bn if the gait time is less than 0.5 s.
The step Bn is necessary supplement and reinforcement of the step B, and overcomes the technical defect that the follow-up gait feature extraction is difficult due to the fact that the number of frames of the video sequence is small.
Specifically, the steps Bn and B1 are performed after the step B, and may be performed using a trained gait video frame generation module. The gait video frame generation module recognizes the motion curve trend according to the incomplete gait cycle obtained from the video, calculates the incomplete motion curve part by utilizing nonlinear regression, and generates a gait video frame.
In some embodiments, the step d. Compares the gait feature with gait features preset in the storage module to obtain an identification result of:
calculating the distance between the gait feature in a feature space and the gait feature preset in the storage module through a measurement function, wherein if the distance is in a range [ a,1], the gait feature is the gait feature of different angles of a human body to which the gait feature preset in the storage module belongs, and if the distance is in a range [0, a), the gait feature is not matched with the gait feature preset in the storage module; the gait characteristics of the same human body preset in the storage module are in multi-Gaussian distribution in the characteristic space, the measurement function is a probability density function corresponding to the multi-Gaussian distribution, and the specific calculation method can be applied to the calculation method of the probability density function in the prior art. Specifically, the critical value a can be adjusted in practical application, and the user can make adjustment according to the error condition in the application. Thereby, accuracy and uniqueness of gait feature recognition are ensured.
Specifically, the storage module may be established by referring to the method for establishing the storage module in the gait feature recognition method.
The terminal 203 is used to monitor the gait recognition process and output the recognition result. The monitoring application range includes, but is not limited to, viewing acquired video, conditions of human body identified in the video, identification results of gait characteristics and the like. Specifically, the user terminal 203 may be a computer client or a mobile phone APP client.
When the gait feature recognition system is specifically applied, the gait feature recognition system can be applied to a legal certification department of public security, the video acquisition terminal is a data access port of a computer, the terminal is a computer client of a public security office, gait features of a suspected person are prestored in a storage module, a server extracts and recognizes the gait features of the video imported through the data access port of the computer, the gait features are compared with gait features of the suspected person preset in the storage module, the public security person views a recognition result from the computer client, if the recognition is successful, the suspected person can be locked, and if the recognition is not matched, the suspected person is not a target person.
The storage medium provided by the invention is provided with a computer program stored therein, and when the computer program runs on a computer, the computer is caused to execute the gait recognition method based on artificial intelligence. In particular, the storage medium may include, but is not limited to: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.
The server provided by the invention comprises a processor and a memory, wherein a computer program is stored in the memory, and the processor is used for executing the gait recognition method based on artificial intelligence by calling the computer program stored in the memory.
In the description of the present specification, reference to the terms "one embodiment," "certain embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
What has been described above is merely some embodiments of the present invention. It will be apparent to those skilled in the art that various modifications and improvements can be made without departing from the spirit of the invention.
Claims (9)
1. The gait recognition method based on artificial intelligence is characterized by comprising the following steps:
A. identifying a human body in the video;
B. identifying and tracking main joints of the human body, and storing the position information of the main joints in a video frame to form a video sequence;
C. learning and extracting gait features in the video sequence;
the gait feature comprises position information of the primary joint in the video frame, and motion information of the primary joint in the video sequence;
D. comparing the gait characteristics with gait characteristics preset in a storage module to obtain an identification result;
and D, comparing the gait characteristics with gait characteristics preset in a storage module to obtain an identification result as follows: calculating the distance between the gait feature in a feature space and the gait feature preset in the storage module through a measurement function, wherein if the distance is in a range [ a,1], the gait feature is the gait feature of different angles of a human body to which the gait feature preset in the storage module belongs, and if the distance is in a range [0, a), the gait feature is not matched with the gait feature preset in the storage module;
the gait characteristics of the same human body preset in the storage module are distributed in a multi-element Gaussian mode in a characteristic space.
2. The artificial intelligence based gait recognition method of claim 1, further comprising, after step B, step Bn. interpolating the video to generate gait video frames having a frame number greater than or equal to 20;
and before executing the step Bn, executing the step B1, judging the time length of the human body gait in the video, and if the time length of the human body gait is less than 0.5s, executing the step Bn.
3. The artificial intelligence based gait recognition method of claim 1, wherein the primary joints include ankle joints, hip joints, knee joints, wrist joints, elbow joints and shoulder joints.
4. The artificial intelligence based gait recognition method according to claim 1, wherein the a. The human body in the recognition video is generated with a corresponding ID number from the human body recognized in the video.
5. An artificial intelligence based gait recognition device for performing the artificial intelligence based gait recognition method as claimed in any one of claims 1 to 4, comprising a moving body recognition module, a body joint tracking module, a self-encoding learning module, a gait feature recognition module and a storage module;
the moving human body identification module is used for identifying human bodies in the video;
the human body joint tracking module is connected with the moving human body identification module and is used for identifying and tracking main joints of the human body and storing the position information of the main joints in a video frame to form a video sequence;
the self-coding learning module is connected with the human joint tracking module and is used for learning and extracting gait characteristics in the video sequence; the gait feature comprises position information of the primary joint in a video frame, and motion information of the primary joint in a video sequence;
the gait feature recognition module is connected with the self-coding learning module and is used for comparing the gait features with gait features preset in the storage module to obtain recognition results;
the storage module is connected with the gait feature recognition module and is used for storing preset gait features.
6. The artificial intelligence based gait recognition device of claim 5, further comprising a gait video frame generation module, wherein the gait video frame generation module is connected to the moving body recognition module, and is configured to interpolate a gait video frame with a frame number greater than or equal to 20 in the video when the duration of the human gait in the video is less than 0.5 s.
7. An artificial intelligence based gait recognition system for performing the artificial intelligence based gait recognition method as claimed in any one of claims 1 to 4, comprising a video acquisition terminal, a use terminal and a server;
the video acquisition terminal is used for acquiring videos;
the server is used for extracting gait characteristics in the video, comparing the gait characteristics with gait characteristics preset in the storage module and obtaining an identification result;
the using terminal is used for monitoring the gait recognition process and outputting a recognition result.
8. A storage medium having stored therein a computer program which, when run on a computer, causes the computer to perform the artificial intelligence based gait recognition method of any of claims 1 to 4.
9. A server comprising a processor and a memory, said memory having stored therein a computer program for executing the artificial intelligence based gait recognition method of any of claims 1 to 4 by invoking said computer program stored in said memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010334982.3A CN111476198B (en) | 2020-04-24 | 2020-04-24 | Gait recognition method, device, system, storage medium and server based on artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010334982.3A CN111476198B (en) | 2020-04-24 | 2020-04-24 | Gait recognition method, device, system, storage medium and server based on artificial intelligence |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111476198A CN111476198A (en) | 2020-07-31 |
| CN111476198B true CN111476198B (en) | 2023-09-26 |
Family
ID=71755578
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010334982.3A Active CN111476198B (en) | 2020-04-24 | 2020-04-24 | Gait recognition method, device, system, storage medium and server based on artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111476198B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111476198B (en) * | 2020-04-24 | 2023-09-26 | 广西安良科技有限公司 | Gait recognition method, device, system, storage medium and server based on artificial intelligence |
| CN112022166A (en) * | 2020-08-08 | 2020-12-04 | 司法鉴定科学研究院 | Human body identity recognition method and system based on medical movement disorder feature recognition |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101558996A (en) * | 2009-05-15 | 2009-10-21 | 天津大学 | Gait recognition method based on orthogonal projection three-dimensional reconstruction of human motion structure |
| CN103377366A (en) * | 2012-04-26 | 2013-10-30 | 哈尔滨工业大学深圳研究生院 | Gait recognition method and system |
| CN104463099A (en) * | 2014-11-05 | 2015-03-25 | 哈尔滨工程大学 | Multi-angle gait recognizing method based on semi-supervised coupling measurement of picture |
| KR101732402B1 (en) * | 2015-12-29 | 2017-05-08 | 건국대학교 산학협력단 | Gait monitoring apparatus by video observation and method of thereof |
| WO2017215618A1 (en) * | 2016-06-14 | 2017-12-21 | 中兴通讯股份有限公司 | Gait recognition method and apparatus |
| US10154228B1 (en) * | 2015-12-23 | 2018-12-11 | Amazon Technologies, Inc. | Smoothing video panning |
| CN111476198A (en) * | 2020-04-24 | 2020-07-31 | 广西安良科技有限公司 | Gait recognition method, device and system based on artificial intelligence, storage medium and server |
| WO2020249025A1 (en) * | 2019-06-12 | 2020-12-17 | 杭州海康威视数字技术股份有限公司 | Identity information determining method and apparatus, and storage medium |
| CN114821786A (en) * | 2022-04-27 | 2022-07-29 | 南京邮电大学 | Gait recognition method based on human body contour and key point feature fusion |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2438103B8 (en) * | 2005-01-14 | 2010-09-15 | Ultra Scan Corp | Multimodal fusion decision logic system. |
| JP4513819B2 (en) * | 2007-03-19 | 2010-07-28 | 株式会社日立製作所 | Video conversion device, video display device, and video conversion method |
| CN106331778B (en) * | 2015-07-06 | 2020-08-14 | 腾讯科技（深圳）有限公司 | Video recommendation method and device |
-
2020
- 2020-04-24 CN CN202010334982.3A patent/CN111476198B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101558996A (en) * | 2009-05-15 | 2009-10-21 | 天津大学 | Gait recognition method based on orthogonal projection three-dimensional reconstruction of human motion structure |
| CN103377366A (en) * | 2012-04-26 | 2013-10-30 | 哈尔滨工业大学深圳研究生院 | Gait recognition method and system |
| CN104463099A (en) * | 2014-11-05 | 2015-03-25 | 哈尔滨工程大学 | Multi-angle gait recognizing method based on semi-supervised coupling measurement of picture |
| US10154228B1 (en) * | 2015-12-23 | 2018-12-11 | Amazon Technologies, Inc. | Smoothing video panning |
| KR101732402B1 (en) * | 2015-12-29 | 2017-05-08 | 건국대학교 산학협력단 | Gait monitoring apparatus by video observation and method of thereof |
| WO2017215618A1 (en) * | 2016-06-14 | 2017-12-21 | 中兴通讯股份有限公司 | Gait recognition method and apparatus |
| WO2020249025A1 (en) * | 2019-06-12 | 2020-12-17 | 杭州海康威视数字技术股份有限公司 | Identity information determining method and apparatus, and storage medium |
| CN111476198A (en) * | 2020-04-24 | 2020-07-31 | 广西安良科技有限公司 | Gait recognition method, device and system based on artificial intelligence, storage medium and server |
| CN114821786A (en) * | 2022-04-27 | 2022-07-29 | 南京邮电大学 | Gait recognition method based on human body contour and key point feature fusion |
Non-Patent Citations (1)
| Title |
|---|
| 步态序列图像中人体下肢关节点的定位;王健等;《计算机仿真》;20110215(第02期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111476198A (en) | 2020-07-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8135220B2 (en) | Face recognition system and method based on adaptive learning | |
| Wang et al. | Silhouette analysis-based gait recognition for human identification | |
| US20180129873A1 (en) | Event detection and summarisation | |
| CN111476198B (en) | Gait recognition method, device, system, storage medium and server based on artificial intelligence | |
| JP2006133946A (en) | Moving object recognition device | |
| Zhang et al. | A novel serial multimodal biometrics framework based on semisupervised learning techniques | |
| CN111144165B (en) | Gait information identification method, system and storage medium | |
| US20220148335A1 (en) | Disentangled Representations For Gait Recognition | |
| Shajina et al. | Human gait recognition and classification using time series shapelets | |
| CN110599129A (en) | Campus attendance checking method, device, identification terminal and system based on image tracking | |
| CN113920326A (en) | Tumble behavior identification method based on human skeleton key point detection | |
| CN111862413A (en) | Method and system for realizing epidemic situation resistant non-contact multidimensional identity rapid identification | |
| Yan et al. | Human-object interaction recognition using multitask neural network | |
| CN114821786A (en) | Gait recognition method based on human body contour and key point feature fusion | |
| CN115100684A (en) | Clothes-changing pedestrian re-identification method based on attitude and style normalization | |
| Li et al. | Dynamic long short-term memory network for skeleton-based gait recognition | |
| Bakchy et al. | Human identification on the basis of gait analysis using Kohonen self-organizing mapping technique | |
| CN115578749A (en) | Gait feature extraction and training method, gait recognition method and related equipment | |
| CN115797451A (en) | Acupuncture point identification method, device and equipment and readable storage medium | |
| WO2022134916A1 (en) | Identity feature generation method and device, and storage medium | |
| Baptista et al. | Anticipating suspicious actions using a small dataset of action templates | |
| Sivapalan | Human identification from video using advanced gait recognition techniques | |
| Ahmed et al. | Survey: gait recognition based on image energy | |
| Matta et al. | A behavioural approach to person recognition | |
| Nandy et al. | A novel approach to human gait recognition using possible speed invariant features |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |