CN111738060A - Human gait recognition system based on millimeter wave radar - Google Patents
Human gait recognition system based on millimeter wave radar Download PDFInfo
- Publication number
- CN111738060A CN111738060A CN202010378975.3A CN202010378975A CN111738060A CN 111738060 A CN111738060 A CN 111738060A CN 202010378975 A CN202010378975 A CN 202010378975A CN 111738060 A CN111738060 A CN 111738060A
- Authority
- CN
- China
- Prior art keywords
- subsystem
- gait
- wave radar
- millimeter wave
- classification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
- G06V40/25—Recognition of walking or running movements, e.g. gait recognition
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G06F18/24155—Bayesian classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
Abstract
The invention belongs to the technical field of security monitoring, and particularly relates to a human body gait recognition system based on a millimeter wave radar. The system of the invention comprises: the system comprises a millimeter wave radar subsystem, a data processing subsystem, a characteristic database subsystem, a classification identification subsystem and an interactive interface subsystem; the millimeter wave radar subsystem is used for transmitting and receiving millimeter waves to obtain digital intermediate frequency original data containing human gait information; the data processing subsystem is used for carrying out digital signal processing on the original data to extract radar features; the characteristic database subsystem is used for storing and updating a characteristic data set of the set gait; the classification and identification subsystem is used for performing gait classification and identification by using a traditional classification method and a deep learning method; the interactive interface subsystem is used for interactively controlling and displaying the gait recognition result. The gait characteristic analysis method is focused on gait characteristic analysis of indoor and outdoor human bodies, and is suitable for human body walking state analysis, walking characteristic identification, identity identification and the like in intelligent home and intelligent security scenes.
Description
Technical Field
The invention belongs to the technical field of security monitoring, and particularly relates to a human body gait recognition system based on a millimeter wave radar.
Background
The existing main monitoring and identifying technology in the field of security monitoring is based on monitoring and identifying of a camera, however, the camera is easily influenced by illumination conditions and shelters, is poor in concealment and easy to damage, and is not suitable for privacy sensitive scenes.
Gait recognition is a non-contact biological feature recognition technology, and identity recognition and walking characteristic analysis can be carried out through the walking posture habit of people. Compared with other biological feature recognition technologies such as fingerprint recognition, iris recognition, face recognition and the like, the gait recognition has the advantages of non-contact, long distance and difficulty in disguising. The feasibility of gait recognition technology derives from the specificity of different human walking poses, and studies by professor mackerson at university of south ampton, uk, showed that each person has different gait characteristics because of subtle differences in muscle strength, tendon and bone length, bone density, visual acuity, coordination ability, experience, weight, center of gravity, degree of muscle or bone damage, physiological conditions, and the "style" in which the person walks. The existing gait recognition products and solutions are mainly based on visible light cameras, such as gait retrieval all-in-one machine equipment-water drip magic of the galaxy water drip science and technology company, and can be used for carrying out target tracking and gait recognition on people in videos. However, the gait recognition scheme based on the visible light camera is limited by the defects of camera monitoring, and the application scenes are limited.
How to overcome the not enough of current gait recognition scheme, promote the accuracy and the practicality of gait recognition technology are the problem that awaits solution at present.
Disclosure of Invention
The invention aims to provide a system for identifying the walking state, the walking mode and the identity attribute of a freely walking human body indoors and outdoors by utilizing millimeter waves.
The invention provides a human body gait recognition system based on a millimeter wave radar, which comprises: the system comprises a millimeter wave radar subsystem, a data processing subsystem, a characteristic database subsystem, a classification identification subsystem and an interactive interface subsystem; the millimeter wave radar subsystem is used for transmitting and receiving millimeter waves to obtain digital intermediate frequency original data containing human gait information; the data processing subsystem is used for carrying out digital signal processing on the original data to extract radar features; the characteristic database subsystem is used for storing and updating a characteristic data set of the set gait; the classification and identification subsystem is used for performing gait classification and identification by using a traditional classification method and a deep learning method; the interactive interface subsystem is used for interactively controlling and displaying the gait recognition result.
Each subsystem of the system has real-time working capability, namely the millimeter wave radar subsystem can output original data in real time; the data processing subsystem can process the original data in real time, detect a moving target and extract the gait characteristics of the target; the characteristic database subsystem can store and update target gait characteristics in real time; the classification and identification subsystem can classify and identify gait in real time and update the classification model through online learning; the interactive interface subsystem can display the gait recognition result in real time.
The feature data set for a set gait comprises: gait with different walking speeds, such as fast walking, slow walking, normal walking, running, random walking and the like; gait in different walking modes, such as outward splayed, inward splayed, lameness, clubbing, normal walking and the like; and the gait of different human bodies corresponds to the identity attribute.
In the millimeter wave radar subsystem, radar parameters mainly comprise the number N of transmitting antennasTxNumber of receiving antennas NRxNumber of FM cycles per frame NcFrequency-modulated starting frequency f1Frequency modulation slope KsFrequency modulation period TcFrame period TfADC sampling rate FsNumber of ADC samples per FM periodadcEtc., which may be based on the maximum measured distance d of the application scenariomaxMaximum measurement velocity vmaxDistance resolution dresVelocity resolution vresFrame rate frateEtc. are determined by the requirements.
In the invention, the main process of acquiring the original data by the millimeter wave radar subsystem is as follows: n is a radical ofTxThe transmitting antenna periodically transmits linear frequency-modulated continuous wave, which is reflected by human body and then passes through NRxA receiving antenna receives the signal and then converts NTx×NRxThe echo signals of each channel and the corresponding transmitting signals are subjected to frequency mixing to obtain intermediate frequency signals, and the intermediate frequency signals are subjected to low-pass filtering and analog-digital conversion to obtain original data of human gait.
In the invention, the process of processing the original data and extracting the gait characteristics by the data processing subsystem mainly comprises the following steps: fast Fourier Transform (FFT), target detection, target clustering, target positioning and tracking, a series of feature extraction and the like, wherein the extracted radar gait features mainly comprise: the method comprises the following steps of obtaining a distance Doppler image, a Doppler-time image, a distance-time image, an azimuth-time image, a pitch angle-time image, a 3D point cloud and other characteristics, and further estimating the characteristics of pace, step length, height, body type and the like on the basis of the characteristics.
In the invention, the characteristic database subsystem stores data according to asynchronous attributes, and the gait attributes comprise: the identity, walking speed, walking mode and the like of the human body, and effective data obtained after feature comparison and screening by the classification and identification subsystem are stored.
In the invention, the classification and identification subsystem comprises a traditional method and a deep learning method; when the number of samples is less and is not enough to carry out deep learning training, the traditional method (such as dynamic time warping [1] and Bayesian classification [2]) is adopted to carry out less sample classification; and when the data quantity is enough, a deep learning method is adopted for classification.
In the classification and identification subsystem, the deep learning method comprises the design and optimization of an artificial neural network, a classification model is obtained by training when a data set meets the requirement of data quantity, and the process of retraining an update model, calling the classification model obtained by training to perform gait identification and the like is carried out when the data set meets the online learning update condition.
In the invention, the system has a low power consumption mode with a low frame rate and a working mode with a high frame rate; after the system is started, the system is in a low power consumption mode by default, and only the millimeter wave radar subsystem and the signal processing subsystem work at a low frame rate and are used for detecting whether a moving target exists in a detectable range; the characteristic database subsystem and the classification and identification subsystem do not work; when the moving target is detected to exist, the operation mode is switched to be the working mode, the millimeter wave radar subsystem and the signal processing subsystem work at a high frame rate, and the characteristic database subsystem and the classification and identification subsystem work normally.
In the invention, the interactive interface subsystem also comprises other applications for displaying the gait recognition result, such as identity recognition, walking state recognition, leg and foot problem detection and the like.
In the invention, the hardware of the system mainly comprises a millimeter wave radar module, a signal processing module, a data storage module, a deep learning module, a Graphical User Interface (GUI) module and the like. The millimeter wave radar module mainly comprises a millimeter wave radar chip, a transmitting and receiving antenna, a phase-locked loop, a frequency converter, an analog-to-digital converter, an MCU, a communication interface and the like; the signal processing module can be an MCU, a DSP, an embedded device, a smart phone, a computer and other devices with enough signal processing capacity, and has an interface and a function for communicating with the millimeter wave radar module and the data storage module; the data storage module can be a mechanical hard disk, a solid state hard disk and other devices with data storage capacity, and simultaneously has interfaces and functions for communicating with the signal processing module, the deep learning module and the GUI module; the deep learning module can be embedded equipment, a smart phone, a computer, a server and other equipment with deep learning support, and simultaneously has an interface and a function for communicating with the signal processing module, the data storage module and the GUI module; the GUI module can be a display, a display screen and other equipment capable of providing display and interaction functions, and is provided with an interface and a function for communicating with the data storage module and the deep learning module.
The system of the invention has the following working procedures:
(1) establishing wired or wireless communication between subsystems;
(2) setting a gait type, radar parameters and an artificial neural network according to the application scene requirements;
(3) the millimeter wave radar subsystem periodically transmits linear frequency modulation continuous waves, and processing such as frequency mixing, low-pass filtering, analog-to-digital conversion and the like is carried out on echoes reflected by a human body, and intermediate-frequency original data containing human body gait information are output;
(4) the digital signal processing subsystem carries out operations such as FFT, target detection, target clustering, target positioning and tracking, a series of feature extraction and the like on the original data, and extracts various features representing gait; the characteristic database subsystem is used for storing and updating a characteristic data set of the set gait;
(5) when the digital signal processing subsystem detects a moving target, the millimeter wave radar subsystem and the digital signal subsystem are switched from a low power consumption mode to a normal working mode, and the characteristic database subsystem starts to store effective gait characteristic data;
(6) when the number of gait special diagnosis samples is small and insufficient for deep learning training, the classification and identification subsystem adopts a traditional method to carry out gait identification of few sample matching; and when the data volume is enough, sending the data set into a designed artificial neural network for training and fitting to obtain and store a classification model, and then calling the classification model to combine with a traditional method for gait recognition.
The invention focuses on the gait characteristic analysis of indoor and outdoor human bodies, is suitable for the application of human body walking state analysis, walking characteristic identification, identity identification and the like in intelligent home and intelligent security scenes, and has the following advantages:
(1) the low-power millimeter wave which does not generate ionization reaction is adopted, so that the method is harmless to human bodies;
(2) the method is sensitive to the characteristics of the movement gait, and the discrimination and the accuracy for identifying the asynchronous state are high;
(3) non-contact sensing, sensing distance is long;
(4) the device is not influenced by illumination, can penetrate through a non-metal shelter with a certain thickness, and is wide in applicable scene;
(5) the human body which walks freely can be identified;
(6) the concealment is high, and the detection and the identification can be finished under the condition that the detected human body target is unknown;
(7) low-dimensional features are extracted, human body image features are not extracted, and privacy is good;
(8) the system core component has small size and strong integration capability, and can be integrated with a plurality of devices with data processing.
Drawings
Fig. 1 is a schematic view of an application scenario in an embodiment of the present invention.
Fig. 2 is a flow chart of an implementation process in an embodiment of the invention.
FIG. 3 is a schematic diagram of a system architecture in an embodiment of the invention, wherein the reference numbers: the system comprises a millimeter wave radar subsystem 1, a data processing subsystem 2, a characteristic database subsystem 3, a classification identification subsystem 4 and an interactive interface subsystem 5.
Figure 4 is a set gait signature diagram in an embodiment of the invention. Wherein, the walking stick is used for (a) walking slowly, (b) walking quickly, (c) running, (d) walking freely, (e) walking normally, (f) lameness, and (g) splayed outside; (A) the gait characteristics of 6 persons are shown as (B), (C), (D), (E) and (F).
Fig. 5 is a flow diagram of raw data signal processing in an embodiment of the invention.
Fig. 6 is a diagram of a convolutional neural network structure in an embodiment of the present invention.
Fig. 7 is a gait recognition confusion matrix (gaits corresponding to different walking speeds include fast walking, running, slow walking, and random forward walking) in an embodiment of the invention.
Fig. 8 is a gait recognition confusion matrix (gait includes three types of normal walking, lameness and out-toed eight corresponding to different walking modes) in the embodiment of the invention.
Fig. 9 is a gait recognition confusion matrix (corresponding to 6 persons) in an embodiment of the invention.
FIG. 10 is a system workflow diagram in an embodiment of the invention.
Detailed Description
The invention is further illustrated by the following examples and figures. The scope of the invention is not limited to the examples described below.
An application scenario of an embodiment of the present invention is shown in fig. 1, and a specific implementation process is shown in fig. 2, including:
(1) arranging a millimeter wave radar and signal processing integrated module in a scene, connecting a computer through a USB data line, establishing communication and starting;
(2) when a pedestrian target passes through the detection area, the millimeter wave radar detects the moving target and starts to acquire original data containing gait information of the pedestrian;
(3) utilizing a signal processing DSP chip to perform digital signal processing on the original data and extract various gait characteristics, wherein the Doppler-time diagram characteristics are taken as an example in the embodiment;
(4) transmitting the gait feature data to a computer through a USB3.0 interface to establish a feature data set;
(5) inputting a sufficient number of feature data sets into a designed convolutional neural network training classification model on a computer;
(6) and calling a classification model on a computer to classify the gait features transmitted in real time to realize gait recognition, and displaying the recognized identity, walking speed type and walking mode type of the pedestrian.
The system structure of this embodiment is shown in fig. 3, and is composed of a millimeter wave radar, a signal processor, and a computer. In the structure, a millimeter wave radar subsystem and a data processing subsystem are integrated into a terminal, and the millimeter wave radar subsystem is used for transmitting and receiving millimeter waves to obtain digital intermediate frequency original data containing human gait information; the data processing subsystem is used for carrying out digital signal processing on the original data to extract radar features; the characteristic database subsystem, the classification identification subsystem and the interactive interface subsystem are integrated in a computer, and the characteristic database subsystem is used for storing and updating a characteristic data set of the set gait; the classification and identification subsystem is used for performing gait classification and identification by using a traditional classification method and a deep learning method; the interactive interface subsystem is used for interactively controlling and displaying the gait recognition result.
The gait definitions and corresponding characteristic diagrams of different walking speeds and different walking modes of the embodiment are shown in fig. 4, the gait of different walking speeds comprises four types of walking quickly, running slowly and walking forward randomly, and the gait of different walking modes comprises three types of normal walking, limping and out-toed splayfoot. There are also gait characteristics of 6 persons (i.e., A, B, C, D, E, F).
The radar parameter setting of the embodiment comprises two transmitting antennas and four receiving antennas, the frequency modulation starting frequency is 77.666GHz, the frequency modulation slope is 96.80009, the frequency modulation period is 300us, the number of frequency modulation periods of each frame is 64, the frame period is 20ms, the ADC sampling rate is 3.515MHz, the number of ADC sampling points of each frequency modulation period is 128, the maximum measurement distance of the radar determined by the parameters is 5.44m, the maximum measurement speed is 3.14m/s, the distance resolution is 4.25cm, the speed resolution is 9.83cm/s, and the frame rate is 20.
The original data signal processing flow of this embodiment is as shown in fig. 5, first perform range FFT and doppler FFT on one frame of data of each channel to obtain range-doppler complex amplitude distribution of each channel, calculate an incoherent superimposed channel average range-doppler plot, perform target detection on the channel average range-doppler plot to obtain a target point with the maximum amplitude, splice the doppler distributions corresponding to the target point according to a time sequence to obtain a doppler-time plot of the target, and capture the doppler-time plot of a fixed frame length as a classification feature.
The deep learning method of this embodiment employs a Convolutional Neural Network (CNN), which has a structure shown in fig. 6, where INPUT, COV, MAX-POOL, RELU, BN, FC, SOFTMAX, OUTPUT respectively represent an INPUT layer, a convolutional layer, a MAX pooling layer, an activation function layer, a batch normalization layer, a full connection layer, a classification layer, and an OUTPUT layer. The processing process of the gait feature map in the CNN is as follows: firstly, inputting feature maps (corresponding to N two-dimensional features) with N channels and 64 x 64 sizes into an input layer; then, performing convolution and zero padding operation of 'same' by a convolution kernel of 3 multiplied by 3 on a convolution layer with the depth of 16 to obtain a feature map of 64 multiplied by 16; after passing through the activation function layer and the batch normalization layer, entering a 1 st maximum pooling layer with a sampling core of 2 multiplied by 2 and a step length of 2 for down-sampling to obtain a characteristic diagram of 32 multiplied by 16; then, the 2 nd convolutional layer with the depth of 32 is convoluted by a 3 multiplied by 3 convolutional kernel and zero padding is carried out on the 'same', and a characteristic diagram of 32 multiplied by 32 is obtained; after passing through the activation function layer and the batch normalization layer, entering a 2 nd largest pooling layer with a sampling core of 2 multiplied by 2 and a step length of 2 for down-sampling to obtain a 16 multiplied by 32 characteristic diagram; then, the data enters a 3 rd convolution layer with the depth of 64, is convoluted by a 3 x 3 convolution kernel, and is subjected to zero filling by 'same', so that a 16 x 64 characteristic diagram is obtained; after passing through the activation function layer and the batch normalization layer, entering a 3 rd maximum pooling layer with a sampling core of 2 multiplied by 2 and a step length of 2 for down-sampling to obtain a characteristic diagram of 8 multiplied by 64; then the data enters a 1 st full-connection layer with the neuron number of 128, and after 0.5 times of random discard (Dropout), the data is sent to a 2 nd full-connection layer with the neuron number of Nclass, and the data is sent to a classification layer after passing through a 4 th activation function layer, and a classification result is obtained and sent to an output layer.
In this embodiment, 100 data samples of each gait of different walking speeds and different walking modes are collected as shown in fig. 4, a convolutional neural network as shown in fig. 6 is adopted, and a training test ratio is set to be 6: fig. 7 and 8 show the classification results obtained. The average classification accuracy of four walking speed gaits of forward walking freely, fast walking, slow walking and running is 97.5%, wherein only 4 samples of forward walking freely are classified into slow walking by mistake, and other samples are classified correctly, which shows that the designed gait recognition system and method in the embodiment can be used for well distinguishing different walking speed gaits of people; the average classification accuracy of the gaits of the three walking modes of the splayfoot, the normal walking and the lameness is 97.5%, wherein the classification of the lameness is larger than that of the other two walking modes, 40 test samples are classified correctly, 1 sample of the normal walking gaits is classified into the splayfoot and the lameness, and 1 sample of the splayfoot is classified into the lameness, so that the designed gait recognition system and method in the embodiment can be used for well distinguishing the gaits of different walking modes of people. In addition, about 50 data samples of each of the six human (A, B, C, D, E, F) gaits were collected as shown in fig. 4, and a convolutional neural network as shown in fig. 6 was used to set the training test ratio to 8: 2, the obtained classification result is shown in fig. 9, the average accuracy of identification recognition of six persons is 95%, the gait test samples of A, B and E are all correctly recognized, one test sample of C is classified as B, one test sample of D is classified as F, and one test sample of F is classified as B, which shows that the gait recognition system and the method designed in the embodiment can be used for well recognizing the identities of a plurality of persons by using the gait.
The overall work flow of the system in this embodiment is shown in fig. 10, and the work flow is as follows:
1. the method comprises the steps of firstly starting a radar, transmitting a linear Frequency Modulation Continuous Wave (FMCW) of periodic frequency sweep after parameter configuration, receiving target scene echo data, obtaining intermediate frequency data after frequency mixing, filtering and ADC sampling, judging whether effective data judgment conditions are met, if not, closing the radar and restarting, and if so, sending the intermediate frequency data to a data processing subsystem;
2. performing operations such as distance FFT, interesting distance range extraction (ROI), moving target Display (DTI), Doppler FFT and the like on the intermediate frequency data of each channel obtained by the transceiving antenna array, calculating an average distance Doppler spectrum of the channel, then performing target detection and target Doppler spectrum calculation, filling the Doppler spectrum into a time frame window with a fixed length, if the frame window is not full, continuously sending new Doppler spectrum data, updating the frame window according to a first-in-queue principle when the frame window is full, then performing effective action judgment, if the effective action is not obtained, continuously updating the frame window, and if the effective action is obtained, sending the Doppler-time spectrum data in the frame window at the moment into a subsystem integrating a feature database, a classification identification and an interactive interface;
3. standardizing the frame window characteristics, storing characteristic images into a gait data set, sending the gait data set into a convolutional neural network to train to obtain a classification model when the number of samples in the data set reaches the requirement of the minimum number of samples of each type, classifying the standardized frame window characteristics when the classification model exists, and displaying the gait characteristics and the classification result on an interactive interface.
Reference to the literature
[1] Chinese academy of sciences remote sensing and digital Earth institute, remote sensing image time series clustering method based on cloud pixel number division, China, CN201410235145. XP 2014-08-13.
[2] SiAn university of science and technology dynamic data flow classification method based on Bayesian network, China, CN201910571906.1[ P ]. 2019-10-15.
Claims (10)
1. A human gait recognition system based on millimeter wave radar is characterized by comprising: the system comprises a millimeter wave radar subsystem, a data processing subsystem, a characteristic database subsystem, a classification identification subsystem and an interactive interface subsystem; the millimeter wave radar subsystem is used for transmitting and receiving millimeter waves to obtain digital intermediate frequency original data containing human gait information; the data processing subsystem is used for carrying out digital signal processing on the original data and extracting radar features; the characteristic database subsystem is used for storing and updating a characteristic data set of the set gait; the classification and identification subsystem is used for performing gait classification and identification by using a traditional classification method and a deep learning method; the interactive interface subsystem is used for interactively controlling and displaying the gait recognition result;
each subsystem of the system has real-time working capability, namely the millimeter wave radar subsystem can output original data in real time; the data processing subsystem can process the original data in real time, detect a moving target and extract the gait characteristics of the target; the characteristic database subsystem can store and update target gait characteristics in real time; the classification and identification subsystem can classify and identify gait in real time and update the classification model through online learning; the interactive interface subsystem can display the gait recognition result in real time;
wherein the feature data set for the set gait comprises: characteristic data sets of gaits with different walking speeds, gaits with different walking modes and gaits of different human bodies; the gait with different walking speeds comprises fast walking, slow walking, normal walking, running and random walking; the gaits of different walking modes comprise an external splayfoot, an internal splayfoot, lameness, a pestle walking stick and normal walking; different human body gaits correspond to identity attributes;
in the millimeter wave radar subsystem, radar parameters mainly comprise the number N of transmitting antennasTxNumber of receiving antennas NRxNumber of FM cycles per frame NcFrequency-modulated starting frequency f1Frequency modulation slope KsFrequency modulation period TcFrame period TfADC sampling rate FsNumber of ADC samples per FM periodadcThese parameters are dependent on the maximum measured distance d of the application scenariomaxMaximum measurement velocity vmaxDistance resolution dresVelocity resolution vresFrame rate frateAnd (4) determining the requirement.
2. The millimeter wave radar-based human body gait recognition system according to claim 1, characterized in that the millimeter wave radar subsystem mainly processes to acquire raw data as follows: n is a radical ofTxThe transmitting antenna periodically transmits linear frequency-modulated continuous wave, which is reflected by human body and then passes through NRxA receiving antenna receives the signal and then converts NTx×NRxAnd the echo signals of each channel and the corresponding transmitting signals are subjected to frequency mixing to obtain intermediate frequency signals, and the intermediate frequency signals are subjected to low-pass filtering and analog-digital conversion to obtain the original data of human gait.
3. The millimeter wave radar-based human body gait recognition system according to claim 1, wherein the data processing subsystem processes the raw data to extract gait features, including Fast Fourier Transform (FFT), target detection, target clustering, target location tracking and a series of feature extraction, and the extracted radar gait features include: and further estimating the pace, step length, height and body shape characteristics on the basis of the characteristics.
4. The millimeter-wave radar-based human body gait recognition system according to claim 1, characterized in that the characteristic database subsystem stores data according to different gait attributes, the gait attributes comprise human body identity, walking speed and walking mode, and the valid data after feature comparison and screening by the classification recognition subsystem is stored.
5. The millimeter-wave radar-based human body gait recognition system according to claim 1, wherein the classification recognition subsystem comprises a traditional method and a deep learning method, and when the number of samples is less and insufficient for deep learning training, the traditional method is adopted for classification; and when the data quantity is enough, a deep learning method is adopted for classification.
6. The millimeter-wave radar-based human body gait recognition system according to claim 5, characterized in that the deep learning method in the classification recognition subsystem comprises the design and optimization of an artificial neural network, the classification model is trained when the data set meets the data volume requirement, the update model is retrained when the data set meets the online learning update condition, and the trained classification model is called to perform the gait recognition process.
7. The millimeter wave radar-based human body gait recognition system according to claim 1, characterized by a low power consumption mode with a low frame rate and a high frame rate operation mode; after the system is started, the system is in a low power consumption mode by default, and only the millimeter wave radar subsystem and the signal processing subsystem work at a low frame rate and are used for detecting whether a moving target exists in a detectable range; the characteristic database subsystem and the classification and identification subsystem do not work; when the moving target is detected to exist, the operation mode is switched to be the working mode, the millimeter wave radar subsystem and the signal processing subsystem work at a high frame rate, and the characteristic database subsystem and the classification and identification subsystem work normally.
8. The millimeter wave radar-based human gait recognition system according to claim 1, characterized in that the interactive interface subsystem further comprises displaying other applications depending on the gait recognition result.
9. The millimeter wave radar-based human body gait recognition system according to claim 1, characterized in that the hardware components of the system comprise a millimeter wave radar module, a signal processing module, a data storage module, a deep learning module, and a Graphical User Interface (GUI) module; the millimeter wave radar module mainly comprises a millimeter wave radar chip, a transmitting and receiving antenna, a phase-locked loop, a frequency converter, an analog-to-digital converter, an MCU and a communication interface; the signal processing module can be an MCU, a DSP, an embedded device, a smart phone, a computer and other devices with enough signal processing capacity, and has an interface and a function for communicating with the millimeter wave radar module and the data storage module; the data storage module is equipment with data storage capacity, comprises a mechanical hard disk and a solid state hard disk, and simultaneously has interfaces and functions for communicating with the signal processing module, the deep learning module and the GUI module; the deep learning module is embedded equipment, a smart phone, a computer, a server and other equipment with deep learning support, and has an interface and a function for communicating with the signal processing module, the data storage module and the GUI module; the GUI module is a display, a display screen and other equipment capable of providing display and interaction functions, and is provided with an interface and a function for communicating with the data storage module and the deep learning module.
10. The millimeter wave radar-based human body gait recognition system according to claim 1, characterized in that the workflow is as follows:
(1) establishing wired or wireless communication between subsystems;
(2) setting a gait type, radar parameters and an artificial neural network according to the application scene requirements;
(3) the millimeter wave radar subsystem periodically transmits linear frequency modulation continuous waves, and processing such as frequency mixing, low-pass filtering, analog-to-digital conversion and the like is carried out on echoes reflected by a human body, and intermediate-frequency original data containing human body gait information are output;
(4) the digital signal processing subsystem carries out operations such as FFT, target detection, target clustering, target positioning and tracking, a series of feature extraction and the like on the original data, and extracts various features representing gait; the characteristic database subsystem is used for storing and updating a characteristic data set of the set gait;
(5) when the digital signal processing subsystem detects a moving target, the millimeter wave radar subsystem and the digital signal subsystem are switched from a low power consumption mode to a normal working mode, and the characteristic database subsystem starts to store effective gait characteristic data;
(6) when the number of gait special diagnosis samples is small and insufficient for deep learning training, the classification and identification subsystem adopts a traditional method to carry out gait identification of few sample matching; and when the data volume is enough, sending the data set into a designed artificial neural network for training and fitting to obtain and store a classification model, and then calling the classification model to combine with a traditional method for gait recognition.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010378975.3A CN111738060A (en) | 2020-05-07 | 2020-05-07 | Human gait recognition system based on millimeter wave radar |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010378975.3A CN111738060A (en) | 2020-05-07 | 2020-05-07 | Human gait recognition system based on millimeter wave radar |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111738060A true CN111738060A (en) | 2020-10-02 |
Family
ID=72648175
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010378975.3A Pending CN111738060A (en) | 2020-05-07 | 2020-05-07 | Human gait recognition system based on millimeter wave radar |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111738060A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112274140A (en) * | 2019-07-25 | 2021-01-29 | 北京海益同展信息科技有限公司 | Abnormity monitoring method, device and system |
| CN112444805A (en) * | 2020-11-01 | 2021-03-05 | 复旦大学 | Distributed multi-target detection, positioning tracking and identity recognition system based on radar |
| CN112528752A (en) * | 2020-11-17 | 2021-03-19 | 芜湖美的厨卫电器制造有限公司 | Method, device, processor and water heater for recognizing gesture |
| CN112541398A (en) * | 2020-11-17 | 2021-03-23 | 芜湖美的厨卫电器制造有限公司 | Method, device, processor and water heater for identifying user based on gait |
| CN112556193A (en) * | 2020-11-17 | 2021-03-26 | 芜湖美的厨卫电器制造有限公司 | Control method and device for water heater, processor and water heater |
| CN112700619A (en) * | 2020-12-29 | 2021-04-23 | 潍坊医学院 | Intelligent monitoring method and system for falling of old people |
| CN113591760A (en) * | 2021-08-09 | 2021-11-02 | 路晟悠拜(重庆)科技有限公司 | Far-field multi-human-body gait monitoring method based on millimeter waves |
| CN113822241A (en) * | 2021-11-22 | 2021-12-21 | 江苏邦邦智能科技有限公司 | Gait data acquisition method, walking aid device and computer readable storage medium |
| CN114857916A (en) * | 2022-05-24 | 2022-08-05 | 佛山市顺德区美的电子科技有限公司 | Ground processing method, ground processing device, electrical equipment and medium |
| CN115049039A (en) * | 2021-03-08 | 2022-09-13 | 北京金茂绿建科技有限公司 | State recognition method based on neural network, neural network training method and device |
| CN115291184A (en) * | 2022-10-08 | 2022-11-04 | 四川启睿克科技有限公司 | Attitude monitoring method combining millimeter wave radar and deep learning |
| CN117148309A (en) * | 2023-11-01 | 2023-12-01 | 德心智能科技(常州)有限公司 | Millimeter wave radar human body sensing method and system applied to community grid inspection |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180322351A1 (en) * | 2016-11-21 | 2018-11-08 | George Shaker | System and method for sensing with millimeter waves |
| CN110687816A (en) * | 2019-10-31 | 2020-01-14 | 复旦大学 | Intelligent household control system and method based on millimeter wave radar |
| CN110765974A (en) * | 2019-10-31 | 2020-02-07 | 复旦大学 | Micro-motion gesture recognition method based on millimeter wave radar and convolutional neural network |
-
2020
- 2020-05-07 CN CN202010378975.3A patent/CN111738060A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180322351A1 (en) * | 2016-11-21 | 2018-11-08 | George Shaker | System and method for sensing with millimeter waves |
| CN110687816A (en) * | 2019-10-31 | 2020-01-14 | 复旦大学 | Intelligent household control system and method based on millimeter wave radar |
| CN110765974A (en) * | 2019-10-31 | 2020-02-07 | 复旦大学 | Micro-motion gesture recognition method based on millimeter wave radar and convolutional neural network |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112274140A (en) * | 2019-07-25 | 2021-01-29 | 北京海益同展信息科技有限公司 | Abnormity monitoring method, device and system |
| CN112444805A (en) * | 2020-11-01 | 2021-03-05 | 复旦大学 | Distributed multi-target detection, positioning tracking and identity recognition system based on radar |
| CN112528752A (en) * | 2020-11-17 | 2021-03-19 | 芜湖美的厨卫电器制造有限公司 | Method, device, processor and water heater for recognizing gesture |
| CN112541398A (en) * | 2020-11-17 | 2021-03-23 | 芜湖美的厨卫电器制造有限公司 | Method, device, processor and water heater for identifying user based on gait |
| CN112556193A (en) * | 2020-11-17 | 2021-03-26 | 芜湖美的厨卫电器制造有限公司 | Control method and device for water heater, processor and water heater |
| CN112700619A (en) * | 2020-12-29 | 2021-04-23 | 潍坊医学院 | Intelligent monitoring method and system for falling of old people |
| CN115049039B (en) * | 2021-03-08 | 2023-11-14 | 北京金茂绿建科技有限公司 | Neural network-based state identification method, neural network training method and device |
| CN115049039A (en) * | 2021-03-08 | 2022-09-13 | 北京金茂绿建科技有限公司 | State recognition method based on neural network, neural network training method and device |
| CN113591760A (en) * | 2021-08-09 | 2021-11-02 | 路晟悠拜(重庆)科技有限公司 | Far-field multi-human-body gait monitoring method based on millimeter waves |
| CN113591760B (en) * | 2021-08-09 | 2024-01-05 | 路晟悠拜(重庆)科技有限公司 | Gait monitoring method of far-field multiple human bodies based on millimeter waves |
| CN113822241A (en) * | 2021-11-22 | 2021-12-21 | 江苏邦邦智能科技有限公司 | Gait data acquisition method, walking aid device and computer readable storage medium |
| CN113822241B (en) * | 2021-11-22 | 2022-07-01 | 江苏邦邦智能科技有限公司 | Gait data acquisition method, walking aid device and computer readable storage medium |
| CN114857916A (en) * | 2022-05-24 | 2022-08-05 | 佛山市顺德区美的电子科技有限公司 | Ground processing method, ground processing device, electrical equipment and medium |
| CN114857916B (en) * | 2022-05-24 | 2023-12-26 | 佛山市顺德区美的电子科技有限公司 | Ground processing method, ground processing device, electrical equipment and medium |
| CN115291184A (en) * | 2022-10-08 | 2022-11-04 | 四川启睿克科技有限公司 | Attitude monitoring method combining millimeter wave radar and deep learning |
| CN117148309A (en) * | 2023-11-01 | 2023-12-01 | 德心智能科技(常州)有限公司 | Millimeter wave radar human body sensing method and system applied to community grid inspection |
| CN117148309B (en) * | 2023-11-01 | 2024-01-30 | 德心智能科技(常州)有限公司 | Millimeter wave radar human body sensing method and system applied to community grid inspection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111738060A (en) | Human gait recognition system based on millimeter wave radar | |
| Fu et al. | Sensing technology for human activity recognition: A comprehensive survey | |
| Yang et al. | MU-ID: Multi-user identification through gaits using millimeter wave radios | |
| Maitre et al. | Fall detection with UWB radars and CNN-LSTM architecture | |
| CN110456320B (en) | Ultra-wideband radar identity recognition method based on free space gait time sequence characteristics | |
| CN115345908B (en) | Human body posture recognition method based on millimeter wave radar | |
| CN111505632A (en) | Ultra-wideband radar action attitude identification method based on power spectrum and Doppler characteristics | |
| CN113311428B (en) | Human body falling intelligent monitoring system and falling identification method based on millimeter wave radar | |
| CN114038012A (en) | Fall detection method and system based on millimeter wave radar and machine learning | |
| CN114814775A (en) | Radar fall detection method and equipment based on ResNet network | |
| CN114814832A (en) | Millimeter wave radar-based real-time monitoring system and method for human body falling behavior | |
| CN112782664A (en) | Toilet fall detection method based on millimeter wave radar | |
| CN112444805A (en) | Distributed multi-target detection, positioning tracking and identity recognition system based on radar | |
| CN116602663A (en) | Intelligent monitoring method and system based on millimeter wave radar | |
| CN115859078A (en) | Millimeter wave radar fall detection method based on improved Transformer | |
| CN113591760A (en) | Far-field multi-human-body gait monitoring method based on millimeter waves | |
| Ding et al. | A fall detection method based on K-nearest neighbor algorithm with MIMO millimeter-wave radar | |
| Yao et al. | mmSignature: Semi-supervised human identification system based on millimeter wave radar | |
| CN116561700A (en) | Indoor human body posture recognition method based on millimeter wave radar | |
| Hu et al. | Agricultural robot for intelligent detection of pyralidae insects | |
| CN109948481A (en) | A kind of passive human body recognition method based on the sampling of narrow radio frequency link | |
| Kittiyanpunya et al. | Millimeter-Wave Radar-Based Elderly Fall Detection Fed by One-Dimensional Point Cloud and Doppler | |
| CN113283415B (en) | Sedentary and recumbent detection method based on depth camera | |
| Raeis et al. | InARMS: Individual activity recognition of multiple subjects with FMCW radar | |
| Li et al. | Gait recognition using spatio-temporal information of 3D point cloud via millimeter wave radar |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20201002 |