CN116821778B - Gait recognition method and device based on WIFI signals and readable storage medium - Google Patents

Gait recognition method and device based on WIFI signals and readable storage medium Download PDF

Info

Publication number
CN116821778B
CN116821778B CN202311103044.2A CN202311103044A CN116821778B CN 116821778 B CN116821778 B CN 116821778B CN 202311103044 A CN202311103044 A CN 202311103044A CN 116821778 B CN116821778 B CN 116821778B
Authority
CN
China
Prior art keywords
gait
environment
time
human
channel state
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311103044.2A
Other languages
Chinese (zh)
Other versions
CN116821778A (en
Inventor
陈升
潘爱民
杨非
梅哲炜
李振廷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Lab
Original Assignee
Zhejiang Lab
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Lab filed Critical Zhejiang Lab
Priority to CN202311103044.2A priority Critical patent/CN116821778B/en
Publication of CN116821778A publication Critical patent/CN116821778A/en
Application granted granted Critical
Publication of CN116821778B publication Critical patent/CN116821778B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00Reducing energy consumption in communication networks
    • Y02D30/70Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

The application relates to a gait recognition method and device based on WIFI signals and a readable storage medium, wherein the gait recognition method based on the WIFI signals comprises the following steps: training the gait feature extractor, the environment discriminator and the gait recognizer by using training samples acquired under different environments, so that the human gait features extracted by the feature extractor comprise human gait information and do not comprise environment information at the same time, obtaining a trained gait recognition model, and outputting identity information corresponding to the object to be tested by inputting the first time-frequency diagram into the trained gait recognition model. The problem that the gait recognition model is sensitive to environmental changes in the related technology is solved, and the cross-environment application of the gait recognition model is realized.

Description

Gait recognition method and device based on WIFI signals and readable storage medium
Technical Field
The present disclosure relates to the field of wireless communications technologies, and in particular, to a gait recognition method and device based on WIFI signals, and a readable storage medium.
Background
When a human body performs walking activities, in order to ensure the coordination and stability of the whole body, not only the lower limbs do exercises, but also the whole body moves along with the lower limbs, so that the walking activities of the human body have strong individual characteristics, namely the gait of the human body. With the continuous development of internet technology, the interaction means between intelligent devices and human bodies are becoming more and more diversified. The human gait is used for realizing individual identification, and has important practical significance. For example, in the current large context of everything interconnection, a user may confirm his identity by gait when approaching his smart home, thereby activating subsequent smart operations.
Conventional gait recognition methods are typically based on wearable devices such as smartwatches, smartphones, cameras, floor sensors, etc. These conventional methods have several drawbacks: the wearable device-based gait sensing method can recognize human gait through specific changes of sensor readings when a human walks, but the human is required to wear the sensors all the time, so that a great deal of attention of the wearer is consumed on the device; secondly, when the light is insufficient, the gait sensing method based on the camera is difficult to realize, and the method is extremely easy to be influenced by a shielding object and has some privacy problems; furthermore, the deployment cost and the susceptibility to false positives are both unavoidable issues for floor sensors.
In view of the shortcomings of the above methods, new wireless sensing technologies based on WIFI signals are getting more and more attention. Because of the high throughput and easy deployment characteristics of WIFI devices, WIFI networks have widely entered each family, and therefore ubiquitous WIFI signals have become an important research medium in the human gait recognition direction.
In an indoor environment, the transmission of WIFI signals is constrained by a physical space, and when a person exists in the physical space, the WIFI signals reach a receiving end from a transmitting end through direct radiation, human body reflection/diffraction, and physical space reflection/diffraction, and these effects are recorded by the WIFI signals of the receiving end and recorded by channel state information. Based on this, a large number of sensing systems based on WIFI signals are proposed, wherein the human gait sensing system based on WIFI signals is very representative in this. The human gait sensing system based on the WIFI signals generally utilizes a deep learning method, performs training by collecting a large amount of data, and establishes a mapping relation between the change of WIFI channel state information and the gaits of different human bodies, so that human gait sensing is realized.
However, environmental information in a physical space, such as furniture and walls, is also described by changes in channel state information, that is, the channel state information carries environmental information, so that the channel state information collected under a certain environment carries environmental information, and thus the gait recognition model obtained by training can have larger fluctuation in testing and application performance under other environments, that is, the gait recognition model is very sensitive to environmental changes and cannot be suitable for different environments.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a gait recognition method, device and readable storage medium based on WIFI signals.
In a first aspect, an embodiment of the present application provides a gait recognition method based on WIFI signals, the method including:
acquiring first channel state information associated with movement of an object to be detected in a WIFI environment, and converting the first channel state information into a first time-frequency diagram;
inputting the first time-frequency diagram to a gait recognition model after training is completed, and outputting identity information corresponding to the object to be tested; the gait recognition model comprises a gait feature extractor, an environment discriminator and a gait recognizer; the gait feature extractor is used for extracting human gait features based on the time-frequency diagram; the environment discriminator is used for judging the environment according to the human gait characteristics; the gait recognition device is used for carrying out identity recognition based on the human gait characteristics, and the training process of the gait recognition model comprises the following steps:
Taking a second time-frequency diagram as a training sample, wherein the second time-frequency diagram is obtained by converting second channel state information associated with movement of a target object in different WIFI environments;
and training the gait feature extractor, the environment discriminator and the gait identifier by using the training sample to obtain the gait recognition model, wherein the human gait feature extracted by the feature extractor comprises human gait information and does not comprise environment information.
In one embodiment, the gait feature extractor is configured to extract human gait features based on the time-frequency diagram, and includes:
inputting the time-frequency diagram into a convolutional neural network, and extracting continuous human gait local characteristics;
and inputting the continuous human gait local characteristics into a gating circulation unit, and extracting human gait characteristics.
In one embodiment, the training the gait feature extractor, the environment discriminator, and the gait identifier with the training sample with the object that the human gait feature extracted by the feature extractor includes human gait information and does not include environment information, and obtaining the gait recognition model includes:
Training parameters of the environment discriminator in each round of counter-propagation process of the gait recognition model training, so that the environment discriminator judges the environment where the target object is located based on the human gait characteristics; the parameters of the environment discriminant are parameters corresponding to the minimum loss of the environment discriminant;
updating parameters of the gait feature extractor and parameters of the gait identifier based on the loss of the environment discriminator; and the human gait features extracted by the feature extractor contain human gait information and do not contain environment information until the human gait features contain human gait information.
In one embodiment, the updating the parameters of the gait feature extractor and the parameters of the gait identifier based on the loss of the environment identifier is as follows:
wherein θ F θ is a parameter of the gait feature extractor G Lambda is a parameter of the gait recognizer G Weighting lambda for the gait identifier D Weight of the environment discriminant, L F L is the loss of the feature extractor G L for loss of the gait recognition device D Is a loss of the environment discriminator.
In one embodiment, the obtaining the first channel state information associated with the movement of the object to be measured in the WIFI environment includes:
Acquiring initial channel state information of an object to be tested in a WIFI environment;
calculating the variance of the initial channel state information in a moving window by using a moving window method;
if the variance is larger than a dynamic threshold, determining the moment corresponding to the moving window as the movement starting time or the movement ending time of the object to be detected;
determining the movement start time and the movement end time of the object to be detected based on the time sequence;
and acquiring first channel state information associated with the movement of the object to be detected in the WIFI environment based on the movement starting time and the movement ending time.
In one embodiment, the dynamic threshold is set to a preset multiple of the variance estimate L (t), where the variance estimate L (t) is determined as follows:
wherein L (t) is the variance estimation of the initial channel state information in the moving window at the time t, L (t-1) is the variance estimation of the initial channel state information in the moving window at the time t-1, the coefficient γ is set to 0.1, and var (t) is the variance of the initial channel state information in the moving window at the time t.
In one embodiment, the converting the first channel state information into a first time-frequency diagram includes:
And sequentially carrying out short-time Fourier transform and Hilbert yellow transform on the first gait channel state information to obtain a first time-frequency diagram.
In one embodiment, after the obtaining the first channel state information associated with the movement of the object to be measured in the WIFI environment, the method further includes:
and carrying out interpolation, noise reduction, static component removal and dimension reduction on the first channel state information.
In a second aspect, embodiments of the present application further provide a gait recognition device based on WIFI signals, where the device includes:
the conversion module is used for acquiring first channel state information associated with movement of an object to be detected in a WIFI environment and converting the first channel state information into a first time-frequency diagram;
the recognition module is used for inputting the first time-frequency diagram into a gait recognition model after training is completed and outputting identity information corresponding to the object to be tested; the gait recognition model comprises a gait feature extractor, an environment discriminator and a gait recognizer; the gait feature extractor is used for extracting human gait features based on the time-frequency diagram; the environment discriminator is used for judging the environment according to the human gait characteristics; the gait recognition device is used for carrying out identity recognition based on the human gait characteristics, and the training process of the gait recognition model comprises the following steps:
Taking a second time-frequency diagram as a training sample, wherein the second time-frequency diagram is obtained by converting second channel state information associated with movement of a target object in different WIFI environments;
and training the gait feature extractor, the environment discriminator and the gait identifier by using the training sample to obtain the gait recognition model, wherein the human gait feature extracted by the feature extractor comprises human gait information and does not comprise environment information.
In a third aspect, embodiments of the present application also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method as described in the first aspect above.
According to the gait recognition method, the device and the computer readable storage medium based on the WIFI signal, the gait feature extractor, the environment discriminator and the gait recognizer are trained by using the second time-frequency diagrams acquired under different environments as training samples, so that the human gait features extracted by the feature extractor comprise human gait information and do not comprise environment information, a trained gait recognition model is obtained, and identity information corresponding to the object to be tested is output by inputting the first time-frequency diagrams into the trained gait recognition model. The problem that the gait recognition model is sensitive to environmental changes in the related technology is solved, and the cross-environment application of the gait recognition model is realized.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the other features, objects, and advantages of the application.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:
fig. 1 is an application environment schematic diagram of a gait recognition method based on WIFI signals in one embodiment;
FIG. 2 is a flow chart of a gait recognition method based on WIFI signals in one embodiment;
FIG. 2-1 is a block diagram of a gait recognition model in one embodiment;
FIG. 3 is a flow diagram of extracting human gait features in an embodiment;
FIG. 4 is a flow diagram of gait recognition model training in one embodiment;
FIG. 5 is a flow diagram of acquiring first channel state information associated with a motion in one embodiment;
FIG. 6 is a time-frequency diagram of CSI data obtained using a short-time Fourier transform, in one embodiment;
FIG. 7 is a time-frequency diagram of CSI data obtained using a Hilbert-Huang transform in one embodiment;
FIG. 8 is a CSI data waveform without data pre-processing in one embodiment;
FIG. 9 is a waveform of CSI data filtered using a Hampel filter in one embodiment;
FIG. 10 is a waveform of CSI data after denoising using wavelets in one embodiment;
FIG. 11 is a waveform of CSI data filtered using a Butterworth filter in one embodiment;
fig. 12 is a block diagram of a gait recognition device based on WIFI signals in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described and illustrated below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden on the person of ordinary skill in the art based on the embodiments provided herein, are intended to be within the scope of the present application.
It is apparent that the drawings in the following description are only some examples or embodiments of the present application, and it is possible for those of ordinary skill in the art to apply the present application to other similar situations according to these drawings without inventive effort. Moreover, it should be appreciated that while such a development effort might be complex and lengthy, it would nevertheless be a routine undertaking of design, fabrication, or manufacture for those of ordinary skill having the benefit of this disclosure, and thus should not be construed as having the benefit of this disclosure.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is to be expressly and implicitly understood by those of ordinary skill in the art that the embodiments described herein can be combined with other embodiments without conflict.
Unless defined otherwise, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this application belongs. Reference to "a," "an," "the," and similar terms herein do not denote a limitation of quantity, but rather denote the singular or plural. The terms "comprising," "including," "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to only those steps or elements but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. The terms "connected," "coupled," and the like in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" as used herein refers to two or more. "and/or" describes an association relationship of an association object, meaning that there may be three relationships, e.g., "a and/or B" may mean: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship. The terms "first," "second," "third," and the like, as used herein, are merely distinguishing between similar objects and not representing a particular ordering of objects.
The gait recognition method based on the WIFI signal can be applied to an application environment shown in fig. 1. The terminal 104 communicates with the WIFI signal transceiver device 102 in different environments (environment 1, environment 2 and … … N) through a network, where the WIFI signal transceiver device 102 is configured to obtain first channel state information associated with movement of an object to be tested in the WIFI environment, and the terminal 104 is configured to convert the first channel state information into a first time-frequency diagram; and inputting the first time-frequency diagram to a gait recognition model after training is completed, and outputting identity information corresponding to the object to be tested.
Where terminal 104 may include one or more processors (e.g., a single-chip processor or a multi-chip processor). By way of example only, the terminal 104 may include a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), a special instruction set processor (ASIP), an image processing unit (GPU), a physical arithmetic processing unit (PPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), a controller, a microcontroller unit, a Reduced Instruction Set Computer (RISC), a microprocessor, or the like, or any combination thereof.
The embodiment of the application provides a gait recognition method based on WIFI signals, as shown in fig. 2, the method comprises the following steps:
Step S201, obtaining first channel state information associated with movement of an object to be detected in a WIFI environment, and converting the first channel state information into a first time-frequency diagram;
specifically, a pair of a WIFI signal transmitter and a WIFI signal receiver respectively equipped with Na omni-directional antennas are respectively arranged in Ne different indoor environments, and first channel state information including Ns subcarriers associated with movement of an object to be measured in the WIFI environments is collected at a fixed sampling rate.
Step S202, inputting the first time-frequency diagram to a gait recognition model after training is completed, and outputting identity information corresponding to the object to be tested; 2-1, the gait recognition model comprises a gait feature extractor, an environment discriminator and a gait recognizer; the gait feature extractor is used for extracting human gait features based on the time-frequency diagram; the environment discriminator is used for judging the environment according to the human gait characteristics; the gait recognition device is used for carrying out identity recognition based on the human gait characteristics, and the training process of the gait recognition model comprises the following steps:
taking a second time-frequency diagram as a training sample, wherein the second time-frequency diagram is obtained by converting second channel state information associated with movement of a target object in different WIFI environments;
Exemplary, the second channel state information collection process associated with the movement of the target object in different WIFI environments is as follows:
a WIFI signal transmitter with 3 omni-directional antennas and a WIFI signal receiver with 3 omni-directional antennas are arranged as WIFI signal transceiver devices in 4 different indoor environments. And the WIFI signal transceiver acquires corresponding channel state information (Channel State Information, CSI for short) in the period of time at the same time. For CSI, in the IEEE 802.11N/ac standard, in a device supporting mimo communication, the WIFI channel is divided into a plurality of subcarriers by the ofdm technique, and this embodiment is 30 subcarriers, so that CSI measurement of one CSI physical frame is composed of 30 matrices, and the dimension of the matrix is N Tx ×N Rx Wherein N is Tx Indicating the number of antennas of the WIFI signal transmitter, N Rx Indicating the number of antennas of the WIFI signal receiver. Furthermore, the present invention requires that the CSI data sampling frequency is not lower than 500Hz. And converting second channel state information associated with the motion acquired by the target object in different WIFI environments into a time-frequency diagram, and taking the time-frequency diagram as a training sample of the gait recognition model.
And training the gait feature extractor, the environment discriminator and the gait identifier by using the training sample to obtain the gait recognition model, wherein the human gait feature extracted by the feature extractor comprises human gait information and does not comprise environment information.
Step S201 to step S202 are performed with training samples collected under different environments to train the gait feature extractor, the environment discriminator and the gait recognizer, so that the human gait feature extracted by the feature extractor includes human gait information and does not include environment information, a trained gait recognition model is obtained, and identity information corresponding to the object to be tested is output by inputting the first time-frequency diagram into the trained gait recognition model. The problem that the gait recognition model is sensitive to environmental changes in the related technology is solved, and the cross-environment application of the gait recognition model is realized.
In one embodiment, as shown in fig. 3, the gait feature extractor is configured to extract human gait features based on the time-frequency diagram, including:
step S301, inputting the time-frequency diagram into a convolutional neural network, and extracting continuous human gait local features;
Step S302, inputting the continuous human gait local features into a gating circulation unit, and extracting human gait features.
Specifically, the input gait feature extractor is a continuous time-frequency pattern book for a period of time, because the human walking activity is decomposed into a sequence of tiny actions, which is a time-sequence behavior, the time-frequency pattern book itself has a certain local correlation, so the characteristic can be extracted by a convolution neural network, the time-frequency pattern book firstly passes through N c The layer convolution neural network extracts continuous human gait local characteristics and uses N g The layer gating unit extracts global time-frequency characteristics, namely human gait characteristics, from the continuous human gait local characteristic sequence. N in the present embodiment c The value is 6, N g The value is 2.
In one embodiment, as shown in fig. 4, the training the gait feature extractor, the environment discriminator and the gait identifier by using the training sample with the object of making the human gait feature extracted by the feature extractor include human gait information and not include environment information, and obtaining the gait recognition model includes:
step S401, training parameters of the environment discriminator in each round of counter-propagation process of the gait recognition model training, so that the environment discriminator judges the environment where the target object is located based on the human gait characteristics; the parameters of the environment discriminant are parameters corresponding to the minimum loss of the environment discriminant;
Specifically, each round of counter propagation process of gait recognition model training is divided into two steps, the first step trains parameters of the environment discriminator, so that the environment discriminator can better judge which environment the target object is in, the environment discriminator receives human gait characteristics, the human gait characteristics are input into a full-connection layer of 2 layers, and finally recognition of the environment where the target object is located is completed through a softmax function, wherein the parameters theta D The method comprises the following steps:
wherein L is D Representing the loss of the environment discriminator, θ D Representing parameters in the environment discriminant, i.e. L D The corresponding parameter when the value is minimum;
the softmax function is as follows:
wherein Z is i The output value of the ith node in the environment discriminator is C, which is the number of output nodes, namely the number of classified categories.
Step S402, updating parameters of the gait feature extractor and parameters of the gait identifier based on the loss of the environment identifier; and the human gait features extracted by the feature extractor contain human gait information and do not contain environment information until the human gait features contain human gait information.
Specifically, training parameters of a gait feature extractor and a human gait identifier in the second step, and updating the parameters of the gait feature extractor and the parameters of the gait identifier according to the loss of an environment discriminator in the back propagation process; and the human gait features extracted by the feature extractor contain human gait information and do not contain environment information until the human gait features contain human gait information. That is, after model training is completed, the human gait features extracted by the feature extractor after updating parameters include human gait information and do not include environmental information, at this time, the environment identifier cannot determine the environment in which the target object is located based on the human gait features including no environmental information, and the gait identifier can identify the identity of the target object based on the human gait features including human gait information. Therefore, the gait recognizer after training can eliminate the influence of the environment information on the gait recognition, so that the human gait recognition can be realized under different environments.
Wherein the human gait recognition device receives the human gait characteristics and inputs the human gait characteristics to N f1 Full connection layer of layers, N in this embodiment f1 And finally, the human gait recognition is completed through a softmax function, wherein the softmax function is as follows:
wherein Z is i The output value of the ith node in the human gait recognizer is C, which is the number of the output nodes, namely the number of the classified categories.
In one embodiment, the updating the parameters of the gait feature extractor and the parameters of the gait identifier based on the loss of the environment identifier is as follows:
wherein θ F θ is a parameter of the gait feature extractor G Lambda is a parameter of the gait recognizer G Weighting lambda for the gait identifier D Weight of the environment discriminant, L F L is the loss of the feature extractor G L for loss of the gait recognition device D Is a loss of the environment discriminator.
Specifically, the gait feature extractor in the present embodiment uses the L1 loss (minimum absolute deviation) as a loss function, the human gait recognizer uses the cross entropy loss as a loss function, and the environment discriminator uses the cross entropy loss as a loss function, the loss being L D
Specifically, in order to achieve the object that the human gait feature extracted by the feature extractor includes human gait information and does not include environmental information, it is necessary to train parameters of the gait feature extractor and the human gait identifier while preventing the environmental arbiter from completing tasks, specifically, losing the first-step environmental arbiter L D Substituting the characteristics into formula (1) and updating the parameters of the gait characteristic extractor and the parameters of the gait identifier, so that the characteristic vector extracted by the characteristic extractor cannot identify the current environment of the human body for the whole model, simultaneously training the characteristic extractor and the gait identifier to complete the task of human gait identification, wherein the loss of the characteristic extractor is L F Loss of gait recognition device is L G . In the training process, a self-adaptive moment estimation (Adaptive Moment Estimation, abbreviated as Adam) optimization algorithm is adopted to update model parameters, and the training is iterated until the maximum training times are reached, so that the training of the gait recognition model is finally completed.
It should be noted that, before acquiring the first channel state information associated with the movement of the object to be measured in the WIFI environment, it is necessary to know which part of the CSI data corresponds to the walking activity of the human body. When a human body starts to walk in a stationary environment, the amplitude of the CSI timing data may start to fluctuate drastically, and when the human body stops from walking activities, the amplitude of the CSI timing data may gradually become smooth from the intense fluctuation. Thus, the method using the dynamic threshold value can determine the movement start time or the movement end time of the object to be measured.
In one embodiment, as shown in fig. 5, the obtaining the first channel state information associated with the movement of the object to be measured in the WIFI environment includes:
step S501, obtaining initial channel state information of an object to be detected in a WIFI environment;
all CSI data acquired by the WIFI signal transceiver device in a period of time, that is, the initial channel state information includes channel state information acquired by the object to be measured when the object is not moving.
Step S502, calculating the variance of the initial channel state information in the moving window by using a moving window method;
specifically, a size N is set at time t W Moving window of CSI samples, calculating N in the window at the moment W Variance var (t) of magnitudes corresponding to the CSI samples; i.e. the moving window is centered on the CSI sample at time t, before adding the CSI sampleIndividual CSI samples and post->The composition size of the CSI samples is N W In this embodiment, a moving window with a size of 21 CSI samples is set, and variance var (t) of corresponding amplitudes of 21 CSI samples in the moving window corresponding to each moment is calculated.
Step S503, if the variance is greater than the dynamic threshold, determining the moment corresponding to the moving window as the movement start time or the movement end time of the object to be measured;
If the variance of a certain moment of the moving window is larger than the dynamic threshold value, the moment is considered to be a time point when the human walking activity starts or ends;
step S504, determining the movement start time and the movement end time of the object to be detected based on the time sequence;
the starting or ending time points of the walking activity are determined in the time sequence in which the walking movement occurs.
Step S505, based on the motion start time and the motion end time, obtains first channel state information associated with motion of the object to be measured in the WIFI environment.
Specifically, based on the motion start time and the motion end time, corresponding channel state information, namely first channel state information associated with motion of the object to be detected in the WIFI environment, is obtained.
In one embodiment, the dynamic threshold is set to a preset multiple of the variance estimate L (t), where the variance estimate L (t) is determined as follows:
wherein L (t) is the variance estimation of the initial channel state information in the moving window at the time t, L (t-1) is the variance estimation of the initial channel state information in the moving window at the time t-1, the coefficient γ is set to 0.1, and var (t) is the variance of the initial channel state information in the moving window at the time t. The dynamic threshold is set to 3 times the variance estimate L (t) in this embodiment.
In one embodiment, the converting the first channel state information into a first time-frequency diagram includes:
and sequentially carrying out short-time Fourier transform and Hilbert yellow transform on the first gait channel state information to obtain a first time-frequency diagram.
Specifically, the amplitude waveform of the CSI timing data can only show the effect of human walking activity on the WIFI signal in the time domain, however, each part of the human moves at different rates during walking, and the corresponding WIFI reflected signal has different frequencies. Considering that the Hilbert-Huang transform has better recognition performance in a complex environment, and the short-time Fourier transform has better recognition performance in a clear environment, the combination of the short-time Fourier transform and the Hilbert-Huang transform converts the CSI data into a time-frequency diagram. Illustratively, a time-frequency diagram obtained by short-time fourier transform is shown in fig. 6, and a time-frequency diagram obtained by hilbert yellow transform is shown in fig. 7.
In one embodiment, after the obtaining the first channel state information associated with the movement of the object to be measured in the WIFI environment, the method further includes:
and carrying out interpolation, noise reduction, static component removal and dimension reduction on the first channel state information.
Specifically, the data preprocessing is performed on the first channel state information through interpolation, noise reduction, static component removal and dimension reduction, and the method comprises the following steps:
step 1 interpolation: due to uncertainty of a WIFI signal propagation path, a condition that a CSI physical frame is lost possibly occurs in a CSI acquisition process, so that intervals of CSI data packets are uneven.
Step 2, noise reduction: in a commercial WIFI transceiver, due to the change of the transmission power of the device, some CSI data measurements are abnormal, as shown in fig. 8, so that abnormal values exist in CSI timing signals. Here, an outlier detection filter (Hampel filter) is used to detect these outliers and remove them, and the CSI data curve after outlier removal is shown in fig. 9. In addition, the CSI signal after the outlier removal still contains in-band noise, and at this time, wavelet denoising is used to remove in-band noise, and the CSI data curve after the in-band noise removal is shown in fig. 10. Finally, considering that the speed of the human body during walking is not very fast, that is to say, the frequency of the CSI signal change caused by walking is generally lower, the CSI change frequency caused by the human body walking activity is not more than 70Hz, and the channel noise change speed caused by other hardware such as WIFI transceiver equipment is very fast, and the change frequency is very high. This embodiment employs a Butterworth low pass filter to filter these extraneous high frequency noise. The frequency response curve of the Butterworth filter in the passband is very flat, and drops to 0 in the stopband rapidly, so that out-of-band noise can be largely eliminated, meanwhile, the fidelity of the signal in the passband can be ensured, and the final CSI data curve is shown in fig. 11.
Step 3 removes the static component: the WIFI signal reflected by the human body belongs to a dynamic component, the WIFI signal reflected by the indoor environment belongs to a dynamic component, and the average value of all the CSI data is subtracted from each column of the CSI data, so that the static component in the CSI data is removed, and the CSI matrix is standardized.
Step 4, dimension reduction: the CSI data collected in this embodiment has 30 dimensions due to 30 subcarriers, so that the dimensions are reduced by using a principal component analysis method, and 8 most representative principal components are obtained, so that component information irrelevant to gait recognition in the CSI data is removed, and meanwhile, the calculation efficiency is improved.
According to the embodiment, the first channel state information is subjected to data preprocessing through interpolation, noise reduction, static component removal and dimension reduction, so that the data quality of the acquired channel state information is ensured, and the accuracy of the subsequent human gait recognition can be improved. Note that the present embodiment also performs the same data preprocessing operation on the second channel state information.
In a second aspect, an embodiment of the present application further provides a gait recognition device based on WIFI signals, as shown in fig. 12, where the device includes:
the conversion module 610 is configured to obtain first channel state information associated with movement of an object to be detected in a WIFI environment, and convert the first channel state information into a first time-frequency diagram;
The recognition module 620 is configured to input the first time-frequency chart to a gait recognition model after training is completed, and output identity information corresponding to the object to be tested; the gait recognition model comprises a gait feature extractor, an environment discriminator and a gait recognizer; the gait feature extractor is used for extracting human gait features based on the time-frequency diagram; the environment discriminator is used for judging the environment according to the human gait characteristics; the gait recognition device is used for carrying out identity recognition based on the human gait characteristics, and the training process of the gait recognition model comprises the following steps:
taking a second time-frequency diagram as a training sample, wherein the second time-frequency diagram is obtained by converting second channel state information associated with movement of a target object in different WIFI environments;
and training the gait feature extractor, the environment discriminator and the gait identifier by using the training sample to obtain the gait recognition model, wherein the human gait feature extracted by the feature extractor comprises human gait information and does not comprise environment information.
According to the device provided by the embodiment, the gait feature extractor, the environment discriminator and the gait recognizer are trained by using training samples collected under different environments, so that the human gait features extracted by the feature extractor comprise human gait information and do not comprise environment information at the same time, a trained gait recognition model is obtained, and identity information corresponding to the object to be tested is output by inputting the first time-frequency diagram into the trained gait recognition model. The problem that the gait recognition model is sensitive to environmental changes in the related technology is solved, and the cross-environment application of the gait recognition model is realized.
In one embodiment, the identification module 620 is further configured to:
inputting the time-frequency diagram into a convolutional neural network, and extracting continuous human gait local characteristics;
and inputting the continuous human gait local characteristics into a gating circulation unit, and extracting human gait characteristics.
In one embodiment, the identification module 620 is further configured to:
training parameters of the environment discriminator in each round of counter-propagation process of the gait recognition model training, so that the environment discriminator judges the environment where the target object is located based on the human gait characteristics; the parameters of the environment discriminant are parameters corresponding to the minimum loss of the environment discriminant;
updating parameters of the gait feature extractor and parameters of the gait identifier based on the loss of the environment discriminator; until the human gait feature extracted by the feature extractor contains human gait information and does not contain environmental information
In one embodiment, the identification module 620 is further configured to:
based on the loss of the environment discriminant, the parameters of the gait feature extractor and the parameters of the gait identifier are updated as follows:
wherein θ F θ is a parameter of the gait feature extractor G Lambda is a parameter of the gait recognizer G Weighting lambda for the gait identifier D Weight of the environment discriminant, L F L is the loss of the feature extractor G L for loss of the gait recognition device D Is a loss of the environment discriminator.
In one embodiment, the conversion module 610 is further configured to:
acquiring initial channel state information of an object to be tested in a WIFI environment;
calculating the variance of the initial channel state information in a moving window by using a moving window method;
if the variance is larger than a dynamic threshold, determining the moment corresponding to the moving window as the movement starting time or the movement ending time of the object to be detected;
determining the movement start time and the movement end time of the object to be detected based on the time sequence;
and acquiring first channel state information associated with the movement of the object to be detected in the WIFI environment based on the movement starting time and the movement ending time.
In one embodiment, the dynamic threshold is set to a preset multiple of the variance estimate L (t), where the variance estimate L (t) is determined as follows:
wherein L (t) is the variance estimation of the initial channel state information in the moving window at the time t, L (t-1) is the variance estimation of the initial channel state information in the moving window at the time t-1, the coefficient γ is set to 0.1, and var (t) is the variance of the initial channel state information in the moving window at the time t.
In one embodiment, the conversion module 610 is further configured to:
and sequentially carrying out short-time Fourier transform and Hilbert yellow transform on the first gait channel state information to obtain a first time-frequency diagram.
In one embodiment, the conversion module 610 is further configured to:
and carrying out interpolation, noise reduction, static component removal and dimension reduction on the first channel state information.
The above-described respective modules may be functional modules or program modules, and may be implemented by software or hardware. For modules implemented in hardware, the various modules described above may be located in the same processor; or the above modules may be located in different processors in any combination.
Embodiments of the present application also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of any of the embodiments of the WIFI signal-based gait recognition method described above.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, or the like. Volatile memory can include random access memory (RandomAccess Memory, RAM) or external cache memory. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like.
The technical features of the above-described embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above-described embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.

Claims (8)

1. A gait recognition method based on WIFI signals, the method comprising:
acquiring first channel state information associated with movement of an object to be detected in a WIFI environment, and converting the first channel state information into a first time-frequency diagram;
inputting the first time-frequency diagram to a gait recognition model after training is completed, and outputting identity information corresponding to the object to be tested; the gait recognition model comprises a gait feature extractor, an environment discriminator and a gait recognizer; the gait feature extractor is used for extracting human gait features based on the time-frequency diagram; the environment discriminator is used for judging the environment according to the human gait characteristics; the gait recognition device is used for carrying out identity recognition based on the human gait characteristics, and the training process of the gait recognition model comprises the following steps:
Taking a second time-frequency diagram as a training sample, wherein the second time-frequency diagram is obtained by converting second channel state information associated with movement of a target object in different WIFI environments;
the training sample is used for training the gait feature extractor, the environment discriminator and the gait recognizer to obtain the gait recognition model, wherein the object is that the human gait features extracted by the feature extractor contain human gait information and do not contain environment information, and the method specifically comprises the following steps:
training parameters of the environment discriminator in each round of counter-propagation process of the gait recognition model training, so that the environment discriminator judges the environment where the target object is located based on the human gait characteristics; the parameters of the environment discriminant are parameters corresponding to the minimum loss of the environment discriminant;
updating parameters of the gait feature extractor and parameters of the gait identifier based on the loss of the environment discriminator; until the human gait feature extracted by the feature extractor contains human gait information and does not contain environmental information; wherein, based on the loss of the environment discriminator, the parameters of the gait feature extractor and the parameters of the gait identifier are updated as follows:
Wherein θ F θ is a parameter of the gait feature extractor G Lambda is a parameter of the gait recognizer G Weighting lambda for the gait identifier D Weight of the environment discriminant, L F L is the loss of the feature extractor G L for loss of the gait recognition device D Is a loss of the environment discriminator.
2. The method of claim 1, wherein the gait feature extractor is configured to extract human gait features based on the time-frequency diagram, comprising:
inputting the time-frequency diagram into a convolutional neural network, and extracting continuous human gait local characteristics;
and inputting the continuous human gait local characteristics into a gating circulation unit, and extracting human gait characteristics.
3. The method according to claim 1, wherein the obtaining the first channel state information associated with the movement of the object to be measured in the WIFI environment includes:
acquiring initial channel state information of an object to be tested in a WIFI environment;
calculating the variance of the initial channel state information in a moving window by using a moving window method;
if the variance is larger than a dynamic threshold, determining the moment corresponding to the moving window as the movement starting time or the movement ending time of the object to be detected;
Determining the movement start time and the movement end time of the object to be detected based on the time sequence;
and acquiring first channel state information associated with the movement of the object to be detected in the WIFI environment based on the movement starting time and the movement ending time.
4. A method according to claim 3, characterized in that the dynamic threshold is set to a preset multiple of the variance estimate L (t), wherein the variance estimate L (t) is determined in the following way:
wherein L (t) is the variance estimation of the initial channel state information in the moving window at the time t, L (t-1) is the variance estimation of the initial channel state information in the moving window at the time t-1, the coefficient γ is set to 0.1, and var (t) is the variance of the initial channel state information in the moving window at the time t.
5. The method of claim 1, wherein said converting said first channel state information into a first time-frequency diagram comprises:
and sequentially carrying out short-time Fourier transform and Hilbert yellow transform on the first channel state information to obtain a first time-frequency diagram.
6. The method according to claim 1, wherein after the obtaining the first channel state information associated with the movement of the object to be measured in the WIFI environment, the method further comprises:
And carrying out interpolation, noise reduction, static component removal and dimension reduction on the first channel state information.
7. Gait recognition device based on WIFI signal, characterized in that the device comprises:
the conversion module is used for acquiring first channel state information associated with movement of an object to be detected in a WIFI environment and converting the first channel state information into a first time-frequency diagram;
the recognition module is used for inputting the first time-frequency diagram into a gait recognition model after training is completed and outputting identity information corresponding to the object to be tested; the gait recognition model comprises a gait feature extractor, an environment discriminator and a gait recognizer; the gait feature extractor is used for extracting human gait features based on the time-frequency diagram; the environment discriminator is used for judging the environment according to the human gait characteristics; the gait recognition device is used for carrying out identity recognition based on the human gait characteristics, and the training process of the gait recognition model comprises the following steps:
taking a second time-frequency diagram as a training sample, wherein the second time-frequency diagram is obtained by converting second channel state information associated with movement of a target object in different WIFI environments;
The training sample is used for training the gait feature extractor, the environment discriminator and the gait recognizer to obtain the gait recognition model, wherein the object is that the human gait features extracted by the feature extractor contain human gait information and do not contain environment information, and the method specifically comprises the following steps:
training parameters of the environment discriminator in each round of counter-propagation process of the gait recognition model training, so that the environment discriminator judges the environment where the target object is located based on the human gait characteristics; the parameters of the environment discriminant are parameters corresponding to the minimum loss of the environment discriminant;
updating parameters of the gait feature extractor and parameters of the gait identifier based on the loss of the environment discriminator; until the human gait feature extracted by the feature extractor contains human gait information and does not contain environmental information; wherein, based on the loss of the environment discriminator, the parameters of the gait feature extractor and the parameters of the gait identifier are updated as follows:
wherein θ F θ is a parameter of the gait feature extractor G Lambda is a parameter of the gait recognizer G Weighting lambda for the gait identifier D Weight of the environment discriminant, L F L is the loss of the feature extractor G L for loss of the gait recognition device D Is a loss of the environment discriminator.
8. A computer readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the method of any one of claims 1 to 6.
CN202311103044.2A 2023-08-30 2023-08-30 Gait recognition method and device based on WIFI signals and readable storage medium Active CN116821778B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311103044.2A CN116821778B (en) 2023-08-30 2023-08-30 Gait recognition method and device based on WIFI signals and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311103044.2A CN116821778B (en) 2023-08-30 2023-08-30 Gait recognition method and device based on WIFI signals and readable storage medium

Publications (2)

Publication Number Publication Date
CN116821778A CN116821778A (en) 2023-09-29
CN116821778B true CN116821778B (en) 2024-01-09

Family

ID=88117010

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311103044.2A Active CN116821778B (en) 2023-08-30 2023-08-30 Gait recognition method and device based on WIFI signals and readable storage medium

Country Status (1)

Country Link
CN (1) CN116821778B (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107862295A (en) * 2017-11-21 2018-03-30 武汉大学 A kind of method based on WiFi channel condition informations identification facial expression
CN110337066A (en) * 2019-05-21 2019-10-15 西安电子科技大学 Based on channel state information indoor occupant activity recognition method, man-machine interactive system
CN111259700A (en) * 2018-12-03 2020-06-09 北京京东尚科信息技术有限公司 Method and apparatus for generating gait recognition model
CN113609977A (en) * 2021-08-04 2021-11-05 燕山大学 Pedestrian gait recognition method based on channel state information quotient distance
CN114757237A (en) * 2022-06-13 2022-07-15 武汉理工大学 Speed-independent gait recognition method based on WiFi signals
CN114999643A (en) * 2022-06-01 2022-09-02 暨南大学 WiFi-based intelligent monitoring method for old people
CN115527266A (en) * 2022-09-13 2022-12-27 深圳大学 User identification method and device based on gait information, electronic equipment and storage medium
CN115687894A (en) * 2022-10-27 2023-02-03 天津大学 Tumble detection system and method based on small sample learning
WO2023049962A1 (en) * 2021-10-01 2023-04-06 Newsouth Innovations Pty Limited System and method of detecting environmental changes using wireless sensing
CN116644300A (en) * 2023-02-10 2023-08-25 厦门大学 Through-wall target behavior identification method and device based on channel state information

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190311261A1 (en) * 2018-04-10 2019-10-10 Assured Information Security, Inc. Behavioral biometric feature extraction and verification
US11675066B2 (en) * 2019-08-08 2023-06-13 Syracuse University Motion detection and classification using ambient wireless signals

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107862295A (en) * 2017-11-21 2018-03-30 武汉大学 A kind of method based on WiFi channel condition informations identification facial expression
CN111259700A (en) * 2018-12-03 2020-06-09 北京京东尚科信息技术有限公司 Method and apparatus for generating gait recognition model
CN110337066A (en) * 2019-05-21 2019-10-15 西安电子科技大学 Based on channel state information indoor occupant activity recognition method, man-machine interactive system
CN113609977A (en) * 2021-08-04 2021-11-05 燕山大学 Pedestrian gait recognition method based on channel state information quotient distance
WO2023049962A1 (en) * 2021-10-01 2023-04-06 Newsouth Innovations Pty Limited System and method of detecting environmental changes using wireless sensing
CN114999643A (en) * 2022-06-01 2022-09-02 暨南大学 WiFi-based intelligent monitoring method for old people
CN114757237A (en) * 2022-06-13 2022-07-15 武汉理工大学 Speed-independent gait recognition method based on WiFi signals
CN115527266A (en) * 2022-09-13 2022-12-27 深圳大学 User identification method and device based on gait information, electronic equipment and storage medium
CN115687894A (en) * 2022-10-27 2023-02-03 天津大学 Tumble detection system and method based on small sample learning
CN116644300A (en) * 2023-02-10 2023-08-25 厦门大学 Through-wall target behavior identification method and device based on channel state information

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
From RSSI to CSI:indoor localization via channel response;YANG Z等;《ACM Computing Surveys》;全文 *
一种基于WiFi的室内重点区域监测方法;党小超;张彤;郝占军;段渝;;小型微型计算机系统(02);全文 *
基于WiFi信号的人体动作识别系统;肖玲;潘浩;;北京邮电大学学报(03);全文 *

Also Published As

Publication number Publication date
CN116821778A (en) 2023-09-29

Similar Documents

Publication Publication Date Title
US11902857B2 (en) Handling concept drift in Wi-Fi-based localization
Zou et al. Deepsense: Device-free human activity recognition via autoencoder long-term recurrent convolutional network
Hsieh et al. Deep learning-based indoor localization using received signal strength and channel state information
Liu et al. DeepCount: Crowd counting with WiFi via deep learning
US11070399B1 (en) Filtering channel responses for motion detection
CN112036433A (en) CNN-based Wi-Move behavior sensing method
CN111759305A (en) Living body detection method and living body detection system
CN114222202B (en) Environment self-adaptive activity detection method and system based on WiFi CSI
CN111901028B (en) Human body behavior identification method based on CSI (channel State information) on multiple antennas
CN111474955A (en) Unmanned aerial vehicle image signal system identification method, device, equipment and storage medium
CN116821778B (en) Gait recognition method and device based on WIFI signals and readable storage medium
CN113453180A (en) Intelligent detection method and system for human body tumble and information data processing terminal
Yousefi et al. A survey of human activity recognition using wifi CSI
TWI704364B (en) Method and apparatus for detecting presence of an object in an environment
CN116644300A (en) Through-wall target behavior identification method and device based on channel state information
Sharma et al. Optimised CNN for Human Counting Using Spectrograms of Probabilistic WiFi CSI
Onumanyi et al. A real valued neural network based autoregressive energy detector for cognitive radio application
CN112883905B (en) Human behavior recognition method based on coarse-grained time-frequency features and multi-layer fusion learning
CN114759904A (en) Data processing method, device, equipment, readable storage medium and program product
CN115002703A (en) Passive indoor people number detection method based on Wi-Fi channel state information
CN114358162A (en) Falling detection method and device based on continuous wavelet transform and electronic equipment
CN114501329A (en) Environment self-adaptive training method and device of indoor positioning model based on 1D-CNN
CN113827253A (en) Computing device and method for removing noise from electroencephalogram signals
CN115795377B (en) Respiratory state classifier generation method, respiratory state monitoring method and related devices
Eslami et al. Memory based energy detection spectrum sensing method in cognitive radio driven hospitals

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant