CN110659598B - Human body action layered analysis and identification method and device based on Wi-Fi signals - Google Patents

Human body action layered analysis and identification method and device based on Wi-Fi signals Download PDF

Info

Publication number
CN110659598B
CN110659598B CN201910867333.7A CN201910867333A CN110659598B CN 110659598 B CN110659598 B CN 110659598B CN 201910867333 A CN201910867333 A CN 201910867333A CN 110659598 B CN110659598 B CN 110659598B
Authority
CN
China
Prior art keywords
signal
action
classifier
signals
actions
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910867333.7A
Other languages
Chinese (zh)
Other versions
CN110659598A (en
Inventor
胡海苗
姜永强
李波
蒲养林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beihang University
Original Assignee
Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beihang University filed Critical Beihang University
Priority to CN201910867333.7A priority Critical patent/CN110659598B/en
Publication of CN110659598A publication Critical patent/CN110659598A/en
Application granted granted Critical
Publication of CN110659598B publication Critical patent/CN110659598B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/30—Services specially adapted for particular environments, situations or purposes

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

The invention provides a Wi-Fi signal-based human body action layered analysis and identification method and device. Wherein, the method comprises the following steps: (A) preprocessing the original signal to obtain a principal component analysis signal and a spectrogram; (B) designing action hierarchical relation and corresponding characteristics according to the action characteristics; (C) training a plurality of classifiers by using the features in the step (B), and constructing a classification framework according to the action hierarchical relationship; (D) carrying out steps A to C on the unknown signals to obtain classification results, namely action recognition results; meanwhile, the invention designs a human body action layered analysis and recognition device based on Wi-Fi signals, which comprises two computers respectively used as a receiving and transmitting end, a plurality of antennas, an antenna extension line and a mobile experiment table. The invention provides a complete flow and a method for identifying human body actions by using Wi-Fi signals; the recognition system built by the method can well complete human body action recognition tasks and has robustness to environmental changes.

Description

Human body action layered analysis and identification method and device based on Wi-Fi signals
Technical Field
The invention relates to a Wi-Fi signal-based human body action hierarchical analysis and identification method and device, in particular to a hierarchical human body action classification method and device based on action hierarchical relation, and belongs to the field of wireless perception.
Background
Wi-Fi technology is a wireless local area network technology based on IEEE802.11 standard, and is a common technology in the field of wireless communication. Wi-Fi signals, which are essentially electromagnetic waves, have the condition and potential to be used as sensing devices and are therefore also used as a means of sensing the physical world.
Human behavior recognition is an important subject in Wi-Fi perception, and the principle is that the recognition of human actions is achieved by analyzing different influences of different actions of the human body on signal changes. The Information used is Channel State Information (CSI), which is Information used to describe the transmission State of a wireless Channel in the field of wireless communication. For the Wi-Fi equipment conforming to the IEEE802.11n standard, the orthogonal frequency division multiplexing technology is used in the data transmission process. The technology has the function that for each transmitting-receiving antenna pair, a channel for signal transmission between the antennas is divided into a plurality of sub-channels, and a plurality of sub-carriers are used for jointly completing a data transmission task. Therefore, the expression of CSI is:
H(k，t)＝|H(k，t)|ej∠H(k，t)
where H (k, t) is the CSI signal, which is a function of time and carrier variation, and is a complex value. Taking the modulus of H (k, t) as the signal amplitude; h (k, t) is the phase. Therefore, CSI is a time-varying complex matrix with a size of n × m × k, where n and m are the number of transmit antennas and receive antennas, respectively, and k is the number of subcarriers involved in the received signal.
Current main methods of Wi-Fi based human motion recognition can be divided into two categories, data-driven and model-driven. The data driving method mainly comprises the steps of obtaining a signal change rule from data by analyzing acquired Wi-Fi signals and utilizing a machine learning method so as to complete identification of specific actions; and the model driving method adds a Wi-Fi signal propagation model on the basis, and enables the model to assist in calculation, so that the identification accuracy is higher. In contrast, the data-driven method has poor environmental adaptability, i.e., the accuracy rate of the data-driven method is significantly reduced after the environment is changed, while the model-driven method performs well in the environmental adaptability.
Disclosure of Invention
According to one aspect of the invention, a human body action layered analysis and identification method based on Wi-Fi signals is provided, which identifies seven actions of walking, running, sitting down, standing up, raising hands, kicking legs once and for a plurality of times, and is characterized by comprising the following steps:
1) preprocessing the received channel state information signal, including modulo, bandpass filtering, Principal Component Analysis (PCA), Short Time Fourier Transform (STFT), etc., to convert C to CConversion of SI signals to PCA signals HpcaAnd a spectrogram S;
2) designing hierarchical relation of actions according to characteristics of different actions, and designing according to classification basis from PCA signal H obtained in step 1)pcaExtracting corresponding features from the spectrogram S;
3) training a model and constructing a hierarchical classification architecture, using the characteristics obtained in the step 2), using a support vector machine method to train a plurality of classifiers, and using the obtained classifiers to construct the hierarchical classification architecture according to the hierarchical relationship and the logical relationship of the action;
4) executing steps 1) to 2) on the signals of the unknown type of action to obtain the characteristics of the signals, classifying the obtained characteristics by using a layered classification architecture in step 3), and taking the classification result as an identification result;
wherein:
the step 2) comprises the following steps:
2.1) determining action hierarchical relation according to the characteristics of the actions, wherein the 7 actions of walking, running, sitting, standing, kicking, lifting hands and kicking are performed for multiple times, and the determination operation of the hierarchical relation of the 7 actions comprises the following steps:
2.1.1) dividing all actions into a moving set and a static set according to whether the actions are moving or static;
2.1.2) dividing the moving set into a running set and a walking set according to the speed of the movement of the action;
2.1.3) dividing the static set into a body set and a limb set according to whether the action is body action or limb action;
2.1.4) dividing the body set into a sitting set and a standing set according to the height and the change direction of the action generating points;
2.1.5) dividing the action into a hand lifting set and a leg set according to the height and the change direction of the action generating point;
2.1.6) dividing the leg set into a kicking one-time set and a kicking multi-time set according to the occurrence frequency of the action;
2.2) five groups of characteristics aiming at seven actions of walking, running, sitting, standing, kicking, lifting hands and kicking legs are determined as follows:
a. features of the moving/stationary classifier:
Fms＝[F′ms，F″ms]；
wherein the content of the first and second substances,
Figure BDA0002201644370000021
F″ms＝argmax(Hpca)-argmin(Hpca).
f 'in the formula'msAnd F ″)msIs FmsTwo components of (a); hpcaIs the signal, F ″, obtained after analysis of the principal component obtained in step 1)msIs to HpcaThe squared sum data, argmax (. sup.). sup. -) and argmin (. sup. -) are functions of maximum and minimum values, F ″, respectivelymsIs the difference between the maximum and minimum of the PCA;
b. features of the run/walk classifier:
Fwr＝[argmean(P)，argmax(P)]
wherein, P is a curve composed of frequencies corresponding to the maximum energy when the energy is maximum at each moment in the spectrogram S obtained in the step 1), argmean (. +) and argmax (. +) are respectively an average value and a maximum value function, FwrThe vector is composed of the average value and the maximum value of P;
c. features of the body extremity classifier:
Fbl＝∑fS
wherein S is a spectrogram, FblAdding the frequency spectrums according to the frequency f to obtain a vector;
d. characteristics of sit/stand classifier and leg/arm classifier:
Figure BDA0002201644370000031
where H is the signal data after the band pass filtering in step 1), H1，H2，H3Are respectively three receiving ends Rx1，Rx2，Rx3On the signal passing through the band passThe result obtained after filtering, FacThe ratio of signal energy on three different receiving end antennas is obtained;
e. characteristics of the action frequency classifier:
Fr＝[length[Speak＞argmean(Sc)]，argmean(Sc)]
wherein S iscIs the energy sum curve, S, obtained by adding frequency spectrapeakIs ScThe peak values at (a), argmean (x) and argmax (x) are the mean and maximum functions, respectively, length (x) is the function for calculating the vector dimension, FrHas the meaning of ScThe number of peaks larger than the mean, and ScA vector consisting of the mean values of;
the step 3) comprises the following steps: aiming at 7 actions of walking, running, sitting down, standing up, kicking legs, lifting hands and kicking legs for a plurality of times, the corresponding classifier is trained by using the characteristics in the step 2), and the classification process of the constructed classification framework comprises the following steps:
3.1) classifying by using a mobile/static classifier, and if the classification result is mobile, entering a step 3.2); otherwise, entering step 3.3);
3.2) classifying by using a running/walking classifier, outputting a classification result and ending;
3.3) classifying by using a body/limb classifier, and if the classification result is that the body is the body, entering a step 3.4); otherwise, entering step 3.5);
3.4) classifying by using a sitting/standing classifier and outputting a classification result;
3.5) classifying by using a leg/arm classifier, and if the classification result is that the leg is the leg, entering the step 6); otherwise, outputting a classification result, and ending;
and 3.6) classifying by using a frequency classifier, outputting a classification result and finishing.
Drawings
FIG. 1 is a flow diagram of a commercial Wi-Fi device based human body action recognition method according to one embodiment of the invention;
FIG. 2 is a schematic diagram of a hierarchical classification method according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a CSI signal acquisition apparatus built by using commercial Wi-Fi equipment according to an embodiment of the present invention.
FIG. 4 is a diagram of a signal acquisition scenario according to one embodiment of the present invention;
FIG. 5 is a schematic diagram of an action hierarchy according to one embodiment of the invention;
fig. 6 is a diagram of an original signal, a PCA-processed signal, and a spectrogram according to an embodiment of the present invention.
Detailed Description
As described above, the main methods of Wi-Fi based human motion recognition of the prior art can be classified into data-driven and model-driven. However, both data-driven and model-driven approaches have some problems that are not considered. In the existing Wi-Fi-based human behavior identification method, the commonly adopted flow is as follows: the method comprises the steps of data acquisition, data processing, feature extraction, model training, action recognition and the like. For the feature extraction and model training steps therein, the current approach is to jointly train the model using all extracted features. That is, existing approaches consider all features to be the same in their roles in differentiating between different actions. In fact, however, the different features may have different roles in distinguishing between different actions. Taking the duration of the action as an example, it may be more effective to distinguish between a walk and a jump, but a jump and a sit may not be as effective. Therefore, the existing method has the problem of unreasonable feature use, and the problem may influence the accuracy rate of motion recognition.
On the other hand, as Wi-Fi signals are susceptible to environmental changes, features used in the existing methods cannot be well linked with the characteristics of actions, so that the features cannot be well adapted to the environmental changes, and the problem of insufficient feature validity is caused.
In order to solve the problems of unreasonable feature use, low feature effectiveness and the like in the existing Wi-Fi signal-based human body action recognition technology, the invention provides a Wi-Fi signal-based human body action layered analysis and recognition method and device. In the identification process, corresponding features are designed according to the division in the action hierarchical relationship, extracted features are used for training a plurality of groups of classifiers according to the division in the action hierarchical relationship, and a hierarchical classification framework is constructed to improve the identification accuracy.
According to one aspect of the invention, signals are collected using a commercial Wi-Fi device, and recognition of human actions is achieved by analyzing the collected signals.
According to another aspect of the invention, a human body action layered analysis and identification method based on Wi-Fi signals is disclosed, which is characterized by mainly comprising the following steps:
(1) preprocessing a received Channel State Information (CSI) signal, including operations such as modulus extraction, band-pass filtering, Principal Component Analysis (PCA), short-time Fourier transform (STFT), and the like, so as to convert the CSI signal into a PCA signal HpcaAnd a spectrogram S;
(2) designing hierarchical relation of actions according to characteristics of different actions, and designing PCA signal H obtained from step (1) according to classification basispcaExtracting corresponding features from the spectrogram S;
(3) model training and hierarchical classification architecture construction. Training a plurality of models by using the characteristics obtained in the step (2), and constructing a hierarchical classification architecture by using the obtained models according to the hierarchical relationship and the logical relationship of the actions;
(4) executing steps (1) to (2) to the unknown type action signal to obtain the characteristics of the action signal, then classifying the action signal by using the layered classification architecture in step (3), wherein the classification result is the recognition result,
characterized in that the step 1 comprises:
1.1) obtaining a modulus of the collected signal to obtain a signal amplitude. The original form of the collected CSI signals is a complex matrix of n multiplied by m multiplied by k multiplied by t, wherein n, m, k and t are the number of transmitting antennas, the number of receiving antennas, the number of carriers and time respectively. By performing a modulus operation on the complex number, i.e., performing a square sum and an evolution operation on the real part and the complex part of the complex number, the complex number can be converted into a real number, i.e., the amplitude of the signal. For complex number z ═ a + bi (a, b ∈ R), the modulo formula is:
Figure BDA0002201644370000051
1.2) carrying out band-pass filtering processing on the signals after modulus extraction. For n × m × k pieces of timing signals H obtained in 1.1mAnd performing band-pass filtering operation to filter out high-frequency noise. The signal after the filtering process is recorded as Hp，
1.3) the n × m × k denoised time-series signals H obtained in 1.2 are comparedpDividing the antenna pair number into n × m groups, performing PCA processing according to the carrier number k, and reserving the first 3 principal components. With a set of k timing signals H thereinp1For example, the formula is:
Hpca1＝Hp1×e
wherein e is Hp1Singular matrix of, resulting in Hpca1Is a k x t matrix. According to the size of the singular value, the PCA corresponding to the maximum three singular values is reserved, and then the n multiplied by m multiplied by 3 multiplied by t time sequence signals H are finally obtainedpca；
1.4) to HpcaAnd carrying out short-time Fourier transform to obtain a spectrogram. By using sliding window operation, dividing the signal into subsections according to the time dimension, performing Fourier transform on each subsection, converting the time sequence signal into a frequency domain signal, and then arranging the frequency domain signals of each sub-window according to the time sequence, a spectrogram S with time domain information and frequency domain information at the same time can be obtained, wherein the formula is as follows:
Figure BDA0002201644370000052
where x (t) is the original signal, w is a window function, and τ is time.
According to a further aspect of the present invention, in the step (2), fromPCA signal H obtained in step (1)pcaAnd extracting features from the spectrogram S. Extracting a group of features for each classification basis according to the hierarchical relation of the actions; each set of features is used to train a classifier. The hierarchical relationship of the actions means that the action set is decomposed into different subsets layer by layer through characteristic differences among the actions, and finally, each action only exists in one subset. For example, actions can be classified into stationary and moving classes according to whether the actions occur in a certain place or not; then, the moving class is divided into running and walking according to the moving speed of the movement, and the static class is divided into arm movement, leg movement and the like according to the position of the movement. And different actions are gradually subdivided into a certain type of layer-by-layer decomposition process according to the characteristics of the actions, namely, the hierarchical relation chromatography process of the actions is obtained. And (2) designing and extracting corresponding features according to the classification basis of the hierarchical relationship. Aiming at 7 actions of walking, running, sitting down, standing up, lifting hands, kicking legs once and kicking legs for many times, the hierarchical relationship and classification basis involved in the step (2) are as follows:
2.1) judging whether the motion is moving or static, and if the motion is moving, entering a step 2.2); otherwise step 2.3) is entered.
2.2) judging the movement speed of the action, if the movement speed is high, running is carried out, otherwise, walking is carried out, and ending.
2.3) judging whether the movement is a body movement or a four-limb movement, and if the movement is a body, entering a step 2.4); otherwise step 2.5) is entered.
And 2.4) judging whether the action is sitting down or standing up, and ending.
2.5) judging whether the action is a leg action or an arm action, and if the action is a leg, entering the step 2.6); otherwise, ending.
2.6) judging whether the action is repeated, if so, kicking the leg for a plurality of times, otherwise, kicking the leg once, and ending.
Meanwhile, the classification is based on the 5 groups of features:
a. features of the moving/stationary classifier:
Fms＝[F′ms，F″ms]
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0002201644370000061
F″ms＝argmax(Hpca)-argmin(Hpca)
in the formula HpcaFor the PCA signal obtained in step 1.3), argmax (. times.) and argmin (. times.) are functions of maximum and minimum values, F'msIs the sum of squares F ″, of all PCA signalsmsIs the difference between the maximum and minimum of the PCA;
b. features of the run/walk classifier:
Fwr＝[argmean(P)，argmax(P)]
wherein, P is the curves argmean (. +) and argmax (. +) composed of the frequency with the maximum energy at each moment in the spectrogram obtained in the step 1.4) are respectively the average value and the maximum function, FwrThe vector is composed of the average value and the maximum value of P;
c. features of the body extremity classifier:
Fbl＝ΣfS
wherein S is a spectrogram matrix, FblAdding the frequency spectrograms according to the frequency f;
d. characteristics of sit/stand classifier and leg/arm classifier:
Figure BDA0002201644370000071
where H is the signal data after the band pass filtering in step 1.1), H1，H2，H3Are respectively three receiving ends Rx1，Rx2，Rx3The result of band-pass filtering of the signal above, FacThe ratio of signal energy on three different receiving end antennas is obtained;
e. characteristics of the action frequency classifier:
Fr＝[length[Speak＞argmean(Sc)]，argmean(Sc)]
wherein S iscIs the energy sum curve obtained by adding frequency spectra; argmean and argmax are mean and maximum functions, respectively, length is a function of the calculated vector dimensions, SpeakThe peak value thereon; frHas the meaning of ScThe number of peaks larger than the mean, and ScA vector consisting of the mean values of;
training 6 classifiers through the 5 groups of characteristics to identify 7 actions of walking, running, sitting, standing, kicking once, lifting hands and kicking for many times; in the specific implementation process, the action hierarchical relationship can be extended and modified, so that the action types and characteristics are increased to identify other actions.
The Wi-Fi signal-based human body action layered analysis and recognition method is characterized in that in the step 3, a plurality of different classifiers are trained by using the features in the step 2, and a layered classification architecture is established. Wherein the classifier may use any two classification models. According to the hierarchical relationship of the actions designed in step 2, in step 3, the trained classifier should correspond to each classification basis of the hierarchical relationship. And then, constructing a hierarchical classification framework of action classification according to the logical relationship and the sequence among all classification bases in the action hierarchical relationship. Aiming at seven actions of walking, running, sitting down, standing up, lifting hands, kicking legs once and for many times, the specific classification flow is as follows:
3.1) classifying by using a mobile/static classifier, and if the classification result is mobile, entering a step 3.2); otherwise step 3.3) is entered.
And 3.2) classifying by using the running/walking classifier, outputting a classification result and finishing.
3.3) classifying by using a body/limb classifier, and if the classification result is that the body is the body, entering a step 3.4); otherwise step 3.5) is entered.
And 3.4) classifying by using a sitting/standing classifier and outputting a classification result.
3.5) classifying by using a leg/arm classifier, and if the classification result is that the leg is the leg, entering the step 3.6); otherwise, outputting the classification result and ending.
3.6) classifying by using a frequency classifier, outputting a classification result and ending
The Wi-Fi signal-based human body action hierarchical analysis and identification method is characterized in that in the step (4), the classifier trained in the step (3) and the constructed hierarchical classification architecture are used for realizing identification of different actions. Executing the steps (1) to (2) to the newly collected action signals without the labels to obtain the characteristics of the action signals; and (4) classifying by using the classification framework in the step (3), wherein the classification result is the identification result.
On the other hand, the invention provides a CSI signal acquisition device built by commercial Wi-Fi equipment, and the hardware configuration of the CSI signal acquisition device comprises: a transmitting end (Tx) which comprises a desktop computer provided with a Wi-Fi network card and comprises one antenna and one antenna extension line,
a receiving end (Rx) comprising a desktop computer provided with a Wi-Fi network card and three antennas and three antenna extension lines,
the mobile experiment table is used for placing the antenna and other equipment.
The beneficial effects of the invention include:
the CSI signal acquisition device can acquire CSI signals in different scenes, and can ensure that the antennas are placed in different environments to be consistent while ensuring the convenience of movement;
meanwhile, the human body action recognition method based on the commercial Wi-Fi equipment, disclosed by the invention, designs a layered classification architecture after the hierarchical relation among different actions is considered, so that the extracted features are more representative, the use mode of the features is more efficient, and finally the action recognition accuracy is improved.
The invention is further described below with reference to the accompanying drawings.
A signal acquisition scenario according to an embodiment of the present invention is shown in fig. 4, and within the scenario of motion recognition required, the acquisition platform is placed in the position shown in fig. 4. The device for realizing the human body action recognition method based on the commercial Wi-Fi equipment comprises the following steps: the system comprises two computers provided with the same type of commercial wireless network cards, a plurality of antennas, antenna extension lines and proper platforms for placing the antennas. During the process of placing the device, attention should be paid to ensure that the position of the antenna is as shown in fig. 3, so as to ensure that the acquired signal meets the requirements for signal quality in the hierarchical classification method of the present invention.
As shown in fig. 1, the flowchart of the human body motion recognition method based on commercial Wi-Fi device according to one embodiment of the present invention includes four steps:
(1) preprocessing a received Channel State Information (CSI) signal, including operations such as modulus taking, band-pass filtering, Principal Component Analysis (PCA), short-time fourier transform (STFT), and the like. Through the step, the CSI signal is converted into a PCA signal HpcaAnd a spectrogram S;
(2) designing hierarchical relation of actions according to characteristics of different actions, and designing PCA signal H obtained from (1) according to classification basispcaAnd extracting corresponding features from the spectrogram S.
(3) Model training and hierarchical classification architecture construction. Training a plurality of models by using the characteristics obtained in the step (2), and constructing a hierarchical classification architecture by using the obtained models according to the hierarchical relationship and the logical relationship of the actions;
(4) executing (1) to (2) on the unknown type action signal to obtain the characteristics of the action signal, and then classifying the action signal by using a hierarchical classification framework in the step (3), wherein the classification result is an identification result;
as described above in step (1), the PCA signal H is acquiredpcaAnd a spectrogram S in the form shown in FIG. 6;
meanwhile, as shown in fig. 2, a schematic diagram of a hierarchical classification method according to an embodiment of the present invention is shown, in which classification of 7 actions is involved. In this schematic diagram, the steps to be passed by a feature of an action in the process of classification are:
(1) classifying by using a moving/static classifier, and entering the step (2) if the classification result is moving; otherwise, entering the step (3).
(2) And (4) classifying by using a running/walking classifier, outputting a classification result and ending.
(3) Classifying by using a body/limb classifier, and entering the step (4) if the classification result is a body; otherwise, entering the step (5).
(4) And (4) classifying by using a sitting/standing classifier, and outputting a classification result.
(5) Classifying by using a leg/arm classifier, and if the classification result is that the leg is the leg, entering the step (6); otherwise, outputting the classification result and ending.
(6) And (5) classifying by using a frequency classifier, outputting a classification result and ending.
Through the above 6 steps, it can be determined which of the 7 actions (running, walking, sitting, standing, waving hands, kicking once and repeatedly kicking) the input action is, and the purpose of action identification is completed.
In addition, according to an embodiment of the present invention, as shown in fig. 3, a schematic structural diagram of a CSI signal acquisition apparatus built using a commercial Wi-Fi device is shown. In the figure Rx1，Rx2，Rx3For the receiving-end antenna, TxIs a transmitting end antenna. T isxAnd Rx1，Rx2Arranged at the same height, Rx3Is arranged at Rx2Right below. The specific values are shown in the figure. Meanwhile, T in FIG. 3xAnd Rx1The connecting line of the connecting rod is parallel to the wall surface.
Fig. 4 is a diagram illustrating a signal acquisition scenario, according to an embodiment of the present invention. Where the position and movement path of the motion acquisition is indicated in the figure.
FIG. 5 is a diagram illustrating a hierarchical relationship of actions according to an embodiment of the present invention. The movement can be divided into 4 classes by three classification bases of static/moving, fast/slow speed and body/limb movement. And then, using the three classification bases, designing and extracting features related to the three classification bases, and training a responsive classifier, so that the construction of the hierarchical classification architecture in the step 3 can be completed.
The foregoing disclosure discloses only specific embodiments of the invention. Various changes and modifications can be made by those skilled in the art based on the basic technical concept of the present invention without departing from the scope of the claims of the present invention.

Claims (10)

1. A human action layered analysis and identification method based on Wi-Fi signals identifies seven actions of walking, running, sitting, standing, lifting hands, kicking once and kicking many times, and is characterized by comprising the following steps:
1) preprocessing the received channel state information signal, including modulus extraction, bandpass filtering, principal component analysis, and short-time Fourier transform, to convert the channel state information signal into a principal component analysis signal HpcaAnd a spectrogram S;
2) designing hierarchical relation of actions according to characteristics of different actions, and designing according to classification basis from principal component analysis signal H obtained in step 1)pcaExtracting corresponding features from the spectrogram S;
3) training a model and constructing a hierarchical classification architecture, using the characteristics obtained in the step 2), using a support vector machine method to train a plurality of classifiers, and using the obtained classifiers to construct the hierarchical classification architecture according to the hierarchical relationship and the logical relationship of the actions;
4) performing steps 1) to 2) on the signals of the unknown type of action to obtain the characteristics of the action signals, then classifying the obtained characteristics by using the hierarchical classification architecture in step 3), and taking the classification result as an identification result;
wherein:
the step 2) comprises the following steps:
2.1) determining action hierarchical relation according to the characteristics of the actions, wherein the 7 actions of walking, running, sitting, standing, kicking, lifting hands and kicking are performed for multiple times, and the determination operation of the hierarchical relation of the 7 actions comprises the following steps:
2.1.1) dividing all actions into a moving set and a static set according to whether the actions are moving or static;
2.1.2) dividing the moving set into a running set and a walking set according to the speed of the movement of the action;
2.1.3) dividing the static set into a body set and a limb set according to whether the action is body action or limb action;
2.1.4) dividing the body set into a sitting set and a standing set according to the height and the change direction of the action generating points;
2.1.5) dividing the action into a hand lifting set and a leg set according to the height and the change direction of the action generating point;
2.1.6) dividing the leg set into a kicking one-time set and a kicking multi-time set according to the occurrence frequency of the action;
2.2) five groups of characteristics aiming at seven actions of walking, running, sitting, standing, kicking, lifting hands and kicking legs are determined as follows:
a. features of the moving/stationary classifier:
Fms＝[F′ms，F″ms]
wherein, the first and the second end of the pipe are connected with each other,
Figure FDA0003599923430000021
F″ms＝argmax(Hpca)-argmin(Hpca)
f 'in the formula'msAnd F ″)msIs FmsTwo components of (a); hpcaIs the principal component analysis signal, F'msIs to HpcaThe squared sum data, argmax (. sup.). sup. -) and argmin (. sup. -) are functions of maximum and minimum values, F ″, respectivelymsThe difference between the maximum value and the minimum value of the principal component analysis signal;
b. features of the run/walk classifier:
Fwr＝[argmean(P)，argmax(P)]
wherein, P is a curve composed of frequencies corresponding to the maximum energy when the energy is maximum at each moment in the spectrogram S obtained in the step 1), argmean (. +) and argmax (. +) are respectively an average value and a maximum value function, FwrThe vector is composed of the average value and the maximum value of P;
c. features of the body extremity classifier:
Fbl＝∑fS
wherein S is a spectrogram, FblAdding the frequency spectrums according to the frequency f to obtain a vector;
d. characteristics of sit/stand classifier and leg/arm classifier:
Figure FDA0003599923430000022
where H is the signal data after the band pass filtering in step 1), H1，H2，H3Are respectively three receiving ends Rx1，Rx2，Rx3The result of band-pass filtering of the signal above, FacThe ratio of signal energy on three different receiving end antennas is obtained;
e. characteristics of the action frequency classifier:
Fr＝[length[Speak＞argmean(Sc)]，argmean(Sc)]
wherein S iscIs the energy sum curve, S, obtained by adding frequency spectrapeakIs ScThe peak values at (a), argmean (x) and argmax (x) are the mean and maximum functions, respectively, length (x) is the function for calculating the vector dimension, FrHas the meaning of ScThe number of peaks larger than the mean, and ScA vector consisting of the mean values of;
the step 3) comprises the following steps: aiming at 7 actions of walking, running, sitting down, standing up, kicking legs, lifting hands and kicking legs for a plurality of times, the corresponding classifier is trained by using the characteristics in the step 2), and the classification process of the constructed classification framework comprises the following steps:
3.1) classifying by using a mobile/static classifier, and if the classification result is mobile, entering a step 3.2); otherwise, entering step 3.3);
3.2) classifying by using a running/walking classifier, outputting a classification result and ending;
3.3) using a body/limb classifier for classification, and if the classification result is that the body is the body, entering a step 3.4); otherwise, entering step 3.5);
3.4) classifying by using a sitting/standing classifier and outputting a classification result;
3.5) classifying by using a leg/arm classifier, and if the classification result is that the leg is the leg, entering the step 6); otherwise, outputting a classification result, and ending;
and 3.6) classifying by using a frequency classifier, outputting a classification result and finishing.
2. The Wi-Fi signal-based human motion hierarchy analysis and recognition method according to claim 1, wherein in the step 1), the principal component analysis operation is performed on the received data of each receiving antenna, that is, the principal component analysis is performed on 30 subcarrier data received on each receiving antenna, the first three principal components are retained, and finally, 3 × 3 principal components are obtained from 3 receiving antennas.
3. The Wi-Fi signal-based human body action hierarchical analysis and identification method according to claim 1, wherein in step 1), a short-time Fourier transform operation is performed on all principal component analysis signals simultaneously, and each piece of action data obtains one spectrogram.
4. The Wi-Fi signal-based human motion hierarchy parsing and recognition method according to claim 1, wherein:
a transmitting end including a computer with wireless network card, a transmitting antenna TxAnd an antenna extension;
a receiving end including a computer with wireless network card, three receiving antennas Rx1，Rx2，Rx3And an antenna extension;
a platform for placing the antenna thereon, wherein,
wherein:
said three receiving antennas R for the receiving endx1，Rx2，Rx3With said one transmitting antenna T of the transmitting endxShould be placed at different horizontal heights;
two of the receiving end antennasRx1，Rx2Should be compatible with the transmitting antenna TxAt the same level, another receiving antenna Rx3Lower than the other three antennas and located at Rx1Or Rx2Right below.
5. The Wi-Fi signal based human body action hierarchical parsing and recognition method according to claim 1, wherein the step 1) comprises:
1.1) taking the modulus of the collected signal to obtain the amplitude of the signal, the original form of the collected channel state information signal is a complex matrix of n × m × k × t, where n, m, k, t are the number of transmitting antennas, the number of receiving antennas, the number of carriers and time, the complex is subjected to modulus operation, i.e. the square sum and the square root operation are performed on the real part and the complex part of the complex, the complex can be converted into a real number, i.e. the amplitude of the signal, and for the complex z ═ a + bi, where a, b ∈ R, the modulus formula is:
Figure FDA0003599923430000041
1.2) carrying out band-pass filtering processing on the signals after modulus taking, and carrying out band-pass filtering processing on the n multiplied by m multiplied by k time sequence signals H obtained in the 1.1mPerforming band-pass filtering operation to filter out high-frequency noise, and recording the signal after filtering as Hp，
1.3) the n × m × k denoised time-series signals H obtained in 1.2 are comparedpDividing the antenna pair number into n × m groups, analyzing the principal components according to the number k of carriers, retaining the first 3 principal components, and using one group of k time sequence signals Hp1For example, the formula is:
Hpca1＝Hp1×e
wherein e is Hp1Singular matrix of, resulting in Hpca1For k × t matrix, according to the size of singular value, retaining principal component analysis signals corresponding to the largest three singular values, and finally obtaining n × m × 3 × t time sequence signals Hpca；
1.4) to HpcaFor a short timePerforming Fourier transform to obtain a spectrogram, using sliding window operation, dividing signals into subsections according to time dimension, performing Fourier transform on each subsection, converting time sequence signals into frequency domain signals, and arranging the frequency domain signals of each sub-window according to time sequence to obtain the spectrogram S simultaneously having time domain information and frequency domain information, wherein the formula is as follows:
Figure FDA0003599923430000042
where x (t) is the original signal, w is a window function, and τ is time.
6. Storage medium storing a computer program enabling a processor to execute the Wi-Fi signal based human motion hierarchy interpretation and recognition method according to one of claims 1-5.
7. The utility model provides a human action layering is analyzed and recognition device based on Wi-Fi signal, its is to walking, running, sitting down, stand up, lifting hands, kicking one time, kicking seven kinds of actions many times and discerning, its characterized in that includes:
6.1) a signal preprocessing section for preprocessing the received channel state information signal including modulus taking, band pass filtering, principal component analysis, short time Fourier transform, thereby converting the channel state information signal into a principal component analysis signal HpcaAnd a spectrogram S;
6.2) a feature extraction section for designing hierarchical relationship of the actions according to the characteristics of the different actions and for analyzing the principal component analysis signal H obtained from the signal preprocessing section according to the classification basis designpcaExtracting corresponding features from the spectrogram S;
6.3) a model training and hierarchical classification architecture constructing part, using the features obtained by the feature extracting part, using a support vector machine method to train a plurality of classifiers, and using the obtained classifiers to construct a hierarchical classification architecture according to the hierarchical relationship and the logical relationship of the actions;
the method comprises the steps that for signals of unknown types of actions, the characteristics of the unknown types of actions are obtained through a signal preprocessing part and a characteristic extraction part, then the obtained characteristics are classified according to a hierarchical classification framework through a model training and hierarchical classification framework construction part, and classification results are used as recognition results;
wherein:
the operation performed by the feature extraction section includes:
6.3.1) determining action hierarchical relation according to the characteristics of the actions, wherein the 7 actions of walking, running, sitting, standing, kicking, lifting hands and kicking are performed for multiple times, and the determination operation of the hierarchical relation of the 7 actions comprises the following steps:
6.3.1.1) dividing the total motion into a moving set and a stationary set according to whether the motion is moving or stationary;
6.3.1.2) dividing the moving set into running set and walking set according to the speed of the movement;
6.3.1.3) dividing the static set into a body set and a limb set according to whether the action is body action or limb action;
6.3.1.4) dividing the body set into a sitting set and a standing set according to the height and the change direction of the action points;
6.3.1.5) dividing the action into a hand lifting set and a leg set according to the height and the change direction of the action generating point;
6.3.1.6) dividing the leg set into a kicking one-time set and a kicking multiple-time set according to the occurrence frequency of the action;
6.3.2) five groups of characteristics aiming at seven actions of walking, running, sitting down, standing up, kicking legs, lifting hands and kicking legs are determined as follows:
a. features of the moving/stationary classifier:
Fms＝[F′ms，F″ms]
wherein, the first and the second end of the pipe are connected with each other,
Figure FDA0003599923430000051
F″ms＝argmax(Hpca)-argmin(Hpca)
f 'in the formula'msAnd F ″)msIs FnsTwo components of (a); hpcaIs the signal F 'obtained after analysis of the principal component obtained in step 1)'msIs to HpcaThe squared sum data, argmax (. sup.). sup. -) and argmin (. sup. -) are functions of maximum and minimum values, F ″, respectivelymsThe difference between the maximum value and the minimum value of the principal component analysis signal;
b. features of the run/walk classifier:
Fwr＝[argmean(P)，argmax(P)]
wherein, P is a curve composed of frequencies corresponding to the maximum energy when the energy is maximum at each time in the spectrogram S obtained in step 6.1), argmean (—) and argmax (—) are functions of the average value and the maximum value respectively, and FwrThe vector is composed of the average value and the maximum value of P;
c. features of the body limb classifier:
Fbl＝∑fS
wherein S is a spectrogram, FblAdding the frequency spectrums according to the frequency f to obtain a vector;
d. characteristics of sit/stand classifier and leg/arm classifier:
Figure FDA0003599923430000061
where H is the signal data after the band-pass filtering in step 6.1), H1，H2，H3Are respectively three receiving ends Rx1，Rx2，Rx3The result of band-pass filtering of the signal above, FacThe ratio of signal energy on three different receiving end antennas is obtained;
e. characteristics of the action frequency classifier:
Fr＝{length[Speak＞argmean(Sc)]，argmean(Sc)}
wherein S iscIs the sum of the energies and curves obtained by adding frequency spectraLine, SpeakIs ScThe peak values at (a), argmean (x) and argmax (x) are the mean and maximum functions, respectively, length (x) is the function for calculating the vector dimension, FrHas the meaning of ScThe number of peaks larger than the mean, and ScA vector consisting of the mean values of;
6.3.4) the operations performed by the model training and hierarchical classification architecture construction part include: aiming at 7 actions of walking, running, sitting, standing, kicking, lifting hands and kicking, the corresponding classifier is trained by using the feature extraction part, and the classification flow of the constructed classification framework comprises the following steps:
6.3.4.1) using a moving/stationary classifier, if the classification result is moving, then go to step 6.3.4.2); otherwise, entering step 6.3.4.3);
6.3.4.2) classifying by using a running/walking classifier, outputting a classification result, and finishing;
6.3.4.3) using a body/limb classifier to classify, and if the classification result is body, entering step 6.3.4.4); otherwise, entering step 6.3.4.5);
6.3.4.4) classifying by using a sitting/standing classifier, and outputting a classification result;
6.3.4.5) using a leg/arm classifier to classify, and if the classification result is a leg, entering the step 6.3.4.6); otherwise, outputting a classification result, and ending;
6.3.4.6) using a frequency classifier to classify, outputting a classification result and finishing.
8. The Wi-Fi signal-based human action hierarchy parsing and recognition device of claim 7, wherein:
in the signal preprocessing part, the principal component analysis operation is respectively operated aiming at the received data of each receiving antenna, namely, the principal component analysis is carried out on the 30 pieces of subcarrier data received on each receiving antenna, the first three principal components are reserved, and finally 3 multiplied by 3 principal components are obtained from 3 receiving antennas,
short-time Fourier transform operation is carried out on all principal component analysis signals simultaneously, and each piece of action data obtains a spectrogram.
9. The Wi-Fi signal-based human body action hierarchy parsing and recognition device according to claim 7, wherein a hardware configuration thereof comprises:
a sending end including a computer with wireless network card, a sending antenna TkAnd an antenna extension;
a receiving end including a computer with wireless network card, three receiving antennas Rx1，Rx2，Rx3And an antenna extension;
a platform for placing the antenna thereon,
wherein:
said three receiving antennas R for the receiving endx1，Rx2，Rx3With said one transmitting antenna T of the transmitting endxShould be placed at different horizontal heights;
two R in receiving end antennax1，Rx2Should be compatible with the transmitting antenna TxAt the same level, another receiving antenna Rx3Lower than the other three antennas and located at Rx1Or Rx2Right below.
10. The Wi-Fi signal-based human motion hierarchy parsing and recognition device of claim 7, wherein the signal pre-processing section comprises:
9.1) a modulus taking part, configured to take a modulus of the acquired signal to obtain a signal amplitude, where the original form of the acquired CSI signal is an n × m × k × t complex matrix, where n, m, k, and t are the number of transmit antennas, the number of receive antennas, the number of carriers, and time, and by performing a modulus taking operation on the complex number, that is, performing a sum of squares and an evolution operation on a real part and a complex part of the complex number, the complex number can be converted into a real number, that is, the amplitude of the signal, and for a complex number z ═ a + bi, where a, b ∈ R, and a modulus formula is:
Figure FDA0003599923430000071
9.2) a band-pass filtering part for performing band-pass filtering processing on the signals after modulus extraction and obtaining n multiplied by m multiplied by k time sequence signals H of the modulus extraction partmPerforming band-pass filtering operation to filter out high-frequency noise therein to obtain a signal H after filtering processingp，
9.3) a grouping and principal component analysis processing section for performing noise reduction on the n × m × k pieces of time series signals H obtained in 1.2pDividing the antenna pair number into n × m groups, analyzing and processing the principal components according to the carrier number k, and reserving the first 3 principal components, including:
a set of k timing signals Hp1Characterized in that:
Hpca1＝Hp1×e
wherein e is Hp1Singular matrix of, resulting in Hpca1For k × t matrix, according to the size of singular value, retaining principal component analysis signals corresponding to the largest three singular values, and finally obtaining n × m × 3 × t time sequence signals Hpca；
9.4) short-time Fourier transform part for H pairpcaCarrying out short-time Fourier transform to obtain a spectrogram, using sliding window operation, dividing signals into subsections according to time dimension, carrying out Fourier transform on each subsection, converting time sequence signals into frequency domain signals, and then arranging the frequency domain signals of each sub-window according to time sequence to obtain the spectrogram S simultaneously having time domain information and frequency domain information, wherein the spectrogram S is characterized by a formula:
Figure FDA0003599923430000081
where x (t) is the original signal, w is a window function, and τ is time.
CN201910867333.7A 2019-09-12 2019-09-12 Human body action layered analysis and identification method and device based on Wi-Fi signals Active CN110659598B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910867333.7A CN110659598B (en) 2019-09-12 2019-09-12 Human body action layered analysis and identification method and device based on Wi-Fi signals

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910867333.7A CN110659598B (en) 2019-09-12 2019-09-12 Human body action layered analysis and identification method and device based on Wi-Fi signals

Publications (2)

Publication Number Publication Date
CN110659598A CN110659598A (en) 2020-01-07
CN110659598B true CN110659598B (en) 2022-07-01

Family

ID=69037040

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910867333.7A Active CN110659598B (en) 2019-09-12 2019-09-12 Human body action layered analysis and identification method and device based on Wi-Fi signals

Country Status (1)

Country Link
CN (1) CN110659598B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112218303B (en) * 2020-09-28 2022-02-18 上海交通大学 Signal conversion method based on Wi-Fi identification system

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109635872A (en) * 2018-12-17 2019-04-16 上海观安信息技术股份有限公司 Personal identification method, electronic equipment and computer program product

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170024660A1 (en) * 2015-07-23 2017-01-26 Qualcomm Incorporated Methods and Systems for Using an Expectation-Maximization (EM) Machine Learning Framework for Behavior-Based Analysis of Device Behaviors

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109635872A (en) * 2018-12-17 2019-04-16 上海观安信息技术股份有限公司 Personal identification method, electronic equipment and computer program product

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
WiFi action recognition via vision-based methods;Jen-Yin Chang;《 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)》;20160519;全文 *
基于肌电信号层级分类的手部动作识别方法;赵漫丹;《北京生物医学工程》;20141031;全文 *

Also Published As

Publication number Publication date
CN110659598A (en) 2020-01-07

Similar Documents

Publication Publication Date Title
Wang et al. Wi-Fi CSI-based behavior recognition: From signals and actions to activities
Zhang et al. Data augmentation and dense-LSTM for human activity recognition using WiFi signal
CN106295684B (en) A kind of dynamic based on micro-Doppler feature is continuous/discontinuous gesture recognition methods
CN110348288B (en) Gesture recognition method based on 77GHz millimeter wave radar signal
US20190087654A1 (en) Method and system for csi-based fine-grained gesture recognition
CN109188414A (en) A kind of gesture motion detection method based on millimetre-wave radar
Liu et al. Deep learning and recognition of radar jamming based on CNN
CN106899968A (en) A kind of active noncontact identity identifying method based on WiFi channel condition informations
CN105044701B (en) Ground target sorting technique based on robustness time-frequency characteristics
CN103294199B (en) A kind of unvoiced information identifying system based on face's muscle signals
KR101293446B1 (en) Electroencephalography Classification Method for Movement Imagination and Apparatus Thereof
CN107527016B (en) User identity identification method based on motion sequence detection in indoor WiFi environment
Shi et al. Human activity recognition using deep learning networks with enhanced channel state information
CN110059612A (en) A kind of gesture identification method and system that the position based on channel state information is unrelated
CN110414468B (en) Identity verification method based on gesture signal in WiFi environment
CN109512390B (en) Sleep staging method and wearable device based on EEG time domain multi-dimensional features and M-WSVM
Miao et al. Underwater acoustic signal classification based on sparse time–frequency representation and deep learning
CN109325399A (en) A kind of stranger's gesture identification method and system based on channel state information
CN112580486B (en) Human behavior classification method based on radar micro-Doppler signal separation
WO2023029390A1 (en) Millimeter wave radar-based gesture detection and recognition method
CN115343704A (en) Gesture recognition method of FMCW millimeter wave radar based on multi-task learning
CN110659598B (en) Human body action layered analysis and identification method and device based on Wi-Fi signals
Fang et al. Writing in the air: recognize letters using deep learning through WiFi signals
Shi et al. Deep learning networks for human activity recognition with CSI correlation feature extraction
Zhang et al. WiEnhance: Towards data augmentation in human activity recognition using WiFi signal

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant