CN116318297A - Wireless gesture sensing solution method and device - Google Patents
Wireless gesture sensing solution method and device Download PDFInfo
- Publication number
- CN116318297A CN116318297A CN202310083717.6A CN202310083717A CN116318297A CN 116318297 A CN116318297 A CN 116318297A CN 202310083717 A CN202310083717 A CN 202310083717A CN 116318297 A CN116318297 A CN 116318297A
- Authority
- CN
- China
- Prior art keywords
- signal
- sub
- carrier
- window
- csi
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 230000009471 action Effects 0.000 claims abstract description 54
- 230000002159 abnormal effect Effects 0.000 claims abstract description 34
- 239000011159 matrix material Substances 0.000 claims abstract description 21
- 239000000969 carrier Substances 0.000 claims abstract description 16
- 238000006467 substitution reaction Methods 0.000 claims description 7
- 239000013598 vector Substances 0.000 claims description 6
- 238000012216 screening Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 10
- 230000007474 system interaction Effects 0.000 abstract description 4
- 230000008569 process Effects 0.000 description 35
- 230000003993 interaction Effects 0.000 description 14
- 239000000243 solution Substances 0.000 description 13
- 230000033001 locomotion Effects 0.000 description 9
- 230000008859 change Effects 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000007781 pre-processing Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 240000007651 Rubus glaucus Species 0.000 description 2
- 235000011034 Rubus glaucus Nutrition 0.000 description 2
- 235000009122 Rubus idaeus Nutrition 0.000 description 2
- 238000007792 addition Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000009499 grossing Methods 0.000 description 2
- 238000002347 injection Methods 0.000 description 2
- 239000007924 injection Substances 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B7/00—Radio transmission systems, i.e. using radiation field
- H04B7/02—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas
- H04B7/04—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas
- H04B7/06—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station
- H04B7/0613—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station using simultaneous transmission
- H04B7/0615—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station using simultaneous transmission of weighted versions of same signal
- H04B7/0619—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station using simultaneous transmission of weighted versions of same signal using feedback from receiving side
- H04B7/0621—Feedback content
- H04B7/0626—Channel coefficients, e.g. channel state information [CSI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W72/00—Local resource management
- H04W72/04—Wireless resource allocation
- H04W72/044—Wireless resource allocation based on the type of the allocated resource
- H04W72/0453—Resources in frequency domain, e.g. a carrier in FDMA
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W84/00—Network topologies
- H04W84/02—Hierarchically pre-organised networks, e.g. paging networks, cellular networks, WLAN [Wireless Local Area Network] or WLL [Wireless Local Loop]
- H04W84/10—Small scale networks; Flat hierarchical networks
- H04W84/12—WLAN [Wireless Local Area Networks]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
The invention provides a wireless gesture sensing solving method and a wireless gesture sensing solving device, wherein the method comprises the following steps: calculating a CSI signal based on a WiFi signal generated by a WiFi transmitting end and a WiFi signal received by a WiFi receiving end, wherein the CSI signal comprises multiple sub-carriers at the same time; sequentially checking the CSI signals by adopting a sliding window, calculating the variance of each sub-carrier based on the signal value of each signal point in the sub-carrier, determining whether the sub-carrier in the sliding window is an abnormal sub-carrier based on the variance of each sub-carrier, and determining an action starting window based on the number of the abnormal sub-carriers in the sliding window; based on a preset acquisition length, acquiring a corresponding CSI signal segment by taking a starting window as a marking window, acquiring a signal value of each signal point in the CSI signal segment, and constructing a signal matrix; inputting the signal matrix into a preset classifier to obtain an action classification result. The scheme does not need multipath antenna signal processing, and reduces time delay, thereby improving the real-time performance of system interaction.
Description
Technical Field
The invention relates to the technical field of motion sensing, in particular to a wireless gesture sensing solution method and a wireless gesture sensing solution device.
Background
With the development of the internet of things and smart home, the requirement for controlling home equipment such as televisions, lamps, sound equipment and the like through human body actions is increasing, and how to realize the perception of human body actions becomes a challenging problem. At present, human body actions are mainly perceived in two modes of contact type and non-contact type, wherein contact type means that the actions are perceived by acquiring data such as speed, acceleration and the like through wearable special equipment or sensors and calculating the direction, speed and the like of limb movements; non-contact motion sensing also includes computer vision-based, radio signal-based, and other technical means. Compared with the contact type motion sensing, the non-contact type motion sensing does not need a user to wear any equipment, so that the non-contact type motion sensing is easy to use, and long-time and uninterrupted sensing can be realized.
In the non-contact motion sensing technology, a Computer Vision (CV) based method has high accuracy, but the sensing range is limited by the viewing distance and is easily affected by light, obstacles, etc., and users have a privacy concern. Compared with the method based on computer vision, the method based on the radio signal is not easy to be interfered by factors such as weather, light rays and the like, and has good protection on user privacy.
The existing system for realizing human body action recognition by using the CSI mainly considers the accuracy and the anti-interference performance of recognition, forms MIMO by using a plurality of receiving and transmitting antennas, extracts different characteristics from a plurality of antennas positioned at different positions for fusion, extracts physical quantities by using the CSI signals and constructs a physical model, and improves the recognition accuracy and the robustness of the system by using a plurality of different schemes such as a neural network with a more complex structure, but has a certain problem in real-time performance: increasing the number of antennas increases the amount of raw data, extracting physical quantity features or using a neural network with a more complex structure increases the complexity of the algorithm, which leads to an increase in the amount of computation. In the smart home scene, the commonly used physical equipment has low power consumption and weaker performance, and inevitably generates larger time delay when facing large calculation amount, so that the real-time performance of system interaction is reduced, and the user experience is greatly influenced.
Disclosure of Invention
In view of this, embodiments of the present invention provide a wireless gesture sensing solution that obviates or mitigates one or more of the disadvantages of the prior art.
One aspect of the present invention provides a wireless gesture sensing solution, the method comprising the steps of:
calculating a CSI signal based on a WiFi signal generated by a WiFi transmitting end and a WiFi signal received by a WiFi receiving end, wherein the CSI signal comprises multiple sub-carriers at the same time;
sequentially checking the CSI signals by adopting a sliding window, calculating the variance of each sub-carrier based on the signal value of each signal point in the sub-carrier, determining whether the sub-carrier in the sliding window is an abnormal sub-carrier based on the variance of each sub-carrier, and determining an action starting window based on the number of the abnormal sub-carriers in the sliding window;
based on a preset acquisition length, acquiring a corresponding CSI signal segment by taking the starting window as a marking window, acquiring a signal value of each signal point in the CSI signal segment, and constructing a signal matrix;
inputting the signal matrix into a preset classifier to obtain an action classification result.
By adopting the scheme, the CSI signal can be acquired by adopting the WiFi transmitting end and the WiFi receiving end, and the scheme does not need multipath antenna signal processing, so that the time delay is reduced, and the real-time performance of system interaction is improved; the method and the device further determine the abnormal subcarriers based on the variance of each subcarrier, determine the action starting window based on the number of the abnormal subcarriers in the sliding window, accurately acquire the CSI signal segments containing the action interference, further accurately construct a signal matrix, judge the action type, accurately identify the CSI signal segments containing the action interference and accurately determine the action type.
In some embodiments of the present invention, the step of calculating the CSI signal based on the WiFi signal generated by the WiFi transmitting end and the WiFi signal received by the WiFi receiving end further includes calculating a mean value and a standard deviation of signal values based on the signal values of each path of the subcarriers, calculating a standard score for each signal point in the subcarriers based on the mean value and the standard deviation, and removing an outlier in the signal point based on the standard score.
In some embodiments of the present invention, in the step of calculating a standard score for each signal point in the subcarrier based on the mean and standard deviation, the standard score is calculated according to the following formula:
z＝(x-μ)σ；
where z represents the standard fraction, μ represents the mean, σ represents the standard deviation, and x represents the signal value of the signal point.
In some embodiments of the invention, the step of removing outliers in the signal points based on the standard score comprises:
and if not, calculating a replacement value based on the mean value and the standard deviation of the subcarriers where the signal points are located, and modifying the abnormal value into the replacement value.
In some embodiments of the present invention, in the step of calculating the substitution value based on the mean value and standard deviation of the subcarrier in which the signal point is located, the substitution value is calculated according to the following formula:
δ＝μ±2σ；
where δ represents the substitution value, μ represents the mean value, and σ represents the standard deviation.
In some embodiments of the present invention, the step of calculating the CSI signal based on the WiFi signal generated by the WiFi transmitting end and the WiFi signal received by the WiFi receiving end further includes:
and filtering the CSI signal.
In some embodiments of the present invention, the step of calculating the CSI signal based on the WiFi signal generated by the WiFi transmitting end and the WiFi signal received by the WiFi receiving end further includes:
constructing a subcarrier vector of each subcarrier based on a signal value of each signal point of each subcarrier in the CSI signal;
respectively calculating the distance between each sub-carrier and other sub-carriers based on the sub-carrier vectors, and summing to obtain the distance sum of each sub-carrier;
and screening the distance of the subcarriers and a third smaller threshold number of subcarriers.
In some embodiments of the present invention, the step of determining whether the subcarriers within the sliding window are abnormal subcarriers based on the per-subcarrier variance includes:
calculating a dynamic threshold for each sub-carrier within a sliding window based on the variance of the sub-carrier;
determining whether the sub-carrier within the sliding window is an abnormal sub-carrier based on the dynamic threshold.
In some embodiments of the present invention, the step of calculating a dynamic threshold for each sub-carrier within a sliding window based on the variance of that sub-carrier is to calculate the dynamic threshold based on the following formula:
Tn, i ＝γ((1-α)Tn- 1,i +αVi)；
wherein T is n,i Dynamic threshold value, tn, representing ith sub-carrier in nth sliding window -1,i Representing the dynamic threshold of the ith sub-carrier in the n-1 st sliding window, wherein gamma and alpha are respectively preset adjustment coefficients and importance coefficients, V i Representing the variance of the ith sub-carrier.
In some embodiments of the present invention, in the step of determining the action start window based on the number of abnormal subcarriers in the sliding window, a ratio of the number of abnormal subcarriers to the total number of subcarriers in the same sliding window is calculated, and if the calculated ratio is greater than a preset second threshold, the sliding window is the action start window.
In some embodiments of the present invention, based on a preset acquisition length, the step of acquiring a signal value of each signal point in the CSI signal segment by using the start window as a marking window and constructing the signal matrix includes:
the acquisition length comprises a first length extending in a time reverse sequence and a second length extending in a time sequence, the initial window is used as the first length extending in the time reverse sequence direction of the marking window, and the initial window is used as the second length extending in the time reverse sequence direction of the marking window, so that the CSI signal segment is acquired;
and taking each sub-carrier in the CSI signal section as a transverse row of the matrix, and taking the signal value of each signal point in the sub-carrier as a parameter in the transverse row to construct the signal matrix.
The second aspect of the present invention also provides a wireless gesture sensing solution apparatus comprising a computer device comprising a processor and a memory, the memory having stored therein computer instructions for executing the computer instructions stored in the memory, the apparatus implementing the steps implemented by the method as described above when the computer instructions are executed by the processor.
The third aspect of the present invention also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor realizes the steps implemented by the aforementioned wireless gesture awareness solution.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and drawings.
It will be appreciated by those skilled in the art that the objects and advantages that can be achieved with the present invention are not limited to the above-described specific ones, and that the above and other objects that can be achieved with the present invention will be more clearly understood from the following detailed description.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate and together with the description serve to explain the invention.
FIG. 1 is a schematic diagram of an embodiment of a wireless gesture sensing solution of the present invention;
fig. 2 is a schematic diagram of processing an acquired CSI signal;
FIG. 3 is a schematic diagram of actions to be identified;
FIG. 4 is a schematic view of a Fresnel zone;
FIG. 5 is a schematic view of an installation mode of the present embodiment;
FIG. 6 is a schematic diagram of the process flow of the present embodiment;
fig. 7 is a functional schematic diagram of a transmitting end and a receiving end.
Detailed Description
The present invention will be described in further detail with reference to the following embodiments and the accompanying drawings, in order to make the objects, technical solutions and advantages of the present invention more apparent. The exemplary embodiments of the present invention and the descriptions thereof are used herein to explain the present invention, but are not intended to limit the invention.
It should be noted here that, in order to avoid obscuring the present invention due to unnecessary details, only structures and/or processing steps closely related to the solution according to the present invention are shown in the drawings, while other details not greatly related to the present invention are omitted.
It should be emphasized that the term "comprises/comprising" when used herein is taken to specify the presence of stated features, elements, steps or components, but does not preclude the presence or addition of one or more other features, elements, steps or components.
It is also noted herein that the term "coupled" may refer to not only a direct connection, but also an indirect connection in which an intermediate is present, unless otherwise specified.
Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.
In order to solve the above problems, as shown in fig. 1, the present invention proposes a wireless gesture sensing solution, which includes the steps of:
step S100, calculating a CSI signal based on a WiFi signal generated by a WiFi transmitting end and a WiFi signal received by a WiFi receiving end, wherein the CSI signal comprises multiple sub-carriers at the same time;
in a specific implementation process, a CSI (Channel State Information ) signal is obtained through a nexmon, where nexmon is a firmware patch framework developed by using a C language and oriented to a WiFi chip, and in this scheme, a WiFi chip firmware patch of the nexmon can be developed based on the nexmon, so as to activate a Monitor mode of the WiFi chip or implement a Frame Injection (Frame Injection) function. The nexmon CSI software developed based on nexmon can extract CSI of channels through the WiFI chip, which supports 4x4 MIMO (Multi-Input Multi-Output system), bandwidth of 80MHz, and 256 subcarriers at the highest. The abundant CSI information can better reflect the state of the channel, and stronger wireless sensing capability is brought.
In a specific implementation process, the CSI is obtained by calculating the difference between the signal received by the receiving end and the signal sent by the transmitting end. The transmitting end periodically sends out WiFi frames, and in order to ensure that enough information is acquired and the stability of the acquired signals, the transmitting end sends ECHO messages of ICMP (Internet Control Message Protocol ) at the frequency of 500 Hz. The raspberry group at the receiving end modifies BCM42455C0 network card firmware on the raspberry group through a nexmon patch framework, decodes the received signals at the Linux kernel driving layer, can obtain original signals sent by the sending end, and then makes a difference with actual signals received by the WiFi receiving end, so that the CSI of the channel can be calculated. The CSI acquisition process of the receiving end continuously acquires the CSI of the WiFi channel from the network card, and the received CSI data is forwarded from the Linux kernel to the application layer for subsequent processing by using a UDP (User Datagram Protocol ) protocol through a socket.
At the WiFi receiving end, CSI data acquired from the WiFi channel is transmitted to the main process through the socket in the form of UDP datagram, and the format of the datagram is shown in table 1 below.
TABLE 1
The CSI preprocessing module of the main process starts a 5500 port of a UDP server for monitoring a local loop address, and obtains CSI data from the port;
the CSI preprocessing module takes out a plurality of UDP messages from the socket each time, and parses the UDP messages according to the message structure described in table 1. Since the MIMO technology is used for transmission between the transmitting end and the receiving end, wiFi frames with the same Sequence Number are sent by different spatial streams and received by different antennas, so that the CSI preprocessing module needs to assemble multiple messages with the same Sequence Number together and discard incomplete data caused by packet loss. One WiFi channel contains a plurality of subcarriers, and the number of subcarriers contained in channels with different bandwidths is also different. The 20MHz bandwidth WiFi channel used by the system comprises 64 subcarriers, wherein part of subcarriers do not carry any data, so that the subcarriers are removed during processing; in addition, because the paths of different subcarriers are similar in the propagation process, the contained data have certain similarity, so in order to reduce the data volume of the system and improve the system performance, 40 subcarriers actually carrying the data are finally selected from 64 subcarriers to be used for subsequent processing.
In some embodiments of the present invention, the WiFi transmitting terminal is provided with at least one transmitting antenna.
Step S200, sequentially checking the CSI signals by adopting a sliding window, calculating the variance of each sub-carrier based on the signal value of each signal point in the sub-carrier, determining whether the sub-carrier in the sliding window is an abnormal sub-carrier based on the variance of each sub-carrier, and determining an action starting window based on the number of the abnormal sub-carriers in the sliding window;
in a specific implementation process, a sliding window is adopted to sequentially check the CSI signals based on time sequence, and each window comprises the same number of subcarriers.
In the implementation process, the signal value of each signal point in one path of subcarrier is added to the calculation of variance.
Step S300, based on a preset acquisition length, acquiring a corresponding CSI signal segment by taking the initial window as a marking window, acquiring a signal value of each signal point in the CSI signal segment, and constructing a signal matrix;
in a specific implementation process, the acquisition length comprises a first length extending in a time reverse order and a second length extending in a time sequence, the initial window is used as the first length extending in the time reverse order direction of the marking window, and the initial window is used as the second length extending in the time reverse order direction of the marking window, so that the CSI signal segment is acquired;
the first length may be 1 window length, each window covering 100 data packets; the second length may be 7 window lengths.
As shown in fig. 3, in step S400, the signal matrix is input into a preset classifier, so as to obtain an action classification result.
In the specific implementation process, the classifier can be a Bottleneck structure of the MobileNet V3, the system adopts a design of a lightweight network, the Bottleneck structure of the MobileNet V3 is mainly adopted on the network structure, the scale of the network is greatly reduced on the premise of maintaining unchanged output dimension through deep separable convolution operation, the parameter quantity of the network is reduced, and the calculation amount required in the training and reasoning process is reduced. The Bottleneck structure can also comprise a SE (Squeeze and Excitation) module which is similar to a lightweight attention mechanism and is connected between two convolution operations of the Bottleneck, and the importance of each channel output by the previous convolution layer is acquired through learning to modify the weight of each channel, so that the recalibration of the network is realized and the robustness of the network is increased. The motion recognition model adopted by the system is formed by stacking Bottleneck structures comprising different convolution kernel sizes, channel numbers, activation functions and the like, and the Bottleneck is specifically shown in table 2.
TABLE 2
By adopting the scheme, the CSI signal can be acquired by adopting the WiFi transmitting end and the WiFi receiving end, and the scheme does not need multipath antenna signal processing, so that the time delay is reduced, and the real-time performance of system interaction is improved; the method and the device further determine the abnormal subcarriers based on the variance of each subcarrier, determine the action starting window based on the number of the abnormal subcarriers in the sliding window, accurately acquire the CSI signal segments containing the action interference, further accurately construct a signal matrix, judge the action type, accurately identify the CSI signal segments containing the action interference and accurately determine the action type.
As shown in fig. 2, in some embodiments of the present invention, after the step of calculating CSI signals based on the WiFi signal generated by the WiFi transmitting end and the WiFi signal received by the WiFi receiving end, step S110 further includes calculating a mean value and a standard deviation of signal values based on the signal values of each of the sub-carriers, calculating a standard score for each of the signal points in the sub-carriers based on the mean value and the standard deviation, and removing abnormal values in the signal points based on the standard score.
In some embodiments of the present invention, in the step of calculating a standard score for each signal point in the subcarrier based on the mean and standard deviation, the standard score is calculated according to the following formula:
z＝(x-μ)σ；
where z represents the standard fraction, μ represents the mean, σ represents the standard deviation, and x represents the signal value of the signal point.
In some embodiments of the invention, the step of removing outliers in the signal points based on the standard score comprises:
whether the standard fraction of the signal point is in a preset first threshold range or not, if not, calculating a replacement value based on the mean value and standard deviation of the subcarriers where the signal point is located, and modifying the abnormal value into the replacement value; if so, no modification is required.
In some embodiments of the present invention, in the step of calculating the substitution value based on the mean value and standard deviation of the subcarrier in which the signal point is located, the substitution value is calculated according to the following formula:
δ＝μ±2σ；
where δ represents the substitution value, μ represents the mean value, and σ represents the standard deviation.
In a specific implementation process, if the calculated standard score is lower than the lower limit value of the first threshold value, delta=mu-2σ is calculated to obtain a replacement value; if the calculated standard score is higher than the upper limit value of the first threshold, δ=μ+2σ is calculated, and a replacement value is obtained.
In an implementation, the first threshold may be [ -2,2].
By adopting the scheme, the abnormal value is proposed, and the accuracy of motion recognition is improved.
In some embodiments of the present invention, the step of calculating the CSI signal based on the WiFi signal generated by the WiFi transmitting end and the WiFi signal received by the WiFi receiving end further includes:
step S120, filtering the CSI signal.
In a specific implementation process, an input CSI data stream is smoothed by an FIR filter based on a Hanning window.
In the specific implementation process, a sliding window with a fixed size of 41 is used to slide along the input CSI data stream after abnormal value removal, in each window, data in the window are symmetrically extended along two sides by taking a window starting point and a window ending point as axes for each subcarrier, then convolution operation is carried out on the generated Hanning window and the extended window, and finally the middle 41 data of the convolved result are output as output windows. Clutter in the input CSI data stream is eliminated and the output CSI data stream is smoothed through continuous sliding and calculation of a window.
In some embodiments of the present invention, the step of calculating the CSI signal based on the WiFi signal generated by the WiFi transmitting end and the WiFi signal received by the WiFi receiving end further includes: in step S130 of the process of the present invention,
constructing a subcarrier vector of each subcarrier based on a signal value of each signal point of each subcarrier in the CSI signal;
respectively calculating the distance between each sub-carrier and other sub-carriers based on the sub-carrier vectors, and summing to obtain the distance sum of each sub-carrier;
and screening the distance of the subcarriers and a third smaller threshold number of subcarriers.
In the implementation process, dynamic time warping (Dynamic TimeWarping, DTW) is adopted to calculate the distance between different subcarriers, then the distance between any two subcarriers is expressed as a matrix, and the matrix is summed according to rows, namely the sum of the distances from each subcarrier to all other subcarriers is calculated.
With the above scheme, the original CSI is affected by noise in the environment, hardware thermal noise, and the like, and many spikes formed by abnormal values may occur, and these spikes may affect the subsequent action segmentation and recognition, so that removal is required. The method does not use wavelet reconstruction method to remove outliers, but uses statistic-based method to remove noise with significant difference from other data in original signal, thus eliminating most outliers without excessive calculation.
In some embodiments of the present invention, the step of determining whether the subcarriers within the sliding window are abnormal subcarriers based on the per-subcarrier variance includes:
calculating a dynamic threshold for each sub-carrier within a sliding window based on the variance of the sub-carrier;
determining whether the sub-carrier within the sliding window is an abnormal sub-carrier based on the dynamic threshold.
In some embodiments of the present invention, the step of calculating a dynamic threshold for each sub-carrier within a sliding window based on the variance of that sub-carrier is to calculate the dynamic threshold based on the following formula:
Tn, i ＝γ((1-α)Tn- 1,i +αVi)；
wherein Tn ,i Dynamic threshold value, tn, representing ith sub-carrier in nth sliding window -1,i The dynamic threshold value of the ith sub-carrier in the n-1 sliding window is represented, gamma and alpha are respectively preset adjustment coefficients and importance coefficients, and Vi represents the variance of the ith sub-carrier.
In some embodiments of the present invention, in the step of determining the action start window based on the number of abnormal subcarriers in the sliding window, a ratio of the number of abnormal subcarriers to the total number of subcarriers in the same sliding window is calculated, and if the calculated ratio is greater than a preset second threshold, the sliding window is the action start window.
In the implementation process, the second threshold may be 0.5, and if the ratio of the number of abnormal subcarriers to the total number of subcarriers in the same sliding window is greater than 0.5, the sliding window is an action start window.
By adopting the scheme, the CSI data stream after abnormal value removal and smoothing is input into the action detection module, the data stream comprises a section with human action and a blank section without any action, in order to avoid system performance waste and time delay increase caused by identifying the blank section, the action detection module detects the input CSI data stream, finds the starting position of the effective action of the human body, and then intercepts the section containing the human action data for subsequent action identification. Since the CSI signal is relatively stable when the human body does not move, and the CSI signal fluctuates by a large extent when the human body starts to move, the variance can be used to determine whether the human body starts to move.
The method uses a dynamic threshold variance checking method, continuously checks the variance of a current window on a CSI data stream through a sliding window with a fixed size, and judges the current window as a starting window of human actions if the variance exceeds a threshold. The dynamic threshold method considers variance change caused by background noise change in past time and variance change caused by human body action at the same time, and adjusts the variance change through an importance factor; in addition, the dynamic threshold method also considers a plurality of subcarriers simultaneously, and decides a final judgment result through a voting mechanism.
In some embodiments of the present invention, based on a preset acquisition length, the step of acquiring a signal value of each signal point in the CSI signal segment by using the start window as a marking window and constructing the signal matrix includes:
the acquisition length comprises a first length extending in a time reverse sequence and a second length extending in a time sequence, the initial window is used as the first length extending in the time reverse sequence direction of the marking window, and the initial window is used as the second length extending in the time reverse sequence direction of the marking window, so that the CSI signal segment is acquired;
and taking each sub-carrier in the CSI signal section as a transverse row of the matrix, and taking the signal value of each signal point in the sub-carrier as a parameter in the transverse row to construct the signal matrix.
In the specific implementation process, the starting window is used as a first length of the marking window extending towards the time reverse sequence direction, and the starting window is used as a second length of the marking window extending towards the time reverse sequence direction, so that the CSI signal segment is acquired;
the first length may be 1 window length, each window covering 100 data packets; the second length may be 7 window lengths.
In the specific implementation process, the classifier is trained in advance by utilizing the data set, and when the classifier is acquired, gesture actions are continuously performed for a plurality of times, and the action time is slightly stopped, so that a section of longer CSI data comprising the gesture actions for a plurality of times is formed. Finally, 1200 sets of data are acquired for each action, and if four recognition actions exist, 4800 sets of sample data are taken as a data set.
As shown in fig. 7, in the implementation process, the WiFi frame transmission process of the transmitting end is to generate a WiFi frame, and periodically send the WiFi frame at a fixed frequency through the wireless network card. The CSI acquisition process of the receiving end is responsible for acquiring the CSI from the WiFi channel and transmitting the CSI to each module of the main process for subsequent processing; the CSI preprocessing module of the main process performs preprocessing on received original CSI data, and comprises the steps of merging and extracting data from a plurality of spatial streams, removing noise in an original signal, performing further smoothing on the denoised signal and the like; the action detection module of the main process needs to find the interval generated by the effective action from the continuously input CSI data stream, and intercepts the interval for further identification; the action recognition module uses a recognition model based on deep learning, takes the intercepted single-section action data as input, and judges a specific action type. The interaction process is responsible for connecting the system core function module and the external equipment, the external equipment can register itself to the interaction process through specific connection, the interaction process can send the identification result to all registered external equipment, and then the external equipment realizes respective interaction logic to realize the decoupling of interaction and core functions.
As shown in fig. 4 and 5, according to the fresnel zone model theory, when the distance between the transmitting end T and the receiving end R is 2m, the first fresnel zone may represent the following formula, where λ is a wavelength, and when a WiFi channel with a frequency of 2.4GHz is used, λ is about 125mm.
|TP|+|PR|-2＝λ/2
Since the diffraction phenomenon of the wireless signal is more than the reflection phenomenon when the object is located in the first fresnel zone, and the reflection phenomenon of the wireless signal is more than the diffraction phenomenon when the object is located outside the first fresnel zone, the sensing area should be selected at least outside the first fresnel zone. When an object passes through the fresnel zone boundary, the difference between the reflected path and the direct path length will change continuously, resulting in a change in the phase difference between the signal arriving along the reflected path and the signal arriving along the direct path, causing the CSI waveform to fluctuate dramatically, reflecting the characteristics of different actions, so the perceived area needs to at least cross multiple fresnel zone boundaries. Therefore, assuming that any point in the sensing region is P, P should satisfy the condition:
|TP|+|PR|＝2+βλ/2,β＝2,3,4...
that is, when using a WiFi channel of 2.4GHz, the distance from P to the transmitting end T and the distance from P to the receiving end R should be 2.0625m,2.1250m,2.1885m … …, taking the factors such as the device deployment position, the user operation position, the signal strength and the like into consideration, and finally selecting a point on the boundary of the eighth fresnel zone as the center of the sensing area, as shown in fig. 5, that is, at this time, n=8, the sum of the distances from the point P to the transmitting end T and the receiving end R is 2+8×0.0625 m=2.5 m, and the sensing area is within a square range of 0.5m each in length and width, so when performing gesture in the sensing area, the maximum can pass through the boundary of 8 fresnel zones, so that the CSI change is more severe, and more effective information can be contained.
In the implementation process, an interaction module running in the interaction process is responsible for processing interaction with various external devices, maintains a WebSocket connection pool, and once the external devices establish WebSocket connection with the interaction process, relevant information of the devices is registered to the interaction module. Meanwhile, the interaction module also establishes WebSocket connection with the main process, the main process sends real-time CSI data flow, detected effective action data and action recognition results to the interaction module through the WebSocket, and then the interaction module broadcasts to external equipment registered in a connection pool, and the interaction process is shown in fig. 6.
The embodiment of the invention also provides a wireless gesture perception solving device, which comprises computer equipment, wherein the computer equipment comprises a processor and a memory, the memory is stored with computer instructions, the processor is used for executing the computer instructions stored in the memory, and the device realizes the steps realized by the method when the computer instructions are executed by the processor.
Embodiments of the present invention also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps performed by the wireless gesture sensing solution described above. The computer readable storage medium may be a tangible storage medium such as Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disk, a removable memory disk, a CD-ROM, or any other form of storage medium known in the art.
Those of ordinary skill in the art will appreciate that the various illustrative components, systems, and methods described in connection with the embodiments disclosed herein can be implemented as hardware, software, or a combination of both. The particular implementation is hardware or software dependent on the specific application of the solution and the design constraints. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the invention are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave.
It should be understood that the invention is not limited to the particular arrangements and instrumentality described above and shown in the drawings. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present invention are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after appreciating the spirit of the present invention.
In this disclosure, features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, and various modifications and variations can be made to the embodiments of the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A wireless gesture-aware solution, the method comprising the steps of:
calculating a CSI signal based on a WiFi signal generated by a WiFi transmitting end and a WiFi signal received by a WiFi receiving end, wherein the CSI signal comprises multiple sub-carriers at the same time;
sequentially checking the CSI signals by adopting a sliding window, calculating the variance of each sub-carrier based on the signal value of each signal point in the sub-carrier, determining whether the sub-carrier in the sliding window is an abnormal sub-carrier based on the variance of each sub-carrier, and determining an action starting window based on the number of the abnormal sub-carriers in the sliding window;
based on a preset acquisition length, acquiring a corresponding CSI signal segment by taking the starting window as a marking window, acquiring a signal value of each signal point in the CSI signal segment, and constructing a signal matrix;
inputting the signal matrix into a preset classifier to obtain an action classification result.
2. The wireless gesture sensing solution according to claim 1, wherein after the step of calculating CSI signals based on the WiFi signal generated by the WiFi transmitting end and the WiFi signal received by the WiFi receiving end, further comprises calculating a mean value and a standard deviation of signal values based on the signal values of each of the sub-carriers, calculating a standard score for each signal point in the sub-carriers based on the mean value and the standard deviation, and removing an abnormal value in the signal point based on the standard score.
3. The wireless gesture sensing solution of claim 2, wherein the step of removing outliers in the signal points based on the standard score comprises:
and if not, calculating a replacement value based on the mean value and the standard deviation of the subcarriers where the signal points are located, and modifying the abnormal value into the replacement value.
4. A wireless gesture sensing solution according to claim 3, wherein in the step of calculating the replacement value based on the mean and standard deviation of the sub-carriers in which the signal points are located, the replacement value is calculated according to the following formula:
δ＝μ±2σ；
where δ represents the substitution value, μ represents the mean value, and σ represents the standard deviation.
5. The wireless gesture sensing solution according to claim 1, wherein after the step of calculating CSI signals based on the WiFi signal generated by the WiFi transmitting end and the WiFi signal received by the WiFi receiving end, the method further comprises:
constructing a subcarrier vector of each subcarrier based on a signal value of each signal point of each subcarrier in the CSI signal;
respectively calculating the distance between each sub-carrier and other sub-carriers based on the sub-carrier vectors, and summing to obtain the distance sum of each sub-carrier;
and screening the distance of the subcarriers and a third smaller threshold number of subcarriers.
6. The wireless gesture sensing solution of claim 1, wherein the step of determining whether a subcarrier within the sliding window is an abnormal subcarrier based on the per-subcarrier variance comprises:
calculating a dynamic threshold for each sub-carrier within a sliding window based on the variance of the sub-carrier;
determining whether the sub-carrier within the sliding window is an abnormal sub-carrier based on the dynamic threshold.
7. The wireless gesture sensing solution of claim 6, wherein the step of calculating a dynamic threshold for each sub-carrier within a sliding window based on the variance of the sub-carrier is based on the formula:
T n,i ＝γ((1-α)T n-1,i +αV i )；
wherein T is n,i A dynamic threshold value T representing the ith sub-carrier in the nth sliding window n-1,i Representing the dynamic threshold of the ith sub-carrier in the n-1 st sliding window, wherein gamma and alpha are respectively preset adjustment coefficients and importance coefficients, V i Representing the variance of the ith sub-carrier.
8. The wireless gesture sensing solution according to claim 1, wherein in the step of determining the action start window based on the number of abnormal subcarriers in the sliding window, a ratio of the number of abnormal subcarriers to the total number of subcarriers in the same sliding window is calculated, and if the calculated ratio is greater than a preset second threshold, the sliding window is the action start window.
9. The wireless gesture sensing solution according to any one of claims 1 to 8, wherein the step of acquiring the signal value of each signal point in the CSI signal segment based on the preset acquisition length and using the start window as a marking window to acquire the corresponding CSI signal segment, and constructing the signal matrix includes:
the acquisition length comprises a first length extending in a time reverse sequence and a second length extending in a time sequence, the initial window is used as the first length extending in the time reverse sequence direction of the marking window, and the initial window is used as the second length extending in the time reverse sequence direction of the marking window, so that the CSI signal segment is acquired;
and taking each sub-carrier in the CSI signal section as a transverse row of the matrix, and taking the signal value of each signal point in the sub-carrier as a parameter in the transverse row to construct the signal matrix.
10. A wireless gesture aware resolution apparatus, characterized in that the apparatus comprises a computer device comprising a processor and a memory, the memory having stored therein computer instructions for executing the computer instructions stored in the memory, the apparatus realizing the steps of the method according to any of claims 1-9 when the computer instructions are executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310083717.6A CN116318297A (en) | 2023-02-08 | 2023-02-08 | Wireless gesture sensing solution method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310083717.6A CN116318297A (en) | 2023-02-08 | 2023-02-08 | Wireless gesture sensing solution method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116318297A true CN116318297A (en) | 2023-06-23 |
Family
ID=86784161
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310083717.6A Pending CN116318297A (en) | 2023-02-08 | 2023-02-08 | Wireless gesture sensing solution method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116318297A (en) |
-
2023
- 2023-02-08 CN CN202310083717.6A patent/CN116318297A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107968689B (en) | Perception identification method and device based on wireless communication signals | |
| Ali et al. | Recognizing keystrokes using WiFi devices | |
| Ahmed et al. | Device free human gesture recognition using Wi-Fi CSI: A survey | |
| WO2017038100A1 (en) | Monitoring server, method for determining distributed processing, and non-temporary computer-readable medium in which program is stored | |
| Guo et al. | WiReader: Adaptive air handwriting recognition based on commercial WiFi signal | |
| Tavakkoli et al. | Non-parametric statistical background modeling for efficient foreground region detection | |
| WO2022148477A1 (en) | Wifi-based human body detection method, and smart device | |
| Dou et al. | Edge computing-enabled deep learning for real-time video optimization in IIoT | |
| KR20210012012A (en) | Object tracking methods and apparatuses, electronic devices and storage media | |
| CN110852200B (en) | Non-contact human body action detection method | |
| Ozer et al. | Offloading deep learning powered vision tasks from UAV to 5G edge server with denoising | |
| Tong et al. | A fine-grained channel state information-based deep learning system for dynamic gesture recognition | |
| CN116318297A (en) | Wireless gesture sensing solution method and device | |
| CN101344920B (en) | Method and device for detecting specific area in video data frame | |
| Chen et al. | WiTT: Modeling and the evaluation of table tennis actions based on WIFI signals | |
| Khan et al. | Data rate estimation based on scene complexity for dynamic vision sensors on unmanned vehicles | |
| CN117056717A (en) | Behavior detection model training method, detection method, device and storage medium | |
| Yang et al. | Recognition for human gestures based on convolutional neural network using the off-the-shelf Wi-Fi routers | |
| CN113918004A (en) | Gesture recognition method, device, medium, and system thereof | |
| CN116644300A (en) | Through-wall target behavior identification method and device based on channel state information | |
| CN109753934A (en) | A kind of method and identification device identifying image true-false | |
| Liu et al. | Deep learning based encryption policy intrusion detection using commodity WiFi | |
| CN114037741A (en) | Adaptive target detection method and device based on event camera | |
| Bhat et al. | Gesture Recognition with mmWave Wi-Fi Access Points: Lessons Learned | |
| CN116800382B (en) | Index modulation method and device based on visual sensor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |