WO2017156492A1

WO2017156492A1 - Methods, apparatus, servers, and systems for vital signs detection and monitoring

Info

Publication number: WO2017156492A1
Application number: PCT/US2017/021963
Authority: WO
Inventors: Chen Chen; Beibei Wang; Yi Han; Feng Zhang; Hung-Quoc Duc LAI; Yan Chen; K.J. Ray LIU
Original assignee: Origin Wireless, Inc.
Priority date: 2016-03-11
Filing date: 2017-03-10
Publication date: 2017-09-14
Also published as: CN108778106A; CN108778106B; EP3426137A4; JP2019512304A; JP6900390B2; EP3426137A1

Abstract

The present teaching relates to vital sign detection and monitoring based on channel state information (CSI). In one example, an apparatus for vital sign detection is disclosed. The apparatus comprises a receiver, a processor and a memory communicatively coupled with the processor. The receiver is configured for receiving at least one wireless signal from a multipath channel that can be impacted by a vital sign of at least one living being. The processor is configured for: extracting a time series of channel state information (CSI) for the multipath channel from the at least one wireless signal, obtaining one or more periodic fluctuations based on a spectral analysis of the time series of CSI, and determining whether the vital sign is present based on the one or more periodic fluctuations.

Description

METHODS, APPARATUS, SERVERS, AND SYSTEMS FOR VITAL SIGNS

DETECTION AND MONITORING

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] The present application claims priority to U.S. Patent Application No. 62/307,081 , filed March 1 1 , 2016, entitled "TR-BREATH: TIME-REVERSAL BREATHING RATE ESTIMATION AND DETECTION," and U.S. Patent Application No. 62/316,850, filed April 1 , 2016, entitled "TR-BREATH: TIME-REVERSAL BREATHING RATE ESTIMATION AND DETECTION," which are incorporated herein by reference in their entireties.

TECHNICAL FIELD

[0002] The present teaching generally relates to vital sign detection and monitoring. More specifically, the present teaching relates to vital sign detection and monitoring based on channel state information of wireless channels.

BACKGROUND

[0003] Vital signs are important indicators of a person's health and wellbeing as well as predictors of acute medical conditions and chronic disease states for a person. Breathing rate is one of the most important vital signs, which can be measured by the number of exhalation and inhalation a person takes per minute. In addition, the breathing pattern may highly correlate to psychological conditions of a human being, such as stress and anxiety.

[0004] Most traditional approaches for breathing monitoring are invasive in that they need physical contact of the human bodies. For instance, in hospitals, the patients are required to wear oxygen masks, Nasal cannulas, chest straps, or wearable sensors such as thermistors and pressure sensors. Another example is Polysomnography (PSG) used in sleep medicine, which typically requires a minimum of 22 wire attachments to the patient. These dedicated devices are often costly and bulky, create discomfort to the human bodies, and are limited only to clinical settings.

[0005] Currently existing non-invasive (contact- free) breathing monitoring solutions can be categorized as below. [0006] (1) Radar-based breathing monitoring: Doppler radars are often used in breathing monitoring. They are operated by transmitting a signal and receiving a signal with a Doppler shift due to a periodic motion of objects. The breathing rates can be extracted from the Doppler shift. As a drawback, these systems use high transmission power, rely on sophisticated and expensive hardware, and use extremely large bandwidths. A vital sign monitoring system was disclosed utilizing frequency modulated continuous radar (FMCW). It used Universal Software Radio Peripheral (USRP) as the RF front-end to transmit a frequency-sweeping signal. But the additional cost and complexity of the dedicated hardware limited a large-scale deployment of FMCW radar.

[0007] (2) Wireless-sensor based breathing monitoring: The received signal strength (RSS) measurements from 802.15.4 compliant sensors on multiple 802.15.4 channels were also used for breathing detection and breathing rate estimation. Dense deployment of wireless sensors is required in these methods as additional wireless infrastructures. In addition, the specific design of frequency-hopping mechanism is required to support multiple channel measurements.

[0008] (3) Wi-Fi-based breathing monitoring: RSS is commonly used in the Wi-Fi-based breathing monitoring due to its availability on most commercial Wi-Fi network interface controllers ( ICs). Measurements were also used with Wi-Fi devices for breathing estimation. Yet, this method is accurate only when the users hold the Wi-Fi-enabled devices in close proximity to their chests.

[0009] In addition to the drawbacks mentioned above, methods (1) and (2) require design and manufacturing of special devices such as specialized radar devices or sensor network nodes; while method (3) has very low accuracy and sensitivity.

[0010] Therefore, there is a need for methods and apparatus for vital sign detection and monitoring to solve the above-mentioned problems and to avoid the above-mentioned drawbacks.

SUMMARY

[0011] The present teaching generally relates to vital sign detection and monitoring. More specifically, the present teaching relates to vital sign detection and monitoring based on channel state information of wireless channels. [0012] In one example, an apparatus for vital sign detection is disclosed. The apparatus comprises a receiver, a processor and a memory communicatively coupled with the processor. The receiver is configured for receiving at least one wireless signal from a multipath channel that can be impacted by a vital sign of at least one living being. The processor is configured for: extracting a time series of channel state information (CSI) for the multipath channel from the at least one wireless signal, obtaining one or more periodic fluctuations based on a spectral analysis of the time series of CSI, and determining whether the vital sign is present based on the one or more periodic fluctuations.

[0013] In another example, an apparatus for vital sign detection is disclosed. The apparatus comprises a plurality of receivers, a processor and a memory communicatively coupled with the processor. Each of the plurality of receivers is configured for receiving at least one wireless signal from a multipath channel that can be impacted by a vital sign of at least one living being. The processor is configured for: extracting, regarding each of the plurality of receivers, a time series of CSI for the multipath channel from the at least one wireless signal, obtaining, from each of the plurality of receivers, one or more periodic fluctuations based on a spectral analysis of the time series of CSI, determining, with respect to each of the plurality of receivers, one or more vital sign candidates based on the one or more periodic fluctuations, combining the vital sign candidates obtained from the plurality of receivers to generate a set of vital sign candidates, and determining whether the vital sign is present based on the one or more periodic fluctuations.

[0014] In yet another example, an apparatus for breathing rate estimation is disclosed. The apparatus comprises a plurality of receivers, a processor and a memory communicatively coupled with the processor. Each of the plurality of receivers is configured for receiving at least one wireless signal from a multipath channel that can be impacted by breathing of at least one living being. The processor is configured for: extracting, regarding each of the plurality of receivers, a time series of CSI for the multipath channel from the at least one wireless signal, obtaining, from each of the plurality of receivers, one or more periodic fluctuations based on a spectral analysis of the time series of CSI, determining, with respect to each of the plurality of receivers, one or more breathing rate candidates based on the one or more periodic fluctuations, combining the breathing rate candidates obtained from the plurality of receivers to generate a set of breathing rate candidates, partitioning the breathing rate candidates into a plurality of clusters each with an associated likelihood that indicates a possibility for that cluster to represent a breathing rate, merging two clusters in the plurality of clusters into one cluster when a centroid distance between the two clusters is less than a threshold, recalculating an estimated breathing rate and its associated likelihood for each merged cluster, and determining one or more breathing rates of one or more living beings based on one or more clusters having the highest associated likelihoods among the clusters after merging.

[0015] In one embodiment, at least one CSI is missing in the time series of CSI. In another embodiment, the processor is further configured for projecting at least one CSI in the time series of CSI into two or more distinct path delays of the multipath channel. In yet another embodiment, the processor is further configured for extracting at least one of high-order harmonics of the one or more periodic fluctuations based on the spectral analysis of the time series of CSI, wherein the one or more vital sign candidates are determined based on the at least one of the high-order harmonics.

[0016] In still another example, a method for vital sign detection is disclosed. The method may be implemented on a machine including at least a receiver, a processor and a memory communicatively coupled with the processor for vital sign detection. The method may comprise: receiving at least one wireless signal from a multipath channel that can be impacted by a vital sign of at least one living being; extracting a time series of CSI for the multipath channel from the at least one wireless signal; obtaining one or more periodic fluctuations based on a spectral analysis of the time series of CSI; and determining presence or absence of the vital sign based on the one or more periodic fluctuations.

[0017] Other examples in the present teaching include systems, methods, medium, devices, servers, and other implementations directed to vital sign detection and monitoring based on channel state information. For example, a system or server may include the above mentioned apparatus, or have a similar structure as the above mentioned apparatus for vital sign detection and monitoring based on channel state information.

[0018] Other concepts relate to software for implementing the present teaching on exploring computation, storage, application, and processing of vital sign detection and monitoring based on channel state information. A software product, in accord with this concept, includes at least one machine-readable non-transitory medium and information carried by the medium. The information carried by the medium may be executable program code data, parameters in association with the executable program code, and/or information related to a user, a request, content, or information related to a social group, etc.

[0019] Additional novel features will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and the accompanying drawings or may be learned by production or operation of the examples. The novel features of the present teachings may be realized and attained by practice or use of various aspects of the methodologies, instrumentalities and combinations set forth in the detailed examples discussed below.

BRIEF DESCRIPTION OF DRAWINGS

[0020] FIG. 1 shows an exemplary application of a disclosed breathing monitoring system, according to an embodiment of the present teaching;

[0021] FIG. 2 shows a flow chart for an exemplary process of the breathing monitoring system, according to an embodiment of the present teaching;

[0022] FIG. 3 shows an exemplary diagram for the breathing monitoring system, according to an embodiment of the present teaching;

[0023] FIG. 4 shows an exemplary process for breathing detection and monitoring using a Multiple Signal Classification (MUSIC) algorithm with machine learning, according to an embodiment of the present teaching;

[0024] FIG. 5 shows an exemplary process for breathing detection and monitoring using a MUSIC algorithm without machine learning, according to an embodiment of the present teaching;

[0025] FIG. 6 shows an exemplary process for breathing detection and monitoring using a Root-MUSIC algorithm with machine learning, according to an embodiment of the present teaching;

[0026] FIG. 7 illustrates a process of likelihood assignment and cluster merging, according to an embodiment of the present teaching;

[0027] FIG. 8A shows an exemplary diagram of general implementation of a breathing monitoring system, according to an embodiment of the present teaching; [0028] FIG. 8B shows an exemplary diagram of general implementation of another breathing rate monitoring system, according to an embodiment of the present teaching.

[0029] FIG. 9 shows an exemplary process for temporal smoothing, according to an embodiment of the present teaching;

[0030] FIG. 10 shows an exemplary experimental setting for multi -person breathing monitoring under a line-of-sight scenario, according to an embodiment of the present teaching;

[0031] FIG. 11 shows an exemplary experimental setting for multi-person breathing monitoring under a non-line-of-sight scenario, according to an embodiment of the present teaching;

[0032] FIG. 12 shows results of an exemplary experiment for breathing detection using machine learning algorithm, according to an embodiment of the present teaching;

[0033] FIG. 13 shows results of an exemplary experiment for breathing monitoring under an exemplary experimental setting in a line-of-sight environment, according to an embodiment of the present teaching;

[0034] FIG. 14 shows accuracy performance of an exemplary experiment for breathing monitoring under an exemplary experimental setting in a non-line-of-sight environment, according to an embodiment of the present teaching;

[0035] FIG. 15 shows performance of an exemplary experiment about people counting based on breathing, according to an embodiment of the present teaching;

[0036] FIG. 16 shows accuracy performance of an exemplary experiment in the presence of packet loss with and without compensation, according to an embodiment of the present teaching; and

[0037] FIG. 17 shows cumulative density functions of accuracy performance of an exemplary experiment using different schemes, according to an embodiment of the present teaching.

DETAILED DESCRIPTION

[0038] In the following detailed description, numerous specific details are set forth by way of examples in order to provide a thorough understanding of the relevant teachings. However, it should be apparent to those skilled in the art that the present teachings may be practiced without such details. In other instances, well known methods, procedures, components, and/or circuitry have been described at a relatively high-level, without detail, in order to avoid unnecessarily obscuring aspects of the present teachings.

[0039] The present teaching discloses a breathing rate monitoring system that can be used to a) measure breathing rates of a person, who is possibly moving, b) measure breathing strength of people, and c) estimate the number of people in a particular enclosure.

[0040] The disclosed breathing rate monitoring system is based on off-the-shelf Wi-Fi NICs, which are easily and cheaply available. Therefore, it can be deployed, managed and maintained easily and can be obtained at low cost. The disclosed system may work in both line-of-sight (LOS) and non-line-of-sight ( LOS) propagation conditions, and possibly for through-the-wall detection as well. A LOS environment means there is a direct LOS path between the device (transmitter or receiver) and a potential subject (human being or animal) to be detected or whose breathing rate is to be tested. In contrast, in a NLOS environment, there are some blockages, e.g. walls, between the device and the test subject, such that no light can directly pass through the straight path between the device and the potential subject.

[0041] The disclosed method implemented on the disclosed system in the present teaching does not require a physical proximity of the users of the system and the wireless transceivers, e.g. Wi-Fi devices. Moreover, the disclosed method can offer a contact-free solution, thus allowing for in-home and long-term health monitoring.

[0042] In one embodiment, the disclosed breathing rate monitoring system may comprise a receiver, a processor and a memory communicatively coupled with the processor. The receiver is configured for receiving at least one wireless signal from a multipath channel that can be impacted by a vital sign of at least one living being. The processor is configured for: extracting a time series of CSI for the multipath channel from the at least one wireless signal, obtaining one or more periodic fluctuations based on a spectral analysis of the time series of CSI, and determining presence or absence of the vital sign based on the one or more periodic fluctuations. [0043] In one embodiment, the vital sign is represented by at least one of: breathing and heartbeat; and the at least one living being includes at least one of: a human being and an animal. In another embodiment, the vital sign is breathing and the processor is further configured for determining one or more breathing rates of one or more living beings, upon determining presence of the vital sign. The processor may be further configured for: cleaning each of the time series of CSI by subtracting background information from the CSI; normalizing each of the time series of CSI; and calculating a degree of similarity based on each pair of CSIs among the time series of CSI, wherein the spectral analysis is performed based on the calculated degrees of similarity. The degree of similarity may be calculated based on at least one of: a time-reversal resonance strength (TRRS), a cross-correlation, an inner product of two vectors, a similarity score, a distance score, a phase correction, a timing correction, a timing

compensation, and a phase offset compensation, of a pair of CSIs.

[0044] In one embodiment, the processor is further configured for: calculating a TRRS matrix based on the time series of CSI; smoothing the TRRS matrix in time domain to generate a smoothed TRRS matrix, wherein the spectral analysis is performed based on the smoothed TRRS matrix. The processor may also be configured for: performing an eigenvalue

decomposition on the TRRS matrix; obtaining a noise subspace based on the eigenvalue decomposition; generating a pseudo-spectrum based on the noise subspace; performing spectrum folding on the pseudo-spectrum to generated an updated pseudo-spectrum; trimming the updated pseudo-spectrum into a range related to breathing rate; and detecting one or more peaks of the updated pseudo-spectrum within the range, wherein the one or more breathing rates of one or more living beings are determined based on the detected one or more peaks. The processor may further be configured for: performing an eigenvalue decomposition on the TRRS matrix; obtaining a noise subspace based on the eigenvalue decomposition; obtaining a matrix based on the noise subspace; generating a polynomial equation based on the obtained matrix; and selecting one or more complex roots of the polynomial equation, wherein the one or more breathing rates of one or more living beings are determined based on information related to at least one of angle and phase of the one or more complex roots.

[0045] In one example, the spectral analysis may be performed based on at least one of: a MUSIC based algorithm and a Root-MUSIC based algorithm. In another example, at least one of the one or more living beings is moving. In yet another example, at least one of the one or more living beings is not within a line-of-sight of the receiver, such that no light can directly pass through a straight path between the at least one living being and the receiver.

[0046] In one embodiment, determining the one or more breathing rates of one or more living beings may comprise: determining breathing rate estimations based on the spectral analysis of the time series of CSI; partitioning the breathing rate estimations into a plurality of clusters each with an associated likelihood which indicates a possibility for that cluster to represent a breathing rate; merging two clusters in the plurality of clusters into one cluster when a centroid distance between the two clusters is less than a threshold; recalculating the breathing rate estimation and its associated likelihood for each merged cluster; and determining the one or more breathing rates based on one or more clusters having the highest associated likelihoods among the clusters.

[0047] The processor may be further configured for estimating a quantity of the one or more living beings based on the number of clusters after merging, or monitoring the continuous one or more breathing rates of the one or more living beings as they perform at least one of doing exercises, working, sleeping, eating, drinking, standing, sitting, walking, running, and any other activities.

[0048] In one embodiment, the at least one wireless signal is received through a network that is at least one of: Internet, an Internet-protocol network, and another multiple access network; and the receiver is associated with a physical layer of at least one of: a wireless PAN, IEEE 802.15.1 (Bluetooth), a wireless LAN, IEEE 802.1 1 (Wi-Fi), a wireless MAN, IEEE 802.16 (WiMax), WiBro, HiperMAN, mobile WAN, GSM, GPRS, EDGE, HSCSD, iDEN, D-AMPS, IS-95, PDC, CSD, PHS, WiDEN, CDMA2000, UMTS, 3GSM, CDMA, TDMA, FDMA, W- CDMA, HSDPA, W-CDMA, FOMA, IxEV-DO, IS-856, TD-SCDMA, GAN, UMA, HSUPA, LTE, 2.5G, 3G, 3.5G, 3.9G, 4G, 5G, 6G, 7G and beyond, another wireless system and another mobile system.

[0049] FIG. 1 illustrates an application for the disclosed breathing monitoring system, where one or more patients lying in bed while their breathings are being monitored. The patients' chest movements can impact the wireless signals transmitted from a Bot, which may be a transceiver, to an Origin, which may be another transceiver. The disclosed system may capture the signals via the Origin and extract feature about the breathing patterns from the captured signals. The breathing monitoring system can calculate the patient's breathing rate, number of people in the room, and may trigger an alarm if the breathing rate drops/changes suddenly.

[0050] Applications of the disclosed breathing monitoring system may include monitoring the breathing rates of newborns, detecting abnormal breathing patterns such as apnea for the elderly, and studying the sleeping quality of inhabitants of a house during nights. Moreover, the disclosed breathing monitoring system can be applied in life detection, for instance, searching and rescuing survivors in the debris of collapsed buildings at sites suffered by earthquake. In such scenarios, breathing is considered as the most crucial type of motion that can be detected due to its presence even when a person is unconscious.

[0051] The present teaching can exploit the channel state information (CSI), a fine-grained portrait of the environment that can capture the variations caused by inhalations and exhalations of breathing. The disclosed system may project CSIs onto a time-reversal (TR) resonating strength feature space, which is analyzed via spectral analyzer (e.g. based on the MUSIC/Root- MUSIC based algorithms) for breathing detection and breathing rate estimation.

[0052] FIG. 2 shows a flow chart showing a general process of breathing monitoring. At 202, a transceiver/transmitter/Bot can transmit radio signals (e.g., a pulse or a pseudo random sequence) via a multipath channel. Then at 204, a receiver/another transceiver/Origin can receive the signals from the multipath channel that are impacted by the breathing of a person. The CSIs are extracted at 206 from the received radio signals using channel estimation. To improve the breathing monitoring, the system can also process the CSIs to obtain processed CSIs, e.g., formulate CSI fingerprints using the CSI fingerprint formulating methods disclosed in patent applications PCT/US2017/015909 and PCT/US2016/066015, which are incorporated herein by reference in their entireties. The phase distortion of the CSIs can be cleaned using phase offset compensation methods disclosed in patent applications PCT/US2017/015909 and PCT/US2016/066015.

[0053] Interference/background in the CSIs can be removed at 210 by background subtraction, and the CSIs can be normalized to a unit power before and after the background subtraction, at 208 and 212 respectively. Then a TRRS (time-reversal resonating strength) matrix is calculated at 214 based on the processed CSIs. The TRRS matrix can also be smoothed (e.g., averaged over time) to improve the rank condition of the matrix. After performing spectral analysis at 216 on the TRRS matrix, a set of breathing rate candidates are generated at 218, based on which the existence of breathing human can be detected at 220. For example, by comparing a peak in the spectrum of MUSIC-based spectral analysis with a threshold, or by comparing a distribution of roots after the Root-MUSIC -based spectral analysis with a threshold, the existence of breathing can be detected at 220. If breathing is detected at 221, the breathing rates can be estimated at 222. Otherwise, the process ends at 224.

[0054] According to one embodiment, some extensive experiment results demonstrated that the disclosed system can work well in around a minute of CSI measurements. More

specifically, it can achieve a mean accuracy of 98.65% in breathing rate estimation for a dozen people under the line-of-sight (LOS) scenario and a mean accuracy of 97.3% in breathing rate estimation up to 7 people under the NLOS scenario, both with 63 seconds of measurement. Moreover, the disclosed system can estimate the number of people with an error around 1. The disclosed system is robust against packet loss.

[0055] The disclosed system can employ a number of wireless transmitters and receivers working together to implement the breath monitoring functionality. One possible embodiment of the disclosed system is to use off-the-shelf Wi-Fi wireless transceivers. But the disclosed system is not limited to such devices. It can be used with any wireless devices that can provide channel state information (CSI) as a part of its operation (e.g. Bluetooth, 3GPP LTE

transceivers, or any custom-designed non-standard-compliant wireless transceiver). The present teaching discloses methods, apparatus, devices, systems, and servers that take this CSI as input, extract features relevant to breathing (or other vital signs having a periodic pattern) that describe the wireless channel behavior between the transmitter(s) and the receivers(s), and generate outputs that quantify the breathing rates of people being in the vicinity of the wireless transceivers.

Channel Model [0056] In an exemplary Orthogonal Frequency Division Multiplexing (OFDM) system, such as the current IEEE 802.1 lg, 802.1 In, 802.1 lad, or 802.1 l ac or the 3 GPP LTE wireless standard, the information-bearing signal is sent over the available frequency band over a number of parallel subcarriers. Each subcarrier can be treated as an independent narrow-band channel, and the total transmitted signal is the superposition of all the signals on all subcarriers. The CSI may summarize the behavior of the wireless channel (transmission medium) in the frequency domain by describing how the amplitude and phase of the transmitted signal is affected over each subcarrier. When the breathing is not present, the time series of CSIs on subcarrier k, k€ K where K is the set of usable subcarriers with K elements, can be written as H_k(t) expressed as below:

■ ^dt

H_fc(t) =∑^L _e=1 a_te^~]27l~ _k + e_fe(t), (1) where L is the total number of multipath components (MPC), a₍ is the complex gain of path , is the delay of path £, and A_k is the wavelength of the k-t subcarrier given by A_k— — .

Here, f_c is the carrier frequency, c is the speed of the light which is 3 X 10⁸m/s, T_s is the sampling interval which is 25 nanoseconds under 40 MHz Wi-Fi bandwidth and 50

nanoseconds under 20 MHz bandwidth, N is the size of FFT which is 128 for 40 MHz and 64 for 20 MHz, and e_fc(t) is the channel noise on subcarrier k at time t. Since the channel is assumed to be static as no breathing is present, the gains and delays of the MPCs are static. Due to the presence of breathing, part of the channel gains and path delays may change periodically with time t. For simplicity, one can assume that breathing affects path 1. The channel gain and path delay for path 1 can then be written as a_x (t) = a x (l + ^ sin0sin(27r/^"t + φ))⁷ (2) where a_x is the channel gain without breathing, d is the path length without impact of breathing, Δί^ is the additional positional displacement of path 1 with breathing, Θ is the incident angle between the subject and the impinging electromagnetic wave, / is the breathing frequency, φ is the initial phase of breathing,. One can see that, with the approximation of d- » Adi, a_x(t) « -i, and the variation in the path gain a_x is negligible. On the other hand, breathing affects the phase of path 1 as well. The path length d₁(t) can be expressed as d₁(t)— d_t + Ad₁sin0sin(2^t + φ)

Now, H_fc(t) takes the form

H_fe(t) = α_{ι β} ¾ +∑ ₌₂ α,β ¾ + e_k(t) which can be further written as

H_k(t) = _aie ^J ¾e ¾ +∑^₂ a,e ¾ + e_fe(t) (5)

Notice that the first term on the right hand side (RHS) of H_k (t) as shown in (5) can be

decomposed into an infinite summation according to the Jacobi-Anger expansion, as

∑^= - oo (-l J_miy eW^fteⁱ™*, (6) where v_k— 2nsin6Ad₁/A_k and J_m ) is the m-th order Bessel function with argument x. It can be seen that in addition to the spectral line at /, there also exist an infinite number of harmonics with spectral lines at mf with m being an integer. However, in practice, J_m( _k) may decay quickly for \m\≥ 2 given the typical values of v_k. Therefore, equation (6) can be written as

which includes two spectral lines located at +/ as well as a DC component. Therefore, H_k(t) can be expressed as

H_k(t)

(8)

Skit) _k with v_k— 2nsm6Ad₁/A_k. [0057] Notice that the dynamic model of ¾(t) shown in equation (8) can be changed accordingly under different scenarios, such as multiple people breathing, one person affecting a number of multipath components at the same time, or different people affecting the same multipath component at the same time. In practice, one may consider the following two random disturbances. First random disturbance is the random phase distortion caused by the differences between the local oscillators of the Wi-Fi transmitter and receiver. Second random disturbance is the random amplitude variation due to the automatic gain control (AGC), which tunes the input voltage to fully utilize the dynamic range of the analog-to-digital converter (ADC).

[0058] With these two effects, H_k(t) can be modified as

H_k{t) = r_t(S_fc(t) + /_fe)e^⁺/W + e_fe(t) (9) where r_t is the real AGC gain at time t; a_t is the initial phase distortion at time t; /?_t is the linear phase distortion at time t.

Background Subtraction

[0059] One can view the three parts of H (t) in equation (8) as (i) signal, (ii) interference, and (iii) noise, respectively. It is important to mitigate the impact of the interference I_k to improve the decodability of S_k(t). For convenience, I_k is called the background of subcarrier k. Besides directly using H_k(t), the system can also formulate CSI fingerprint using methods disclosed in patent applications PCT/US2017/015909 and PCT/US2016/066015. Also, one may sanitize the CSI phase of H t) to compensate phase distortions due to imperfect hardware using methods disclosed in patent applications PCT/US2017/015909 and PCT/US2016/066015. For example, the phase distortion compenstation can be done by: CSI folding and phase de-linearization.

[0060] One can assume that the phase of H_k(t) has been sanitized as ff'_fcC = H_fe(t)e^{" (¾+¾fe)} = r_t(S_fc(t) + /_fc)_e (A«_t+A ?_tfc) ₊ e_fc (_t)_e - (2_t+¾^fe⁾ _{t ( 10}) where a_t is the estimated initial phase distortion at time t and /?_t is the estimated linear phase distortion at time t. The channel power at time t is the £₂ norm of H' (t), or, l | H'(t) | |₂ = (∑_fcea: |H'_fc(t)|²)^1/2 . (i i) can be written as

H'_k(t) \ ² = T_t ² \S_k(t) + I_k \ ² + | e_fe(t) | ² + [Cross Terms] (12)

Then, the summation of |H'_fc(t) l ² i^s

*t =∑ ^,'k=u₁ T_t ² \S_k(t) + 7_fe 1² + | e_fe(t) | ² + [Cross Terms]

,U_K

>k=u₁ l¾(t) + /_fe | ² + | e_fe(t) |² +∑ 'k=u₁ [Cross Terms]_fc (13)

Given a high SNR scenario, the system can normalize H'(t) such that E_t— 1, Vt, which is equivalent to adjust Γ² to the same Γ². Here, one can denote the normalized CSI as H"(t).

[0061] There are two ways to mitigate I^. The first is by Principle Component Analysis (PCA). PCA does not require normalization. First of all, one can assemble the K X 1 CSIs into a matrix H with N columns, where N is the number of CSI snapshots, i.e., CSIs received at N different time instances. As long as N > K, one could fully exploit the inherent structure of H using PCA. One can simply fit H into the following model in the least-square sense:

H = σΒΒ' (14)

Here, σ is a scaling factor and B stands for the unit norm CSI that representing the background. Equation (14) is simply the rank-1 fitting of H, which can be done easily by taking the first principle component of H, i.e., finding the largest eigenvalue and the corresponding singular vector using singular value decomposition (SVD).

[0062] The second method is to take average over H"(t) to derive the background CSI and subtract it from all CSIs. For example, one can simply take average over all N_t time instances of each subcarrier k of the normalized CSI H"(t), i.e.,

\l_tl H"_k(t) (15)

Using real experiment data, these two methods may lead to highly similar results. However, method 2 is much simpler than method 1 where SVD is involved, which could be slow for H with a large dimension. Therefore, one can simply use equation (15) to calculate the background over a time period for each subcarrier. The CSI at time t and subcarrier k with background removed can be written as

H_k(t) = H"_k(t) - B_k (16)

Given uniform sampling and that t_N— t₁— cT, where T is the interval of breathing and c≠ 0 is an integer, it can be proved that H_k (t) only contains the harmonics of / where / is the breathing frequency. In other words, under certain assumptions, the background subtraction could perfectly mitigate the interference term I_k while leaving the useful terms intact, i.e.,

= a₁e-^^7rdi/^Afc [/₁( _fc)(e^^'2^^te-^ - e^^e^)] + e_fc(t)

[0063] In practice, perfect background subtraction can hardly be achieved due the following reasons. First, the interference term I_k can be time- varying and should be written as I_k(t), due the presence of motions in the environment caused by the movement of objects and people.

Second, since the breathing frequency f is unknown, the period of breathing, T, is also unknown. If t_N— t-L≠ cT, then, equation (17) should be modified as

H_fc(t) = -2 (v_k)jsm(2nft + φ) + r_fe(t) + e_fe(t) (18) where r_k(t) is the remaining term due to the fact that t_N— t_x≠ cT. Breathing Rate Estimation

[0064] The breathing rates can be extracted by spectral analysis from the collected CSI information. Spectral analysis algorithms can be divided into two categories: non-parametric and parametric.

[0065] The non-parametric algorithms do not make any assumption on the covariance structure of the CSIs, leading to their inherent lower resolution in comparison with the

parametric algorithms. The non-parametric algorithms include but are not limited to the periodogram, Blackman-Tukey, and Welch algorithms. The spectral resolution of these algorithms is strictly limited by 1/T, where T is the total time of measurement in seconds.

[0066] On the other hand, the parametric algorithms make assumptions on the covariance structure of the CSIs. For instance, in the MUSIC and Root-MUSIC algorithms, one may assume the knowledge of the signal dimension. The breathing monitoring method disclosed in the present teaching can adopt MUSIC and Root-MUSIC as the spectral analysis algorithms due to their higher resolution in comparison to the non-parametric algorithms, especially in a dynamic environment. It can be understood that other spectral analysis algorithms can also be adopted to estimate the breathing rates by the disclosed system.

[0067] Since human breathing may not be strictly stationary, the breathing monitoring system can operate on overlapping time windows. On link d from a transmit antenna to a receive antenna, for each time window of W seconds, the breathing monitoring system can calculate the resonating strength matrix R using the CSIs after phase calibration and background subtraction. The calculations may be repeated for each time window shifted by P seconds. Given M CSI measurements with packet rate f_s, the breathing monitoring system can formulate a total of

V_max— floor(^M^ ^W) time windows. Steps for breathing monitoring using CSI samples inside each time window are described in detail as below.

[0068] After background subtraction, one can calculate the TRRS matrix R with its (t— j^')-th entry defined as the TRRS between the t-th sampled CSI and the j^'-th sampled CSI, expressed as below, assuming a CSI sampling rate of T_samp, i.e., obtaining one CSI every 1/T_samp seconds, γγπ\χι ⁱ)ⁱ _samp), n^ ⁱ)ⁱ _samp)\ | |H((i-i)r_samp) | |₂ | |H₍a-i)r_samp)| |₂ ^> where K is the index of usable subcarriers. Notice that e^~^^ⁱⁿⁱ⁺^^link^ is introduced to

compensate the effect of linear phase distortions, where φ\_ίη is calculated as

Φΐί_η = argmax∑_fce H_fc((t - l)T_samp)H_k(( - l T_samp)e-^^k , (20) while φΙ_ηί is given as Φΐ_ηί = (∑_kex H_fc((i - l)T_samp)H_k{{] - l T_samp e-i*lin*) . (21)

The denominator of (19) normalizes the TRRS such that |y[H((i— V)T_samp), H((y—

l)T_sarrip)] I€ [0,1]; I |x| \₂ is the £₂ norm of the vector x. Assuming a total of N CSI snapshots, the system can calculate the N X N TRRS matrix R. Then, the system can perform spectral analysis, e.g., by using MUSIC and/or Root-MUSIC algorithm to extract breathing rates.

[0069] FIG. 3 shows an exemplary diagram for the breathing monitoring system, according to an embodiment of the present teaching. As shown in FIG. 3, to capture CSIs, the system can levarege Wi-Fi NICs to support the channel sounding mechanism between the access point (AP) and station (STA), or between a Bot and a Origin. Both AP and STA may be equipped with one or more antennas resulting in a total of D transmitting-receiving links. CSIs captured on each transmitting-receiving link are calibrated to mitigate the impact of residual synchronization errors. The calibrated CSIs are further processed by the breathing feature extraction module which generates two sets: K_d and S_d. S_d is a set containing the breathing rate estimations on link d obtained by solving a polynomial that will be detailed later in the present teaching, while K_d is a set containing a number of empty sets indicating the occurrences that the polynomial is insolvable. The sets K_d and S_d with d— 1, · ·· , D are combined into the set K and S. The disclosed breathing monitoring system can learn from the statistics of S and J\f to train a logistic regression binary classifier for breathing detection.

[0070] If breathing is detected, the sytem can perform clustering on S, which partitions the breathing rate estimations into different clusters. Then, it can assign likelihood to each cluster, indicating the possibility of that cluster in representing the breathing rate. The centroid of each cluster is interpreted as the estimated breathing rate of one person. Because closely separated breathing rate estimations might actually correspond to the same person, the system can merge clusters when their centroid differences fall below a threshold and recalculate the breathing rate estimations as well as their likelihoods. Then the system may output the breathing rate estimates with the highest likelihoods.

Eigenvalue Decomposition and Obtaining Noise Subspace [0071] Eigenvalue Decomposition: The first step to the MUSIC and/or Root-MUSIC

algorithm is the eigenvalue decomposition (EVD), which leads to

R - UAU^† (22) where U is a N X N orthonormal matrix, Λ is a N X N diagonal matrix with diagonal entries the eigenvalues of R, and† is the Hermitian operator.

[0072] Obtaining Noise Subspace: For a signal dimension of p, the system can partition U into a signal subspace and a noise subspace. The first p columns of U constructs the signal subspace U_s, while the last N— p columns of U constructs the noise subspace U_n.

MUSIC -based Breathing Estimation with Machine Learning

[0073] According to one embodiment of the present teaching, the disclosed system may perform breathing estimation using the MUSIC algorithm with machine learning. The MUSIC algorithm may estimate the frequency based on the distance from the signal subspace to the noise subspace. More specifically, it can evaluate the pseudo-spectrum P_mu(f) with /€ T where 7 is the possible region of / and is in general discrete considering that the system cannot achieve infinite resolution in real-life. P_mu ( ) is expressed as

where is the steering vector parameterized by /, given as s_f = [l e^j27IfTs e^j47IfTs ··· _ei(N-i)2nfT_sY (24) where T is the transpose operator. Notice that / in equation (23) is limited in the range of

[0, /_SOMnd/2] where f_SOund is ^me sounding rate, i.e., the rate at which CSI samples are acquired which is in general in the range of 5~200 Hz. Thus, f_SOund/2 is the critical Nyquist rate.

[0074] Spectrum Folding: according to equation (8), the pseudo-spectrum is symmetric around f = 0. Thus, to improve the performance of MUSIC, one can fold the spectrum around f = O as P'mu n = iPmuif) + Pmu(-n ■ (25)

Clearly, one can observe from equation (25) that P'_mu(0)— P_mu (0).

[0075] Spectrum Sifting: one may realize that the range of human breathing rates is limited in a certain range, say, f_min to f_max, where f_max≥ f_min≥ 0 and f_max≤ f_S0Und/2. Therefore, one can trim P'_mil(f) into the range of f_min to f_max. For convenience, it is written as ^ human—

[fmin' fmax] -

[0076] Peak Detection: By localizing the peaks of P_mu(f) with f€ T^_uman, one can estimate f as f. Based on some certain criterion, e.g., persistence, minimum peak distance, minimum peak height, peak prominence, peak width, and/or a mixture of these criterion, one can choose K peaks from P_mu(f), denoted as {fi, i2,^■·· , } which are ranked according to the chosen criterion.

[0077] Partitioning Breathing Rate Estimation: For Wi-Fi systems with multiple transmitting and receiving antennas (known as the Multiple-Input-Multiple-Output (MIMO) system) and/or Wi-Fi systems capable of frequency hopping and working on multiple channels, with a total of D antenna links / Wi-Fi channels, one could formulate several sets of breathing rate estimations, say, f = [?_! f₂ f₃ · ·· ¾]_> where is the vector standing for the breathing rate estimations extracted from the pseudo-spectrum of the i-th link/Wi-Fi channel, i.e., P_mu,i (f), f £ ^human-

[0078] Then, the system can perform generic clustering method to partition f into multiple clusters, e.g., affinity propagation and k-means clustering, with each cluster contains several breathing rate estimations in f close to each other and stands for the breathing rate estimations associated with one specific person.

[0079] For example, clustering with affinity propagation is detailed as below. In case that breathing is detected, the estimations in S, as shown in FIG. 3, are fed into the affinity

propagation clustering algorithm, which takes a similarity matrix as input. Affinity propagation works by passing the responsibility message to decide which estimations are exemplars, and the availability message to determine the membership of an estimation to one of the clusters. The similarity between estimated breathing rates f; and f_j can be given as exp(— |fj— f_j |). [0080] When there are more than one person and their breathing rates are different, then, the system can observe multiple clusters as well. For each cluster, the system can calculate the following statistics: a) the normalized population ( P) of each cluster, indicating the

concentration of elements of each cluster, b) the normalized variance (NV) of each cluster, indicating the tightness of each cluster, and c) the cluster centroid (CC) of each cluster. Clusters with higher centroids are less likely than the lower ones considering the nature of human breathing.

[0081] By evaluating the aforementioned statistics for each cluster, one could assign

likelihoods, denoted as \_lr 1₂,^■·· , lu to all clusters with a total of U clusters, to determine the likelihood of each cluster in representing the breathing rate associated with one person. For the Root-MUSIC -based approaches which would be discussed later, this step can be slightly modified to harness the complex roots produced by the Root-MUSIC algorithm. Instead of performing clustering on f, the system can partition the complex roots z directly, where z stands for the set of all complex roots. Compared with clustering using the breathing rates f, using the complex roots could potentially enhance the clustering results since the information about the radius p is also fully utilized in addition to the breathing rates, which are simply the angles (4-) of the complex roots z.

[0082] One example of likelihood assignment for each cluster is discussed in detail below. When affinity propagation returns U clusters, the disclosed system may evaluate the population, variance, and centroid of each cluster, denoted as p;, v_; , and respectively. Then, p_; and v_; are normalized as p.— — and Vj = — . The likelihood for cluster i may be calculated as

with

(27)

∑ί_{= 1} ^ρ(ω_ρρ_Γω_νν_ί-ω _ί) where ω_ρ, ω_ν, and < are positive weighting factors to account for different scales of the corresponding terms. The denominator of ^(P_j, V{, /;) is a normalization factor.

[0083] As can be seen from the expression of L(p_j, V_j, fj) in equation (26), the likelihood assignment incorporates a term related to the estimated breathing rate fj. One insight is that high breathing rate should be less likely than low breathing rate, and high breathing rate might correspond to the harmonics of actual breathing rates. As can be seen from the expression of 1; above in equation (27), clusters with V; = 0 and p; = 1 are singletons that must be removed since they are caused by interference or noise. Thus, one can assign likelihoods of 0 to singletons.

[0084] Cluster Merging: In practice, due to the existence of noise, it is possible that two adjacent clusters are actually associcated with the breathing of the same person. Thus, the present teaching introduces a parameter denoted as Γ and known as the merging radius: as long as the centroid distance between two different clusters falls below Γ, one can merge these clusters into one single cluster. When the NPs of these two clusters are p_x and p₂, NVs as v₁ and v₂ , centroids as c and c₂ , the likelihoods as l_x and 1₂ , then, the merged cluster has a NP of P'i— Pi + P₂- Oⁿ the other hand, the variance of the merged cluster is re-evaluated based on the elements in the merged cluster. The cluster centroid is calculated as c — +

l₂c₂)/0i + 1₂)- The likelihood 1Ί is re-assigned based upon the updated NP, NV, and CC.

[0085] An example of clustering merging is shown in detail below. For instance, when cluster i and i + 1 are to be merged since \ — f_i+11 < γ, one can denote the new cluster as i', whose population is calculated as p_;, = p_; + p_i+ The variance of cluster i' is re-evaluated based on the breathing rate estimations in cluster i'. The breathing rate estimation of cluster i' is updated as:

Then, the disclosed system can evaluate the likelihood of cluster i' by ^(P_j,, ^;,, ;,) in equation (27). Merging of more than two clusters can be generalized from the aforementioned steps, which is omitted here for brevity. When the number of clusters is K after merging and K is known, the disclosed system may directly output K₀ = min(K, K) breathing estimations with the highest likelihoods.

[0086] The procedures for likelihood assignment and cluster merging are illustrated in FIG. 7. As shown in FIG. 7, the breathing rate candidates may be processed at 702 using a clustering algorithm to generate 5 different clusters. Then, the disclosed system can assign likelihood at 704 to each of these clusters. The clusters with only one element, i.e., singletons, may be removed at this step. After 704, a total of 4 clusters remained in this example. Given that the centroid distance between cluster 1 and cluster 2 is smaller than the merging radius, the system can merge these two clusters at 706 into a single larger cluster. Finally, the disclosed system may use the centroids of merged clusters as the breathing rate estimations.

[0087] Breathing Rate Estimation: After performing the above calculations, the breathing rates can be estimated as below. When the total number of clusters after the step of cluster merging is U', and the likelihoods for these clusters are given as \ , \'₂, · ·· , l'u,. then, one can produce the breathing rate estimations under two different scenarios: (a) the number of people K is known, and (b) the number of people K is unknown.

[0088] For case (a), there may be two sub-cases: (a_x) U' < K, and (a₂) U' > K. For a_{l 5} one can detect U' out of K people. In other words, the miss detection rate is (K— U') /K. For a₂, the miss detection rate is 0 and one can use the centroids of the K clusters with the K highest likelihoods as the breathing rate estimations for K people.

[0089] For case (b), one may formulate an estimation on the people number K first, which would be elaborated later below. The rest steps may be identical to case (a).

[0090] FIG. 4 shows an exemplary process for breathing detection and monitoring using a MUSIC algorithm with machine learning, according to an embodiment of the present teaching. As shown in FIG. 4, Eigenvalue Decomposition (EVD) is performed at 402 on a TRRS matrix that is obtained based on CSIs, where the CSIs may be extracted from received wireless signals that may be impacted by breathing of a person. A TRRS, as strength of the TR resonance, is a measurement of a similarity between two CSI samples. TRRS can be used as a matching score between any two CSIs. The higher the TRRS is, the more similar two CSIs are. It can be understood that besides TRRS, a degree of similarity between two CSIs can also be represented by other measurements, e.g. a cross-correlation, an inner product of two vectors, a similarity score, a distance score, a phase correction, a timing correction, a timing compensation, and a phase offset compensation, of two CSIs.

[0091] Noise subspace is obtained at 404. Pseudo spectrum, e.g. P_mu( ), may be evaluated at 406. Spectrum folding around f=0 is performed at 408 to get updated pseudo spectrum P'_mil(f)- Spectrum sifting is performed at 410 and P'_mu( i^s trimmed into human breathing rate range

^human = [fmin_> fmax] - Peaks of P'_mu( ^are detected at 412 with f€ T_human, and K peaks are chosen and ranked as candidates. It can be understood that the steps 402-412, referred together as 420, may be repeated for each transmitter-receiver antenna link, e.g. for a multi-antenna system.

[0092] At 422, it is determined whether breathing is detected, e.g. based on breathing rate candidates determined at 412. If breathing is detected at 422, then a jumbo set of breathing rate candidates may be formulated at 424 for a multi-antenna system by repeating the steps 420, then the process moves on to 426. Otherwise, if breathing is not detected at 422, the process may end at 434, or in some embodiments go back to 402 to start over.

[0093] Breathing rate candidates are partitioned at 426 by clustering, and criteria are calculated for each cluster based on statistics. Clusters are merged at 428 when centroid distance between two clusters is less than a threshold. Statistics of the merged clusters are recalculated at 430. Breathing rates are estimated at 432 according to the statistics of the merged clusters. In can be understood that because the centroid of each cluster represents an estimation of the breathing rate associated with one person. The system can detect one or more breathing rates of one or more persons at the same time, based on the centroid of each cluster.

MUSIC -based Breathing Estimation without Machine Learning

[0094] FIG. 5 shows an exemplary process for breathing detection and monitoring using a MUSIC algorithm without machine learning, according to an embodiment of the present teaching. An algorithm using machine learning can make the system computationally mor expensive, especially for the application which needs local computations using low-end embedded devices, than an algorithm without need of machine learning. For the algorithm without need of machine learning, one may avoid the situation of multiple TRRS matrix R, i.e., due to the presence of multiple antenna links and/or multiple channels. For each link or channel t, one can calculate the TRRS matrix Rj. Then, one may take average of these TRRS matrix as

The rest steps an be identical to the steps of the breathing rate estimation algorithm with machine learning. Since there is only one pseudo-spectrum P_mu(f) calculated from R, no further procedures, like steps related to machine learning, are needed.

[0095] As shown in FIG. 5, EVD is performed at 502 on an averaged TRRS matrix over TRRS matrices of all antenna links/channels, where each TRRS matrix was obtained based on CSIs extracted from received wireless signals that may be impacted by breathing of one or more persons, through a wireless link/channel. Noise subspace is obtained at 504. Pseudo spectrum, e.g. P_mu(f), may be evaluated at 506. Spectrum folding around f=0 is performed at 508 to get updated pseudo spectrum P'_mu(f). Spectrum sifting is performed at 510 and P'_mu(f) is trimmed into human breathing rate range T human— [ min_> max]- Peaks of P'_mu(f) are detected at 512 with /€ ^human^ ^and K peaks are chosen and ranked as candidates at 512 as well. At 522, it is determined whether breathing is detected, e.g. based on breathing rate candidates determined at 512. If breathing is detected at 522, then breathing rates may be estimated at 524 according to the peaks detected at 512. The locations of the peaks in the MUSIC pseudo- spectrum may imply the estimated frequencies that are most likely to produce such a pseudo- spectrum. As such, these peak locations can be directly translated into frequencies indicating estimated breathing rates. Otherwise, if breathing is not detected at 522, the process may end at 526, or in some embodiments go back to 502 to start over.

Root-MUSIC -based Breathing Estimation with Machine Learning

[0096] FIG. 6 shows an exemplary process for breathing detection and monitoring using a Root-MUSIC algorithm with machine learning, according to an embodiment of the present teaching. In this case, one can first calculate the matrix Q = U_nU^. Then, one may formulate the polynomial given as

where [Q]_mjn is the (m, n)-th element of matrix Q, and z = e^~j2rtf^Ts. Since the polynomial in equation (30) is of order 2N— 1 , solving f(z)— 0 in equation (30) can lead to 2N— 1 complex roots. Discarding the trivial solution z— 0, there are 2N— 2 complex roots. Considering that Q is Hermitian, N— 1 out of the 2N— 2 complex roots are in the unit circle, while the rest N— 1 complex roots are outside the unit circle. Since the N— 1 complex roots inside the unit circle and the N— 1 complex roots outside the unit circle are symmetric around the unit circle and can be paired, one may just consider the N— 1 complex roots inside the unit circle. For a signal dimension of p, one can pick the p complex roots closest to the unit circle as long as p < N— 1, denoted as z₁, z₂, · ·· , z_p. The breathing frequency estimation can be formulated as

The following steps may be identical to the steps described previously for the breathing detection and monitoring using a MUSIC algorithm with machine learning.

[0097] Note that p can be an estimate on the maximum number of people. In case that one can solve f(z)— 0 in equation (30), one may put an empty set into N_d . Otherwise, one put the breathing rate estimations {f_\, fz,^■" , f_p} into S_d. Because the human breathing rates are limited to a certain range, say, _min to _max, one can discard estimations in S_d if they are not in the range of [ min' max] - After the breathing monitoring system processes all D links, it can calculate S as u ^ S_d and JV^* as u ^ J _d, where U denotes the union operator.

[0098] As shown in FIG. 6, EVD is performed at 602 on a TRRS matrix that is obtained based on CSIs, where the CSIs may be extracted from received wireless signals that may be impacted by breathings of one or more persons. Noise subspace is obtained at 604. A matrix is obtained at 606 based on noise subspace U_n as Q = υ_ηυ„. A polynomial equation may be solved at 608, e.g. the equation f(z)

with z = e^"i2ltfTs. For a signal dimension of p, p complex roots closest to the unit circle z_lt z₂, · ·· , z_p is picked at 610. At 612, the breathing rates are estimated based on the angle/phase of the roots as f; = -^- . It can be understood that the steps 602-612, referred together as 620, may be repeated for each transmitter-receiver antenna link, e.g. for a multi-antenna system.

[0099] At 622, it is determined whether breathing is detected, e.g. based on breathing rate candidates determined at 612. If breathing is detected at 622, then a jumbo set of breathing rate candidates may be formulated at 624 for a multi-antenna system by repeating the steps 620, then the process moves on to 626. Otherwise, if breathing is not detected at 622, the process may end at 634, or in some embodiments go back to 602 to start over.

[00100] Breathing rate candidates are partitioned at 626 by clustering, and criteria are calculated for each cluster based on statistics. Clusters are merged at 628 when centroid distance between two clusters is less than a threshold. Statistics of the merged clusters are recalculated at 630. Breathing rates are estimated at 632 according to the statistics of the merged clusters. For example, as shown in FIG. 7, clusters are assigned with likelihoods, which can be used as statistics leading to estimated breathing rates.

[00101] FIG. 8A shows an exemplary diagram of general implementation of a breathing rate monitoring system, according to an embodiment of the present teaching. As shown in FIG. 8A, users 802 may communicate with a breathing rate monitoring system 810. The breathing rate monitoring system 810 in this example includes a receiver 815, a CSI generator 822, a CSI fetcher 824, and a breathing estimator 826.

[00102] The CSI generator 822 in this example may receive a request from the users 802. In one embodiment, as shown in FIG. 8 A, the request is for estimation of breathing rates of one or more persons. In another embodiment, the request may be for determination of presence of breathing, or any vital sign of a person. In yet another embodiment, the request may be for determination of a quantity of people in a particular area.

[00103] The CSI generator 822 may generate CSI based on the request. In one embodiment, the CSI generator 822 may obtain wireless signals received by the receiver 815 via the multipath channel that may be impacted by breathing of one or more persons. Based on the received wireless signals, the CSI generator 822 may generate CSIs that represent information of a profile of the multipath channel which may be embedded with breathing information of the one or more persons. The CSI generator 822 may send the generated CSIs to the CSI fetcher 824. In one embodiment, the breathing rate monitoring system 810 may also include one or more transmitters configured to transmit wireless signals via the multipath channel.

[00104] The CSI fetcher 824 in this example may be configured for receiving wireless signals transmitted via a multipath environment on multiple antennas and extracting the channel profiles based on the received signals. The breathing estimator 826 in this example may be configured for resolving multiple breathing rates given the extracted channel profiles, providing the breathing rate estimations to the users in response to the request.

[00105] FIG. 8B shows an exemplary diagram of general implementation of another breathing rate monitoring system 81 1 , according to an embodiment of the present teaching. As shown in FIG. 8B, the receiver 815 may be replaced by a plurality of receivers 816 to make use of technology of multiple antennas. CSIs from multiple antennas, e.g. from the plurality of receivers, can be combined or fused using various methods such as those disclosed in

PCT/US2017/015909 and PCT/US2016/066015.

[00106] In one embodiment, as shown in FIG. 8A and FIG. 8B, the CSI generator 822, the CSI fetcher 824, and the breathing estimator 826 may all be implemented on a processor 820. In addition, each of the breathing rate monitoring systems 810, 81 1 may include a memory communicatively coupled with the processor 820.

Breathing Detection

[00107] The disclosed system, with various algorithms disclosed above, can also be used for breathing detection, that is, to detect if there are breathing humans located in the vicinity of the wireless transmission and reception devices.

[00108] In one embodiment, the process for breathing detection using a MUSIC -based method may comprise the following steps. When there is no breathing, the disclosed MUSIC -based methods would not produce any estimation in general due to insignificant peaks in terms of the chosen criterion in the pseudo-spectrum. On the other hand, when there is breathing, the disclosed MUSIC -based schemes would generate estimations satisfying the chosen criterion.

[00109] In another embodiment, the process for breathing detection using a Root-MUSIC- based method may comprise the following steps. When there is no breathing, the disclosed Root-MUSIC -based method would not produce many roots, i.e., the polynomial cannot be solved under most circumstances, and when the polynomial is solvable under this case, the number of roots would be small in general. On the contrary, when there is breathing, the polynomial can be solved to lead to many estimated values.

[00110] For the MUSIC-based schemes, one may set a threshold on the chosen criterion to detect the breathing. For the Root-MUSIC -based schemes, the system can calculate two metrics a and β, where a stands for a likelihood that the polynomial cannot be solved, while β stands for a richness of the roots calculated as the ratio between the number of obtained roots to the total number of possible roots.

[00111] An example of breathing detection is discussed as below. Extensive experiments showed that statistics of S and JV can be indicators of the presence or absence of breathing. In the absence of breathing, it is highly possible that the polynomial of Root-MUSIC in equation (30) does not yield any breathing rate estimation in the range [f_mi_n, f_max] > which leads to a large JV^* and a small S in terms of cardinalities, i.e., the number of elements in the two sets. In the presence of breathing, solving equation (30) could produce many breathing rate estimations and lead to a small JV and a large S.

[00112] The breathing monitoring system leverages such observations for breathing detection. First of all, the system formulates the a and β statistics given by a = _EI- _{f ?} = J£L _f

\S\ + \M\ pVD where the denominator of β is the total number of possible breathing rate estimations for V time windows, D links, p estimations per link per time window, and | · | denotes the cardinality of a set, a indicates the insolvability of equation (30), while β indicates the diversity of solutions in solving equation (30).

[00113] The correlation between θ = (α, β) and the existence of breathing is utilized to train the logistic regression binary classifier, if the ground-truth label y associated with Θ is also available, with y = 1 representing the presence of breathing and 0 otherwise. Since the disclosed breathing monitoring system is designed to be an autonomous system, it may have no access to the ground-truth label.

[00114] To learn the label y automatically from Θ, the system can develop an unsupervised learning scheme. For example, the system may run for T times and produce Q_lt θ₂,^■·· , θ_τ.

Then, the labels associated with Θ can be determined by the following two steps: (i) perform k- means clustering on θ_1; θ₂,^■·· , θ_χ with k = 2; and (ii) select, out of the 2 clusters generated by k -means, the one with the larger averaged a value and label all members within that cluster with 1. Meanwhile, label all members within the other cluster with 0.

[00115] The labels {Υί)ί=ι,2_,-_,τ ^{can men} be fed into a logistic regression model together with {θί}ί=ι,2_,···_,τ f°^r learning the parameters 0⁾ _c, ω_α, α⁾ of the sigmoid function f(a, β) =

1

using gradient descent. After the training is done, given any unseen

1+θχρ(-ω_ε-ω_αα-ω_β β) ^{6 6}

Θ = (α, β), the breathing monitoring system can decide that breathing exists if f(a, β) > κ and that breathing does not exist otherwise, where κ is a predetermined threshold.

People Counting Based on Breathing

[00116] Based on the breathing detection and breathing rate monitoring, the system can also count the number of people detected. For the MUSIC -based scheme, the system can count the number of significant peaks in the pseudo-spectrum based on some certain criterion. Then, the number of significant peaks may indicate the number of people having detected breathing. For the Root-MUSIC-based scheme, the system can count the number of clusters with high likelihoods compared to a certain threshold on the likelihood.

[00117] For example, in case that the number of people is unknown, the disclosed system can be used to estimate how many individuals are in the vicinity of the wireless devices. The disclosed system may use the likelihood of each cluster after merging clusters to formulate an estimation of K given by

K— arg miti_j ^∑™^^K'^ l ≥ λ (32) where Z; is the likelihood of cluster i after merging, and λ is the threshold on the cumulative likelihood. Then, the system can output K₀— K breathing estimations with the highest likelihoods. Here, K₀ stands for an estimated number of people who breathed in the coverage area of the disclosed system.

Estimating Breathing Strength and Geometric Extraction of Breathing

[00118] In addition to the breathing rate estimations, the strength of breathing can also be an important indicator of medical conditions. For example, fast and shallow breathing can be an indicator of Tachypnea, an abnormal breathing caused by fever, fear, and other factors.

Meanwhile, it is useful to know the source location of the breathing signal, e.g., geometric extraction to infer the position of the people under monitoring. This could be helpful in a scenario of two people breathing with identical breathing rates, since they may not be located at the same position and thus can be potentially distinguished from each other by different locations.

[00119] Assume that perfect background subtraction can be achieved as shown in (17) and further assume that the noise can be ignored. Then, the amplitude and phase of H_k (t) can be written as

\H_k(t) \ = 2 \_ai \ \ (v_fc)sin(27T/t + φ) \ (33) and the phase is given as

4H_fc(t) = - f + 4a₁ - ^ⁱ (34)

[00120] Both the amplitude and phase are functions of the subcarrier index k. One can observe that: the amplitude |H_fc(t) | changes periodically with the breathing frequency / across time t; the amplitude \H_k(t) | almost changes periodically with the subcarrier index k given a fixed time t; the phase .H_fc(t) contains crucial information about the path delay d₁. Here, the periodic means the zero-crossing points show a periodic pattern. This is due to the fact that the zero-corssing points of the Bessel function A( ) are nearly periodic with the argument x. Estimating the Effective Breathing Strength

[00121] Based on the periodicity of \H_k(t) \ , the system can estimate the effective breathing strength , given as

Ad₁ - AdsinS. (35)

This may include estimating the period of \H_k(t) | for each fixed t over the subcarriers k, say, T, and estimating Ad₁ (t). First, the zero-crossing period of \H_k(t) | may be estimated for each fixed t over the subcarriers k, denoted as T. For instance, if \H_k(t) \ crosses zero for every 5 subcarriers, then the system cand determine that T— 5. The Ad₁(t) may be estimated as

. , , cNT,

(36) where c is the speed of light, N is the size of FFT which is 128 for 40 MHz and 64 for 20 MHz, T_s is the sampling interval which is 25ns for 40 MHz and 50ns for 20 MHz. The same operation can be repeated for all time instances t.

Estimating the Geometric Information about Breathing

[00122] From -H_k(t), the system can estimate d_x which is related to the position of the person under monitoring. For instance, if the system estimates d_x as 3 meters, then, the person under monitoring may be significantly affecting a multipath component with length of 3 meters. In other words, the person is very close to the wireless transmission pairs of the disclosed system.

[00123] To estimate d_{l 5} one can realize that .H (t) is linear in terms of the subcarrier index k. In reality, the phase noise always exists, and .H_fc(t) equals to Sk + ζ + e(k) where e(k) is the phase noise on subcarrier k. In this case, least-square estimation can be used to estimate the slope of .H_fc(t) for a given t over all subcarriers.

[00124] This can be seen by rewriting .H_k(t) as H_fe(t) = - f + ^ Sk + ζ (37) c where S = is the slope of -H_k(t), and ζ is the interception. Then, one can write -H_k(t) cNT_s

for a fixed t over all subcarriers into a K X 1 column vector h where K is the number of usable subcarriers, to obtain h = A[S ζ]^τ + e (38) where A = []¾ k₂ ... k_K; 1 1 ... 1] is a K X 2 matrix with kj representing i-th subcarrier index, and e is a K X 1 column vector representing the phase error. Then, [S ζ] can be estimated as

[S ζ]^τ = (A^TA ¹A^Th (39)

When the slope is estimated as S, d₁ can be obtained as

The same operation can be repeated for all time instances t to formulate a time series of the estimated path delays affected by the breathing the most.

[00125] The estimation of effective breathing strength and the extraction of geometric information about breathing can also be extended to multi-person breathing scenario.

Dealing with Missing Packets

[00126] During practical utilization of radio signals from commercial chips, e.g., from a Wi-Fi system, packet missing may be inevitable due to the following reasons. First, there may be large RF interference from nearby wireless systems, e.g., another Wi-Fi network running on the same center frequency. Second, the access point capturing CSIs may also be in charge of providing internet access to associated users. When users are occupying a large bandwidth continuously, the packet loss rate would increase significantly, e.g., when the users are watching high- definition online movies. Thrid, there are multiple Wi-Fi stations associated with the access point capturing CSIs, which formulates an uplink scheme. The Wi-Fi stations would compete with each other to send sounding packets to the same access point, which can incur packet loss. [00127] Due to the packet loss, the received CSI timestamps at the access point side cannot be considered as uniform, which happens to be a fundamental assumption in most spectrum analyzers, i.e., MUSIC and Root-MUSIC. It can be expected that when the packet loss is significant enough, the MUSIC and Root-MUSIC algorithm would deteriorate greatly.

[00128] To solve this issue, the system can consider the following problem. After the system receives M CSIs, N of them are missing, where T'—

t'₂, · ·· , t'_N} denotes the set containing the sequence number of the missing CSI packets, and T— t_lr t₂,^■·· , t_N+M denotes the full time index if there is no packet loss at all. Then, the remaining time index under which the M CSIs are received is T— T/T', where / stands for the set difference operator.

[00129] In practice, T is known at the Wi-Fi receiver. Then, one could reconstruct the set T as {minT, min + 1, min + 2, · ·· , max — 1, max }, where min is the minimum value of the sequence number set T while max is the maximum value of the sequence number set T.

Therefore, the missing index set T' can be reconstructed as T/T.

[00130] Then, given N CSIs captured with sequence numbers indicated by T, the system can calculate the N X N TRRS matrix R using the background-subtracted CSIs H, where each

Ύ

column of M representing a snapshot of a CSI vector H(t) = [H_x (t) H₂ ■·· H_K(t)] .

More specifically, the (t j)— th entry of R is given as y[H(t_j), H(t )], where /[·,·] is the TRRS operator. Then, for subspace-based analysis such as MUSIC and/or Root-MUSIC, one can perform EVD on R which leads to

R = UAU^†, (41) where U is an N X N orthomormal matrix, Λ is an N X N diagonal matrix with diagonal entries representing all of the eigenvalues of R, and† stands for the conjugate-transpose operator. In reality, due to the existence of channel noise, R is full rank, and thus U and Λ is also full rank. Therefore, the columns of U span the entire space, with some of them being in the signal subspace and the rest being in the noise subspace. When the signal subspace is p, and the columns of U are sorted based on the descending order of the eigenvalues on the diagonal of Λ, the first p columns ofU span the signal subspace and the rest N— p columns ofU span the noise subspace.

[00131] Then, the system can incorporate the knowledge of T' before proceeding based on the following procedure: first, construct the extended (N + ) X (N + ) all-zero eigenmatrix U'; then let U'f. f .— Uj_j, where _t represents the i-th element of T. Now, one can obtain the interpolated signal and noise eigenspace U'_s and U'_n respectively by taking the first p and last N + M - p columns ofU'. Following similar steps disclosed in MUSIC/Root-MUSIC with machine learning, the system can perform breathing monitoring when there are missing packets.

Smoothing of the TRRS matrix R

[00132] The calculation of R may be based on the N snapshots of CSIs. To improve the accuracy of the MUSIC and Root-MUSIC algorithms, the system can perform temporal smoothing on the TRRS matrix R, which leads to the extended TRRS matrix R. The detail of the algorithm is discussed below.

[00133] Given the knowledge of the sequence number T, the system can construct the extended (N + ) x (N + ) all-zero matrix R. Let Rf. f .— R; , where _t represents the t-th element of T. Meanwhile, the system can construct a counting matrix C based on the extended matrix R. The entry of C is given as below:

, Otherwise (42)

Given a smoothing parameter N'≠ (N + ), i.e., number of snapshots for each submatrix, one can construct a total of N + M - N' submatrix denoted by R_j, i = 1,2,^■■· , N + M— N'. The i- th N' X N' submatrix Rj contains the entries of R with row index in the range of [t— 1, i + N'— 1] and column index in the range of [t— 1, i + N'— 1] as well. The system may construct the submatrix for the counting matrix C in a similar way, written as C;. Accumulating Rj and C; over t, one can get the smoothed N' X N' matrix R = Ν+Μ-Νΐ _{a§ wgll a§ thg N}> _{χ N}> smoothed counting matrix C = ^ ^{1 N}' Cj. To further improve the result, the system can use a forward-backward smoothing, leading to the improved N' X N' matrix j given by

E_ = i (E + JE⁺J), (43) and the improved N' X N' counting matrix as = l (C + JCj) , (44) where J is a N' X N' permutation matrix with unity entries on the anti-diagonal. [00134] For instance, a 2 X 2 permutation matrix is:

For matrix and C_, one can delete the rows and columns in and where at least one entry of that row or column is zero, leading to a N" X N" matrix R and C where N" < N'. The system can then obtain the temporal smoothed N" X N" matrix R_sm with entries given by

®

As long as R_sm is non-empty, it can be used for the analysis using MUSIC and/or Root-MUSIC. On the other hand, if R_sm is empty, then it may indicate that smoothing cannot be performed successfully. This can be due to the missing of a majority of Wi-Fi packets, say, 60%.

[00135] FIG. 9 shows an exemplary process for temporal smoothing, according to an embodiment of the present teaching, where a total of 4 CSIs with one missing CSI is assumed. The TRRS matrix R is formulated first, then, zeros are inserted into R to construct the extended TRRS matrix R. From R, the disclosed system further formulates submatrix, denoted as R_x , R₂, C-L and C₂, respectively. The submatrix are accumulated to produce R and C, which are further processed by backward-forward smoothing which yields R^ and C\ Finally, R_sm is calculated based on R' and C. [00136] The steps disclosed above for smoothing of the TRRS matrix can further improve the performance of breathing monitoring if there are missing packets, when combined with the steps disclosed above for dealing with missing packets.

Breathing Estimation with Projection (Geometric Embedding)

[00137] In some situations, one may want to investigate breathing only for a certain multipath component with a specific path length/delay. For instance, knowing two persons in two positions with very distinct path delays, by projecting the CSI according to the two distinct path delays, the system can separate the two persons' breathing monitoring, even if their breathing rates may be close.

[00138] To implement this feature, the system can perform spatial filtering on the snapshots of CSIs. Given a background subtracted CSI snapshot H(t), the system can project H(t) into H_d (t) with path delay d expressed as

where s_d = [l _e -j2nd/ _{k e} -j nd/ _k . .. _e -j^'2⁽ f-i⁾re^d Afe] ₅ i_>e>5 the steering vector with a path length d. Then, the system can calculate the TRRS matrix R and its smoothed version R_sm based upon the projected CSI time series H_d (t) and the smoothing technique introduced in the present teaching. After that, breathing rates can be estimated using MUSIC and/or Root- MUSIC analysis.

Impact of Motion on Breathing Monitoring

[00139] Unavoidably, the environment may be dynamic. In other words, besides the desirable component in the CSIs relevant to the vital signs, there may also exist other components in the CSIs that are irrelevant to the vital signs.

[00140] More specifically, the CSI model _k t) defined in equation (9) may be dynamic in the sense that the signal component S_fc(t) changes with time t. However, the interference term I_k is not a function of time t, representing a static environment except the dynamics imposed by the breathing. When there exists dynamics due to motions, equation (9) should be modified as H_k(t) = r_t(S_fc(t) +

+ _{efc (t) >} (48) where the interference caused by motion denoted by 7_fc(t) also becomes a function of time t. Even after background subtraction disclosed in the present teaching, one could hardly mitigate the DC component of the motion interference 7_fc(t) while leaving the other spectral components intact. The existence of motion interference would introduce both false alarms and miss detection.

[00141] For example, if some spectral components due to the motion interference 7_fc(t) is very strong, say, f,, then the disclosed system would generate some breathing rate estimations at which is in fact not caused by breathing but by some motions. This can increase false alarm.

[00142] For example, if some spectral components due to 7_fc(t) overlaps with the breathing frequency /, then it is possible that the interference would cancel the spectral component at /, which leads to miss detection.

[00143] To tackle this issue, the system can perform motion detection to detect which CSI samples are affected by large motions instead of minor motions such as breathing, e.g. based on methods disclosed in patent application PCT/US2016/066015. Then, one can treat these CSI samples as if they are simply missing, and use the method dealing with missing packets dosclosed earlier in the present teaching to estimate the breathing rates.

Distinguishing People with Close Breathing Rates

[00144] In reality, the breathing rates of different people could be very close to each other and thus makes the estimation very difficult. According to equation (7), the difference between two breathing rates may be more distinct for higher harmonics. For example, when there are two breathing rates f and f₂ with f₂— f±— Δ, the difference for the second-order harmonic is 2Δ, which is two times the difference for the first-order harmonic. In this situation, the system can use the harmonics to distinguish two breathing rates very close to each other.

[00145] The power of the harmonic may decay rapidly with the order m. Thus, the high-order harmonics, i.e. harmonics with orders larger than one, could be too weak to be used for breathing rate estimation, when using geometric information to extract breathing rate. However, if knowing the rough path delays of the different positions where the people being monitored are at, the system can project the collected CSIs to those path delays and utilize breathing estimation method with projection disclosed earlier to estimate the breathing rates.

Use Cases

Human Detection and Monitoring

[00146] The disclosed breathing monitoring method can be used to detect the presence of human, since it is capable of breathing detection. This is useful for smart home applications as well as life detection in inaccessible areas, e.g., mines or wreckage. Another example is when an elevator has some emergency and gets trapped between two floors, if devices with the disclosed breathing monitoring function are pre-installed inside/around the elevator, the breathing of the people inside the elevator can be closely monitored. Yet another example is when some vehicle is about to leave a jail, cross a country border, or move around a crime scene, and the policemen need to check if there is any suspect inside the vehicle, the disclosed breathing monotiring device can be put in a proper position on/around the vehicle to detect whether there is indeed any suspect inside the vehicle. Yet another example is there may be lack of enough monitoring whether the prisoners are live or in good health condition especially during the night. The disclosed breathing monitoring device may also assist a prison to monitor their prisoners.

Counting the Number of People

[00147] The disclosed breathing monitoring devices can estimate the number of people around the deployed devices. Therefore, it could be used for crowd control applications, e.g., counting the number of people on the bus, in a room, etc.

Healthcare and Wellness

[00148] The disclosed breathing monitoring devices can be used to detect the abnormality of breathing for people. By tracking the variations in the breathing rates, doctors could diagnose the patients with richer information on his/her breathing pattern. For example, after background subtraction, the background-subtracted CSI can be checked and by pattern matching the CSIs' waveform with breathing patterns from some database, it can be found whether the patients have problems in their respiratory systems. Another example is by deploying the disclosed devices in an elder care center, or a nursery, or a home, the elder people/babies' health condition can be closely monitored.

[00149] The disclosed breathing monitoring can also be used for monitoring other vital signs that exhibits a periodically changing pattern. For instance, when the breathing monitoring devices operate in high-center frequency, e.g., in 60 GHz, they can also monitor human heartbeat. Similarly, the disclosed devices can be used for vital signs monitoring of animals such as animals in a zoo.

Alternative Approaches for Breathing Monitoring

[00150] Breathing incurs positional displacement of human chests which can change the path delays reflected by the chests, causing the phases of subcarriers on different antenna links also changing in accordance with breathing. Therefore, the phase changes can be utilized to track human breathing.

[00151] The disclosed breathing monitoring system could select the representative subcarrier on a specific antenna link to demonstrate the time- varying phase changes caused by breathing. For instance, the criterion of subcarrier selection can be (i) the variance of a subcarrier in a period of time; or (ii) the energy of the dominant spectral component within a period of time.

[00152] After selecting the representative subcarrier, by studying the periodically changing pattern of the CSI on that subcarrier, the breathing rate of a person can be estimated. For instance, the amplitude of the CSI on the representative subcarrier may exhibit a sinusoid-like pattern. Given a certain time period TO, by counting the number of zero-crossing points of the CSI amplitude, the system can know how many breathing cycles exist in the time period TO and further estimate the breathing rate.

[00153] Other types of machine learning approaches can also be used for breathing estimation.

Experimental Results [00154] Extensive experiments have been conducted to evaluate the performance of the disclosed monitoring system. The experiments are conducted in two different rooms in an office suite with dimensions 8m x 7m and 8m x 5m, respectively. One pair of prototypes equipped with off-the-shelf WiFi cards was built with 3 omnidirectional antennas to obtain CSIs. Thus, the total number of links D is 9. One of the prototypes works as the ORIGIN, while the other works as the BOT. The center frequency is configured in the 5.8GHz Wi-Fi band with a bandwidth of 40 MHz. The transmit power is 20 dBm.

[00155] The performance was evaluated in both LOS and NLOS scenarios. For the LOS scenarios, the ORIGIN and the BOT are placed in the same room with the people, while for the NLOS scenarios, they are placed outside the room blocked by two walls. The locations of both Wi-Fi devices under both scenarios are marked in FIG. 10 and FIG. 1 1.

[00156] A total of 17 different participants were invited. During the experiments, slight movements, e.g., head or limb movements, were allowed. In FIG. 10, 12 people were seated in a conference room, while in FIG. 1 1 , up to 7 people were seated in another room.

[00157] The performance of the proposed monitoring system is evaluated by comparing the breathing rate estimations against the ground-truths. To obtain the ground-truths, each participant was asked to synchronize his/her breathing according to a metronome application on his/her cellphone. After the controlled breathing experiments, one may conduct experiments in a more practical setting where the participants are asked to breathe naturally according to their personal habits and count their own breathing rates manually.

[00158] To assess the accuracy of breathing rate estimations, one can use a metric called accuracy which is defined as

where K₀— min(/f, /f).

[00159] FIG. 12 demonstrates the breathing detection performance of the proposed system using SVM as the classifier. First of all, one can observe that the labels y can be inferred from θ = (α, β) without errors. Secondly, one can observe that SVM returns a hyperplane that partitions θ = (α, β) perfectly, implying a 100% detection rate. This is further validated by performing K-fold cross-validation on the results, leading to a 100% detection without false alarm for each cross-validation.

[00160] Then, one can evaluate the disclosed system for multi-person breathing monitoring in a LOS scenario. A total of 12 people are invited into the conference room as shown in FIG. 10 under the LOS scenario. The normalized population, variance, likelihood, and centroid for each cluster are presented in FIG. 13. It can be seen from FIG. 13 that the proposed system resolves the breathing rates of 9 out of a dozen people with an accuracy of 98.65%.

[00161] The disclosed system was further evaluated under multi-person NLOS scenario.

Different number of people (up to 7 people) were invited into one conference room with two devices placed under the NLOS scenario as shown in FIG. 1 1. The accuracy performances are shown in FIG. 14, indicating that an accuracy of 99.1% can be achieved when K=7 and a mean accuracy of 97.3% averaging over all 7 cases.

[00162] FIG. 15 demonstrates the performance of people counting of the disclosed system. The mean error of people counting was assessed as Ρ(λ) as a function of the threshold λ. One can observe that the optimal Ρ(λ) is 1.15 when λ = 0.88. Thus, the proposed system can estimate the number of people with an error around 1.

[00163] Then, the performance of the disclosed system was studied in the presence of packet loss due to ambient Wi-Fi traffic incurred by co-existing Wi-Fi systems. One can consider two different packet loss mechanisms, i.e., bursty packet loss and random packet loss. The bursty packet loss may be mainly caused by the continuous data transmission among few Wi-Fi devices which fully jams the medium for a long time. On the other hand, the random packet loss may be due to the random access of a large number of nearby WiFi devices which occupy the medium occasionally.

[00164] The results with different packet loss rate with the aforementioned two mechanisms are shown in FIG. 16. One can observe that the consequence of random packet loss is much more severe than the bursty packet loss when the packet loss compensation is not enabled. With 10% random packet loss, the accuracy drops to 88.35% from 99.35%. The accuracy further deteriorates to 74.13% and 62.83% with 20% and 30% packet loss, respectively. The advantage of packet loss compensation is obvious, as the disclosed system maintains an accuracy of 99.70% even with 30% packet loss. On the contrary, bursty packet loss does not degrade the accuracy greatly. It can be justified by the fact that most CSIs are still sampled uniformly under this scenario.

[00165] FIG. 17 shows cumulative density function performances of multi -person NLOS scenario with various schemes. As shown in FIG. 17, using both the amplitude and phase of CSIs outperform the case which uses CSI amplitude alone. Meanwhile, one can also conclude that using the Root-MUSIC scheme outperforms conventional non-parametric schemes, for example, the Welch's estimator.

[00166] To implement various modules, units, and their functionalities described in the present disclosure, computer hardware platforms may be used as the hardware platform(s) for one or more of the elements described herein (e.g., the components of the system described with respect to any of FIGs. 1- 17). The hardware elements, operating systems and programming languages of such computers are conventional in nature, and it is presumed that those skilled in the art are adequately familiar therewith to adapt those technologies to explore vital sign detection and monitoring based on channel state information as described herein. A computer with user interface elements may be used to implement a personal computer (PC) or other type of work station or terminal device, although a computer may also act as a server if appropriately programmed. It is believed that those skilled in the art are familiar with the structure, programming and general operation of such computer equipment and as a result the drawings should be self-explanatory.

[00167] The disclosed system can be realized by a specialized system having a functional block diagram illustration of a hardware platform which includes user interface elements. The computer may be a general purpose computer or a special purpose computer. Both can be used to implement a specialized system for the present teaching. This computer may be used to implement any component of the techniques of vital sign detection and monitoring based on channel state information, as described herein. For example, the system in FIG. 8 may be implemented on a computer, via its hardware, software program, firmware, or a combination thereof.

[00168] Hence, aspects of the methods of vital sign detection and monitoring based on channel state information, as outlined above, may be embodied in programming. Program aspects of the technology may be thought of as "products" or "articles of manufacture" typically in the form of executable code and/or associated data that is carried on or embodied in a type of machine readable medium. Tangible non-transitory "storage" type media include any or all of the memory or other storage for the computers, processors or the like, or associated modules thereof, such as various semiconductor memories, tape drives, disk drives and the like, which may provide storage at any time for the software programming.

[00169] All or portions of the software may at times be communicated through a network such as the Internet or various other telecommunication networks. Such communications, for example, may enable loading of the software from one computer or processor into another. Thus, another type of media that may bear the software elements includes optical, electrical and electromagnetic waves, such as used across physical interfaces between local devices, through wired and optical landline networks and over various air-links. The physical elements that carry such waves, such as wired or wireless links, optical links or the like, also may be considered as media bearing the software. As used herein, unless restricted to tangible "storage" media, terms such as computer or machine "readable medium" refer to any medium that participates in providing instructions to a processor for execution.

[00170] Hence, a machine-readable medium may take many forms, including but not limited to, a tangible storage medium, a carrier wave medium or physical transmission medium. Nonvolatile storage media include, for example, optical or magnetic disks, such as any of the storage devices in any computer(s) or the like, which may be used to implement the system or any of its components as shown in the drawings. Volatile storage media include dynamic memory, such as a main memory of such a computer platform. Tangible transmission media include coaxial cables; copper wire and fiber optics, including the wires that form a bus within a computer system. Carrier-wave transmission media may take the form of electric or electromagnetic signals, or acoustic or light waves such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media therefore include for example: a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD or DVD-ROM, any other optical medium, punch cards paper tape, any other physical storage medium with patterns of holes, a RAM, a PROM and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave transporting data or instructions, cables or links transporting such a carrier wave, or any other medium from which a computer may read programming code and/or data. Many of these forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to a physical processor for execution.

[00171] Those skilled in the art will recognize that the present teachings are amenable to a variety of modifications and/or enhancements. For example, although the implementation of various components described above may be embodied in a hardware device, it may also be implemented as a software only solution— e.g., an installation on an existing server. In addition, the vital sign detection and monitoring based on channel state information as disclosed herein may be implemented as a firmware, firmware/software combination, firmware/hardware combination, or a hardware/firmware/software combination.

[00172] While the foregoing has described what are considered to constitute the present teachings and/or other examples, it is understood that various modifications may be made thereto and that the subject matter disclosed herein may be implemented in various forms and examples, and that the teachings may be applied in numerous applications, only some of which have been described herein. It is intended by the following claims to claim any and all applications, modifications and variations that fall within the true scope of the present teachings.

Claims

WE CLAIM:

1. An apparatus (810) for vital sign detection, comprising:

a receiver (8 5) configured for receiving at least one wireless signal from a multipath channel that can be impacted by a vital sign of at least one living being; a processor (820); and

a memory communicatively coupled with the processor, wherein the processor is configured for:

extracting a time series of channel state information (CSI) for the multipath channel from the at least one wireless signal,

obtaining one or more periodic fluctuations based on a spectral

analysis of the time series of CSI, and

determining whether the vital sign is present based on the one or more periodic fluctuations,

2. The apparatus of claim 1 , wherein:

the vital sign is represented by at least one of breathing and heartbeat; and the at least one living being includes at least one of a human being and an animal.

3. The apparatus of claim 1 , wherein the vital sign is breathing and the processor is further configured for determining one or more breathing rates of one or more living beings, upon determining the vital sign is present.

4. The apparatus of claim 3, wherein the processor is further configured for:

cleaning each of the time series of CSI by subtracting background information from the CSI;

normalizing each of the time series of CSI; and

calculating a degree of similarity based on each pair of CSIs among the time series of CSI, wherein the spectral analysis is performed based on the calculated degrees of similarity.

5. The apparatus of claim 4, wherein the degree of similarity is calculated based on at least one of: a time-reversal resonance strength (TRRS), a cross-correlation, an inner product of two vectors, a similarity score, a distance score, a phase correction, a timing correction, a timing compensation, and a phase offset compensation, of a pair of CSIs.

6. The apparatus of claim 3, wherein the processor is further configured for:

calculating a TRRS matrix based on the time series of ( Si:

smoothing the TRRS matrix in time domain to generate a smoothed TRRS matrix, wherein the spectral analysis is performed based on the smoothed TRRS matrix,

7. The apparatus of claim 6, wherein the processor is further configured for:

performing an eigenvalue decomposition on the TRRS matrix;

obtaining a noise subspace based on the eigenvalue decomposition;

generating a pseudo-spectrum based on the noise subspace;

performing spectrum folding on the pseudo-spectrum to generated an updated pseudo-spectrum;

trimming the updated pseudo-spectrum into a range related to breathing rate; and

detecting one or more peaks of the updated pseudo-spectrum within the range, wherein the one or more breathing rates of the one or more living beings are determined based on the detected one or more peaks.

8. The apparatus of claim 6, wherein the processor is further configured for:

performing an eigenvalue decomposition on the TRRS matrix;

obtaining a noise subspace based on the eigenvalue decomposition;

obtaining a matrix based on the noise subspace;

generating a polynomial equation based on the obtained matrix; and

selecting one or more complex roots of the polynomial equation, wherein the one or more breathing rates of the one or more living beings are determ ined based on information related to at least one of angle and phase of the one or more complex roots.

9. The apparatus of claim 1, wherein the spectral analysis is performed based on at least one of a MUSIC based algorithm and a Root-MUSIC based algorithm.

10. The apparatus of claim 3, wherein at least one of the one or more living beings is

moving.

1 1. The apparatus of claim 3, wherein the at least one of the one or more living beings is not within a line-of-sight of the receiver, such that no light can directly pass through a straight path between the at least one living being and the receiver.

12. The apparatus of claim 3, wherein determining the one or more breathing rates of the one or more living beings comprises:

determining breathing rate estimations based on the spectral analysis of the time series of CSI;

partitioning the breathing rate estimations into a plurality of clusters each with an associated likelihood which indicates a possibility for that cluster to represent a breathing rate;

merging two clusters in the plurality of clusters into one cluster when a centroid distance between the two clusters is less than a threshold;

recalculating the breathing rate estimation and its associated likelihood for each merged cluster; and

determining the one or more breathing rates based on one or more clusters having the highest associated likelihoods among the clusters after merging,

13. The apparatus of claim 12, wherein the processor is further configured for estimating a quantity of th e one or more living beings based on the number of cl usters after merging.

14. The apparatus of claim 3, wherein the processor is further configured for monitoring the continuous one or more breathing rates of the one or more living beings as they perform at least one of doing exercises, working, sleeping, eating, drinking, standing, sitting, walking, running, and any other activities.

15. The apparatus of claim 1, wherein:

the at least one wireless signal is received through a network that is at least one of: Internet, an Internet-protocol network, and another multiple access network;

the receiver is associated with a physical layer of at least one of: a wireless PAN, IEEE 802.15.1 (Bluetooth), a wireless LAN, IEEE 802.11 (Wi-Fi), a wireless MAN, IEEE 802.16 (WiMax), WiBro, HiperMAN, mobile WAN, GSM, GPRS, EDGE, HSCSD, iDEN, D-AMPS, IS-95, PDC, CSD, PHS, WiDEN, CDMA2000, UMTS, 3GSM, CDMA, TDMA, FDMA, W-CDMA, HSDPA, W-CDMA, FOMA, lxEV-DO, IS-856, TD-SCDMA, GAN, UMA, HSUPA, LTE, 2.5G, 3G, 3.5G, 3.9G, 4G, 5G, 6G, 7G and beyond, another wireless system and another mobile system.

16. A method, implemented on a machine including at least a receiver, a processor and a memory communicatively coupled with the processor for vital sign detection, comprising:

receiving at least one wireless signal from a muitipath channel that can be impacted by a vital sign of at least one living being;

extracting a time series of CSI for the muitipath channel from the at least one wireless signal;

obtaining one or more periodic fluctuations based on a spectral analysis of the time series of CSI; and

determining whether the vital sign is present based on the one or more periodic fluctuations.

17. The method of claim 16, wherein:

the vital sign is represented by at least one of breathing and heartbeat; and the at least one living being includes at least one of: a human being and an animal,

18. The method of claim 16, wherein the vital sign is breathing and the method further comprises determining one or more breathing rates of one or more living beings, upon determining the vital sign is present.

19. The method of claim 18, further comprising:

normalizing each of the time series of CSI; and

20. The method of claim 19, wherein the degree of similarity is calculated based on at least one of: a TRRS, a cross-correlation, an inner product of two vectors, a similarity score, a distance score, a phase correction, a timing correction, a timing compensation, and a phase offset compensation, of a pair of CSIs.

21. The method of claim 18, further comprising:

calculating a TRRS matrix based on the time series of CSI;

smoothing the TRRS matrix in time domain to generate a smoothed TRRS matrix, wherein the spectral analysis is performed based on the smoothed TRRS matrix.

22. The method of claim 21 , further comprising:

performing an eigenvalue decomposition on the TRRS matrix;

obtaining a noise subspace based on the eigenvalue decomposition;

generating a pseudo-spectrum based on the noise subspace; performing spectrum folding on the pseudo-spectrum to generated an updated pseudo-spectrum;

trimming the updated pseudo-spectrum into a range related to breathing rate;

and

23. The method of claim 23 , further comprising:

performing an eigenvalue decomposition on the TRRS matrix;

obtaining a noise subspace based on the eigenvalue decomposition;

obtaining a matrix based on the noise subspace;

generating a polynomial equation based on the obtained matrix; and

selecting one or more complex roots of the polynomial equation, wherein the one or more breathing rates of the one or more living beings are determined based on information related to at least one of angle and phase of the one or more complex roots.

24. The method of claim 16, wherein the spectral analysis is performed based on at least one of a MUSIC based algorithm and a Root-MUSIC based algorithm.

25. The method of claim 18, wherein at least one of the one or more living beings is

moving.

26. The method of claim 18, wherein the at least one of the one or more living beings is not within a line-of-sight of the receiver, such that no light can directly pass through a straight path between the at least one living being and the receiver.

27. The method of claim 18, wherein determining the one or more breathing rates of the one or more living beings comprises: determining breathing rate estimations based on the spectral analysis of the time series of CSI;

28. The method of claim 27, further comprising estimating a quantity of the one or more living beings based on the number of clusters after merging.

29. The method of claim 18, further comprising monitoring the continuous one or more breathing rates of the one or more living beings as they perform at least one of doing exercises, working, sleeping, eating, drinking, standing, sitting, walking, running, and any other activities.

30. The method of claim 16, wherein;

31. An apparatus (811) for vital sign detection, comprising:

a plurality of receivers (816) each of which is configured for receiving at least one wireless signal from a multipath channel that can be impacted by a vital sign of at least one living being;

a processor (820); and

extracting, regarding each of the plurality of receivers, a time series of CSI for the multipath channel from the at least one wireless signal,

obtaining, from each of the plurality of receivers, one or more periodic fluctuations based on a spectral analysis of the time series of CSI,

determining, with respect to each of the plurality of receivers, one or more vital sign candidates based on the one or more periodic fluctuations,

combining the vital sign candidates obtained from the plurality of receivers to generate a set of vital sign candidates, and

determining whether the vital sign is present based on the set of vital sign candidates.

32. The apparatus of claim 31 , wherein at least one CSI is missing in the time series of CSI.

33. The apparatus of claim 31, wherein the processor is further configured for projecting at least one CSI in the time series of CSI into two or more distinct path delays of the multipath channel.

34. The apparatus of claim 31, wherein the processor is further configured for extracting at least one of high-order harmonics of the one or more periodic fluctuations based on the spectral analysis of the time series of CSI, wherein the one or more vital sign candidates are determined based on the at least one of the high-order harmonics.

An apparatus (811) for breathing rate estimation, comprising: a plurality of receivers (816) each of which is configured for receiving at least one wireless signal from a multipath channel that can be impacted by breathing of at least one living being;

a processor (820); and

determining, with respect to each of the plurality of receivers, one or more breathing rate candidates based on the one or more periodic

fluctuations,

combining the breathing rate candidates obtained from the plurality of receivers to generate a set of breathing rate candidates,

partitioning the breathing rate candidates into a plurality of clusters each with an associated likelihood that indicates a possibility for that

cluster to represent a breathing rate,

merging two clusters in the plurality of clusters into one cluster when a centroid distance between the two clusters is less than a threshold,

recalculating an estimated breathing rate and its associated likelihood for each merged cluster, and

determining one or more breathing rates of one or more living beings based on one or more clusters having the highest associated likelihoods among the clusters after merging.

36. The apparatus of claim 35, wherein the processor is further configured for estimating a antity of the one or more living beings based on the number of clusters after merging.