WO2021086809A1 - Methods and systems for remote sleep monitoring - Google Patents

Methods and systems for remote sleep monitoring Download PDF

Info

Publication number
WO2021086809A1
WO2021086809A1 PCT/US2020/057452 US2020057452W WO2021086809A1 WO 2021086809 A1 WO2021086809 A1 WO 2021086809A1 US 2020057452 W US2020057452 W US 2020057452W WO 2021086809 A1 WO2021086809 A1 WO 2021086809A1
Authority
WO
WIPO (PCT)
Prior art keywords
signal
respiratory
radar
cardiac
monitoring
Prior art date
Application number
PCT/US2020/057452
Other languages
French (fr)
Inventor
Yu Rong
Alex CHIRIYATH
Arindam Dutta
Daniel W. Bliss
Original Assignee
Arizona Board Of Regents On Behalf Of Arizona State University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Arizona Board Of Regents On Behalf Of Arizona State University filed Critical Arizona Board Of Regents On Behalf Of Arizona State University
Priority to US17/772,844 priority Critical patent/US11690563B2/en
Publication of WO2021086809A1 publication Critical patent/WO2021086809A1/en

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/02Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
    • G01S7/41Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
    • G01S7/415Identification of targets based on measurements of movement associated with the target
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/48Other medical applications
    • A61B5/4806Sleep evaluation
    • A61B5/4812Detecting sleep stages or cycles
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/02Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/0205Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/02Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/024Detecting, measuring or recording pulse rate or heart rate
    • A61B5/02444Details of sensor
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/05Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves 
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/05Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves 
    • A61B5/0507Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves  using microwaves or terahertz waves
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/113Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb occurring during breathing
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/48Other medical applications
    • A61B5/4806Sleep evaluation
    • A61B5/4809Sleep detection, i.e. determining whether a subject is asleep or not
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/48Other medical applications
    • A61B5/4806Sleep evaluation
    • A61B5/4818Sleep apnoea
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/68Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6887Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient mounted on external non-worn devices, e.g. non-medical devices
    • A61B5/6891Furniture
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235Details of waveform analysis
    • A61B5/7264Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/0209Systems with very large relative bandwidth, i.e. larger than 10 %, e.g. baseband, pulse, carrier-free, ultrawideband
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/50Systems of measurement based on relative movement of target
    • G01S13/52Discriminating between fixed and moving objects or between objects moving at different speeds
    • G01S13/522Discriminating between fixed and moving objects or between objects moving at different speeds using transmissions of interrupted pulse modulated waves
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/0002Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network

Definitions

  • the present disclosure relates to vital sign estimation and monitoring.
  • Sleep monitoring tracks the activity of the human body during sleep and provides information on brain activity and other physiological factors during sleep. Hence, sleep monitoring has been used to diagnose or understand the underlying causes of sleeping disorders such as sleep apnea, sleep related seizures, insomnia, etc., and also to qualitatively measure different aspects of sleep such as quality, activity, etc.
  • Non-contact sleep monitoring approaches also exist, such as camera-based and sensor-based approaches.
  • One example uses a low power radar-based biomotion sensor that detects movement to measure sleep/wake patterns during sleep and perform sleep/wake classification.
  • Methods and systems for remote sleep monitoring are provided. Such methods and systems provide non-contact sleep monitoring via remote sensing or radar sensors.
  • a breathing motion magnification effect is observed from mattress surface displacement due to human respiratory activity.
  • This undesirable motion artifact causes existing approaches for accurate heart-rate estimation to fail.
  • Embodiments of the present disclosure use a novel active motion suppression technique to deal with this problem by intelligently selecting a slow-time series from multiple ranges and examining a corresponding phase difference. This approach facilitates improved sleep monitoring, where one or more subjects can be remotely monitored during an evaluation period (which corresponds to an expected sleep cycle).
  • An exemplary embodiment provides a method for remote sleep monitoring.
  • the method includes transmitting a radar signal toward a subject and receiving a radio frequency (RF) response signal corresponding to the radar signal.
  • the method further includes monitoring vital signs of a subject, which includes processing the RF response signal to produce one or more vital sign signals and monitoring the vital sign signals over an evaluation period.
  • the method further includes classifying a sleep state of the subject based on results from monitoring the vital signs.
  • the sleep monitoring device includes a radar sensor configured to receive an RF response signal to a radar signal and a processing circuit coupled to the radar sensor.
  • the processing circuit is configured to monitor vital signs of a subject from the RF response signal by processing the RF response signal to produce a plurality of vital sign signals and classifying a sleep state of the subject based on monitoring the plurality of vital sign signals over an evaluation period.
  • Another exemplary embodiment provides a system for remote sleep monitoring.
  • the system includes a radar sensor configured to receive an RF response signal to a radar signal transmitted toward one or more subjects, a database, and a processing circuit coupled to the radar sensor and the database.
  • the processing circuit is configured to monitor vital signs of the one or more subjects over an evaluation period by processing the RF response signal to produce vital sign signals for each of the one or more subjects, storing the vital sign signals in the database, and classifying a sleep state of the subject based on the vital sign signals stored in the database.
  • Figure 1 A is a graphical representation of a vital sign micro-Doppler signature in slow time for a subject lying on a rigid surface.
  • Figure 1 B is a graphical representation of a vital sign micro-Doppler signature in slow time for a subject lying on a non-rigid surface.
  • Figure 2 is a schematic diagram of an exemplary system for remote sleep monitoring.
  • Figure 3 is an image of an embodiment of the system for remote sleep monitoring of Figure 2.
  • Figure 4 is an in-phase/quadrature (l/Q) data constellation plot from an example noisy channel due to excessive phase noise.
  • Figure 5 is an l/Q data constellation plot of an example range bin used to actively suppress breathing motion.
  • Figure 6 is a graphical representation of a temporal wave containing both respiration and heartbeat signals while a subject is piece-wise breathing.
  • Figure 7 is a graphical representation of a vital sign spectrum obtained with a conventional approach and a reference heartbeat signal spectrum.
  • Figure 8 is a graphical representation of phase variations from two range bins along with the phase difference of the two range bins.
  • Figure 9 is a graphical representation of a vital sign spectrum obtained with active motion suppression and a reference heartbeat signal spectrum.
  • Figure 10 is a flow diagram illustrating a process for remote sleep monitoring.
  • Figure 11 is a schematic diagram of a generalized representation of an exemplary computer system that could be used to perform any of the methods or functions described above, such as remote sleep monitoring.
  • Methods and systems for remote sleep monitoring are provided. Such methods and systems provide non-contact sleep monitoring via remote sensing or radar sensors.
  • a breathing motion magnification effect is observed from mattress surface displacement due to human respiratory activity.
  • This undesirable motion artifact causes existing approaches for accurate heart-rate estimation to fail.
  • Embodiments of the present disclosure use a novel active motion suppression technique to deal with this problem by intelligently selecting a slow-time series from multiple ranges and examining a corresponding phase difference. This approach facilitates improved sleep monitoring, where one or more subjects can be remotely monitored during an evaluation period (which corresponds to an expected sleep cycle).
  • Remote vital sign detection (e.g., for remote sleep monitoring) can be characterized with a signal model.
  • a synthesized transmitted pulse for remote vital sign detection can be modeled as a cosine wave with a Gaussian envelope:
  • R tx ( ⁇ ) P 0 ( ⁇ ) cos( 2 ⁇ F c t) Equation 1
  • r 0 (t) denotes the Gaussian pulse envelope and is designed to satisfy an emission mask.
  • F c denotes the nominal operating frequency.
  • a receive signal in response to the transmitted pulse can be modeled as:
  • P rx (t) A T p 0 (t - t D (t)) cos(2 ⁇ F c (t - t D (t))) Equation 2
  • a T denotes the target response
  • 3 ⁇ 4(t) denotes time-varying time-delay due to vital sign motion
  • t and t denote slow-time and fast-time scales, respectively.
  • Equation 4 A(t) denotes a time-varying amplitude.
  • the respiration signal in the radar return is much stronger than that of heartbeat signal for two factors: radar cross section and physical displacement. Even though a radar sensor is illuminating the entire body of a subject, only a small portion of this response contains skin motion or vasomotion due to cardiac activity.
  • the physical displacement due to normal breathing ranges from 1 mm (shallow) to 1 cm (deep), while the physical displacement near the skin surface as a result of heartbeat motion is on the order of 0.1 mm.
  • a non-rigid mattress tends to support and conform to the back of the human body very well. This results in the mattress moving with the human body during respiration, causing a motion magnification effect due to respiration.
  • Figures 1 A and 1 B demonstrate the motion magnification effect due to mattress surface displacement during respiration.
  • Figure 1A is a graphical representation of a vital sign micro-Doppler signature in slow time for a subject lying on a rigid surface.
  • Figure 1 B is a graphical representation of a vital sign micro-Doppler signature in slow time for a subject lying on a non-rigid surface.
  • a test subject was instructed to lie down on both a rigid firm surface (e.g., a wooden bed frame) and on a non-rigid surface (e.g., a soft mattress) and breathe normally. Radar response signals for both situations were recorded and processed for comparison. As illustrated in Figures 1 A and 1 B, the non-rigid surface resulted in a relatively large magnification of breathing motion artifacts.
  • Embodiments described herein provide a new active motion suppression technique to recover the heartbeat signal in the presence of magnified respiration interference.
  • This approach exploits the fact that the human body is an extended dynamic target, and thus vital signs can be observed in multiple range bins. Across these multiple range bins, breathing energy spreads across a larger number of range bins when compared to heartbeat or pulse energy. The total number of range bins of interest can be divided into two categories: breathing only, and a mixture of respiration and heartbeat signals.
  • a goal of the active motion suppression technique is to find two ‘best’ range bins such that their phase difference in the slow-time series generates a maximum pulse signal to noise ratio (SNR).
  • SNR pulse signal to noise ratio
  • FIG. 2 is a schematic diagram of an exemplary system 10 for remote sleep monitoring.
  • the system includes a radar sensor 12 positioned at a distance from a subject 14.
  • the radar sensor 12 is in communication with a processing device 16, such as a computer system.
  • the subject 14 lies on a rigid or non-rigid surface 18 (e.g., a mattress and/or a rigid frame) during a sleep evaluation (e.g., an evaluation period corresponding to an expected sleep cycle, such as four or more hours, six or more hours, or eight or more hours).
  • the radar sensor 12 is configured to receive (e.g., sense) a radio frequency (RF) response signal 20 which corresponds to a radar signal 22 emitted toward the subject 14.
  • RF radio frequency
  • the radar sensor 12 is a transceiver which transmits the radar signal 22 and receives the RF response signal 20.
  • the radar signal 22 is emitted from another device (e.g., a transmitter which may be in communication with the radar sensor 12).
  • the processing device 16 records and/or analyzes the RF response signal 20 to produce and/or monitor vital signs of the subject.
  • the processing device 16 is further used to classify a sleep state of the subject (e.g., awake, asleep, depth of sleep, etc.), and may also provide a diagnosis of sleep conditions (e.g., sleep apnea, insomnia, etc.).
  • FIG 3 is an image of an embodiment of the system 10 for remote sleep monitoring of Figure 2.
  • the subject 14 lies on a non- rigid surface 18 (e.g., a mattress) which is on an elevated platform.
  • the radar sensor 12 is suspended above the subject 14 via a ceiling mount or frame 24 and is pointed at the torso of the subject 14.
  • a test subject 14 was also fitted with a photoplethysmogram (PPG) sensor, which was used as a reference signal in a sleep monitor study.
  • PPG photoplethysmogram
  • embodiments may omit such contact sensors.
  • the radar sensor 12 in the embodiment of Figure 3 includes an impulse-based ultra-wide band (UWB) radar.
  • the UWB radar operates at 1 .4 gigahertz (GHz) in the X band with low power emission, satisfying FCC part 15.
  • GHz gigahertz
  • a coherent transceiver is achieved through precise timing control.
  • the theoretical range resolution is about 10 cm. In a higher SNR regime, the actual range resolution is much smaller and on the order of 1 cm.
  • the radar system parameters may be configured to operate at the highest possible pulse integration level while meeting the minimum range requirement for vital sign monitoring. Fine ranging information from the UWB radar provides multiple spatially independent observations at the same time. Thus, embodiments may effectively turn heartbeat detection into a multi-channel source separation problem.
  • the radar sensor 12 can include a different radar, such as a having a bandwidth between 1 .0 GHz and 2.0 GHz. It should also be understood that the radar sensor 12 may in other embodiments be positioned differently, such as wall-mounted or as a freestanding device placed on a surface and aimed toward the subject 14.
  • range bin selection for the active motion suppression technique is described.
  • the initial group of range bins is selected by range gating spatio-temporal data (e.g., such as represented by a spatio-temporal heat map).
  • range gating spatio-temporal data e.g., such as represented by a spatio-temporal heat map.
  • complex signals from each candidate bin e.g., spatial channel
  • curve fitting is to correct for direct current (DC) offset, Q, as shown in Equation 6:
  • Equation 6 where A 0 is a nominal signal amplitude.
  • Figure 4 is an in-phase/quadrature (l/Q) data constellation plot from an example noisy channel due to excessive phase noise. Clean phase samples should be mostly located closely to the unit circle in the l/Q constellation plot.
  • RMSE root mean square error
  • the upper limit is selected as thirty percent of the unit radius, which means that if the distance error from the l/Q samples is over 0.3, then this channel is too noisy and thus is ignored.
  • Figure 5 is an l/Q data constellation plot of an example range bin used to actively suppress breathing motion. It is observed that the above method works very well. However, some embodiments focus on adaptive motion suppression, in which range bins with phase arc ⁇ arc 0 and range bins with phase arc > arc 0 (as seen in Figure 4) are used to actively suppress breathing motion. Note that different range bins V will have different phase offsets as the static phase is linearly related to the distance.
  • phase-based motion cancellation to suppress the magnified respiration signal is to select one channel from the breathing only range bins and select another one from the range bins containing both respiration and heartbeat signals:
  • Equation 8 Equation 9 where the DC offsets in the m-th channel and the n-th channel are assumed to be compensated perfectly. Ideally, the phase difference is only a function of the heartbeat signal: Equation 10
  • Equation 10 representing motion residual noise
  • Equation 11 [0049] The two most appropriate channels are chosen based on the following equation: argma x m n SNR H Equation 12
  • phase difference of any combination from the set of channels V produces the best pulse SNR, SN H , and this combination gets selected.
  • SNR H is defined as the signal to noise ratio at the heartbeat frequency of interest by inspecting the spectrum of the phase differences
  • Figure 6 is a graphical representation of a temporal wave containing both respiration and heartbeat signals while a subject is piece-wise breathing. This illustrates a snapshot of the temporal wave containing both respiration and heartbeat signals at the range bin which has the strongest motion energy while the subject is piece-wise breathing. When the test subject stops breathing, the heartbeat pulse is clearly visible. There is no motion artifact present when breathing stops.
  • Figure 7 is a graphical representation of a vital sign spectrum obtained with a conventional approach and a reference heartbeat signal spectrum.
  • the range bin containing the maximum motion energy is used in Figure 7. Only the possible breathing spectrum and its strong harmonic spectral features can be seen, along with an elevated noise floor. Flowever, embodiments using the proposed active motion suppression technique can significantly improve the heartbeat detectability.
  • Figure 8 is a graphical representation of phase variations from two range bins along with the phase difference of the two range bins. This figure compares phase variations from the two range bins in Equation 12, as well as their phase difference. It can be seen that the overall trend of breathing patterns is almost removed, and thus the phase residual has an almost constant, pulse like shape.
  • Figure 9 is a graphical representation of a vital sign spectrum obtained with active motion suppression and a reference heartbeat signal spectrum.
  • Equation 10 In order to demonstrate the effectiveness of motion suppression through the proposed technique, no filter is applied in generating Figure 9. Additionally, the two phases in Equation 10 are normalized to a similar scale before taking the difference. In other embodiments, additional filtering is applied to the reference heartbeat signal spectrum to further improve results.
  • FIG 10 is a flow diagram illustrating a process for remote sleep monitoring.
  • the process begins at operation 1000, with transmitting a radar signal toward a subject.
  • the process continues at operation 1002, with receiving an RF response signal corresponding to the radar signal.
  • the radar sensor 12 of Figures 2 and 3 sends a series of UWB radar impulses and receives the RF response signal.
  • the RF response signal may include one or more vital sign micro-Doppler spatial signatures (e.g., respiratory and cardiac signatures), at least some of which may be enhanced by displacement of a mattress or other non-rigid surface in contact with the subject.
  • the process continues at operation 1004, with monitoring vital signs of a subject.
  • Operation 1004 includes suboperation 1006, with processing the RF response signal to produce one or more vital sign signals.
  • processing the RF response signal includes processing the RF response signal to produce a respiratory signal (e.g., from a respiratory channel) and a mixed respiratory and cardiac spatial channel and extracting a cardiac signal from the mixed respiratory and cardiac spatial channel.
  • Operation 1004 further includes suboperation 1008, with monitoring the vital sign signals over an evaluation period.
  • the evaluation period corresponds to an expected sleep cycle of the subject.
  • the process continues at operation 1010, with classifying a sleep state of the subject based on results from monitoring the vital signs.
  • the sleep state of the subject e.g., awake, asleep, depth of sleep, etc.
  • This analysis of the vital sign signals, including classification of the sleep state may also be used to provide a diagnosis of sleep conditions (e.g., sleep apnea, insomnia, etc).
  • FIG 11 is a schematic diagram of a generalized representation of an exemplary computer system 1100 that could be used to perform any of the methods or functions described above, such as remote sleep monitoring.
  • the radar sensor 12 and/or the processing device 16 of Figures 2 and 3 are implemented as the computer system 1100.
  • the computer system 1100 may be a circuit or circuits included in an electronic board card, such as, a printed circuit board (PCB), a server, a personal computer, a desktop computer, a laptop computer, an array of computers, a personal digital assistant (PDA), a computing pad, a mobile device, or any other device, and may represent, for example, a server or a user’s computer.
  • PCB printed circuit board
  • PDA personal digital assistant
  • the exemplary computer system 1100 in this embodiment includes a processing device 1102 or processor, a main memory 1104 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), such as synchronous DRAM (SDRAM), etc.), and a static memory 1106 (e.g., flash memory, static random access memory (SRAM), etc.), which may communicate with each other via a data bus 1108.
  • main memory 1104 e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), such as synchronous DRAM (SDRAM), etc.
  • static memory 1106 e.g., flash memory, static random access memory (SRAM), etc.
  • the processing device 1102 may be connected to the main memory 1104 and/or static memory 1106 directly or via some other connectivity means.
  • the processing device 1102 could be used to perform any of the methods or functions described above.
  • the processing device 1102 represents one or more general-purpose processing devices, such as a microprocessor, central processing unit (CPU), or the like. More particularly, the processing device 1102 may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor implementing other instruction sets, or other processors implementing a combination of instruction sets.
  • the processing device 1102 is configured to execute processing logic in instructions for performing the operations and steps discussed herein.
  • processing device 1102 which may be a microprocessor, field programmable gate array (FPGA), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or other programmable logic device, a discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein.
  • the processing device 1102 may be a microprocessor, or may be any conventional processor, controller, microcontroller, or state machine.
  • the processing device 1102 may also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration).
  • a combination of a DSP and a microprocessor e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.
  • the computer system 1100 may further include a network interface device 1110.
  • the computer system 1100 also may or may not include an input 1112, configured to receive input and selections to be communicated to the computer system 1100 when executing instructions.
  • the input 1112 may include, but not be limited to, a touch sensor (e.g., a touch display), an alphanumeric input device (e.g., a keyboard), and/or a cursor control device (e.g., a mouse).
  • the radar sensor 12 of Figures 2 and 3 is an input 1112 to the computer system 1100.
  • the computer system 1100 also may or may not include an output 1114, including but not limited to a display, a video display unit (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), or a printer.
  • a video display unit e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)
  • some or all inputs 1112 and outputs 1114 may be combination input/output devices.
  • the radar sensor 12 of Figures 2 and 3 is also an output 1114 of the computer system 1100.
  • the computer system 1100 may or may not include a data storage device that includes instructions 1116 stored in a computer-readable medium 1118.
  • the instructions 1116 may also reside, completely or at least partially, within the main memory 1104 and/or within the processing device 1102 during execution thereof by the computer system 1100, the main memory 1104, and the processing device 1102 also constituting computer-readable medium.
  • the instructions 1116 may further be transmitted or received via the network interface device 1110.
  • computer-readable medium 1118 is shown in an exemplary embodiment to be a single medium, the term “computer-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions 1116.
  • the term “computer-readable medium” shall also be taken to include any medium that is capable of storing, encoding, or carrying a set of instructions for execution by the processing device 1102 and that causes the processing device 1102 to perform any one or more of the methodologies of the embodiments disclosed herein.
  • the term “computer- readable medium” shall accordingly be taken to include, but not be limited to, solid-state memories, optical medium, and magnetic medium.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Biomedical Technology (AREA)
  • Pathology (AREA)
  • Biophysics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Physiology (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Cardiology (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Physics & Mathematics (AREA)
  • Pulmonology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Psychiatry (AREA)
  • Signal Processing (AREA)
  • Anesthesiology (AREA)
  • Dentistry (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

Methods and systems for remote sleep monitoring are provided. Such methods and systems provide non-contact sleep monitoring via remote sensing or radar sensors. In this regard, when processing backscattered radar signals from a sleeping subject on a normal mattress, a breathing motion magnification effect is observed from mattress surface displacement due to human respiratory activity. This undesirable motion artifact causes existing approaches for accurate heart-rate estimation to fail. Embodiments of the present disclosure use a novel active motion suppression technique to deal with this problem by intelligently selecting a slow-time series from multiple ranges and examining a corresponding phase difference. This approach facilitates improved sleep monitoring, where one or more subjects can be remotely monitored during an evaluation period (which corresponds to an expected sleep cycle).

Description

METHODS AND SYSTEMS FOR REMOTE SLEEP MONITORING
Related Applications
[0001] This application claims the benefit of provisional patent application serial number 62/926,717, filed October 28, 2019, the disclosure of which is hereby incorporated herein by reference in its entirety.
Field of the Disclosure
[0002] The present disclosure relates to vital sign estimation and monitoring.
Background
[0003] Sleep monitoring tracks the activity of the human body during sleep and provides information on brain activity and other physiological factors during sleep. Hence, sleep monitoring has been used to diagnose or understand the underlying causes of sleeping disorders such as sleep apnea, sleep related seizures, insomnia, etc., and also to qualitatively measure different aspects of sleep such as quality, activity, etc.
[0004] Common methods of sleep monitoring require the use of wearables or other contact sensors to measure vital signs. Such wearables can be uncomfortable to wear and can cause difficulty in falling asleep or even affect sleep behavior itself. Furthermore, several of these approaches also require overnight monitoring in clinics, increasing cost and discomfort of patients. For these reasons, non-contact approaches for sleep monitoring have tremendous utility.
[0005] Research into sleep monitoring approaches has been varied. As mentioned above, sleep monitoring using wearables is one of the most common approaches. Non-contact sleep monitoring approaches also exist, such as camera-based and sensor-based approaches. One example uses a low power radar-based biomotion sensor that detects movement to measure sleep/wake patterns during sleep and perform sleep/wake classification. Summary
[0006] Methods and systems for remote sleep monitoring are provided. Such methods and systems provide non-contact sleep monitoring via remote sensing or radar sensors. In this regard, when processing backscattered radar signals from a sleeping subject on a normal mattress, a breathing motion magnification effect is observed from mattress surface displacement due to human respiratory activity. This undesirable motion artifact causes existing approaches for accurate heart-rate estimation to fail. Embodiments of the present disclosure use a novel active motion suppression technique to deal with this problem by intelligently selecting a slow-time series from multiple ranges and examining a corresponding phase difference. This approach facilitates improved sleep monitoring, where one or more subjects can be remotely monitored during an evaluation period (which corresponds to an expected sleep cycle).
[0007] An exemplary embodiment provides a method for remote sleep monitoring. The method includes transmitting a radar signal toward a subject and receiving a radio frequency (RF) response signal corresponding to the radar signal. The method further includes monitoring vital signs of a subject, which includes processing the RF response signal to produce one or more vital sign signals and monitoring the vital sign signals over an evaluation period. The method further includes classifying a sleep state of the subject based on results from monitoring the vital signs.
[0008] Another exemplary embodiment provides a sleep monitoring device. The sleep monitoring device includes a radar sensor configured to receive an RF response signal to a radar signal and a processing circuit coupled to the radar sensor. The processing circuit is configured to monitor vital signs of a subject from the RF response signal by processing the RF response signal to produce a plurality of vital sign signals and classifying a sleep state of the subject based on monitoring the plurality of vital sign signals over an evaluation period.
[0009] Another exemplary embodiment provides a system for remote sleep monitoring. The system includes a radar sensor configured to receive an RF response signal to a radar signal transmitted toward one or more subjects, a database, and a processing circuit coupled to the radar sensor and the database. The processing circuit is configured to monitor vital signs of the one or more subjects over an evaluation period by processing the RF response signal to produce vital sign signals for each of the one or more subjects, storing the vital sign signals in the database, and classifying a sleep state of the subject based on the vital sign signals stored in the database.
[0010] Those skilled in the art will appreciate the scope of the present disclosure and realize additional aspects thereof after reading the following detailed description of the preferred embodiments in association with the accompanying drawing figures.
Brief Description of the Drawing Figures
[0011] The accompanying drawing figures incorporated in and forming a part of this specification illustrate several aspects of the disclosure, and together with the description serve to explain the principles of the disclosure.
[0012] Figure 1 A is a graphical representation of a vital sign micro-Doppler signature in slow time for a subject lying on a rigid surface.
[0013] Figure 1 B is a graphical representation of a vital sign micro-Doppler signature in slow time for a subject lying on a non-rigid surface.
[0014] Figure 2 is a schematic diagram of an exemplary system for remote sleep monitoring.
[0015] Figure 3 is an image of an embodiment of the system for remote sleep monitoring of Figure 2.
[0016] Figure 4 is an in-phase/quadrature (l/Q) data constellation plot from an example noisy channel due to excessive phase noise.
[0017] Figure 5 is an l/Q data constellation plot of an example range bin used to actively suppress breathing motion.
[0018] Figure 6 is a graphical representation of a temporal wave containing both respiration and heartbeat signals while a subject is piece-wise breathing. [0019] Figure 7 is a graphical representation of a vital sign spectrum obtained with a conventional approach and a reference heartbeat signal spectrum. [0020] Figure 8 is a graphical representation of phase variations from two range bins along with the phase difference of the two range bins.
[0021] Figure 9 is a graphical representation of a vital sign spectrum obtained with active motion suppression and a reference heartbeat signal spectrum.
[0022] Figure 10 is a flow diagram illustrating a process for remote sleep monitoring.
[0023] Figure 11 is a schematic diagram of a generalized representation of an exemplary computer system that could be used to perform any of the methods or functions described above, such as remote sleep monitoring.
Detailed Description
[0024] The embodiments set forth below represent the necessary information to enable those skilled in the art to practice the embodiments and illustrate the best mode of practicing the embodiments. Upon reading the following description in light of the accompanying drawing figures, those skilled in the art will understand the concepts of the disclosure and will recognize applications of these concepts not particularly addressed herein. It should be understood that these concepts and applications fall within the scope of the disclosure and the accompanying claims.
[0025] It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of the present disclosure. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items. [0026] It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. In contrast, when an element is referred to as being "directly connected" or "directly coupled" to another element, there are no intervening elements present. [0027] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises," "comprising," "includes," and/or "including" when used herein specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
[0028] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms used herein should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
[0029] Methods and systems for remote sleep monitoring are provided. Such methods and systems provide non-contact sleep monitoring via remote sensing or radar sensors. In this regard, when processing backscattered radar signals from a sleeping subject on a normal mattress, a breathing motion magnification effect is observed from mattress surface displacement due to human respiratory activity. This undesirable motion artifact causes existing approaches for accurate heart-rate estimation to fail. Embodiments of the present disclosure use a novel active motion suppression technique to deal with this problem by intelligently selecting a slow-time series from multiple ranges and examining a corresponding phase difference. This approach facilitates improved sleep monitoring, where one or more subjects can be remotely monitored during an evaluation period (which corresponds to an expected sleep cycle). I. Signal Model
[0030] Remote vital sign detection (e.g., for remote sleep monitoring) can be characterized with a signal model. For example, a synthesized transmitted pulse for remote vital sign detection can be modeled as a cosine wave with a Gaussian envelope:
Rtx(Τ) = P0(Τ) cos( 2πFct) Equation 1 where r0(t) denotes the Gaussian pulse envelope and is designed to satisfy an emission mask. Fc denotes the nominal operating frequency. A receive signal in response to the transmitted pulse can be modeled as:
Prx(t) = ATp0(t - tD(t)) cos(2πFc(t - tD(t))) Equation 2 where AT denotes the target response and ¾(t) denotes time-varying time-delay due to vital sign motion, t and t denote slow-time and fast-time scales, respectively.
[0031] The vital sign motion V(t) of a subject at a nominal distance d0 can be modeled as a sum of two non-stationary and periodic-like signals, XB(t) for respiratory activity and XH(t) for cardiac activity: d(t) = d0 + V(t) = d0 + XB(t) + XH(t) Equation 3 Then the motion modulated time-delay is written as tD(t) = d(t)/c, where c is the speed of light.
[0032] The complex baseband signal is obtained by mixing with a term ej2π Fct and low pass filtering: Equation 4
Figure imgf000008_0001
where A(t) denotes a time-varying amplitude. The ideal phase signal is then derived as: Equation 5
Figure imgf000008_0002
where l = c/Fc represents wavelength and φ0 is the initial phase term.
II. Breathing Magnification Effect
[0033] An interesting motion artifact has been observed during a sleep monitoring study. This motion magnification effect on the respiration signal makes heartbeat detection impossible using existing methods, such as radar- based methods.
[0034] In general, the respiration signal in the radar return is much stronger than that of heartbeat signal for two factors: radar cross section and physical displacement. Even though a radar sensor is illuminating the entire body of a subject, only a small portion of this response contains skin motion or vasomotion due to cardiac activity. The physical displacement due to normal breathing ranges from 1 mm (shallow) to 1 cm (deep), while the physical displacement near the skin surface as a result of heartbeat motion is on the order of 0.1 mm. Furthermore, in supine position, a non-rigid mattress tends to support and conform to the back of the human body very well. This results in the mattress moving with the human body during respiration, causing a motion magnification effect due to respiration.
[0035] Figures 1 A and 1 B demonstrate the motion magnification effect due to mattress surface displacement during respiration. Figure 1A is a graphical representation of a vital sign micro-Doppler signature in slow time for a subject lying on a rigid surface. Figure 1 B is a graphical representation of a vital sign micro-Doppler signature in slow time for a subject lying on a non-rigid surface. [0036] In order to validate this important observation, a test subject was instructed to lie down on both a rigid firm surface (e.g., a wooden bed frame) and on a non-rigid surface (e.g., a soft mattress) and breathe normally. Radar response signals for both situations were recorded and processed for comparison. As illustrated in Figures 1 A and 1 B, the non-rigid surface resulted in a relatively large magnification of breathing motion artifacts.
III. Active Motion Suppression Technique
[0037] Prior work has shown that radar-based heartbeat detection is respiration-interference limited. In supine position on a non-rigid surface, this problem gets even more challenging, as explained above with respect to Figures 1 A and 1 B. As a result, the spectral representation of the slow-time series at the range bin of interest, which contains significant motion energy, only shows respiration spectral energy with elevated sidelobes around the main lobe as well as an increased noise-floor. The weak heartbeat spectral feature is overwhelmed by the enhanced respiration signal.
[0038] Embodiments described herein provide a new active motion suppression technique to recover the heartbeat signal in the presence of magnified respiration interference. This approach exploits the fact that the human body is an extended dynamic target, and thus vital signs can be observed in multiple range bins. Across these multiple range bins, breathing energy spreads across a larger number of range bins when compared to heartbeat or pulse energy. The total number of range bins of interest can be divided into two categories: breathing only, and a mixture of respiration and heartbeat signals. A goal of the active motion suppression technique is to find two ‘best’ range bins such that their phase difference in the slow-time series generates a maximum pulse signal to noise ratio (SNR).
[0039] Figure 2 is a schematic diagram of an exemplary system 10 for remote sleep monitoring. The system includes a radar sensor 12 positioned at a distance from a subject 14. The radar sensor 12 is in communication with a processing device 16, such as a computer system.
[0040] In an exemplary aspect, the subject 14 lies on a rigid or non-rigid surface 18 (e.g., a mattress and/or a rigid frame) during a sleep evaluation (e.g., an evaluation period corresponding to an expected sleep cycle, such as four or more hours, six or more hours, or eight or more hours). The radar sensor 12 is configured to receive (e.g., sense) a radio frequency (RF) response signal 20 which corresponds to a radar signal 22 emitted toward the subject 14. In some examples, the radar sensor 12 is a transceiver which transmits the radar signal 22 and receives the RF response signal 20. In other examples, the radar signal 22 is emitted from another device (e.g., a transmitter which may be in communication with the radar sensor 12). The processing device 16 records and/or analyzes the RF response signal 20 to produce and/or monitor vital signs of the subject. The processing device 16 is further used to classify a sleep state of the subject (e.g., awake, asleep, depth of sleep, etc.), and may also provide a diagnosis of sleep conditions (e.g., sleep apnea, insomnia, etc.).
[0041] Figure 3 is an image of an embodiment of the system 10 for remote sleep monitoring of Figure 2. In this embodiment, the subject 14 lies on a non- rigid surface 18 (e.g., a mattress) which is on an elevated platform. The radar sensor 12 is suspended above the subject 14 via a ceiling mount or frame 24 and is pointed at the torso of the subject 14. For evaluation purposes, a test subject 14 was also fitted with a photoplethysmogram (PPG) sensor, which was used as a reference signal in a sleep monitor study. However, embodiments may omit such contact sensors.
[0042] The radar sensor 12 in the embodiment of Figure 3 includes an impulse-based ultra-wide band (UWB) radar. In some examples, the UWB radar operates at 1 .4 gigahertz (GHz) in the X band with low power emission, satisfying FCC part 15. A coherent transceiver is achieved through precise timing control. For slow motion monitoring (e.g., vital signs), many pulses can be coherently combined to achieve sufficient processing gain. The theoretical range resolution is about 10 cm. In a higher SNR regime, the actual range resolution is much smaller and on the order of 1 cm. In order to maximize the SNR of the response signal, the radar system parameters may be configured to operate at the highest possible pulse integration level while meeting the minimum range requirement for vital sign monitoring. Fine ranging information from the UWB radar provides multiple spatially independent observations at the same time. Thus, embodiments may effectively turn heartbeat detection into a multi-channel source separation problem.
[0043] It should be understood that in other embodiments the radar sensor 12 can include a different radar, such as a having a bandwidth between 1 .0 GHz and 2.0 GHz. It should also be understood that the radar sensor 12 may in other embodiments be positioned differently, such as wall-mounted or as a freestanding device placed on a surface and aimed toward the subject 14.
[0044] With reference to Figures 4 and 5, range bin selection for the active motion suppression technique is described. In an exemplary aspect, the initial group of range bins is selected by range gating spatio-temporal data (e.g., such as represented by a spatio-temporal heat map). To obtain a correct phase variation, which is linearly related to motion, complex signals from each candidate bin (e.g., spatial channel) are further processed by applying a circle fitting algorithm. The purpose of curve fitting is to correct for direct current (DC) offset, Q, as shown in Equation 6:
Equation 6
Figure imgf000012_0001
where A0 is a nominal signal amplitude.
[0045] Figure 4 is an in-phase/quadrature (l/Q) data constellation plot from an example noisy channel due to excessive phase noise. Clean phase samples should be mostly located closely to the unit circle in the l/Q constellation plot. One empirical criterion to evaluate the quality of phase information in each channel is to check the root mean square error (RMSE) of the distance between the sample points and the unit circle:
Equation 7
Figure imgf000012_0002
Here the upper limit is selected as thirty percent of the unit radius, which means that if the distance error from the l/Q samples is over 0.3, then this channel is too noisy and thus is ignored.
[0046] By now, a set of channels or range bins, V, with clean phase is obtained with outliers eliminated. But each set of l/Q samples from the selected range bins can generate a different phase trajectory with varying arc lengths and starting phases in the constellation plot. An example is shown in Figure 4 to demonstrate the results. The example demonstrates the magnified phase response because of the motion magnification effect described above in Section II. Based on the radar parameters, the expected phase rotation is about 90 degrees assuming the motion amplitude of breathing is 5 mm. This observation implies that one or a small set of range bins in V can be selected for heartbeat detection based on the expected phase rotation arc0. Mathematically, this process of choosing range bins, V, that most likely have clean vital sign signals is expressed as
Figure imgf000013_0004
[0047] Figure 5 is an l/Q data constellation plot of an example range bin used to actively suppress breathing motion. It is observed that the above method works very well. However, some embodiments focus on adaptive motion suppression, in which range bins with phase arc < arc0 and range bins with phase arc > arc0 (as seen in Figure 4) are used to actively suppress breathing motion. Note that different range bins V will have different phase offsets as the static phase is linearly related to the distance.
[0048] The basic idea of phase-based motion cancellation to suppress the magnified respiration signal is to select one channel from the breathing only range bins and select another one from the range bins containing both respiration and heartbeat signals:
Equation 8 Equation 9
Figure imgf000013_0001
where the DC offsets in the m-th channel and the n-th channel are assumed to be compensated perfectly. Ideally, the phase difference is only a function of the heartbeat signal: Equation 10
Figure imgf000013_0002
But in reality the motion interference cannot be perfectly cancelled out due to phase noise. This leads to an extra term in Equation 10, representing motion residual noise
Figure imgf000013_0005
Equation 11
Figure imgf000013_0003
[0049] The two most appropriate channels are chosen based on the following equation: argma xm nSNRH Equation 12
[0050] The phase difference of any combination from the set of channels V (obtained as described above with respect to Figure 4) produces the best pulse SNR, SN H, and this combination gets selected. SNRH is defined as the signal to noise ratio at the heartbeat frequency of interest by inspecting the spectrum of the phase differences
Figure imgf000014_0001
[0051] With reference to Figures 6-8, exemplary results of applying the active motion suppression technique are described. These results were obtained using the system for remote sleep monitoring of Figures 2 and 3. A first example, described below with respect to Figure 6, shows results when a subject is holding a breath (e.g., breathing piece-wise). When the subject is holding a breath, the breathing motion and associated breathing artifacts are not present anymore. A second example, described below with respect to Figures 7-9, compares results from a conventional approach against embodiments of the active motion suppression technique.
[0052] Figure 6 is a graphical representation of a temporal wave containing both respiration and heartbeat signals while a subject is piece-wise breathing. This illustrates a snapshot of the temporal wave containing both respiration and heartbeat signals at the range bin which has the strongest motion energy while the subject is piece-wise breathing. When the test subject stops breathing, the heartbeat pulse is clearly visible. There is no motion artifact present when breathing stops.
[0053] Figure 7 is a graphical representation of a vital sign spectrum obtained with a conventional approach and a reference heartbeat signal spectrum. The range bin containing the maximum motion energy is used in Figure 7. Only the possible breathing spectrum and its strong harmonic spectral features can be seen, along with an elevated noise floor. Flowever, embodiments using the proposed active motion suppression technique can significantly improve the heartbeat detectability. [0054] Figure 8 is a graphical representation of phase variations from two range bins along with the phase difference of the two range bins. This figure compares phase variations from the two range bins in Equation 12, as well as their phase difference. It can be seen that the overall trend of breathing patterns is almost removed, and thus the phase residual has an almost constant, pulse like shape.
[0055] Figure 9 is a graphical representation of a vital sign spectrum obtained with active motion suppression and a reference heartbeat signal spectrum. By inspecting the vital sign spectrum of the phase difference in Figure 8, the respiration signal is significantly suppressed, and the heartbeat is correctly recovered as the detected heartbeat has the same BPM as the reference pulse signal.
[0056] In order to demonstrate the effectiveness of motion suppression through the proposed technique, no filter is applied in generating Figure 9. Additionally, the two phases in Equation 10 are normalized to a similar scale before taking the difference. In other embodiments, additional filtering is applied to the reference heartbeat signal spectrum to further improve results.
IV. Method for Remote Sleep Monitoring
[0057] Figure 10 is a flow diagram illustrating a process for remote sleep monitoring. The process begins at operation 1000, with transmitting a radar signal toward a subject. The process continues at operation 1002, with receiving an RF response signal corresponding to the radar signal. In an exemplary aspect, the radar sensor 12 of Figures 2 and 3 sends a series of UWB radar impulses and receives the RF response signal. The RF response signal may include one or more vital sign micro-Doppler spatial signatures (e.g., respiratory and cardiac signatures), at least some of which may be enhanced by displacement of a mattress or other non-rigid surface in contact with the subject. [0058] The process continues at operation 1004, with monitoring vital signs of a subject. Operation 1004 includes suboperation 1006, with processing the RF response signal to produce one or more vital sign signals. In an exemplary aspect, processing the RF response signal includes processing the RF response signal to produce a respiratory signal (e.g., from a respiratory channel) and a mixed respiratory and cardiac spatial channel and extracting a cardiac signal from the mixed respiratory and cardiac spatial channel. Operation 1004 further includes suboperation 1008, with monitoring the vital sign signals over an evaluation period. In an exemplary aspect, the evaluation period corresponds to an expected sleep cycle of the subject.
[0059] The process continues at operation 1010, with classifying a sleep state of the subject based on results from monitoring the vital signs. For example, the sleep state of the subject (e.g., awake, asleep, depth of sleep, etc.) may be determined throughout the evaluation period (e.g., to determine duration and/or quality of sleep). This analysis of the vital sign signals, including classification of the sleep state may also be used to provide a diagnosis of sleep conditions (e.g., sleep apnea, insomnia, etc).
[0060] Although the operations of Figure 10 are illustrated in a series, this is for illustrative purposes and the operations are not necessarily order dependent. Some operations may be performed in a different order than that presented. Further, processes within the scope of this disclosure may include fewer or more steps than those illustrated in Figure 10.
V. Computer System
[0061] Figure 11 is a schematic diagram of a generalized representation of an exemplary computer system 1100 that could be used to perform any of the methods or functions described above, such as remote sleep monitoring. In some examples, the radar sensor 12 and/or the processing device 16 of Figures 2 and 3 are implemented as the computer system 1100. In this regard, the computer system 1100 may be a circuit or circuits included in an electronic board card, such as, a printed circuit board (PCB), a server, a personal computer, a desktop computer, a laptop computer, an array of computers, a personal digital assistant (PDA), a computing pad, a mobile device, or any other device, and may represent, for example, a server or a user’s computer. [0062] The exemplary computer system 1100 in this embodiment includes a processing device 1102 or processor, a main memory 1104 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), such as synchronous DRAM (SDRAM), etc.), and a static memory 1106 (e.g., flash memory, static random access memory (SRAM), etc.), which may communicate with each other via a data bus 1108. Alternatively, the processing device 1102 may be connected to the main memory 1104 and/or static memory 1106 directly or via some other connectivity means. In an exemplary aspect, the processing device 1102 could be used to perform any of the methods or functions described above.
[0063] The processing device 1102 represents one or more general-purpose processing devices, such as a microprocessor, central processing unit (CPU), or the like. More particularly, the processing device 1102 may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor implementing other instruction sets, or other processors implementing a combination of instruction sets. The processing device 1102 is configured to execute processing logic in instructions for performing the operations and steps discussed herein.
[0064] The various illustrative logical blocks, modules, and circuits described in connection with the embodiments disclosed herein may be implemented or performed with the processing device 1102, which may be a microprocessor, field programmable gate array (FPGA), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or other programmable logic device, a discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. Furthermore, the processing device 1102 may be a microprocessor, or may be any conventional processor, controller, microcontroller, or state machine. The processing device 1102 may also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration).
[0065] The computer system 1100 may further include a network interface device 1110. The computer system 1100 also may or may not include an input 1112, configured to receive input and selections to be communicated to the computer system 1100 when executing instructions. The input 1112 may include, but not be limited to, a touch sensor (e.g., a touch display), an alphanumeric input device (e.g., a keyboard), and/or a cursor control device (e.g., a mouse). In an exemplary aspect, the radar sensor 12 of Figures 2 and 3 is an input 1112 to the computer system 1100. The computer system 1100 also may or may not include an output 1114, including but not limited to a display, a video display unit (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), or a printer. In some examples, some or all inputs 1112 and outputs 1114 may be combination input/output devices. In an exemplary aspect, the radar sensor 12 of Figures 2 and 3 is also an output 1114 of the computer system 1100.
[0066] The computer system 1100 may or may not include a data storage device that includes instructions 1116 stored in a computer-readable medium 1118. The instructions 1116 may also reside, completely or at least partially, within the main memory 1104 and/or within the processing device 1102 during execution thereof by the computer system 1100, the main memory 1104, and the processing device 1102 also constituting computer-readable medium. The instructions 1116 may further be transmitted or received via the network interface device 1110.
[0067] While the computer-readable medium 1118 is shown in an exemplary embodiment to be a single medium, the term “computer-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions 1116. The term “computer-readable medium” shall also be taken to include any medium that is capable of storing, encoding, or carrying a set of instructions for execution by the processing device 1102 and that causes the processing device 1102 to perform any one or more of the methodologies of the embodiments disclosed herein. The term “computer- readable medium” shall accordingly be taken to include, but not be limited to, solid-state memories, optical medium, and magnetic medium.
[0068] The operational steps described in any of the exemplary embodiments herein are described to provide examples and discussion. The operations described may be performed in numerous different sequences other than the illustrated sequences. Furthermore, operations described in a single operational step may actually be performed in a number of different steps. Additionally, one or more operational steps discussed in the exemplary embodiments may be combined.
[0069] Those skilled in the art will recognize improvements and modifications to the preferred embodiments of the present disclosure. All such improvements and modifications are considered within the scope of the concepts disclosed herein and the claims that follow.

Claims

Claims What is claimed is:
1 . A method for remote sleep monitoring, the method comprising: transmitting a radar signal toward a subject; receiving a radio frequency (RF) response signal corresponding to the radar signal; monitoring vital signs of a subject, comprising: processing the RF response signal to produce one or more vital sign signals; and monitoring the one or more vital sign signals over an evaluation period; and classifying a sleep state of the subject based on results from monitoring the vital signs.
2. The method of claim 1 , wherein the radar signal is a radar impulse signal.
3. The method of claim 2, wherein the radar signal is an ultra-wide band (UWB) radar impulse signal.
4. The method of claim 1 , further comprising determining whether the subject has a sleep disorder based on classifying the sleep state.
5. The method of claim 1 , wherein monitoring the vital signs of the subject comprises monitoring a respiratory signal and a cardiac signal.
6. The method of claim 5, wherein processing the RF response signal to produce the one or more vital sign signals comprises: processing the RF response signal to produce the respiratory signal and a mixed respiratory and cardiac spatial channel; and extracting the cardiac signal from the mixed respiratory and cardiac spatial channel.
7. The method of claim 6, wherein processing the RF response signal further comprises: producing spatio-temporal data of motion response in the RF response signal; and range gating the spatio-temporal data to produce a plurality of spatial channels comprising one or more respiratory spatial channels and one or more mixed respiratory and cardiac spatial channels.
8. The method of claim 7, wherein processing the RF response signal further comprises applying a circle fitting algorithm to the plurality of spatial channels to correct for direct current (DC) offset.
9. The method of claim 7, wherein processing the RF response signal further comprises identifying the one or more mixed respiratory and cardiac spatial channels based on phase rotation of the plurality of spatial channels.
10. The method of claim 9, wherein identifying the one or more mixed respiratory and cardiac spatial channels comprises identifying spatial channels having an expected phase rotation arc.
11 . The method of claim 10, wherein processing the RF response signal further comprises identifying the one or more respiratory spatial channels from spatial channels having a phase rotation arc outside the expected phase rotation arc.
12. The method of claim 7, wherein extracting the cardiac signal from the mixed respiratory and cardiac spatial channel comprises: extracting the respiratory signal from the one or more respiratory spatial channels; and removing the respiratory signal from the one or more mixed respiratory and cardiac spatial channels.
13. The method of claim 12, wherein: the respiratory signal is extracted from a selected respiratory spatial channel of the one or more respiratory spatial channels; and the respiratory signal is removed from a selected mixed respiratory and cardiac spatial channel of the one or more mixed respiratory and cardiac spatial channels.
14. The method of claim 13, wherein the selected respiratory spatial channel and the selected mixed respiratory and cardiac spatial channel are selected based on which combination among the one or more respiratory spatial channels and the one or more mixed respiratory and cardiac spatial channels produces a higher signal to noise ratio (SNR).
15. The method of claim 1 , wherein the radar signal comprises ultra-wide band (UWB) radar impulses having a bandwidth between 1.0 gigahertz (GHz) and 2.0 GHz.
16. The method of claim 1 , wherein the evaluation period comprises a predicted sleep cycle of the subject.
17. The method of claim 1 , further comprising monitoring vital signs of multiple subjects over the evaluation period.
18. A sleep monitoring device, comprising: a radar sensor configured to receive a radio frequency (RF) response signal to a radar signal; and a processing circuit coupled to the radar sensor and configured to monitor vital signs of a subject from the RF response signal by: processing the RF response signal to produce a plurality of vital sign signals; and classifying a sleep state of the subject based on monitoring the plurality of vital sign signals over an evaluation period.
19. The sleep monitoring device of claim 18, wherein: the radar sensor comprises an ultra-wide band (UWB) radar transmitter; and the radar signal comprises a series of radar impulses at a bandwidth between 1 .0 gigahertz (GHz) and 2.0 GHz.
20. A system for remote sleep monitoring, the system comprising: a radar sensor configured to receive a radio frequency (RF) response signal to a radar signal transmitted toward one or more subjects; a database; and a processing circuit coupled to the radar sensor and the database and configured to monitor vital signs of the one or more subjects over an evaluation period by: processing the RF response signal to produce vital sign signals for each of the one or more subjects; storing the vital sign signals in the database; and classifying a sleep state of the subject based on the vital sign signals stored in the database.
PCT/US2020/057452 2019-10-28 2020-10-27 Methods and systems for remote sleep monitoring WO2021086809A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/772,844 US11690563B2 (en) 2019-10-28 2020-10-27 Methods and systems for remote sleep monitoring

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201962926717P 2019-10-28 2019-10-28
US62/926,717 2019-10-28

Publications (1)

Publication Number Publication Date
WO2021086809A1 true WO2021086809A1 (en) 2021-05-06

Family

ID=75715489

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2020/057452 WO2021086809A1 (en) 2019-10-28 2020-10-27 Methods and systems for remote sleep monitoring

Country Status (2)

Country Link
US (1) US11690563B2 (en)
WO (1) WO2021086809A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102022208945A1 (en) 2022-08-29 2024-02-29 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung eingetragener Verein System and method for contactless vital parameter recording by creating a body model based on the body subdivision along the radar field of view
US11988772B2 (en) 2019-11-01 2024-05-21 Arizona Board Of Regents On Behalf Of Arizona State University Remote recovery of acoustic signals from passive sources

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116369874B (en) * 2023-03-27 2023-12-12 浙江想能睡眠科技股份有限公司 Human health detection method, system and storage medium based on mattress data

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090203972A1 (en) * 2006-06-01 2009-08-13 Biancamed Ltd. Apparatus, system, and method for monitoring physiological signs
US20130053653A1 (en) * 2011-08-29 2013-02-28 General Electric Company Radar based systems and methods for monitoring a subject
US20160022204A1 (en) * 2013-03-13 2016-01-28 Kirill Mostov An apparatus for remote contactless monitoring of sleep apnea
WO2018234394A1 (en) * 2017-06-22 2018-12-27 Iee International Electronics & Engineering S.A. System and method for breathing monitoring using radar-based sensor systems and the signal autocorrelation function
US20200196866A1 (en) * 2018-12-21 2020-06-25 Wistron Corporation Non-contact vital-sign monitoring system and method

Family Cites Families (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4860014A (en) 1987-11-25 1989-08-22 The United State Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Doppler radar with multiphase modulation of transmitted and reflected signal
US5424749A (en) 1994-03-21 1995-06-13 The United States Of America As Represented By The Secretary Of The Air Force Vibrating object assessment from a pulse operated vibrating sensor
US5565872A (en) 1995-04-25 1996-10-15 Loral Fairchild Corporation System and method for detecting and characterizing vibrating targets
GB2310099B (en) 1996-02-08 2000-08-02 Mecon Limited Radar for vibration detection
US6026340A (en) 1998-09-30 2000-02-15 The Robert Bosch Corporation Automotive occupant sensor system and method of operation by sensor fusion
US7687039B2 (en) 1998-10-28 2010-03-30 Covaris, Inc. Methods and systems for modulating acoustic energy delivery
US6492933B1 (en) 1999-09-02 2002-12-10 Mcewan Technologies, Llc SSB pulse Doppler sensor and active reflector system
US7057516B2 (en) 2001-06-01 2006-06-06 Dimitri Donskoy Device and method for detecting localization, monitoring, and identification of living organisms in structures
WO2004075162A2 (en) 2003-02-20 2004-09-02 Ramot At Tel Aviv University Ltd. Method apparatus and system for processing acoustic signals
US7148836B2 (en) 2004-03-05 2006-12-12 The Regents Of The University Of California Obstacle penetrating dynamic radar imaging system
CN1749583A (en) 2004-09-13 2006-03-22 Smc株式会社 Guide-equipped cylinder
US20080074307A1 (en) 2006-05-17 2008-03-27 Olga Boric-Lubecke Determining presence and/or physiological motion of one or more subjects within a doppler radar system
WO2008001092A2 (en) 2006-06-28 2008-01-03 Cambridge Consultants Limited Radar for through wall detection
US7938010B2 (en) 2006-10-23 2011-05-10 Visidyne, Inc. Passive, remote sensor
US7551519B2 (en) 2006-12-21 2009-06-23 Dan Slater Passive long range acoustic sensor
US20100152600A1 (en) 2008-04-03 2010-06-17 Kai Sensors, Inc. Non-contact physiologic motion sensors and methods for use
US8686362B2 (en) 2009-05-01 2014-04-01 Uchicago Argonne, Llc Millimeter wave sensor for far-field standoff vibrometry
US9772261B2 (en) 2010-02-12 2017-09-26 Fluidion Sas Passive micro-vessel and sensor
US8712069B1 (en) 2010-04-19 2014-04-29 Audience, Inc. Selection of system parameters based on non-acoustic sensor information
US20140194793A1 (en) 2010-05-14 2014-07-10 Kai Medical, Inc. Systems and methods for non-contact multiparameter vital signs monitoring, apnea therapy, apnea diagnosis, and snore therapy
US9442189B2 (en) 2010-10-27 2016-09-13 The Fourth Military Medical University Multichannel UWB-based radar life detector and positioning method thereof
US9000973B2 (en) 2011-04-29 2015-04-07 The Invention Science Fund I, Llc Personal electronic device with a micro-impulse radar
US9445729B2 (en) * 2012-07-20 2016-09-20 Resmed Sensor Technologies Limited Range gated radio frequency physiology sensor
US10129658B2 (en) 2013-07-22 2018-11-13 Massachusetts Institute Of Technology Method and apparatus for recovering audio signals from images
US20160089052A1 (en) 2014-09-26 2016-03-31 Industry-University Cooperation Foundation Hanyang University Method and device for measuring biometric data using uwb radar
US20180000408A1 (en) 2014-12-16 2018-01-04 Koninklijke Philips N.V. Baby sleep monitor
US11129576B2 (en) 2016-04-15 2021-09-28 University Of Florida Research Foundation, Incorporated Accurate heart rate measurement by radar using adaptive harmonics filter
KR102647218B1 (en) 2016-09-19 2024-03-12 레스메드 센서 테크놀로지스 리미티드 Apparatus, system, and method for detecting physiological movement from audio and multimodal signals
CN106264501A (en) 2016-10-13 2017-01-04 杭州电子科技大学 A kind of bio-signal acquisition system based on continuous wave Doppler radar
US20180333103A1 (en) 2017-05-18 2018-11-22 One Health Group, LLC Algorithmic Approach for Estimation of Respiration and Heart Rates
SG11202000330XA (en) 2017-07-14 2020-02-27 Fraunhofer Ges Forschung Concept for generating an enhanced sound field description or a modified sound field description using a multi-point sound field description
US10310073B1 (en) * 2018-02-07 2019-06-04 Infineon Technologies Ag System and method for determining engagement level of a human being using a millimeter-wave radar sensor
WO2020072297A1 (en) 2018-10-03 2020-04-09 Arizona Board Of Regents On Behalf Of Arizona State University Direct rf signal processing for heart-rate monitoring using uwb impulse radar
WO2020191142A1 (en) * 2019-03-20 2020-09-24 Arizona Board Of Regents On Behalf Of Arizona State University Radar cardiography: a precise cardiac data reconstruction method
EP4027879A1 (en) * 2019-09-13 2022-07-20 ResMed Sensor Technologies Limited Systems and methods for detecting movement
US20210093203A1 (en) * 2019-09-30 2021-04-01 DawnLight Technologies Systems and methods of determining heart-rate and respiratory rate from a radar signal using machine learning methods
US20220373646A1 (en) * 2021-05-21 2022-11-24 Samsung Electronics Co., Ltd. Joint estimation of respiratory and heart rates using ultra-wideband radar

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090203972A1 (en) * 2006-06-01 2009-08-13 Biancamed Ltd. Apparatus, system, and method for monitoring physiological signs
US20130053653A1 (en) * 2011-08-29 2013-02-28 General Electric Company Radar based systems and methods for monitoring a subject
US20160022204A1 (en) * 2013-03-13 2016-01-28 Kirill Mostov An apparatus for remote contactless monitoring of sleep apnea
WO2018234394A1 (en) * 2017-06-22 2018-12-27 Iee International Electronics & Engineering S.A. System and method for breathing monitoring using radar-based sensor systems and the signal autocorrelation function
US20200196866A1 (en) * 2018-12-21 2020-06-25 Wistron Corporation Non-contact vital-sign monitoring system and method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
MERCURI MARCO, LORATO ILDE ROSA, LIU YAO-HONG, WIERINGA FOKKO, HOOF CHRIS VAN, TORFS TOM: "Vital-sign monitoring and spatial tracking of multiple people using a contactless radar-based sensor", NATURE ELECTRONICS, vol. 2, no. 6, 1 June 2019 (2019-06-01), pages 252 - 262, XP055932193, DOI: 10.1038/s41928-019-0258-6 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11988772B2 (en) 2019-11-01 2024-05-21 Arizona Board Of Regents On Behalf Of Arizona State University Remote recovery of acoustic signals from passive sources
DE102022208945A1 (en) 2022-08-29 2024-02-29 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung eingetragener Verein System and method for contactless vital parameter recording by creating a body model based on the body subdivision along the radar field of view
WO2024046820A1 (en) 2022-08-29 2024-03-07 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. System and method for the contactless recording of vital parameters by creating a body model based on body subdivision along the radar field of view

Also Published As

Publication number Publication date
US20230018038A1 (en) 2023-01-19
US11690563B2 (en) 2023-07-04

Similar Documents

Publication Publication Date Title
Mercuri et al. Vital-sign monitoring and spatial tracking of multiple people using a contactless radar-based sensor
US11690563B2 (en) Methods and systems for remote sleep monitoring
Chen et al. Contactless electrocardiogram monitoring with millimeter wave radar
US20210353156A1 (en) Direct rf signal processing for heart-rate monitoring using uwb impulse radar
Brüser et al. Robust inter-beat interval estimation in cardiac vibration signals
Costa et al. Broken asymmetry of the human heartbeat: loss of time irreversibility in aging and disease
US8149273B2 (en) System and methods for vital sign estimation from passive thermal video
EP3225158A2 (en) Method and apparatus for heart rate and respiration rate estimation using low power sensor
US9532735B2 (en) Apparatus and method for wireless monitoring using ultra-wideband frequencies
Wu et al. Person-specific heart rate estimation with ultra-wideband radar using convolutional neural networks
Gaudes et al. Paradigm free mapping with sparse regression automatically detects single‐trial functional magnetic resonance imaging blood oxygenation level dependent responses
US11311202B2 (en) Robust real-time heart rate monitoring method based on heartbeat harmonics using small-scale radar
Sahu et al. IoT-enabled cloud-based real-time remote ECG monitoring system
Xu et al. Cardiacwave: A mmwave-based scheme of non-contact and high-definition heart activity computing
US20180271435A1 (en) Learning sleep stages from radio signals
Rong et al. Active breathing suppression for improved sleep monitoring heartbeat detection using uwb radar
CN114305355B (en) Breathing heartbeat detection method, system and device based on millimeter wave radar
Mirmohamadsadeghi et al. Real-time multi-signal frequency tracking with a bank of notch filters to estimate the respiratory rate from the ECG
Zhang et al. Health-radio: Towards contactless myocardial infarction detection using radio signals
Tariq et al. Doppler radar vital signs monitoring using wavelet transform
Wang et al. HeRe: Heartbeat signal reconstruction for low-power millimeter-wave radar based on deep learning
Raheel et al. Breathing and heartrate monitoring system using IR-UWB radar
Varanini et al. Cardiac and respiratory monitoring through non-invasive and contactless radar technique
Qiao et al. Spectral Unmixing Successive Variational Mode Decomposition for Robust Vital Signs Detection Using UWB Radar
Zhou et al. A modified two-dimension EEMD method for breathing and heart rates monitoring via through-wall IR-UWB radar

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20883084

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20883084

Country of ref document: EP

Kind code of ref document: A1