CN112806966B - A non-interference sleep apnea early warning system - Google Patents

Abstract

The invention discloses a non-interference type early warning system for apnea in sleep, which comprises: the system comprises a control center, an infrared temperature measurement module, a micro-pressure sensing module, an alarm module and a user terminal; the infrared temperature measurement module is used for collecting thermal imaging images, the micro-pressure sensing module is used for collecting pressure data, the alarm module is used for transmitting alarm signals of apnea, and the user terminal is used for displaying real-time pressure data, real-time thermal imaging images and alarm information; the control center receives the thermal imaging image and the pressure data, carries out target tracking on the thermal imaging image flow, classifies and records sleeping postures, and judges whether a breathing interference action occurs or not; judging and analyzing the pressure value detected by the micro-pressure sensing module, judging whether the breath of the user is normal or not according to the pressure change value between two breaths, judging whether the sleep breath is paused or not by combining the breath interference action, and outputting an alarm signal. The invention can realize non-interference measurement and multi-aspect combined monitoring.

Description

A non-interference sleep apnea early warning system

technical field

The invention relates to the technical field of sleep monitoring, in particular to a non-interference sleep apnea early warning system.

Background technique

Sleep apnea is a serious sleep disorder that occurs primarily when an individual's breathing is interrupted during sleep. Therefore, it is very necessary to monitor and timely alarm for sleep apnea.

A common limitation of the prior art is that it is impossible to monitor sleep conditions without interference, which will have a certain impact on human sleep. The existing sleep apnea acquisition and analysis system based on ambulatory ECG and respiratory wave acquisition adopts a set of wearable analysis system, which uses a dynamic electrocardiogram recorder to record the dynamic electrocardiogram and respiratory wave synchronously, and uses two indicators of HRV and respiratory wave. Corroborating each other, deducing the collected data and analyzing the image display. Using this method to monitor the sleep breathing state during sleep will have a certain impact on the sleep state of the tested person. Existing monitoring equipment can also be made into portable equipment, so that it can be used at home, but also needs to wear external equipment, which will also affect the sleep status of the tested person.

In addition, some studies have used infrared thermal imaging technology to monitor sleep breathing. The basic principle of this method is to capture the temperature fluctuations around the nose and mouth during breathing, and to determine the breathing status by analyzing the results of the fluctuations. At the same time, some studies advocate that a face capture system should also be used when using infrared thermal imaging technology to monitor sleep breathing conditions; the main limitation of using infrared thermal imaging technology for monitoring is that if the sleeping position changes, it cannot be very accurate. In the case of face tracking, it is difficult to output sleep breathing conditions very accurately. If the face recognition system is added, firstly, it will make the algorithm more difficult to write, requiring point-to-point identification; secondly, the monitoring data will be more abundant, for example, if the face recognition system is not added, when the subject buries his head in the quilt , it is slightly insufficient to only test for temperature fluctuations.

There is also a wearable ring that monitors heart rate, blood oxygen saturation levels, perfusion index, and exercise volume during sleep, all by monitoring blood flow through capillaries in the fingers. Although sleep monitoring can be performed through data such as capillary flow, the limitation of such products is that they cannot provide timely warning. The device uses vibration prompts and can tell the user a complete report of the sleep status, but in terms of alarm function, just using vibration to alarm is not enough to solve the early warning problem of sleep apnea, and it cannot issue an alarm in a timely and effective manner.

To sum up, the existing technologies generally have some deficiencies, including: the inability to achieve completely non-interference monitoring, the monitoring data is too single, and the inability to issue alarms in a timely and effective manner, etc., which greatly limit the use of the above monitoring methods.

SUMMARY OF THE INVENTION

In order to overcome the defects and deficiencies of the prior art, the present invention provides a non-interference sleep apnea early warning system. The present invention solves the problem that the prior art affects the sleep of the tested person, and can be used when the tested person is in a completely natural state. It can monitor the breathing state during sleep under the condition of no interference measurement and multi-faceted joint monitoring.

In order to achieve the above object, the present invention adopts the following technical solutions:

The invention provides a non-interference sleep apnea early warning system, comprising: a control center, an infrared temperature measurement module, a micro-pressure sensing module, an alarm module and a user terminal;

The infrared temperature measurement module, the micro pressure sensing module, the alarm module and the user terminal are all connected with the control center;

The infrared temperature measurement module is used to collect thermal imaging images, the micro pressure sensing module is used to collect pressure data, the alarm module is used to transmit an alarm signal of apnea, and the user terminal is used to display real-time pressure data , real-time thermal imaging images and alarm information;

The control center is used for receiving thermal imaging images and pressure data, tracking the thermal imaging image stream, classifying and recording sleeping positions, and judging whether breathing interference occurs;

Judging and analyzing the pressure value detected by the micro-pressure sensing module, determining whether the user's breathing is normal according to the pressure change value between two breaths, and combining the breathing interference action to determine whether sleep apnea is suspended, and outputting an alarm signal.

As a preferred technical solution, a camera module is also provided, the camera module is connected to the control center, and the camera module is used to collect camera video stream data.

As a preferred technical solution, the pressure sensor adopts a flexible material.

The present invention also provides an early warning method for a non-interference sleep apnea early warning system, comprising the following steps:

The thermal imaging image stream collected by the infrared temperature measurement module is used for target tracking, and a BP neural network is constructed for sleep posture classification training to obtain a sleep posture classifier. The sleep posture classifier classifies and records the sleeping posture, and judges the current sleeping posture. Whether it is the same sleeping position as the previous moment;

Detect the face area as the area of interest of the infrared temperature measurement module, obtain the average gray value of the area of interest, perform Kalman filtering on the average gray value, and calculate the time of adjacent peaks of the filtered data as the breathing cycle;

Set the number of breathing cycles n. If the breathing cycles obtained by continuous n calculations exceed the set normal breathing cycle range, it is determined that the breathing is abnormal;

Judging and analyzing the pressure value detected by the micro-pressure sensing module, if the pressure change value between two breaths is less than the preset value, it is determined that the user is breathing normally; if the pressure change value between two breaths is the preset value Or when the preset value is exceeded, and the next pressure value from the micro pressure sensor is not received within the normal breathing time interval, the current time is recorded;

The sleeping posture classifier judges whether the sleeping posture at the current time is the same as the sleeping posture at the previous moment, if not, it is determined that a breathing interference action occurs; if there is no breathing interference action, it is determined that sleep apnea occurs, and an alarm signal is output. .

As a preferred technical solution, the sleeping posture classifier classifies and records the sleeping posture, and the specific steps include:

Set the user's sleep posture set as C={C ₁ , C ₂ , C ₃ }={ supine, left side, right side}, and establish a corresponding data set;

The BP neural network is trained using the corresponding data set to obtain a sleeping posture classifier;

The sleeping posture classifier outputs the judgment result. When the sleeping posture at the current time is different from the sleeping posture at the previous moment, the time t when the sleeping posture changes is stored, and finally the ordered vector of the sleeping posture changing time in the sleeping process is obtained. S _all = [t ₁ , t ₂ , . . . _N ].

As a preferred technical solution, the specific calculation formula for obtaining the average gray value of the region of interest is:

Among them, rm and rn are the width and height of the region of interest, respectively, and k represents any time.

As a preferred technical solution, the Kalman filter is performed on the average gray value, and the specific calculation formula is:

z ^k =AS ^k +υ ^k

P ^k|k-1 =AP ^k-1|k-1 A ^T +Q

K ^k =P ^k|k-1 A ^T (AP ^k|k-1 A ^T +R) ^-1

P ^k|k =(IK ^k A)P ^k|k-1

表示k时刻的先验状态估计值，

分别表示k-1和k时刻的后验状态估计值；P^k|k-1表示k时刻的先验估计协方差，P^k-1|k-1、P^k|k分别表示k-1、k时刻的后验状态估计协方差；Q表示过程噪音协方差矩阵；K^k表示卡尔曼增益。Among them, A represents the measurement state transition matrix, υ ^k ~N(0, R) represents the observed noise that satisfies the Gaussian distribution, R represents the measurement noise covariance, z ^k represents the observed value at time k,

represents the prior state estimate at time k,

represent the posterior state estimates at time k-1 and k, respectively; P ^k|k-1 represents the prior estimated covariance at time k, and P ^k-1|k-1 and P ^k|k represent k-1, The posterior state estimate covariance at time k; Q is the process noise covariance matrix; K ^k is the Kalman gain.

As a preferred technical solution, the time of the adjacent peaks of the filtered data is calculated as the breathing cycle, and the peak acquisition step includes:

且

则k时刻

为第l个波峰点记作C^l＝k；At the kth moment, if the

and

then k time

Denote the lth peak point as C ^l =k;

The time difference between two adjacent peaks is the breathing cycle calculated from the first peak point, that is, T ^l =C ^l -C ^l-1 ;

The normal breathing cycle range is set to [T _low ,T _high ].

As a preferred technical solution, the output alarm signal specifically adopts any one or more of the following alarm methods:

The method of vibrating and waking up the user, notifying the user's family members through the bound mobile phone, and notifying the local ambulance unit through the network.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

(1) The present invention collects data through an infrared temperature measurement module and a micro pressure sensing module, and the infrared temperature measurement module collects data by not directly contacting the human body, and will not generate noise and light that affect sleep; and the micro pressure sensor The module collects data by embedding it in the mattress. If it is in a normal sleep condition, the vibration wake-up mode will not be activated, which solves the problem that the existing technology affects the sleep of the tested person, and can make the tested person in a completely natural state. Monitor the breathing state during sleep to achieve non-interfering measurement and multi-faceted joint monitoring.

(2) The present invention will immediately issue an alarm when it is detected that the time to stop breathing reaches a dangerous time (irreversible damage to the brain occurs in 4-6 minutes), thereby achieving the effect of timely alarming.

Description of drawings

Fig. 1 is the structural frame schematic diagram of the non-interference type sleep apnea early warning system of the present invention;

2 is a schematic flowchart of a non-interference sleep apnea early warning method of the present invention;

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

Example

As shown in FIG. 1 , this embodiment provides a non-interference sleep apnea early warning system, including: a control center, a camera module, an infrared temperature measurement module, a micro-pressure sensing module, and an alarm module.

In this embodiment, the control center is responsible for processing and analyzing all data monitored in the system. If the analysis result is normal, no action will be taken. If the analysis result is abnormal or even after sleep apnea occurs, it will be delivered to the alarm module to give an alarm in time.

The monitored data includes: camera video stream data captured by the camera module, thermal imaging images obtained by the infrared temperature measurement module, and pressure sensing data.

In this embodiment, the camera module is used to capture the video stream data of the camera. In this embodiment, the camera module may not be used in the process of judging sleep respiration, and the RGB image and the video stream data of the camera may not be used. The addition here is to enrich the function of the entire system.

The processing method of the control center also includes transmitting the analysis data to the user terminal, and the user terminal can directly display the real-time monitoring video, real-time infrared imaging and related analysis information.

In this embodiment, the infrared temperature measurement module covers the bed and the nearby 0.5m area, and there is one on each side of the head of the bed, so that accurate temperature changes can be obtained even when sleeping on the side, and the temperature is affected by staying away from windows, air conditioners, etc. irrelevant elements of change.

As shown in FIG. 2 , this embodiment provides a non-interference sleep apnea early warning method, which performs target tracking on the thermal imaging video stream obtained by the infrared temperature measurement module, classifies user behavior, and analyzes the user's current sleeping posture.

The sleeping position analysis is carried out according to the following steps:

(1) Define the user's sleep posture set as C={C ₁ , C ₂ , C ₃ }={ supine, left side, right side}, and establish a corresponding data set;

(2) Constructing a BP neural network for sleeping posture classification, and using the above data set for training, thereby obtaining a sleeping posture classifier;

(3) Using the classifier in (2) for real-time video analysis, the time-varying classifier result Ψ(t)∈C can be obtained. If Ψ(t)=Ψ(t ^- ), it means that the sleeping position does not exist at time t If Ψ(t)≠Ψ(t ^- ), the sleeping position has changed at time t, Ψ(t ^- ) is the classifier result of the previous moment, and the moment when the sleeping position changed is stored in S _all , Then finally, the ordered vector S _all =[t ₁ , t ₂ , ... t _N ] of the sleeping posture change time during the user's sleeping process can be obtained;

(4) If there is t _k ∈ S _all , k>m satisfies t _k -t _km <τ, m is a set normal number, τ is a set time parameter, it can be regarded as a drastic change in sleeping position, and finally The total number of violent changes in sleeping position is counted as K;

In this embodiment, the face detection adopts a relatively mature algorithm at present, such as the V&J algorithm, which mainly uses the haar feature and the Adaboost classifier.

In this embodiment, the detected face area is taken as the ROI (interesting) area of the infrared temperature measurement module, H∈R ^{rm *rn} , where rm and rn are the width and height of the ROI area, respectively, because the infrared temperature measurement module The temperature in the obtained infrared image will affect the gray value in the H image. Therefore, the period of estimating the temperature change of the nose can be converted into the period of estimating the gray value change of the infrared image; since the video recording frame rate is usually 30fps, every 3 A frame is taken and processed according to the following strategy:

a. Find the average gray value of the ROI area at time k:

b. Calculate the period of the average gray value change in the ROI area. Since human respiration is a period-like physiological activity under normal circumstances, the change in the average gray value is also the corresponding period-like change, so we can calculate the average gray value by calculating the average gray value. The two peaks (or troughs) of the degree value are used to estimate the cycle of respiration;

c. However, due to the existence of noise in the environment and other reasons, in order to obtain the peaks and valleys caused by breathing instead of small peaks and valleys caused by noise or other reasons, Kalman filtering is introduced here to reprocess the obtained average gray value:

z ^k =AS ^k +υ ^k (1)

P ^k|k-1 =AP ^k-1|k-1 A ^T +Q (3)

K ^k =P ^k|k-1 A ^T (AP ^k|k-1 A ^T +R) ^-1 (4)

P ^k|k = (IK ^k A)P ^k|k-1 (6)

是k时刻的先验状态估计值，

分别是k-1和k时刻的后验状态估计值；P^k|k-1是k时刻的先验估计协方差，P^k-1|k-1、P^k|k分别为是k-1、k时刻的后验状态估计协方差；Q为过程噪音协方差矩阵；K^k为卡尔曼增益；Among them, A is the measurement state transition matrix, υ ^k ~ N(0, R) is the observed noise that satisfies the Gaussian distribution, R is the measurement noise covariance, which can generally be observed and is the known condition of the filter; z ^k is The observed value at time k;

is the prior state estimate at time k,

are the posterior state estimates at time k-1 and k, respectively; P ^k|k-1 is the a priori estimated covariance at time k, and P ^k-1|k-1 and P ^k|k are k-1 respectively , the posterior state estimation covariance at time k; Q is the process noise covariance matrix; K ^k is the Kalman gain;

For a set of data obtained continuously, the peaks are obtained according to the following strategy:

且

则k时刻

为第l个波峰点记作C^l＝k；If at time k, satisfy

and

then k time

Denote the lth peak point as C ^l =k;

The time difference between two adjacent peaks is the breathing cycle calculated from the first peak point, that is, T ^l =C ^l -C ^l-1 ;

Set the normal breathing cycle range [T _low ,T _high ] to judge abnormal breathing:

If none of the consecutive n calculated breathing cycles are within the range, it is determined that the breathing is abnormal. The value of n will affect the accuracy of judgment. If n is too small, misjudgment is prone to occur. If n is too large, the judgment time will be delayed. In severe cases, the treatment time may be missed.

In this embodiment, in terms of the micro-pressure sensing module, if the pressure change value between the two breaths is less than the preset value, it is determined that "the user's breathing is normal", and the breathing data continues to be monitored and analyzed; when the two breaths are When the pressure change value between the two is the preset value or exceeds the preset value, and the next pressure value from the micro pressure sensor is not received within the normal breathing time interval, the current time t is recorded, combined with infrared temperature measurement and sleep posture classification If the result of the sleeping posture classifier is Ψ(t)≠Ψ(t ^- ), that is, the user has actions such as turning over, it will determine the "breathing interference action", and continue to monitor and analyze the breathing data; if the imaging module does not display the user If a breathing interference action occurs, it is determined that "user sleep apnea". No matter what sleeping position a person adopts during sleep, they can identify and recognize the heat changed by breathing.

In this embodiment, the methods of alarming in emergencies are as follows: vibrate the user to wake up, notify the user's family members through the bound mobile phone, and notify the ambulance unit such as the hospital where they are located through the network. First aid is carried out by any one or more of the above-mentioned alarm methods;

In the present invention, the monitoring mode of remote monitoring, infrared imaging and micro-pressure sensing is adopted, so that the subject can have multiple variables monitored in one monitoring, so as to realize safe, effective and interference-free monitoring. Respiratory status during sleep is monitored.

The invention also adopts a micro pressure sensor for monitoring to prevent the occurrence of special situations such as: the head is covered by a quilt during sleep, which affects the recognition function of the camera. In addition, the operation and monitoring system of the present invention has no contact with the subject, and the pressure sensor adopts a flexible material, so that the subject is in a natural sleep state, so that the monitoring result is more accurate.

The above-mentioned embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited by the above-mentioned embodiments, and any other changes, modifications, substitutions, combinations, The simplification should be equivalent replacement manners, which are all included in the protection scope of the present invention.

Claims (7)

1. a non-interference type sleep apnea early warning system, is characterized in that, comprises: control center, infrared temperature measurement module, micro pressure sensing module, alarm module and user terminal; The infrared temperature measurement module, the micro pressure sensing module, the alarm module and the user terminal are all connected with the control center; The infrared temperature measurement module is used to collect thermal imaging images, the micro pressure sensing module is used to collect pressure data, the alarm module is used to transmit an alarm signal of apnea, and the user terminal is used to display real-time pressure data , real-time thermal imaging images and alarm information; The control center is used for receiving thermal imaging images and pressure data, tracking the thermal imaging image stream, classifying and recording sleeping positions, and judging whether breathing interference occurs; The thermal imaging image stream is subjected to target tracking, a BP neural network is constructed to perform sleep posture classification training, and a sleep posture classifier is obtained, and the sleep posture classifier classifies and records the sleep posture; The sleeping posture classifier judges whether the sleeping posture at the current time is the same as the sleeping posture at the previous moment, and if it is not the same, it is judged that a breathing interference action occurs; if there is no breathing interference action, it is judged that sleep apnea occurs, and an alarm signal is output. ; The thermal imaging image stream is used for target tracking, the detected face region is used as the region of interest of the infrared temperature measurement module, the average gray value of the region of interest is obtained, the average gray value is subjected to Kalman filtering, and the filtered data is calculated. The time of adjacent peaks is regarded as the breathing cycle; the number of breathing cycles is set to n, if the breathing cycles obtained by consecutive n calculations are beyond the set normal breathing cycle range, it is determined that the breathing is abnormal; The Kalman filter is performed on the average gray value, and the specific calculation formula is: z ^k =AS ^k +υ ^k

P ^k|k-1 =AP ^k-1|k-1 A ^T +Q K ^k =P ^k|k-1 A ^T (AP ^k|k-1 A ^T +R) ^-1

P ^k|k =(IK ^k A)P ^k|k-1 Among them, A represents the measurement state transition matrix, υ ^k ~N(0, R) represents the observed noise that satisfies the Gaussian distribution, R represents the measurement noise covariance, z ^k represents the observed value at time k,

represents the prior state estimate at time k,

represent the posterior state estimates at time k-1 and k, respectively; P ^k|k-1 represents the prior estimated covariance at time k, and P ^k-1|k-1 and P ^k|k represent k-1, The posterior state estimation covariance at time k; Q represents the process noise covariance matrix; K ^k represents the Kalman gain, and ^Sk represents the average gray value of the region of interest; Judging and analyzing the pressure value detected by the micro-pressure sensing module, determining whether the user's breathing is normal according to the pressure change value between two breaths, and combining the breathing interference action to determine whether sleep apnea is suspended, and outputting an alarm signal. 2 . The non-interference sleep apnea early warning system according to claim 1 , further comprising a camera module, the camera module is connected to the control center, and the camera module is used to collect camera video stream data. 3 . 3 . The non-interference sleep apnea early warning system according to claim 1 , wherein the micro-pressure sensor of the micro-pressure sensing module adopts a flexible material. 4 . 4. The non-interfering sleep apnea early warning system according to claim 1, wherein the sleeping posture classifier classifies and records the sleeping posture, and the specific steps include: Set the user's sleep posture set as C={C ₁ , C ₂ , C ₃ }={ supine, left side, right side}, and establish a corresponding data set; The BP neural network is trained using the corresponding data set to obtain a sleeping posture classifier; The sleeping posture classifier outputs the judgment result. When the sleeping posture at the current time is different from the sleeping posture at the previous moment, the time t when the sleeping posture changes is stored, and finally the ordered vector of the sleeping posture changing time in the sleeping process is obtained. S _all = [t ₁ , t ₂ , ... t _N ]. 5. The non-interference type sleep apnea early warning system according to claim 1, wherein the acquisition of the average gray value of the region of interest, the specific calculation formula is:

Among them, rm and rn are the width and height of the region of interest, respectively, and k represents any time. 6. The non-interference type sleep apnea early warning system according to claim 1, wherein the time of the adjacent peaks of the data after the calculation is calculated as a breathing cycle, and the peak acquisition step comprises: At the kth moment, if the

and

then k time

Denote the lth peak point as C ^l =k; The time difference between two adjacent peaks is the breathing cycle calculated from the first peak point, that is, T ^l =C ^l -C ^l-1 ; The normal breathing cycle range is set to [T _low ,T _high ]. 7. non-interference sleep apnea early warning system according to claim 1, is characterized in that, described output alarm signal specifically adopts any one or more in following alarm mode: The method of vibrating and waking up the user, notifying the user's family members through the bound mobile phone, and notifying the local ambulance unit through the network.

Images