WO2020045709A1 - Sleep measurement device, and sleep measurement system having same - Google Patents

Abstract

The present invention relates to a sleep measurement device, and a sleep measurement system having same. A sleep measurement device, according to an embodiment of the present invention, comprises: a radar sensor for receiving a radar signal; a microphone for receiving an audio signal; and a processor for detecting whether a sleeper is snoring and suffering from apnea on the basis of the audio signal from the microphone, detecting the sleeper's movement, respiration, and heart rate signal on the basis of the radar signal, and calculating sleep efficiency on the basis of the sleeper's movement, respiration, and heart rate signal, and data on the presence or absence of snoring that is calculated on the basis of the audio signal from the microphone. Accordingly, it is possible to accurately calculate the sleep efficiency by using a composite sensor.

Description

Sleep measurement apparatus, and sleep measurement system having same

The present invention relates to a sleep measurement device, and a sleep measurement system having the same, and more particularly, to a sleep measurement device that can accurately calculate the sleep efficiency using a complex sensor, and a sleep measurement system having the same.

The sleep measurement device is a device that measures the sleep state of the sleeper.

Recently, with the advent of IOT devices, a method for measuring a sleep state of a sleeper has been studied.

Wearable sleep measurement apparatus of the band type, there is an inconvenience that the sleeper wears the wearable device, rather there is an element that interferes with sleep.

Accordingly, research on a non-contact type sleep measurement device is in progress.

It is an object of the present invention to provide a sleep measurement apparatus capable of accurately calculating sleep efficiency using a complex sensor, and a sleep measurement system having the same.

Another object of the present invention is to provide a sleep measurement apparatus capable of calculating a sleep stage or a sleep score using a non-contact composite sensor, and a sleep measurement system having the same.

Sleep measurement apparatus according to an embodiment of the present invention, and a sleep measurement system having the same, to achieve the above object is based on a radar sensor for receiving a radar signal, a microphone for receiving an audio signal, and the audio signal from the microphone Detects whether the sleeper is snoring and apnea, detects the sleeper's movement, breathing, and heart rate signals based on the radar signal, calculates snoring data calculated on the basis of an audio signal from the microphone, and sleep And a processor for calculating sleep efficiency based on the movement, respiration, and heart rate signals of the child.

On the other hand, when the calculated sleep efficiency is greater than or equal to the first reference value, the processor may calculate the sleep stage of the sleeper.

Meanwhile, the processor may calculate a sleep score based on the calculated sleep efficiency, sleep stage, snoring data, and apnea data.

Meanwhile, the processor extracts the snoring candidate event from the audio signal from the microphone, extracts the snoring feature based on the extracted snoring candidate event, and performs the learning based classification based on the extracted snoring feature. According to, it can calculate whether or not snoring.

On the other hand, if the sleeper is calculated as a snoring, the apnea feature may be extracted from the audio signal from the microphone, and based on the extracted apnea feature, the processor may calculate apnea according to the learning-based classification performed. .

On the other hand, the processor classifies whether the sleeper wakes or sleeps based on the detected sleeper's movement, breathing, and heart rate signals, and when the sleeper sleeps, the processor determines whether it is a rapid eye movement (REM) sleep state, Or you can classify whether you are in a non-REM sleep state.

On the other hand, when the sleeper is in a berm sleep state, the processor may calculate the sleep stage of the sleeper.

On the other hand, the processor detects the sleeper based on the radar signal, and can control such that the period of the output radar signal output when the sleeper is absent increases as compared with when the sleeper is present.

In the meantime, when the power supply unit includes the battery, the processor may control the cycle of the output radar signal to be output according to the power level of the battery.

On the other hand, the processor may perform noise canceling for a certain noise of the audio signal from the microphone, and detect whether the sleeper is snoring and apnea based on the noise canceled audio signal.

Sleep measurement apparatus according to an embodiment of the present invention, and a sleep measurement system having the same, the snoring of the sleeper based on the radar sensor for receiving the radar signal, the microphone for receiving the audio signal, and the audio signal from the microphone Whether or not apnea is detected, and based on the radar signal, the sleeper's movement, breathing, and heartbeat signals are detected, and the snoring data calculated based on the audio signal from the microphone, and the sleeper's movement, breathing, and heart rate. And a processor that calculates sleep efficiency based on the signal. Accordingly, it is possible to accurately calculate the sleep efficiency using the complex sensor.

On the other hand, when the calculated sleep efficiency is greater than or equal to the first reference value, the processor may calculate the sleep stage of the sleeper. Accordingly, the sleep stage can be accurately calculated using the non-contact composite sensor.

Meanwhile, the processor may calculate a sleep score based on the calculated sleep efficiency, sleep stage, snoring data, and apnea data. Accordingly, the sleep score can be accurately calculated using the non-contact composite sensor.

Meanwhile, the processor extracts the snoring candidate event from the audio signal from the microphone, extracts the snoring feature based on the extracted snoring candidate event, and performs the learning based classification based on the extracted snoring feature. According to, it can calculate whether or not snoring. Accordingly, it is possible to accurately calculate whether the snoring using a non-contact composite sensor.

On the other hand, if the sleeper is calculated as a snoring, the apnea feature may be extracted from the audio signal from the microphone, and based on the extracted apnea feature, the processor may calculate apnea according to the learning-based classification performed. . Accordingly, it is possible to accurately calculate whether apnea using a non-contact composite sensor.

On the other hand, the processor classifies whether the sleeper wakes or sleeps based on the detected sleeper's movement, breathing, and heart rate signals, and when the sleeper sleeps, the processor determines whether it is a rapid eye movement (REM) sleep state, Or you can classify whether you are in a non-REM sleep state. Accordingly, the non-contact composite sensor can be used to accurately calculate the sleeping phase.

On the other hand, when the sleeper is in a berm sleep state, the processor may calculate the sleep stage of the sleeper. Accordingly, the sleep stage can be accurately calculated using the non-contact composite sensor.

On the other hand, the processor detects the sleeper based on the radar signal, and can control such that the period of the output radar signal output when the sleeper is absent increases as compared with when the sleeper is present. As a result, unnecessary power consumption can be reduced.

In the meantime, when the power supply unit includes the battery, the processor may control the cycle of the output radar signal to be output according to the power level of the battery. Thereby, power consumption can be reduced.

On the other hand, the processor may perform noise canceling for a certain noise of the audio signal from the microphone, and detect whether the sleeper is snoring and apnea based on the noise canceled audio signal. Accordingly, it is possible to accurately detect whether the sleeper is snoring and whether apnea is present.

1 is a view showing a sleep measurement system having a sleep measurement apparatus according to an embodiment of the present invention.

FIG. 2 is an example of an internal block diagram of the sleep measurement apparatus of FIG. 1.

3 is another example of an internal block diagram of the sleep measurement apparatus of FIG. 1.

4 is an example of a flowchart illustrating a method of operating the sleep measurement apparatus of FIG. 1.

5A to 5B are views referred to for describing the operating method of FIG. 4.

6 is an example of a flowchart illustrating a method of operating the sleep measurement apparatus of FIG. 1.

7A to 7B are views referred to for describing the operating method of FIG. 6.

8 is an example of an internal block diagram of the mobile terminal of FIG. 1.

9A to 9C are views illustrating various information displayed on the mobile terminal of FIG. 1.

Hereinafter, with reference to the drawings will be described the present invention in more detail.

The suffixes "module" and "unit" for components used in the following description are merely given in consideration of ease of preparation of the present specification, and do not impart any particular meaning or role by themselves. Therefore, the "module" and "unit" may be used interchangeably.

1 is a view showing a sleep measurement system having a sleep measurement apparatus according to an embodiment of the present invention.

Referring to the drawings, the sleep measurement system 10 according to the embodiment of the present invention of FIG. 1 may include a sleep measurement apparatus 100, a terminal 600, and a server 200.

The sleep measurement apparatus 100 according to the embodiment of the present invention is a sleep measurement apparatus using a non-contact composite sensor and calculates a sleep efficiency of the sleeper USR.

To this end, the sleep measurement apparatus 100 is preferably disposed in a space where the sleeper USR sleeps. For example, it may be placed on a bed or on a separate table spaced from the bed.

On the other hand, the sleep measurement apparatus 100 may be provided in another home appliance. For example, it is possible to be equipped with an air purifier, an air conditioner, a TV, a telephone, a clock, a speaker, a lighting device, and the like.

Sleep measurement apparatus 100 according to an embodiment of the present invention, based on the radar sensor 130 for receiving a radar signal, the microphone 135 for receiving an audio signal, and the audio signal from the microphone 135, Detecting snoring and apnea of the sleeper, detecting movement of the sleeper, breathing, and heart rate signals based on the radar signal, and calculating snoring data calculated based on an audio signal from the microphone 135; And a processor 170 that calculates sleep efficiency based on the sleeper's movement, breathing, and heart rate signals. Accordingly, it is possible to accurately calculate the sleep efficiency using the complex sensor.

On the other hand, when the calculated sleep efficiency is greater than or equal to the first reference value, the processor 170 may calculate the sleep stage of the sleeper. Accordingly, the sleep stage can be accurately calculated using the non-contact composite sensor.

The processor 170 may calculate a sleep score based on the calculated sleep efficiency, sleep stage, snoring data, and apnea data. Accordingly, the sleep score can be accurately calculated using the non-contact composite sensor.

Meanwhile, the terminal 600 may perform wireless data communication with the sleep measurement apparatus 100 and output information such as sleep efficiency received from the sleep measurement apparatus 100 through a display or an audio output unit. .

Meanwhile, the terminal 600 may output sleep state information, sleep stage information, sleep score information, snoring information, apnea information, etc. received from the sleep measurement apparatus 100.

On the other hand, the server 200 may receive the user-specific, sleep efficiency information, and the like, compare them, and make a database.

Meanwhile, the server 200 may receive, compare, and database each user, sleep state information, sleep stage information, sleep score information, snoring information, apnea information, and the like.

FIG. 2 is an example of an internal block diagram of the sleep measurement apparatus of FIG. 1.

Referring to the drawings, the sleep measurement apparatus 100 according to an embodiment of the present invention, the radar sensor 130, the microphone 135, the communication unit 145, memory 140, audio output unit 160, processor ( 170, and a power supply 190.

The radar sensor 130 may output the output radar to the outside and receive the radar signal SLar reflected by the external object. The received radar signal SLar may be transmitted to the processor 170.

In this case, the radar signal may be a continuous wave (CW) radar signal or a frequency modulated continuous wave (FMCW) radar signal.

The microphone 135 may collect or receive the audio signal Sau. The received audio signal Sau may be transmitted to the processor 170.

The communication unit 145 may provide an interface for communicating with an external device. To this end, the communication unit 145 may include at least one of a mobile communication module (not shown), a wireless Internet module (not shown), a short range communication module (not shown), a GPS module (not shown), and the like.

For example, the communication unit 145 may perform Bluetooth communication or WiFi communication, thereby transmitting information about the sleep measured by the processor 170 to the paired terminal 600. At this time, the terminal 600 may be a mobile terminal such as a smart phone capable of a voice call.

The information on sleep may include sleep efficiency information, sleep stage information, sleep score information, snoring information, apnea information, and the like.

The memory 140 may store a program for processing or controlling the processor 170 in the sleep measurement apparatus 100, or may perform a function for temporarily storing input or output data.

In particular, the memory 140 may store information about sleep measured by the processor 170 in the sleep measurement apparatus 100, together with time information. Furthermore, user information can also be stored.

The audio output unit 160 may output information about the measured sleep as audio. Alternatively, the position adjustment message for the sleep measurement apparatus 100 may be output as audio or a notification message may be output as audio.

The processor 170 may control the operation of each unit in the sleep measurement apparatus 100 to control the overall operation of the sleep measurement apparatus 100.

For example, the processor 170 detects whether the sleeper is snoring and apnea based on the audio signal Sau from the microphone 135, and based on the radar signal, moves the breather, breathing, The heart rate signal may be detected, and the sleep efficiency may be calculated based on the snoring data calculated based on the audio signal from the microphone 135 and the movement, breathing, and heart rate signals of the sleeper. Accordingly, it is possible to accurately calculate the sleep efficiency using the complex sensor.

To this end, the processor 170, based on the audio signal Sau from the microphone 135, the movement of the sleeper USR, the breath, the heartbeat signal, the breath, the heartbeat signal detector 171, The sleep efficiency detector 172 may calculate a sleep efficiency based on the movement, respiration, and heart rate signals of the sleeper USR from the movement, respiration, and heart rate signal detector 171.

In particular, the sleep efficiency detector 172 calculates sleep efficiency based on snoring data calculated based on the movement, respiration, and heart rate signals of the sleeper USR, as well as the audio signal from the microphone 135. can do.

On the other hand, when the calculated sleep efficiency is equal to or greater than the first reference value, the processor 170 may calculate the sleep stage of the sleeper USR. Accordingly, the sleep stage can be accurately calculated using the non-contact composite sensor.

To this end, the processor 170 may further include a sleep detector 173 that calculates a sleep stage based on the sleep efficiency calculated by the sleep efficiency detector 172.

The processor 170 may calculate a sleep score based on the calculated sleep efficiency, sleep stage, snoring data, and apnea data. To this end, the processor 170 may further include a sleep score calculator 175 that calculates a sleep score based on the calculated sleep efficiency, sleep stage, snoring data, and apnea data. Accordingly, the sleep score can be accurately calculated using the non-contact composite sensor.

Meanwhile, the processor 170 extracts the snoring candidate event from the audio signal Sau from the microphone 135, extracts the snoring feature based on the extracted snoring candidate event, and extracts the extracted snoring feature. Based on the learning-based classification that is performed, it can be calculated whether or not snoring. Accordingly, it is possible to accurately calculate whether the snoring using a non-contact composite sensor.

To this end, the processor 170 may calculate whether to snore based on the candidate snoring signal detector 176 that detects the candidate snoring signal from the audio signal Sau from the microphone 135, and the candidate snoring signal. A snoring detector 177 and an apnea detector 178 that detects apnea based on snoring data from the snoring detector 177 may be further provided.

Meanwhile, snoring data calculated by the snoring detector 177 may be input to the sleep efficiency detector 172.

On the other hand, when the sleeper USR is calculated as snoring, the processor 170 extracts an apnea feature from the audio signal Sau from the microphone 135 and performs learning based on the extracted apnea feature. According to the base classification, it can be calculated whether or not apnea. Accordingly, it is possible to accurately calculate whether apnea using a non-contact composite sensor.

Meanwhile, the processor 170 classifies whether the sleeper USR is in a wake state or a sleep state based on the detected sleeper USR movement, respiration, and heart rate signals. Rapid Eye Movement and REM sleep or non-REM sleep. Accordingly, the non-contact composite sensor can be used to accurately calculate the sleeping phase.

On the other hand, when the sleeper USR is in a berm sleep state, the processor 170 may calculate the sleep stage of the sleeper USR. Accordingly, the sleep stage can be accurately calculated using the non-contact composite sensor.

On the other hand, the processor 170 detects the sleeper USR based on the radar signal SLar, and when the sleeper USR is absent, the period of the output radar signal outputted when the sleeper USR is present. In comparison, it can be controlled to increase. As a result, unnecessary power consumption can be reduced.

Meanwhile, when the power supply unit 190 includes a battery, the processor 170 may control the cycle of the output radar signal to be output according to the power level of the battery. Thereby, power consumption can be reduced.

Meanwhile, the processor 170 performs noise canceling on a certain noise among the audio signals Sa from the microphone 135, and based on the noise canceled audio signal Sa, the snoring of the sleeper USR is snoring. Whether or not apnea can be detected. As a result, it is possible to accurately detect whether the sleeping person USR is snoring and whether apnea is present.

The power supply unit 190 may supply power required for the operation of each component under the control of the processor 170.

On the other hand, the power supply unit 190 may be provided with a battery.

3 is another example of an internal block diagram of the sleep measurement apparatus of FIG. 1.

Referring to the drawing, the sleep measurement apparatus 100b of FIG. 2 is similar to the sleep measurement apparatus 100 of FIG. 2, but an audio converter 162 is disposed between the microphone 135 and the processor 170. The difference between the radar sensor 130 and the processor 170 is that the signal amplifier 164 and the converter 157 are further provided.

The audio converter 162 may perform preprocessing on the audio signal Sau from the microphone 135.

Alternatively, the audio converter 162 may perform encoding processing on the audio signal Sau from the microphone 135. Accordingly, the audio converter 162 may perform encoding processing on the audio signal Sau using the stored audio codec.

The signal amplifier 164 may amplify the radar signal SLar from the radar sensor 130. To understand this, the signal amplifier 164 may include an OP AMP.

Next, the converter 167 may convert the radar signal SLar from the signal amplifier 164 into a digital signal. To this end, the converter 167 may include an ADC converter.

The processor 170 may detect snoring based on the audio signal Sau from the microphone 135.

To this end, the candidate snoring signal detector 176 in the processor 170 may detect an event on a predetermined time basis from the audio signal Sau from the sleeper USR.

The candidate snoring signal detector 176 in the processor 170 may classify the snoring event and the non-snoring event among the detected events.

On the other hand, about 90% of sleepers (USR) complaining of apnea symptoms have snoring symptoms, and therefore, a high-performance snoring detection technique is required to detect apnea symptoms.

Accordingly, snoring event detection is essential for apnea symptoms detection, sleep stage calculation, and the like.

Meanwhile, the candidate snoring signal detector 176 in the processor 170 performs noise canceling on various noises that may occur during sleep, such as a tossing sound of the sleeper USR, a refrigerator sound, a fan sound, and snoring. It is desirable to distinguish only events.

On the other hand, when snoring events are detected, rather than a Gaussian mixture model, in the present invention, separate deep neural networks for snoring events and non-snoring events are modeled.

The candidate snoring signal detector 176 in the processor 170 removes background noise using a pre-trained DD model for removing background noise in the sleeping environment, and generates an event based on the signal from which the background noise has been removed. To detect, the energy values for all frames can be found.

In addition, the candidate snoring signal detector 176 in the processor 170 may calculate an histogram after obtaining energy values for all frames, and calculate a threshold value based on the calculated values.

The threshold value at this time may be used to accurately locate the start and end points of the event.

On the other hand, the candidate snoring signal detector 176 in the processor 170 uses a statistical length feature of the snoring event to detect an event, using a predetermined length, for example, 200 ms or more and 3500 ms or less. The remaining events except for the event can be removed by assuming snoring.

On the other hand, there are three factors for snoring event classification, energy, spectral, time.

There are seven kinds of energy-related feature vectors: symmetry and asymmetry for amplitude and energy distribution on the time axis, area for envelope of normalized energy signal, volume density, and slope between first frame and maximum. .

In addition, there are 21 feature vectors related to the spectral, and the median of each order of the 20th order MFCC (mel-frequency cepstral coefficient) obtained for all frames after constructing 16 ms and 50% overlapped frames is used as the feature vector. In this case, only the first to the sixteenth order can be used as the feature vector.

Based on this, dynamic MFCC (DMFCC) distance, spectral flux, frequency centroid can be calculated.

Meanwhile, the difference in frequency centroid between the first half and the other half of an event can be used as a feature vector.

Finally, pitch, pitch force, pitch density may be illustrated as the feature vectors associated with the spectral.

On the other hand, a total of six feature vectors are extracted based on a current event, for example, a RP (rhythm period) and a PS (period strength) before and after a predetermined time, for example, about 10 seconds. The ratio of the feature vectors can be used as the feature vector. Finally, ZCR (zero crossing rate) can be extracted and used for learning.

In the present invention, an eLU function may be used as an activation function.

On the other hand, since the snoring detector 177 considers two cases of the presence and absence of snoring, the output layer of the deep neural network is composed of two nodes, and the target value may be for the presence and absence of the voice. have.

On the other hand, after the feed forward to the output layer, cross entropy between the estimated value and the target value can be used in the output layer to proceed the supervised learning using the back propagation, the parameters of the deep neural network can be updated.

Meanwhile, in order to adaptively learn the deep neural network, estimation through adadelta may be performed. In addition, the probability value may be calculated in the output layer using the sigmoid.

To this end, the slope parameter and the bias parameter of the sigmoid function for estimating the voice presence probability may be used, respectively.

On the other hand, the two parameters of the sigmoid function for estimating the probability of snoring events can be obtained through differential learning to minimize the negative log-likelihood (cross entropy error function).

On the other hand, the deep neural network is learned separately for the snoring event and non-snoring event, the parameters of the sigmoid function for estimating the probability of snoring event may also be learned separately.

Finally, the candidate snoring signal detector 176 in the processor 170 compares the difference between the two probabilities and a specific threshold to classify the snoring as an event if it is greater than the threshold and if the snoring is small, Can be classified as an event.

On the other hand, the snoring detection technique by this method, as a snoring detection technique based on the deep neural network, when compared with the Gaussian mixture model, it will lead to better classification performance.

In particular, the distribution of the likelihood ratio values of the snoring detection technique based on the deep neural network proposed by the present invention is better compared to the distribution of the likelihood ratio values of the snoring detection technique based on the Gaussian mixture model.

As a result, the snoring detection based on the deep neural network proposed in the present invention may be largely divided into a learning step and a classification step.

In the processor 170, the candidate snoring signal detector 176 removes the background noise through the DD in the learning step, and then detects the event in units of a predetermined period based on the energy of the input signal.

In addition, the candidate snoring signal detector 176 in the processor 170 extracts a feature vector associated with energy, a feature vector associated with spectral, and a feature vector associated with time, before entering full-scale learning.

The candidate snoring signal detector 176 in the processor 170 uses the extracted feature vectors to individually train the deep neural network model for snoring and snoring, respectively, which normalizes the feature vector immediately before learning. Go through the process.

On the other hand, the candidate snoring signal detector 176 in the processor 170 skips the previous learning process and learns through a back-propagation algorithm in the fine tuning process.

On the other hand, the candidate snoring signal detector 176 in the processor 170, using a rectified linear unit (ReLU) to solve the problem of the slope disappearance of the activation function for each node, the learning rate in the advanced learning step of the deep neural network You can use adadelta to set this adaptively.

 In the processor 170, the snoring detector 177, in the classification step, removes the background noise with respect to the input signal in the same manner as in the learning step, detects an event through the event detector, and based on the event, the feature vector. After estimating the likelihood ratio value through the deep learning neural network model and the non-snoring deep neural network model previously detected for snoring detection, snoring and snoring can be classified based on a specific threshold value.

Accordingly, the snoring detector 177 in the processor 170 may classify the snoring event as the likelihood event when the likelihood ratio value is less than the specific threshold value and classify the snoring event as the snoring event.

The processor 170 may calculate a sleep stage and detect wake, NREM, or REM sleep based on sleep-related signals extracted from the sleeper USR to detect the sleep stage.

In the present invention, using the continuous wave Doppler radar sensor 130 as the main axis, the processor 170 extracts sleep-related movement, breathing, heart rate-related features, randomized for sleep phase classification A forest classifier is used, and the microphone 135 is used as an auxiliary sensor, and situational recognition technology is applied to greatly improve sleep stage detection performance.

On the other hand, when using the radar sensor 130 as a single sensor, when calculating the sleep stage, the ability to recognize the surrounding situation, which is an important factor in improving the sleeper (USR) sleep stage detection performance, is poor. Do. Accordingly, in the present invention, the sleep stage is calculated using the composite sensor, in particular, the radar sensor 130 and the microphone 135.

To this end, the processor 170 may perform preprocessing on the continuous wave radar signal SLar and the audio signal Sau.

For example, the radar signal SLar transmitted in the time domain may include continuous wave and phase noise of a carrier frequency of 24 GHz generated from a voltage controlled oscillator (VCO).

The Doppler signal reflected from the body of the sleeper USR may be sampled, and the sampled signal may include movement information, respiration information, and heartbeat information of the sleeper USR.

The radar sensor 130 may include a quadrature receiver antenna. The radar signal SLar received through the antenna is divided into a baseband signal, an in-phase signal, and a quadrature signal, which are characterized by a 90 ° phase difference.

The audio signal Sau in the time domain measured from the microphone 135 is sampled at 16000 Hz, and the sampled signal may include an audio signal and noise having sleep state data of the sleeper USR.

The processor 170 may remove noise from the input audio signal Sau by using a data-driven technique.

On the other hand, the processor 170 frames the radar signal SLar extracted from the sleeper USR during a sleep time in units of a predetermined period (for example, 30 seconds), and the radar signal SLar in units of frames. To remove the DC offset, we can remove the mean value of the raw signal.

The processor 170 extracts signals related to movement, respiration, and heartbeat by using three Butterworth filters having different bandwidths, and then min, max, mean, rms, median, 1st-quartile, and 3rd from the signals. -Quartile, standard deviation, interquartile range, number of peaks, average distance between peaks, peak average height, zero cross rate, linear regression slope are extracted. The maximum peak can be extracted by converting the respiratory and heartbeat signals into the frequency domain.

On the other hand, the processor 170 may extract skewness and kurtosis from the raw signal.

Meanwhile, the processor 170 extracts 46 features for every frame for each radar signal (SLar) frame, and then applies singular value decomposition to this matrix to add column vector features located at three maximum eigenvalues for each frame. In addition, principal component analysis can also be applied to add column vector features located at three maximum eigenvalues, frame by frame. Accordingly, 52 feature vectors may be generated per frame.

Meanwhile, the processor 170 may detect sleep related movements and the absence of the sleeper USR.

On the other hand, the processor 170 is an algorithm (WRMEnAD, wake) for detecting the absence of the sleeper (USR), the sleeper (USR) shows a movement meaning a wake state during sleep, or out of the sleeping space such as sleepwalking In order to implement -related movement existence and absence detection, the level cross rate feature is extracted on the frame of the raw radar signal SLar from which only DC-offset is removed, based on the heuristic threshold determined by the experiment. ) Feature can be detected.

Meanwhile, the processor 170 may determine the number of snoring events based on the snoring signal based on a frame unit for a predetermined period (for example, 30 seconds) from the audio signal Sau extracted from the sleeper USR during sleep. And detect respiratory signal based cycle intensity (CI) features.

On the other hand, the processor 170, in order to detect the CI, calculates a spectrogram on the noise-free audio signal (Sau), calculates the time-domain autocorrelation for the signal corresponding to each frequency bin, and as a result The first peak amplification can be defined as the CI for the frame by sorting them in order of magnitude, based on the amplitude of the first peak in.

Meanwhile, the processor 170 may first perform wake / sleep classification of the sleeper USR before classifying the sleep stage as wake, NREM, or REM sleep.

The processor 170 selects 44 feature vectors selected as a result of the distribution analysis for the wake / sleep class from the 52 radar signal SLar based features previously extracted for the wake / sleep classification. In addition, the processor 170 performs normalization of a standardization method for the feature vector.

The result of passing the normalized 44 feature vectors through the wake / sleep binary random forest classifier outputs the probability of wake and sleep for each frame. The processor 170 performs probability smoothing based on these output values. Proceed. However, the first and second frames, that is, the first minute, are assumed to be awake state unconditionally.

Meanwhile, the processor 170 may calculate the wake / sleep likelihood ratio for the entire frame after the probability smoothing is performed, and may know the author.

The processor 170 may calculate a wake / sleep personal-adjusted threshold.

Personal adjusted threshold is a kind of adaptive threshold for optimizing the algorithm for all people because sleep pattern and biosignal pattern are different for each person, and this is the main parameter to be used for final wake / sleep decision.

In order to obtain a personal-adjusted threshold, the processor 170 sorts the probability ratio for each class output from the random forest classifier in ascending order, and then based on the specific percentile parameter obtained through grid-search, The likelihood ratio for that percentile is set to the personal-adjusted threshold.

Meanwhile, before applying the NREM / REM random forest classifier, the processor 170 may improve the wake / sleep detection performance by applying context awareness technology based on the extracted features.

This technique is designed as a result of comparing the distribution of the number of snore events for the wake / sleep class. Therefore, when the wake / sleep random forest classifier decides to wake up in a specific frame, it is corrected to sleep.

Meanwhile, the processor 170 passes the NREM / REM random forest classifier for the frames determined as the sleep class.

The radar feature vectors used in the NREM / REM classifier are 35 feature vectors selected from the analysis of the distribution of 52 feature vectors for NREM and REM classes.

On the other hand, based on an experimental result that no REM sleep occurs for a first period of time, for example, 50 minutes, the processor 170 uses a threshold that lowers the probability of REM detection for a first period of time (50 minutes). After the period, the percentage parameter determined by grid-search can be used.

Meanwhile, the processor 170 classifies wake and sleep based on previously stored wake / sleep related parameters to determine a final sleep stage, and then performs NREM / REM class determination on the frames determined to be sleep.

The processor 170 determines a sleep class when the likelihood ratio of a frame is greater than a wake / sleep personal adjusted threshold for a final determination of wake / sleep, and determines a wake when the frame is small.

Meanwhile, the processor 170 assumes that the sleeper USR is an unconditional wake class for a predetermined period, for example, the first minute.

In this case, when the frame index is larger than a predetermined value (for example, 2), the processor 170 may use the number of snoring events previously extracted to correct an error of a frame designated as a wake class.

On the other hand, when the snoring event number is three or more times, the processor 170 may correct this to sleep. This is determined by the distribution of snoring events for the wake / sleep class.

Meanwhile, the processor 170 may perform the last wake / sleep correction by using the motion-related feature of the sleeper USR previously extracted before determining the frame determined by the Sleep class as the NREM / REM class. have.

For example, when the frame is determined to be sleep but the movement of the facer USR is detected or determined to be absent, the processor 170 may correct the class of the frame to wake.

On the other hand, the processor 170 compares the likelihood ratio based on the NREM / REM personal adjusted threshold of the low REM detection rate when the sleep phase test has been started or less than the first period for the frame determined to be sleep, and the likelihood ratio is large In case of REM, it can be determined by NREM.

Meanwhile, when the first period is exceeded, the processor 170 may determine the NREM or the REM by comparing the likelihood ratio based on the heuristic threshold determined by rid-search.

Meanwhile, the processor 170 may finally perform post-processing before finally determining the sleep stage.

For example, the processor 170 may improve the wake detection performance by applying context awareness technology based on the extracted respiratory signal-based feature CI and the snoring signal-based feature.

In this case, the processor 170 estimates a place where a plurality of CIs are gathered as a wake interval, and then determines a case where the continuous frame length is a predetermined interval (110 to 170) as the wake interval.

 The processor 170 may correct the NREM if the continuous continuation is less than 3 frames in the REM class to reduce the detection error of the NREM and the REM, and correct the adjacent REM frames in one REM sleep interval. .

4 is an example of a flowchart illustrating a method of operating the sleep measurement apparatus of FIG. 1, and FIGS. 5A to 5B are views referred to for describing the method of FIG. 4. In particular, FIGS. 4 to 5B are views referred to for description of the sleep step calculation of the sleeper.

Referring to the figure, the processor 170 detects a movement, respiration, and heart rate signal from the radar signal SLar from the radar sensor 130 (S410).

In addition, the processor 170 receives snoring data calculated based on the audio signal Sau from the microphone 135 (S420), snoring data, movement of the sleeper USR, and breathing. In operation S430, the sleep efficiency is calculated based on the heartbeat signal.

Next, when the calculated sleep efficiency is greater than or equal to the first reference value, the processor 170 calculates a sleep step (S450).

FIG. 5A illustrates detection of the sleeper USR movement signal Wva, the respiration signal Wvb, and the heart rate signal Wvc from the radar signals SLara and SLarb of two channels.

The motion, respiration, and heart rate signal detector 171 in the processor 170 detects the motion signal Wva, the respiration signal Wvb, and the heart rate signal Wvc of the sleeper USR from the radar signals SLara and SLarb. can do.

Next, the wake / sleep classifier 511 in the processor 170 may wake the sleeper USR based on the movement signal Wva, the breathing signal Wvb, and the heartbeat signal Wvc of the sleeper USR. It can be distinguished from the wake state or the sleep state.

Meanwhile, when it is determined that the sleep / slip classifier 511 in the processor 170 is in the sleep state, the motion / absence detector 513 in the processor 170 may move the motion signal Wva of the sleeper USR. , Based on the respiration signal Wvb and the heart rate signal Wvc, it is possible to detect whether the sleeper USR is in a moving state or an absent state.

Next, when it is determined that the REM / NREM classifier 515 in the processor 170 is in the sleep state in the wake / sleep classifier 511, the sleeper USR movement signal Wva and the respiration signal Wvb ), Based on the heartbeat signal Wvc, it is possible to classify whether the sleep state USR is in the REM state or the NREM state.

FIG. 5B is a diagram showing a sleep step in the case where the sleep state of the sleeper USR includes both the REM state and the NREM state.

According to the exemplary embodiment of the present invention, the processor 170 detects whether the sleeper USR is snoring and apnea based on the audio signal Sau from the microphone 135, and detects the radar signal SLar. ) Detects movement of the sleeper USR, breathing, and heart rate signals, and calculates snoring data and movement of the sleeper USR calculated based on the audio signal Sau from the microphone 135. Based on the respiration, heart rate signals, sleep efficiency can be calculated. Accordingly, it is possible to accurately calculate the sleep efficiency using the complex sensor.

On the other hand, when the calculated sleep efficiency is equal to or greater than the first reference value, the processor 170 may calculate the sleep stage of the sleeper USR. Accordingly, the sleep stage can be accurately calculated using the non-contact composite sensor.

The processor 170 may calculate a sleep score based on the calculated sleep efficiency, sleep stage, snoring data, and apnea data. Accordingly, the sleep score can be accurately calculated using the non-contact composite sensor.

On the other hand, when the sleeper USR is in a berm sleep state, the processor 170 may calculate the sleep stage of the sleeper USR.

Meanwhile, the processor 170 classifies whether the sleeper USR is in a wake state or a sleep state based on the detected sleeper USR movement, respiration, and heart rate signals. Rapid Eye Movement and REM sleep or non-REM sleep.

FIG. 6 is an example of a flowchart illustrating a method of operating the sleep measurement apparatus of FIG. 1, and FIGS. 7A to 7B are views referred to for describing the method of FIG. 6. In particular, FIGS. 6 to 7B are views referred to for description of the sleep stage calculation of the sleeper.

Referring to the figure, the processor 170 performs preprocessing on the audio signal Sau from the microphone 135 (S610). For example, the processor 170 may remove noise in the audio signal Sau from the microphone 135 in advance.

Next, the processor 170 detects a snoring candidate event from the audio signal Sau from the microphone 135 (S610).

The processor 170 extracts a snoring characteristic based on the extracted snoring candidate event (S620), and based on the learning-based classification performed based on the extracted snoring characteristic, whether or not the snoring is performed. Can be calculated (S620). Accordingly, it is possible to accurately calculate whether the snoring using a non-contact composite sensor.

Next, the processor 170 may transmit the calculated snoring data for sleep efficiency detection (S630). For example, the snore detector 177 may transmit the calculated snoring data to the sleep efficiency detector 172.

Next, the processor 170 extracts the apnea feature based on the snoring data (S640), and calculates the apnea based on the extracted apnea feature (S650).

7A is a diagram referred to for apnea calculations.

Referring to the figure, the processor 170 performs preprocessing on the audio signal Sau from the microphone 135, and the candidate snoring signal detector 176 in the processor 170, from the microphone 135. A snoring candidate event is detected from the audio signal Sau.

7B is an example of the audio signal Sau 630 of the sleeper USR.

The snoring detector 177 in the processor 170 extracts the snoring feature based on the extracted snoring candidate event, and based on the learning-based classification performed based on the extracted snoring feature. You can operate on this. Accordingly, by using the non-contact composite sensor, it is possible to accurately calculate whether snoring is snoring or not.

Next, when the apnea detector 178 in the processor 170 is computed as snoring, the apnea feature is extracted, and based on the extracted apnea feature, the apnea detector 178 calculates the apnea according to the learning-based classification performed. can do. Accordingly, by using the non-contact composite sensor, it is possible to accurately calculate whether apnea is detected or not apnea.

On the other hand, when the sleeper USR is in a berm sleep state, the processor 170 may calculate the sleep stage of the sleeper USR. Accordingly, the sleep stage can be accurately calculated using the non-contact composite sensor.

On the other hand, the processor 170 detects the sleeper USR based on the radar signal SLar, and when the sleeper USR is absent, the period of the output radar signal outputted when the sleeper USR is present. In comparison, it can be controlled to increase. As a result, unnecessary power consumption can be reduced.

Meanwhile, when the power supply unit 190 includes a battery, the processor 170 may control the cycle of the output radar signal to be output according to the power level of the battery. Thereby, power consumption can be reduced.

Meanwhile, the processor 170 performs noise canceling on a certain noise among the audio signals Sa from the microphone 135, and based on the noise canceled audio signal Sa, the snoring of the sleeper USR is snoring. Whether or not apnea can be detected. As a result, it is possible to accurately detect whether the sleeping person USR is snoring and whether apnea is present.

8 is an example of an internal block diagram of the mobile terminal of FIG. 1.

Referring to the drawings, the mobile terminal 600 includes a wireless communication unit 610, an A / V (Audio / Video) input unit 620, a user input unit 630, a sensing unit 640, an output unit 650, and a memory. 660, an interface unit 625, a control unit 670, and a power supply unit 690.

The wireless communication unit 610 may include a broadcast receiving module 611, a mobile communication module 613, a wireless internet module 615, an acoustic communication unit 617, a GPS module 619, and the like.

The broadcast receiving module 611 may receive at least one of a broadcast signal and broadcast related information from an external broadcast management server through a broadcast channel.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 611 may be stored in the memory 660.

The mobile communication module 613 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data according to voice call signal, video call signal, or text / multimedia message transmission and reception.

The wireless internet module 615 refers to a module for wireless internet access, and the wireless internet module 615 may be embedded or external to the mobile terminal 600.

The acoustic communication unit 617 may perform acoustic communication. The acoustic communication unit 617 may output sound by adding predetermined information data to the audio data to be output in the acoustic communication mode. In addition, in the acoustic communication mode, the acoustic communication unit 617 may extract predetermined information data from a sound received from the outside.

The GPS (Global Position System) module 619 may receive location information from a plurality of GPS satellites.

The A / V input unit 620 is for inputting an audio signal or a video signal, and may include a camera 621 and a microphone 623.

The user input unit 630 generates key input data input by the user for controlling the operation of the terminal. To this end, the user input unit 630 may include a key pad, a dome switch, a touch pad (constant voltage / capacitance), and the like. In particular, when the touch pad forms a mutual layer structure with the display 680, this may be referred to as a touch screen.

The sensing unit 640 detects a current state of the mobile terminal 600 such as an open / closed state of the mobile terminal 600, a location of the mobile terminal 600, a presence or absence of a user contact, and controls the operation of the mobile terminal 600. The sensing signal may be generated.

The sensing unit 640 may include a detection sensor 641, a pressure sensor 643, a motion sensor 645, and the like. The motion sensor 645 may detect a movement or position of the mobile terminal 600 using an acceleration sensor, a gyro sensor, a gravity sensor, or the like. In particular, the gyro sensor is a sensor for measuring the angular velocity, and may sense a direction (angle) returned with respect to the reference direction.

The output unit 650 may include a display 680, a sound output unit 653, an alarm unit 655, and a haptic module 657.

The display 680 displays and outputs information processed by the mobile terminal 600. For example, the information received from the communication unit 610 is displayed.

Meanwhile, as described above, when the display 680 and the touch pad form a mutual layer structure and constitute a touch screen, the display 680 may also be used as an input device capable of inputting information by a user's touch in addition to the output device. Can be.

The sound output unit 653 outputs audio data received from the wireless communication unit 610 or stored in the memory 660. The sound output unit 653 may include a speaker, a buzzer, and the like.

The alarm unit 655 outputs a signal for notifying occurrence of an event of the mobile terminal 600. For example, the signal may be output in the form of vibration. .

The haptic module 657 generates various tactile effects that a user can feel. A representative example of the haptic effect generated by the haptic module 657 is a vibration effect.

The memory 660 may store a program for processing and controlling the controller 670, and may provide a function for temporarily storing input or output data (eg, a phone book, a message, a still image, a video, etc.). It can also be done.

The interface unit 625 serves as an interface with all external devices connected to the mobile terminal 600. The interface unit 625 may receive data from such an external device or receive power and transmit the data to each component inside the mobile terminal 600, and may transmit data within the mobile terminal 600 to the external device. .

The controller 670 typically controls the operations of each unit to control the overall operation of the image display apparatus 600. For example, perform related control and processing for voice calls, data communications, video calls, and the like. In addition, the controller 670 may include a multimedia playback module 681 for multimedia playback. The multimedia playback module 681 may be configured in hardware within the controller 670 or may be configured in software separately from the controller 670. The controller 670 may include an application processor (not shown) for driving an application. Alternatively, the application processor (not shown) may be provided separately from the controller 670.

In addition, the power supply unit 690 receives an external power source and an internal power source under the control of the controller 670 to supply power for operation of each component.

Meanwhile, a block diagram of the mobile terminal 600 shown in FIG. 8 is a block diagram for one embodiment of the present invention. Each component of the block diagram may be integrated, added, or omitted according to the specifications of the mobile terminal 600 that is actually implemented. That is, two or more components may be combined into one component as needed, or one component may be divided into two or more components. In addition, the function performed in each block is for explaining an embodiment of the present invention, the specific operation or device does not limit the scope of the present invention.

9A to 9C are views illustrating various information displayed on the mobile terminal of FIG. 1.

First, FIG. 9A illustrates a screen 910 that includes the calculated sleep score, which may be displayed on the display 680 of the mobile terminal 600. In the figure, it illustrates that the calculated sleep score is 88.

Next, FIG. 9B illustrates a screen 920 that includes sleep status information, which may be displayed on the display 680 of the mobile terminal 600. In the figure, it illustrates that the sleep state information without snoring is displayed in the sleep state.

Next, FIG. 9B illustrates a screen 930 including sleep state information that may be displayed on the display 680 of the mobile terminal 600. In the figure, the sleep state information including snoring information is displayed in the sleep state.

Due to the screens 910, 920, and 930, the user can easily grasp his or her sleeping state.

The sleep measurement apparatus according to the embodiment of the present invention, and the sleep measurement system having the same, are not limited to the configuration and method of the embodiments described as described above, the above embodiments may be variously modified. All or some of the embodiments may be configured to be selectively combined so as to.

In addition, while the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, but the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

Claims (14)

1. A radar sensor for receiving a radar signal;

   A microphone for receiving an audio signal;

   Based on the audio signal from the microphone, the snoring and apnea of the sleeper are detected, and based on the radar signal, the sleeper's movement, respiration, and heart rate signals are detected and based on the audio signal from the microphone. And a processor that calculates sleep efficiency based on the calculated snoring data and the sleeper's movement, respiration, and heart rate signal.
2. The method of claim 1,

   The processor,

   And when the calculated sleep efficiency is equal to or greater than a first reference value, calculating a sleep stage of the sleeper.
3. The method of claim 2,

   The processor,

   And a sleep score is calculated based on the calculated sleep efficiency, sleep stage, snoring data, and apnea data.
4. The method of claim 1,

   The processor,

   Extracts a snoring candidate event from the audio signal from the microphone, extracts a snoring feature based on the extracted snoring candidate event, and based on the extracted snoring feature, according to a learning-based classification performed Sleep measurement device, characterized in that the operation for snoring.
5. The method of claim 4, wherein

   The processor,

   When the sleeper is calculated to be snoring, the apnea feature is extracted from the audio signal from the microphone, and based on the extracted apnea feature, the apnea is calculated according to a learning-based classification performed. Sleep measurement device.
6. The method of claim 1,

   The processor,

   Based on the detected sleeper's movement, breathing, and heart rate signals, the sleeper is classified as a wake or sleep state, and in the sleep state, whether the sleeper is a rapid eye movement (REM) sleep state or a non-rem non-REM) Sleep measuring device characterized in that the classification of sleep.
7. The method of claim 6,

   The processor,

   The sleep measurement device, characterized in that for calculating the sleep stage of the sleeper, when the sleeper is in the berem sleep state.
8. The method of claim 1,

   The processor,

   The sleep measurement device, characterized in that for detecting the sleeper based on the radar signal, controls the period of the output radar signal output when the sleeper is absent to increase as compared with when the sleeper is present.
9. The method of claim 1,

   Further comprising; a power supply for supplying power,

   The processor,

   When the power supply unit is provided with a battery, the sleep measurement device, characterized in that for controlling the cycle of the output radar signal output according to the power level of the battery.
10. The method of claim 1,

    The processor,

    And performing noise canceling on a predetermined noise among the audio signals from the microphone, and detecting whether the sleeper is snoring and apnea based on the noise canceled audio signal.
11. The method of claim 1,

    And an audio converter configured to perform preprocessing of the audio signal from the microphone.
12. The method of claim 1,

    A signal amplifier for amplifying the radar signal from the radar sensor; And

    And a converter for converting the radar signal from the signal amplifier into a digital signal.

    And a radar signal converted by the converter is input to the processor.
13. terminal; And

    Sleep measurement system according to any one of claims 1 to 12; comprising a sleep measurement system.
14. The method of claim 13,

    The terminal,

    Sleep measurement system, characterized in that for outputting the sleep stage or sleep score information calculated by the sleep measurement device.

Images