CN113448438B

CN113448438B - Control system and method based on sleep perception

Info

Publication number: CN113448438B
Application number: CN202110701370.8A
Authority: CN
Inventors: 马瑞强; 刘春雷; 张�浩; 陈星植
Original assignee: Inner Mongolia University of Technology
Current assignee: Inner Mongolia University of Technology
Priority date: 2021-06-25
Filing date: 2021-06-25
Publication date: 2023-02-03
Anticipated expiration: 2041-06-25
Also published as: CN116339509A; CN113448438A

Abstract

The invention relates to a control system and a method based on sleep perception, wherein the control system comprises at least one processor, the processor is connected with at least one terminal intelligent device, and the processor is configured to: receiving sound information collected by at least one audio collection module; extracting at least one type of breathing frequency information from the sound information, and identifying a sleep state based on the extracted model and the breathing frequency information; and sending preset instruction information to at least one associated functional module in the specified sleep state. The invention detects the sleep perception of the user by controlling the functional modules to generate the stimulation micro-change which can not obstruct the sleep, and closes the appointed functional module when the user enters the sleep state, thereby realizing the effect of directly determining the sleep state and controlling the functional module without pulse information, heart information and blood flow information.

Description

Control system and method based on sleep perception

Technical Field

The invention relates to the technical field of sleep perception, in particular to a control system and method based on sleep perception.

Background

Before sleeping, people who listen to music, videos and the like by using a mobile phone are increased, but the situation that people are asleep unconsciously is common, and at the moment, the music and the videos are still in a playing state. This causes the smart terminal to consume a large amount of power and even wake up the user again. Therefore, it is necessary to automatically turn off applications such as music, video, etc. after the user enters a sleep state.

Currently, a timing closing mode is generally adopted to control the closing of modes such as music, video and the like. However, some users do not go to sleep at regular time, and some users go to sleep quickly, but are awakened again by audio and video that are not turned off after sleeping. How to close application functions such as audio and video and the like in a personalized manner according to sleep perception is an unsolved technical problem.

In the prior art, the sleep state of a user is detected even by means of devices such as an intelligent sensing mattress and an intelligent watch, so that a terminal device indirectly acquires the sleep state of the user, but the user is required to configure the intelligent sensing mattress, the intelligent watch and other matching devices, so that the cost for sensing the sleep state of the user by the intelligent device is increased, and the personalized closing of audio and video based on sleep sensing in any place where the user sleeps is not facilitated.

For example, chinese patent CN111772583A discloses a sleep monitoring analysis method and apparatus for an intelligent sound box, and an electronic device, and relates to the technical fields of artificial intelligence, computer vision, and voice interaction. The specific implementation scheme is as follows: acquiring multiframe images and audios, collected by an intelligent sound box, of a user during sleep; generating an image curve of the user for sleeping according to the multi-frame image; generating an audio curve of the user for sleeping according to the audio; obtaining sleep quality information of the user according to the image curve and the audio curve; outputting sleep information of the user, wherein the sleep information comprises at least one of the following items: the image curve, the audio curve, the sleep quality information of the user, the image of the user during sleep, and the audio of the user during sleep. The method and the device enable the user to know the detailed change process of the sleep state and the external factors influencing the sleep, thereby helping the user to better find a method for improving the sleep, greatly improving the experience of the user and solving the problem that the sleep quality of the user cannot be influenced when the audio and video are turned off.

For example, chinese patent CN109920532B discloses a control method of a medical wearable device with a sleep function, in which VR devices are worn on the head of a user, and a device camera corresponds to eyes; identifying the series of actions of the eyes by the VR equipment camera, and matching the characteristic data; processing the matched characteristic data in the S2, converting the characteristic data into a control signal and sending the control signal to a processing center; controlling VR equipment to play corresponding video data according to different scene states in S3; when the sleep signal is matched in the S2, the processing center controls the VR equipment to be closed; the eye capturing system is used for identifying the series of eye actions of the user, matching and identifying the eye actions with the feature data in the feature library, and identifying and matching the exciting eye feature, the calming eye feature, the manic eye feature, the wounded eye feature and the sleeping eye feature to further adjust different image qualities and help the user to sleep in time, so that the eye capturing system has strong practicability. Although the device can correspondingly close the audio and video after detecting the sleep state of the user, the user must wear VR equipment, which undoubtedly affects the sleep of the user, and the device is not convenient and convenient for the intelligent equipment carried by the user to automatically close the audio and video function based on the sleep state.

Therefore, how to enable the intelligent terminal device such as a mobile phone to determine the sleep state directly by sensing the breathing sound of human beings, and then to turn off the functions such as audio and video is a technical problem which is not solved in the prior art.

Furthermore, on the one hand, due to the differences in understanding to the person skilled in the art; on the other hand, as the inventor studies a lot of documents and patents while making the present invention, but the space is not detailed to list all the details and contents, however, this invention doesn't have these prior art features, but this invention has all the features of the prior art, and the applicant reserves the right to add related prior art in the background art.

Disclosure of Invention

In the prior art, terminal intelligent equipment needs to determine the sleep state of a user by means of collected blood pressure information, pulse information and respiratory information, so that intelligent terminal equipment such as a mobile phone which does not have the functions of collecting pulse information, sleep information and the like needs to be connected with equipment with corresponding functions to monitor the sleep state and perform personalized control, the user needs to carry a plurality of functional equipment, and otherwise the sleep state cannot be monitored. For example, the existing smart watch needs to maintain a continuous bluetooth signal connection with a smart phone to monitor the sleep state. Therefore, a user who only owns the smart phone cannot accurately judge the sleep state only by collecting the breathing information and can close the played audio and video according to the sleep state. Moreover, when the user uses the intelligent terminal device to play audio or video, a large amount of noise influencing the breathing information can be generated, and the difficulty in recognizing and extracting the breathing information is further improved.

Therefore, how to accurately judge that the user enters the sleep state and close the played audio and video only through the breathing sound is a technical problem which is not solved at present. The technical problem that accurate breathing information is obtained by filtering breathing sound at present is not solved.

The control system provided by the invention can enable the intelligent terminal equipment to process the breathing information to obtain personalized breathing sound only by collecting the breathing information, so that the user is judged to enter the sleep state according to the breathing sound, and the specified function is closed. The invention enables the user to realize the technical effect of closing control through the intelligent terminal equipment only having the sound collection function, simplifies the use conditions of user control, and enables the user to realize the control of audio and video during sleeping through simple intelligent terminal equipment such as intelligent mobile phones, desktop computers, notebook computers, tablet computers and the like which only have microphones.

In view of the deficiencies of the prior art, the present invention provides a sleep awareness-based control system, which includes at least one processor, the processor establishing a connection with at least one terminal intelligent device, the processor being configured to: receiving sound information collected by at least one audio collection module; extracting at least one type of breathing frequency information from the sound information, and identifying a sleep state based on the extracted model and the breathing frequency information; and sending preset instruction information to at least one associated functional module in the specified sleep state. The control system provided by the invention eliminates the defect that the sleep state needs to be accurately judged after the object characteristic information is acquired by the third equipment or the third equipment, so that the intelligent terminal equipment can determine the sleep state only by acquiring the breathing sound, and can determine whether the user is asleep or not by sensing the stimulus micro-change in the sleep, thereby reducing the error of inaccurate detection, and enabling the control system to accurately judge the sleep state and close the functional module according to the preset instruction.

Preferably, the processor is further configured to: and under the condition that the user is determined to enter the sleep state, sending a stimulation information micro-change instruction to at least one running functional module, and testing the sleep perception of the user in a mode of controlling the stimulation information of the functional module to generate micro-change.

Preferably, the processor is further configured to: and under the condition that the user does not feed back the micro-change generated by the stimulation information of the functional module, the processor sends control information to the functional module according to a preset control instruction, so that the functional module is controlled in a preset mode.

Preferably, the processor is further configured to: in the case that there is feedback from the user about a micro-change in the stimulus information of the function module, the processor restores the micro-change producing stimulus information of the controlled function module.

Preferably, the processor is further configured to: the functional module is controlled in accordance with a trend of decreasing the stimulus intensity of the stimulus information, thereby forming a sleep perception test that does not obstruct sleep.

Preferably, the processor is further configured to: the processor forms and updates a personalized frequency sample based on current respiratory frequency information when the user enters a sleep state and there is no feedback on a micro-change of stimulation information of a currently running functional module.

Preferably, the processor is further configured to: after the user enters the sleep state, the feedback information after the stimulus information of the current operation module generates micro-change at least comprises the movement information, the input instruction information and/or the action information of the intelligent terminal equipment.

The invention also provides a control method based on sleep perception, which at least comprises the following steps:

receiving sound information collected by at least one audio collection module;

extracting at least one type of breathing frequency information from the sound information,

identifying a sleep state based on the extraction model and the respiratory rate information;

and sending preset instruction information to at least one associated functional module in the specified sleep state.

Preferably, the method comprises at least: and under the condition that the user is determined to enter the sleep state, sending a stimulation information micro-change instruction to at least one running functional module, and testing the sleep perception of the user in a mode of controlling the stimulation information of the functional module to generate micro-change.

Preferably, the method comprises at least: and under the condition that the user does not feed back the micro-change generated by the stimulation information of the functional module, the processor sends control information to the functional module according to a preset control instruction, so that the functional module is controlled in a preset mode.

Drawings

FIG. 1 is a logic diagram of a sleep awareness based control system of the present invention.

List of reference numerals

10: a breath collection module; 20: a processor; 21: a database; 30: and a functional module.

Detailed Description

The following detailed description is made with reference to the accompanying drawings.

The invention provides a control system and a control method based on sleep perception, and also provides a self-defined control system based on respiratory frequency and portable intelligent terminal equipment. The control system based on sleep perception can eliminate one or more noise and vibration information while keeping the respiratory information, thereby extracting the accurate information of the respiratory frequency.

The intelligent terminal device in the invention is a using device which has the functions of wirelessly accessing the Internet, processing, displaying and executing partial data. The intelligent terminal device is an intelligent product capable of operating based on data instructions, such as a smart phone, an Ipad, a player, a computer, and the like. The intelligent terminal equipment can install sound collection device by itself, or can connect the sound collection device to collect the breathing sound of the user. The sound collection device is for example a microphone.

Preferably, a microphone of the intelligent terminal device is coupled with a public address set, so that the breathing information of the user can be further clearly acquired.

The processor of the present invention represents one or more general-purpose devices such as a microprocessor, central Processing Unit (CPU), application specific integrated chip, or the like.

The breathing information of the invention is breathing sound information, such as breathing sound, snore and the like. The breathing frequency information refers to the frequency at which the user completes one breath, and the period refers to the time taken for the user to complete one breath.

The sleep perception-based control system can be installed and operated in the intelligent terminal equipment or establish data connection with the intelligent terminal equipment.

The user's breathing is distinguished from other noises in that the breathing sound is periodic, regular, and the breathing frequency is in a certain range. For example, the breathing frequency varies between 0.15 and 0.4 times per second.

The acquisition module comprises a sound acquisition module and an acceleration sensor. The acquisition module belongs to a part of intelligent terminal equipment. The acquisition module is capable of acquiring corresponding information based on instructions of the processor.

Preferably, the control system of the present invention further comprises a database. The database is used to store the acquired respiratory information as well as time, store instructions associated with the neural network, and extract process data of respiratory rate from the respiratory information. And the processor reads data from the database, or the processor sends the data in the processing process to the database for storage.

The database is preferably a memory. Memory represents any type or form of volatile or non-volatile storage device or medium capable of storing data and/or other computer-readable instructions. Memory includes, but is not limited to, RAM, ROM, flash memory, or any other suitable storage device. The memory can be coupled to the processor and can be arranged in a manner to establish a signal connection with the processor.

Preferably, the processor may further include a memory control module, an input/output (I/O) controller, and a communication interface, each of which may be interconnected via a communication infrastructure.

Examples of communication infrastructure include, but are not limited to, a communication bus (e.g., an Industry Standard Architecture (ISA), peripheral Component Interconnect (PCI), PCI Express (PCIe), or similar bus) and a network.

An I/O controller represents any type or form of module capable of coordinating and/or controlling the input and output functions of a computing device. An I/O controller may control or facilitate the transfer of data between one or more elements of a processor.

A communication interface represents any type or form of communication device or adapter capable of facilitating communication between a processor and one or more additional devices.

Fig. 1 is a diagram of an embodiment of the control system of the present invention. As shown in fig. 1, the sleep perception based control system includes at least a processor 20. The processor 20 is connected with the acquisition modules 10, the database 21 and the function module 30 respectively to transmit data information and instructions. The database 21 may drive and store data information via a storage drive, such as a floppy disk drive, a magnetic tape drive, an optical disk drive, a flash memory drive, or the like.

Preferably, the intelligent terminal device is provided with a terminal display module. The terminal display module is connected with the processor in a wired or wireless mode, and the terminal display module is in data connection with the acquisition module. And the terminal display module sends the control instruction to the functional module. The terminal display module can display the collected movement information, the breathing frequency information, the sleeping time, the first time, the second time, the stimulation information micro-variation mode, the closing or pause time and other information. The terminal display module can display information in various ways such as graphs or curves.

The functional module of the invention is a module with various execution functions which is arranged in the intelligent terminal equipment and can be adjusted, opened and closed based on the instruction of the processor. The functional modules include, for example, an audio player, a video player, a power on/off module, a lamp adjustment and switch module, and on/off and adjustment of each application module.

As shown in FIG. 1, the sleep awareness based control system of the present invention comprises at least a processor configured to:

receiving sound information collected by at least one audio collection module;

identifying a sleep state based on the extraction model and the respiratory frequency information;

The respiratory frequency information extraction model in the invention, which can be called as an extraction model for short, is established based on a deep learning model and is used for extracting respiratory frequency information. The deep learning model is, for example, a convolutional neural network model.

Preferably, an analog filter is further disposed in the processor and used for amplifying the received breathing information, so that missing of breathing sounds with small sounds is avoided.

Preferably, an acceleration sensor in the intelligent terminal device is connected with the processor and collects the movement signal of the intelligent terminal device. The movement signal comprises the movement speed of the audio and video in the three-dimensional space.

The processor extracts respiratory rate information from the amplified sound information. The processor eliminates noise and audio and video sound without a periodic rule from the sound information to obtain the breathing information with the periodic rule.

Preferably, the processor is further provided with an analog gain module for an analog gain between 10 and 100 times to cover the range of breathing. Preferably, the processor is also capable of connecting hardware that implements gain, such as filters, to limit the frequency response of the signal to within the respiratory frequency range.

Preferably, the analog gain module sends the filtered breathing information to the extraction module.

The analog filter in the processor performs linear filtering on the sound information in the following way:

signal quality for linear filtering y [ n ]]From the error sequence e [ n ]]＝d[n]-y[n]To be determined. Where n denotes the frequency of the mixed sound and m denotes the sound outside the specified frequency range. Omega _m Representing the weight and x representing the acquired data value.

Preferably, the weights are chosen in such a way as to obtain a minimum mean square error:

E{e ² [n]}＝E{d[n]-y[n] ² }

the way to calculate the selection weights for linear filtering is:

according to the orthogonality principle, when the error e [ n ] and the data value are zero, the following results are obtained:

E{x[n-k](d[n]-y[n])}＝0,k＝0,1,…,M-1

the linear filtering effect is then the best.

Preferably, the weight module within the processor may also derive the correct weights by training based on a training algorithm.

After the processor filters the acquired original sound information to eliminate noise caused by audio and video or other sounds, the respiratory frequency of the respiratory information formed after filtering is extracted. At this time, the respiratory information is not completely noiseless, and also contains a small amount of noise, so that the extraction module is required to perform further extraction based on a deep learning algorithm.

Preferably, the way in which the processor extracts the respiratory rate information comprises:

establishing a respiratory frequency information extraction model,

comparing a first respiration rate extracted by the respiration frequency information extraction model with a verifiable respiration rate of the first test subject to calculate an error;

front-end parameters of one or more layers of the extraction model are adjusted back based on the computational error to improve the extraction model placement prediction accuracy.

After acquiring the breathing frequency information, the processor determines whether the user enters a sleep state or not, and even determines which stage of the sleep stage the user enters based on the breathing frequency information.

Preferably, the respiratory information signal has the following relationship:

after the transmission delay information is introduced, the respiration information signal is expressed as:

finally, the relationship of the hybrid collected respiratory information received by the collection module is:

wherein, the frequency f _b The distance between the object and the acquisition module is directly related, and the distance b is related to the velocity of the object. Frequency f _b And the distance b is calculated by a fourier transform algorithm.

For vital signs of the user, frequency f _b The distance measuring method can be used for measuring the distance section of the user, and the distance b can reflect the displacement change of the sound source of the user.

Preferably, the processor is capable of de-manizing the data of respiratory information prior to extracting the respiratory rate. For example, normalization is performed by the equation f (x) = (x- μ)/σ, where μ denotes the average value of the waveform and σ denotes the standard deviation.

By de-noising the received breathing information, other components and unwanted frequencies in the acoustic information signal can also be removed. For example, the breathing frequency varies between 0.15 and 0.4 per second. Other frequencies that are not within the respiratory frequency range can be eliminated.

In the extraction model of the invention, the convolutional neural network model can be a two-layer convolution and can also be expanded to more than two layers.

Each convolutional layer includes a one-dimensional convolutional layer and a pooling layer. One-dimensional convolutional layers effectively derive noteworthy features from a short segment of the entire dataset, and the locations of the features in the segment do not have a high correlation. The one-dimensional convolutional layer can derive any type of signal data within a fixed length period, and therefore, the efficiency of data screening and derivation can be improved. Thus, convolutional neural networks are trained using one-dimensional or two-dimensional convolutional layers at each layer. Pooling by the pooling layer after the convolutional layer data processing can reduce the complexity of the output and prevent over-fitting of the data.

Preferably, the size of the pooling layer is set to 3, meaning that the output matrix is only one third the size of the input matrix. The pooling layer is used to reduce the input size by mapping the size of the defined window to a single result by taking the maximum value of the elements in the window.

The processor is configured to:

the input three-dimensional waveform information for training the extraction model is subjected to discretization processing, so that the signal-to-noise ratio is improved.

The influence of small fluctuation in the data on the extraction model can be reduced by performing data fitting on the respiratory information, and the small fluctuation is noise in general.

Preferably, the processor is further configured to:

data output by the convolutional layer is directed to the average pooling layer. The average pooling layer is another pooling layer to avoid overfitting. The averaging pooling layer transforms the matrix of the convolutional network outputs into a single vector.

In the invention, the processor adjusts and controls the functional module according to a preset instruction after determining that the user enters the sleep state.

Preferably, the extracted breathing frequency information is compared with the breathing frequency samples by an extraction model in the processor, and the processor obtains the corresponding sleep state according to the breathing frequency information on the basis that the breathing frequency samples are associated with the sleep state.

The breathing frequency samples in the present invention include an initial frequency sample and an individualized frequency sample. The initial frequency samples are set based on sample set data correlating the respiratory frequency characteristics of the general population to sleep states.

The personalized frequency sample is formed by performing personalized adjustment on the initial frequency sample based on the personalized breathing characteristics of the user, so that the information of the personalized frequency sample and the sleep state is formed.

Preferably, the initial frequency samples and the sleep state related information in the extraction model are preset, and can be provided by a third-party data platform, or obtained by learning and obtaining in advance based on a deep learning model according to sleep experiments of a plurality of sample crowds.

The initial frequency samples are associated with the sleep state as follows:

in different sleep stages, the breathing frequency of the sleep state of the user is obviously different, in a rapid eye movement sleep stage (REM), brain waves are changed rapidly, and the sleeper often has rapid eye jumping phenomenon and dreams in the stage, so the breathing frequency of the sleeper in the stage is usually unstable. In the Light Sleep stage (Light Sleep), the eye jump phenomenon stops and the sleeper's breathing frequency tends to be smooth. During the Deep Sleep phase (Deep Sleep), the sleeper's breathing frequency is further slowed down. Therefore, the magnitude of the breathing frequency varies with the stage of sleep in which the sleeper is located.

The sound collection module collects breath sound data according to a sampling frequency of 8KHz, and the breath sound data is divided into N groups of data according to each 400 sample points. N denotes the number of groups of sample points and the sampling frequency is 8kHz. The time interval of 400 sample points is 0.05s.

After the sounds of the audio and the video are filtered through linear filtering to form respiratory information, the respiratory frequency information is extracted according to the extraction model.

The invention does not need to monitor the sleep state of the user in the whole process, and can close the appointed functional module only by determining that the user enters the initial stage of the sleep state. However, it is important to determine how to accurately determine that the user enters the sleep state only through the analysis of the breathing sound, and to turn off the played function module in a manner that does not affect the sleep feeling of the user, which is also difficult to do in the prior art.

When the user enters the sleep state and turns off the functional module, particularly when the audio and video which the user is watching or listening to is turned off, if the user is turned off too early, the user only enters the sleep state quickly, and the turning off of the functional module can lead the user to be awake due to the change of the environment, so that the counter-action effect is achieved. If the switch-off is too late, the function of the sleep-aware control function module is lost.

Based on this deficiency, the processor tests the user's perception level in a sleep state in a test mode. And controlling the functional module when the perception level of the user is reduced to a preset level. The invention tests the perception level of the user through the test mode so as to regulate and control the functional module in a mode of not influencing the sleep feeling of the user and even close the functional module.

The test mode in the present invention tests the perception level of the user in a way that stimulates the micro-variation of information. The stimulation in the invention refers to stimulation in modes of sound stimulation, visual stimulation, playing progress stimulation, playing content change and the like presented by the intelligent terminal equipment when a user uses the functional module. The slight change of the stimulation information in the present invention means a sensory stimulation that does not stimulate the user to be awake, in view of the fact that the user of the present invention is in a sleep state or a state close to sleep. The stimulation information micro-change of the invention is to finely adjust and change the stimulation already applied by the functional module, and new stimulation cannot be added to avoid waking up the user.

Preferably, the embodiments of the micro-variation of the stimulation information of the processor include at least the following.

After the user is judged to enter the sleep state based on the breathing frequency information, the processor reduces the playing volume of the current functional module for the first time by limiting a volume difference value, and gradually reduces the playing volume in a step volume mode under the condition that the intelligent terminal device does not detect any input instruction information within a first limited time after the playing volume is reduced, and directly closes the functional module when the playing volume is reduced to be within a closing threshold range. The step-type reduction of the playing volume can not bring new stimulation information to the user.

When the user does not completely enter the sleep state, the perception capability of the user is strong, the experience is inevitably deteriorated due to the reduction of the playing volume, and the user can move the intelligent terminal device or input related instructions or action signals for checking. When the processor detects a moving signal acquired by an acceleration sensor in the intelligent terminal device, or an action signal of a related input device, or an input instruction, the processor controls the functional module to recover the playing volume.

After the user enters the sleep state, the perception capability of the user is weakened, the processor limits the volume difference to reduce the playing volume of the current functional module for the first time, the intelligent terminal device cannot be moved under the condition that the user does not perceive, or the input device cannot be adopted to input action signals to check the change of the functional module, or instructions cannot be input, therefore, the processor cannot receive the movement signals collected by the acceleration sensor, or the action signals of the relevant input device, or the input instructions, the processor can reduce the playing volume for the second time, and under the condition that any interference information is still not received within the second limited time, the processor closes or suspends the operation of the current appointed functional module.

In the present invention, the operation signal of the input device is, for example, a touch screen operation signal, a mouse movement signal, a keyboard operation signal, or the like. An input command refers to any input command information.

After determining that the user enters the sleep state based on the breathing rate information, the processor detects the perception of the user in a manner that the functional module is suspended. For example, the playing content of the audio module and the video module is temporarily played, and the processor sends a preset control instruction to at least one functional module under the condition that any input instruction information is not detected within a third limited time. The control instruction comprises user preset instructions such as closing the designated function module, pausing and shutting down.

And if the movement information of the intelligent terminal device, any input instruction information and the action signal of the input device are detected within the third limited time, the processor sends an instruction for recovering the operation to the regulated function module.

Preferably, the processor detects the perception of the user by pausing the function module at least once. Preferably, the processor detects the perception of the user by means of two or three functional module pauses.

After sensing is tested in a mode that the functional module is suspended once and user feedback is detected, the processor judges whether the user enters a sleep state again based on the current respiratory frequency information. And the perception detection is repeated after the user enters the sleep state.

Preferably, the processor is further capable of detecting the perception of the user by altering the manner in which the content is played after determining that the user enters the sleep state based on the breathing frequency information. Preferably, the processor alters the content for playback according to a trend that is conducive to sleep. For example, the playing music content is changed into light music without lyric content for helping sleep, and the playing video content is changed into video content with relaxed atmosphere for helping sleep, such as news video, visiting video and the like which are not easy to cause sensory stimulation.

And after the playing content is changed, the processor sends a preset control instruction to at least one functional module under the condition that any input instruction information or movement signal is not detected within the fourth limited time. The control instruction comprises user preset instructions of closing the specified function module, pausing, shutting down and the like.

And if the movement information of the intelligent terminal device, any input instruction information and the action signal of the input device are detected within the fourth limited time, the processor sends an instruction for recovering the operation to the regulated function module.

Preferably, in case the processor controls the function module to pause, the processor can control the function module to display a still picture or a question. For example, the text content of the display screen is: the playing function is to be closed, or a certain picture is used to cover the display picture. And under the condition that the user does not feed back any signal within the fifth limited time, the processor judges that the user perception is weak and enters a sleep state, so that a preset control instruction is executed.

The slight change in the stimulation information according to the present invention is not limited to the above-described embodiments, and any slight change in the stimulation information that can achieve the same effect may be implemented.

The test mode of the present invention can also be used to adapt the initial frequency sample of the breathing frequency to form an individualized individual frequency sample.

And in the time period of the initial use of the control system, the processor can take the breathing frequency when the user feedback signal is not received as the personalized frequency sample of the user after the functional module is subjected to the control of the stimulation information micro-change.

With the prolonging of the use time of the user and the increasing of the detection times, the processor can extract and update the breathing frequency information of the user in the sleep state into personalized frequency samples when the user uses various functional modules, and simultaneously store the personalized frequency samples, so that the personalized frequency samples in the database are more and more abundant. When the extracted respiratory frequency information is compared with the personalized frequency sample by the processor, the accuracy rate of judging whether the sleep state is entered is higher and higher, and a forward cycle is formed.

After the control system is started, the processor sends a command for collecting sound information to the sound collection module and sends a command for collecting movement information to the acceleration sensor. And the processor receives the collected sound information and the mobile information of the intelligent terminal equipment.

The processor of the invention can also be a cloud server connected with the intelligent terminal device through a network. The control system is connected with the intelligent terminal device, on the basis that new hardware is not added, the self assembly of the intelligent terminal device is used for collecting sound information and mobile information, and perception of a user is detected based on the micro change of the stimulation information of the functional module, so that the functional module can be controlled in a preset mode at a proper time to save energy consumption and improve the sleep quality of the user.

In the present invention, the respiratory information includes expiratory sound, inspiratory sound and snore sound. Snoring has certain common characteristics with breathing sound, which has similarities in waveform, both periodic, but differs from snoring by a larger average amplitude than breathing sound, because snoring is generally larger than breathing sound. The frequency distribution of the snore and the snore is obviously different from each other, the frequency distribution of the snore is concentrated in a low frequency band of 0-1000 Hz, and the respiratory sound is more distributed in both low frequency bands and high frequency bands, so that whether the collected sound sample contains the snore or not can be judged according to the ratio of the low frequency band to the high frequency band.

Judging the existence of snore by using the frequency range of the current respiratory sound dominance, and expressing a sound frame to be detected containing n sample points by using f _i If f is represented by the ith data value after fourier transform, the ratio of the low frequency band to the high frequency band is represented as:

when the user turns from a normal breathing state to snoring, the sound frequency segments are gradually inclined to low frequencies, so that the A (f) value is gradually increased. By analyzing the snore data of more than 100 different users, the user is in the snore state when the ratio of the low frequency to the high frequency is gradually increased and A (f) >1.5 is agreed, otherwise, the user is in the normal breathing state. I.e. when a (f) >1.5, it is determined that the user is snoring.

Preferably, since the processor of the present invention is able to distinguish snoring based on the above differences. The processor of the present invention can also be configured to:

and under the condition that the processor determines that the user enters the sleep state and snore is detected, the processor directly controls the functional module according to a preset instruction and does not perform sleep perception detection on the user. For example, some users can snore when entering a sleep state, sensory detection on the users is not needed, the processor can control the functional module to be directly closed or suspended, and the functional module is prevented from continuously releasing stimulation information to disturb the sleep of the users.

Under the condition that the processor determines that the user enters a sleep state and snoring is not detected, the processor detects the user perception by detecting the micro-change of the stimulation information of the user, and controls the functional module under the condition that the user perception is weaker.

Preferably, the level of sleep perception of the present invention can be divided based on the user's feedback of the slight changes in the stimulus information.

For example, the processor determines that the user enters a sleep state based on the breathing rate information. When the stimulation information is sound change, and the intensity of the sound is reduced in the first step range, if the user does not have any feedback information or feedback action, the sensory level of the user is one level. When the user is detected to enter the sleep state for the second time, and the intensity of the sound is reduced within the second gradient range, if the user does not have any feedback information or feedback action, the sensory level of the user is in the second level, namely the sensory is further weakened. The second step range is larger than the first step range. The first step range is 0-5 degrees, and the second step range is 5-10 degrees.

By analogy, the perception grade of the user can be divided into a plurality of grades, and the specific division range and the grade setting can be flexibly adjusted according to the requirement.

Preferably, the processor determines the user's perception level by a mixture of slight changes to the stimulus information. For example, for a playing video module, the processor first detects the perception of the user by masking the playing picture and not changing the sound. If the user does not have any feedback on the micro-change of the first stimulation information, the sensory grade belongs to one grade.

After the first stimulus micro-change, the processor detects the user's perception level a second time by controlling the sound of the video module, e.g. changing the intensity of the sound. If the user does not have any feedback on the micro-change of the second stimulation information, the sensory level belongs to the second level.

After the second stimulus micro-change, the processor detects the perception level of the user for a third time by changing the sound content of the played content. The sound content such as volume video is replaced with song content. If the user does not have any feedback on the micro-change of the third stimulation information, the sensory level belongs to three levels.

Obviously, in the sense intensity, the first-level sense intensity corresponding to the first-level sense level is greater than the second-level sense intensity, and the second-level sense intensity is greater than the third-level sense intensity. When the sleep sensory level of the user enters the secondary or tertiary sensory intensity, the processor sends a pause or close instruction to the video module.

As the examples show, the invention is also able to control the micro-variation of the stimulation information from multiple aspects to form a detection of the user perception. The perception detection of the micro-change of the stimulation information aims to not disturb the sleep of a user.

Preferably, the processor determines at which sensory level the shutdown or pause of the function module is performed based on the sensory level of the user and the corresponding feedback information.

For example, for a perception level, the user always has feedback on the detection of the primary senses during a first time frame and no feedback on the detection of the secondary senses during a second time frame. The processor personalizes the adjustment based on the user's usage. And starting timing after entering the sleep state, and carrying out primary sensory detection when the first time is met. And if no feedback exists in the preset feedback time, the processor performs secondary sensory detection when the second time is met. If no feedback exists in the preset feedback time, the processor confirms that the sleep perception of the user is enough, and sends a closing or pause instruction to the functional module. Wherein the second time duration is greater than the first time duration.

If the user has no feedback after meeting the first time for a long time and the execution times exceed the preset times threshold, the processor can send a closing or pause instruction to the functional module after the first-stage sensory detection is finished and no feedback exists in the preset feedback time.

If the user closes the function module after meeting the second time for a long time and the ratio of the execution times of closing compared with the first-level sense organ exceeds a preset ratio, the processor can skip the first-level sense organ detection, directly perform the second-level sense organ detection and control the function module to close.

Therefore, the method and the device can reduce the influence on the sleep of the user under the condition of reducing the sensory stimulation, and simultaneously determine the perception weakening degree of the user through the judgment on the respiratory frequency information and the sleep perception of the user, so that the sleep of the user is not influenced by the pause or the closing of the functional module.

Preferably, when the breathing frequency information is completely unfamiliar breathing frequency information, the processor may store the new breathing frequency information and the determination result of the sleep state only in a new account by establishing a new account, so that the user can distinguish the new breathing frequency information and the determination result of the sleep state, and the processor executes a control function on the preset function module.

It should be noted that the above-mentioned embodiments are exemplary, and that those skilled in the art, having benefit of this disclosure, may devise various solutions which are within the scope of this disclosure and are within the scope of the invention. It should be understood by those skilled in the art that the present specification and figures are illustrative only and are not limiting upon the claims. The scope of the invention is defined by the claims and their equivalents.

The present specification encompasses multiple inventive concepts and the applicant reserves the right to submit divisional applications according to each inventive concept. The present description contains several inventive concepts, such as "preferably", "according to a preferred embodiment" or "optionally", each indicating that the respective paragraph discloses a separate concept, the applicant reserves the right to submit divisional applications according to each inventive concept.

Claims

1. A control system based on sleep perception, the control system comprises at least one processor, the processor establishes connection with at least one terminal intelligent device,

the processor is configured to:

receiving sound information collected by at least one audio collection module;

sending preset instruction information to at least one associated functional module in a specified sleep state;

under the condition that the user is determined to enter the sleep state, sending a stimulation information micro-change instruction to at least one running functional module, and testing the sleep perception of the user in a mode of controlling the stimulation information of the functional module to generate micro-change;

under the condition that a user does not feed back micro-change generated by stimulation information of the functional module, the processor sends control information to the functional module according to a preset control instruction so as to control the functional module in a preset mode;

after the user is judged to enter the sleep state based on the breathing frequency information, the processor reduces the playing volume of the current functional module for the first time by limiting a volume difference value, gradually reduces the playing volume in a step volume mode under the condition that the intelligent terminal device does not detect any input instruction information within a first limited time after the playing volume is reduced, and directly closes the functional module when the playing volume is reduced to be within a closing threshold range;

the mode of extracting the respiratory frequency information by the processor comprises the following steps:

establishing a respiratory frequency information extraction model,

2. The sleep perception-based control system of claim 1, wherein the processor is further configured to:

in the case where there is feedback from the user regarding a micro-change in the stimulus information of the function module, the processor restores the micro-change producing stimulus information of the controlled function module.

3. The sleep perception-based control system of claim 2, wherein the processor is further configured to:

the functional module is controlled in accordance with a trend of decreasing the stimulus intensity of the stimulus information, thereby forming a sleep perception test that does not obstruct sleep.

4. The sleep perception based control system of claim 1 wherein the processor is further configured to:

the processor forms and updates a personalized frequency sample based on current respiratory frequency information when the user enters a sleep state and there is no feedback on a micro-change of stimulation information of a currently running functional module.

5. The sleep perception-based control system of claim 1, wherein the processor is further configured to:

after the user enters a sleep state, the feedback information after the stimulation information of the current operation module is slightly changed at least comprises the movement information, the input instruction information and/or the action information of the intelligent terminal equipment.

6. A control method based on sleep perception, characterized in that the method at least comprises:

receiving sound information collected by at least one audio collection module;

under the condition that a user does not feed back micro-change generated by stimulation information of the functional module, the processor sends control information to the functional module according to a preset control instruction, so that the functional module is controlled in a preset mode;

establishing a respiratory frequency information extraction model,

front-end parameters of one or more layers of the extraction model are adjusted back based on the computational error to improve extraction model placement prediction accuracy.