KR102387734B1 - Sleep state determination method, apparatus and computer program through non-contact data collection - Google Patents

Abstract

A method performed by one or more processors of a computing device, comprising: obtaining sleep sensing data of a user; sleep analysis information of the user using the sleep sensing data—the sleep analysis information is related to the user's sleep environment Calculating—including sleep stage information on changes in one or more sleep states according to time change; identifying a first time point at which the user's sleep quality deteriorates based on the sleep stage information; the first Sleep through non-contact data collection, comprising: identifying a singularity related to a change amount of indoor environmental information acquired through an environmental sensing module in response to one time point; and generating sleep-degrading factor information based on the identified singularity A state determination method is disclosed.

Description

Sleep state determination method, device and computer program through non-contact data collection {SLEEP STATE DETERMINATION METHOD, APPARATUS AND COMPUTER PROGRAM THROUGH NON-CONTACT DATA COLLECTION}

The present disclosure provides information on the sleep state based on the user's sleep data obtained in the sleep environment, and more specifically, to provide analysis information and health management information corresponding to the user's sleep state by utilizing an artificial neural network. it is for

There are various ways to maintain and improve health, such as exercise and diet, but it is most important to manage sleep, which accounts for more than 30% of the time of the day. However, modern people cannot get a good night's sleep due to irregular eating habits, lifestyle, and stress despite simple replacement of labor and leisure time by machines, and suffer from sleep disorders such as insomnia, hypersomnia, sleep apnea syndrome, nightmares, night terrors, and sleepwalking. are receiving

According to the National Health Insurance Corporation, the number of sleep disorder patients in Korea increased by an average of about 8% per year from 2014 to 2018, and in 2018, about 570,000 patients were treated for sleep disorder in Korea.

In particular, in the case of sleep apnea, the most common but dangerous sleep disorder, one in six Korean adults suffers. Sleep apnea is considered to be a major cause of heart disease, mental disease, and brain disease as well as disturbing sleep.

As deep sleep is recognized as an important factor affecting physical or mental health, interest in good sleep is increasing. However, in order to improve sleep disorders, it is necessary to visit a specialized medical institution in person, a separate examination fee is required, and continuous Due to the difficulty in management, the efforts of users for treatment are insufficient.

Accordingly, Korean Patent Laid-Open Patent No. 2003-0032529 discloses that it is possible to induce optimal sleep by receiving the user's body information and outputting vibration and/or ultrasound in a frequency band detected by repeated learning according to the user's physical condition during sleep. Disclosed are a sleep induction machine and a sleep induction method.

However, in the prior art, there is a concern that the quality of sleep may be reduced due to discomfort caused by body wearable equipment, and periodic maintenance (eg, charging, etc.) of the equipment is required. Additionally, the prior art simply provides information that induces optimal sleep, and as the correlation between information obtainable from sleep and comorbid diseases is not considered, it is possible to provide predictive information on the possibility of future disease occurrence. Therefore, medical diagnosis and information delivery are impossible.

Therefore, in the industry, it is possible to diagnose continuous sleep disorders by collecting data related to the user's sleep environment in a non-contact manner, and at the same time, the occurrence of sleep disorders and accompanying diseases (eg, heart disease, mental disease, and brain disease) There may be a demand for computing devices that can predict the possibilities.

The present disclosure has been devised in response to the above-described background technology, provides information related to a sleep state based on data measured in an individual user's sleep environment, and detects the possibility of occurrence of sleep disorders and accompanying diseases with sleep disorders This is to predict and provide medical diagnosis information to the user.

The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In one embodiment of the present disclosure for solving the above-described problem, a computing device for predicting a sleep state based on data measured in a user's sleep environment is disclosed. The computing device may include a processor configured to receive the user's sleep sensing data, input the sleep sensing data to a sleep evaluation model to calculate the user's sleep analysis information, a memory for storing program codes executable in the processor, and and a network unit for transmitting and receiving data to and from a user terminal, and wherein the sleep sensing data includes the user's respiration information obtained for a predetermined time period in relation to the user's sleep environment, and the sleep analysis information is the It may include at least one of apnea severity information on the user's apnea occurrence degree and disease prediction information on the possibility of disease occurrence.

In an alternative embodiment, further comprising a sensor unit for acquiring the user's sleep sensing data, wherein the sensor unit receives one or more transmission modules for transmitting a radio wave of a specific frequency and a reflected wave generated in response to the radio wave of the specific frequency And by detecting a phase difference or frequency change according to the moving distance of the reflected wave, it is possible to acquire the sleep sensing data from the user in a non-contact manner.

In an alternative embodiment, the processor receives a sleep diagnostic data set including a plurality of sleep diagnostic data corresponding to each of a plurality of users, and from each of the plurality of sleep diagnostic data, the user's breathing for a predetermined time period , by extracting information about heart rate and movement to generate a learning input data set, and extracting at least one of information about an apnea index, information about a sleep state, and information on whether or not a disease occurs from each of the plurality of sleep diagnosis data. generating a training output data set, matching the training input data set with the training output data set to generate a labeled training data set, and learning one or more network functions using the labeled training data set to learn the sleep An evaluation model can be created.

In an alternative embodiment, the sleep evaluation model may include: the processor inputs each of the training input data sets to the one or more network functions, and each of the output data calculated by the one or more network functions and each of the training input data sets An error is derived by comparing each of the learning output data sets corresponding to the label, and the weight of the one or more network functions is adjusted in a backpropagation method based on the error, and the learning of the one or more network functions is pre- When more than the determined epoch is performed, to be generated by determining whether to stop learning using verification data, and determining whether to activate the one or more network functions by testing the performance of the one or more network functions using a test data set can

In an alternative embodiment, the sleep evaluation model is a model for predicting the user's sleep state and disease occurrence probability based on the sleep sensing data, and includes one or more network functions, wherein the one or more network functions include: It may include a Dilated-Convolutional Neural Network for reinforcing a long-term relationship without loss of input data.

In an alternative embodiment, the extended convolutional neural network expands a receptive field of a filter with respect to the inputs of the one or more network functions to enhance the long term association, and a filter with respect to the inputs. It may be characterized in that the length of the input and output of the one or more network functions is maintained by adding zero padding to the .

In an alternative embodiment, the disease prediction information includes prediction information on at least one of a sleep disease, a mental disease, a brain disease, and a cardiovascular disease, and the processor, the user input to the sleep evaluation model A correlation between the sleep sensing data and the disease prediction information may be derived based on the amount of change in the disease prediction information output by changing the respiration information for a predetermined time period included in the sleep sensing data of .

In an alternative embodiment, the sleep sensing data includes movement information and heart rate information for a predetermined time period, and the sleep analysis information includes one or more sleep states according to time changes in relation to the user's sleep environment. It may include sleep stage information about the change.

In an alternative embodiment, a sensor unit including one or more environmental sensing modules for acquiring indoor environment information including information on at least one of the user's body temperature, indoor temperature, indoor humidity, indoor sound, and indoor illuminance is further Including, and the processor, based on the sleep stage information and the indoor environment information may generate sleep degradation factor information.

In an alternative embodiment, the processor identifies a first time point at which the quality of the user's sleep deteriorates based on the sleep stage information, and detects a singularity related to the amount of change in the indoor environment information in response to the first time point and may generate the sleep-decreasing factor information based on the identified singularity.

In an alternative embodiment, the processor relates to the variability of the indoor environment information based on whether a change amount of each of the one or more pieces of information included in the indoor environment information in response to the first time point exceeds a predetermined threshold change amount singularities can be identified.

In an alternative embodiment, further comprising an indoor environment builder for adjusting at least one of temperature, humidity, sound and illuminance to create an indoor environment related to the user's sleep environment, and wherein the processor includes: the sleep degrading factor information An environment control signal for controlling the indoor environment creating unit may be generated based on the .

In an alternative embodiment, the processor, based on the sleep evaluation information, determines to generate and transmit health management information for promoting the user's sleep environment and health to the user terminal of the user, wherein the health management information is , information about eating habits, information about an exercise amount, and information about an optimal sleeping environment may include at least one information.

In another embodiment of the present disclosure, a method of predicting a sleep state and a disease based on data measured in a user's sleep environment performed by one or more processors of a computing device is disclosed. The method includes: the processor receiving the user's sleep sensing data, the processor calculating the user's sleep analysis information by using the sensing data as an input of a sleep evaluation model, and the processor is the sleep analysis information may include determining to transmit to the user terminal of the user.

In another embodiment of the present disclosure, a computer program stored in a computer-readable storage medium is disclosed. When the computer program is executed on one or more processors, it performs the following operations for predicting a sleep state based on data obtained from a user's sleep environment, the operations comprising: receiving the user's sleep sensing data , the processor may include the operation of calculating the sleep analysis information of the user by using the sensing data as an input of the sleep evaluation model, and the operation of the processor determining to transmit the sleep analysis information to the user terminal of the user. .

Other specific details of the present disclosure are included in the detailed description and drawings.

The present disclosure has been devised in response to the above-mentioned background technology, and it is possible to provide information related to sleep disorders based on data measured in each user's individual sleep environment, and at the same time to reduce the possibility of occurrence of comorbidities accompanying sleep disorders. By predicting and providing medical diagnostic information to the user, it is possible to provide an effect of improving health management efficiency.

Effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

Various aspects are now described with reference to the drawings, wherein like reference numbers are used to refer to like elements throughout. In the following example, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. It will be evident, however, that such aspect(s) may be practiced without these specific details.
1 is a conceptual diagram illustrating a system in which various aspects of a computing device for providing sleep analysis information based on sleep environment sensing data related to a user's sleep environment related to an embodiment of the present disclosure can be implemented.
2 is a block diagram of a computing device for providing sleep analysis information based on sleep environment sensing data related to a user's sleep environment related to an embodiment of the present disclosure.
3 illustrates an exemplary diagram of a computing device for acquiring sleep sensing data related to a user's sleep environment according to an embodiment of the present disclosure.
4 shows an exemplary diagram exemplarily illustrating a learning process of a sleep evaluation model related to an embodiment of the present disclosure.
5 is a schematic diagram illustrating one or more network functions related to an embodiment of the present disclosure.
6 is a flowchart illustrating an exemplary method of providing sleep analysis information based on sleep sensing data related to a user's sleep environment related to an embodiment of the present disclosure.

Advantages and features of the present disclosure, and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments are provided so that the disclosure of the present disclosure is complete, and those of ordinary skill in the art to which the present disclosure belongs It is provided to fully inform those skilled in the art of the scope of the present disclosure, which is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present disclosure. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present disclosure.

Unless otherwise defined, all terms (including technical and scientific terms) used herein may have the meaning commonly understood by those of ordinary skill in the art to which this disclosure belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

As used herein, the term “unit” or “module” refers to a hardware component such as software, FPGA, or ASIC, and “unit” or “module” performs certain roles. However, “part” or “module” is not meant to be limited to software or hardware. A “unit” or “module” may be configured to reside on an addressable storage medium or to reproduce one or more processors. Thus, as an example, “part” or “module” refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, Includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. Components and functionality provided within “parts” or “modules” may be combined into a smaller number of components and “parts” or “modules” or as additional components and “parts” or “modules”. can be further separated.

In this specification, a computer refers to all types of hardware devices including at least one processor, and may be understood as encompassing software configurations operating in the corresponding hardware device according to embodiments. For example, a computer may be understood to include, but is not limited to, smart phones, tablet PCs, desktops, notebooks, and user clients and applications running on each device.

Those skilled in the art will further appreciate that the various illustrative logical blocks, configurations, modules, circuits, means, logics, and algorithm steps described in connection with the embodiments disclosed herein may be implemented in electronic hardware, computer software, or combinations of both. It should be recognized that they can be implemented with To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, configurations, means, logics, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application. However, such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

Each step described in this specification is described as being performed by a computer, but the subject of each step is not limited thereto, and at least a portion of each step may be performed in different devices according to embodiments.

1 illustrates an exemplary diagram illustrating a system in which various aspects of a computing device for predicting a sleep state based on data measured in a user's sleep environment related to an embodiment of the present disclosure may be implemented.

A system according to embodiments of the present disclosure may include a computing device 100 , a user terminal 10 , an external server 20 , and a network. The computing device 100 , the user terminal 10 and the external server 20 according to embodiments of the present disclosure may mutually transmit/receive data for the system according to embodiments of the present disclosure through a network.

Networks according to embodiments of the present disclosure include Public Switched Telephone Network (PSTN), x Digital Subscriber Line (xDSL), Rate Adaptive DSL (RADSL), Multi Rate DSL (MDSL), Very High Speed DSL (VDSL). ), a variety of wired communication systems such as Universal Asymmetric DSL (UADSL), High Bit Rate DSL (HDSL), and Local Area Network (LAN) can be used.

In addition, the networks presented herein include Code Division Multi Access (CDMA), Time Division Multi Access (TDMA), Frequency Division Multi Access (FDMA), Orthogonal Frequency Division Multi Access (OFDMA), Single Carrier-FDMA (SC-FDMA) and Various wireless communication systems may be used, such as other systems.

The network according to the embodiments of the present disclosure may be configured regardless of its communication mode, such as wired and wireless, and is composed of various communication networks such as a personal area network (PAN) and a wide area network (WAN). can be In addition, the network may be a well-known World Wide Web (WWW), and may use a wireless transmission technology used for short-range communication, such as infrared (IrDA) or Bluetooth (Bluetooth). The techniques described herein may be used in the networks mentioned above, as well as in other networks.

According to an embodiment of the present disclosure, the computing device 100 may obtain sleep sensing data related to the user's sleep environment, and generate sleep analysis information related to the user's sleep state based on the sleep sensing data. Specifically, the computing device 100 may generate a sleep evaluation model that learns one or more network functions using the labeled training data set to output sleep analysis information based on the sleep sensing data. That is, the computing device 100 may obtain sleep sensing data related to the user's sleep environment, and process the sleep sensing data as an input of a sleep evaluation model that is a learned neural network model to generate sleep analysis information. In this case, the sleep analysis information provided by the computing device 100 may include at least one of: apnea severity information on the user's sleep-related apnea occurrence degree, sleep stage information on a change in sleep state, and disease prediction information on the possibility of disease occurrence. may contain one. In other words, the computing device 100 may provide information related to a sleep disorder and information for identifying a sleep stage based on data measured in an individual user's sleep environment, and at the same time, a comorbidity associated with a sleep disorder. By predicting the possibility of occurrence and providing medical diagnostic information to the user, it is possible to provide an effect of improving health management efficiency. A detailed configuration of the computing device 100 of the present disclosure and effects according to the configuration will be described in detail below with reference to FIG. 2 below.

Also, the computing device 100 may be a terminal or a server, and may include any type of device. The computing device 100 is a digital device, and may be a digital device equipped with a processor, such as a laptop computer, a notebook computer, a desktop computer, a web pad, and a mobile phone, and having a computing power having a memory. The computing device 100 may be a web server that processes a service. The above-described types of servers are merely examples, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the user terminal 10 is a terminal capable of receiving information related to a user's sleep through information exchange with the computing device 100 , and may refer to a terminal possessed by the user. For example, the user terminal 10 may be a terminal related to a user who wants to improve health through information related to his or her sleeping habits. The user may receive information related to his/her sleep, prediction information on the possibility of occurrence of a comorbid disease, information for reducing the likelihood of occurrence of a disease, or information for improving health, through the user terminal 10 .

The user terminal 10 is a customer terminal (UE), a mobile, a PC capable of wireless communication, a mobile phone, a kiosk, a cellular phone, a cellular, a cellular terminal, a subscriber unit, a subscriber station, a mobile station, a terminal, a remote station, a PDA, a remote terminal Any that can use a wireless mechanism, such as an access terminal, user agent, cellular phone, wireless phone, session initiation protocol (SIP) phone, wireless local loop (WLL) station, portable device with wireless access capability, wireless model It may be referred to as a device of, but is not limited to. In addition, the user terminal 10 may be referred to as any device capable of using a wired connection mechanism, such as a wired fax machine, a PC equipped with a wired model, a wired telephone, a terminal capable of wired communication, etc., but is not limited thereto. does not

According to an embodiment of the present disclosure, the external server 20 may be a server that stores health checkup information or sleep checkup information for a plurality of users. For example, the external server 20 may be at least one of a hospital server and an information server, and may be a server that stores information about a plurality of polysomnography records, electronic health records, electronic medical records, and the like. For example, polysomnography records include information on brain waves, eye movements, muscle movements, respiration, electrocardiogram, etc. during sleep of the sleep test subject and sleep diagnosis results corresponding to the information (eg, sleep apnea, sleep disorders, etc.). ) may include information about In addition, the polysomnography record may include information on whether or not the test subject has a disease, for example, the polysomnography subject has heart disease (ischemic myocardial infarction, heart attack, arrhythmia, etc.), brain disease (hemorrhagic stroke, ischemic stroke, transient ischemia, cerebral infarction, etc.) and mental disorders (depression, anxiety disorder, bipolar disorder, panic disorder, dementia, delusional disorder, etc.). Information stored in the external server 20 may be utilized as training data, verification data, and test data for learning the neural network in the present disclosure.

The computing device 100 of the present disclosure may receive health checkup information or sleep checkup information from the external server 20 , and build a learning data set based on the information. The computing device 100 may generate a sleep evaluation model for calculating sleep analysis information corresponding to the user by learning the sleep evaluation model including one or more network functions through the training data set. A configuration for constructing a learning data set for learning a neural network of the present disclosure and a detailed description of a learning method using the learning data set will be described later with reference to FIG. 2 .

The external server 20 is a digital device, and may be a digital device equipped with a processor, such as a laptop computer, a notebook computer, a desktop computer, a web pad, and a mobile phone, and having a computing capability with a memory. The external server 20 may be a web server that processes a service. The above-described types of servers are merely examples, and the present disclosure is not limited thereto.

Hereinafter, a method in which the computing device 100 provides sleep analysis information based on the sleep environment sensing data will be described in more detail below with reference to FIG. 2 .

2 is a block diagram of a computing device for providing sleep analysis information based on sleep environment sensing data related to a user's sleep environment related to an embodiment of the present disclosure.

As shown in FIG. 2 , the computing device 100 may include a network unit 110 , a memory 120 , a sensor unit 130 , an environment creation unit 140 , and a processor 150 . Components included in the aforementioned computing device 100 are exemplary and the scope of the present disclosure is not limited to the aforementioned components. That is, additional components may be included or some of the above-described components may be omitted depending on implementation aspects for the embodiments of the present disclosure.

According to an embodiment of the present disclosure, the computing device 100 may include the user terminal 10 and the network unit 110 for transmitting and receiving data to and from the external server 20 . The network unit 110 may transmit/receive data for performing a method of providing sleep analysis information based on the sleep environment sensing data according to an embodiment of the present disclosure to other computing devices, servers, and the like. That is, the network unit 110 may provide a communication function between the computing device 100 , the user terminal 10 , and the external server 20 . For example, the network unit 110 may receive sleep examination records and electronic health records for a plurality of users from a hospital server. Additionally, the network unit 110 may allow information transfer between the computing device 100 and the user terminal 10 and the external server 20 by calling a procedure to the computing device 100 .

The network unit 110 according to an embodiment of the present disclosure includes a Public Switched Telephone Network (PSTN), x Digital Subscriber Line (xDSL), Rate Adaptive DSL (RADSL), Multi Rate DSL (MDSL), VDSL ( A variety of wired communication systems such as Very High Speed DSL), Universal Asymmetric DSL (UADSL), High Bit Rate DSL (HDSL), and Local Area Network (LAN) can be used.

In addition, the network unit 110 presented herein is CDMA (Code Division Multi Access), TDMA (Time Division Multi Access), FDMA (Frequency Division Multi Access), OFDMA (Orthogonal Frequency Division Multi Access), SC-FDMA ( A variety of wireless communication systems can be used, such as Single Carrier-FDMA) and other systems.

In the present disclosure, the network unit 110 may be configured regardless of its communication mode, such as wired and wireless, and may be composed of various communication networks such as a short-range network (PAN: Personal Area Network) and a local area network (WAN: Wide Area Network). can In addition, the network may be a well-known World Wide Web (WWW), and may use a wireless transmission technology used for short-range communication, such as infrared (IrDA) or Bluetooth (Bluetooth). The techniques described herein may be used in the networks mentioned above, as well as in other networks.

According to an embodiment of the present disclosure, the memory 120 may store a computer program for performing the method for providing sleep analysis information according to an embodiment of the present disclosure, and the stored computer program is executed by the processor 150 . It can be read and driven. In addition, the memory 120 may store any type of information generated or determined by the processor 150 and any type of information received by the network unit 110 . Also, the memory 120 may store data related to the user's sleep. For example, the memory 120 may temporarily or permanently store input/output data (eg, sleep sensing data related to the user's sleep environment, sleep analysis information, health promotion information, etc.).

According to an embodiment of the present disclosure, the memory 120 is a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (eg, a SD or XD memory, etc.), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read (PROM) -Only Memory), a magnetic memory, a magnetic disk, and an optical disk may include at least one type of storage medium. The computing device 100 may operate in relation to a web storage that performs a storage function of the memory 120 on the Internet. The description of the above-described memory is only an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the computing device 100 may acquire sleep sensing data using the sensor unit 130 . The sensor unit 130 may acquire sleep sensing data related to the user's sleep environment. The sleep sensing data may include the user's respiration information obtained for a predetermined time period in relation to the user's sleep environment. In addition, the sleep sensitive data may further include movement information and heart rate information of the user acquired during a predetermined time period in relation to the user's sleep environment. That is, the sensor unit 130 may be configured to include a sensor module for acquiring at least one of respiration information, motion information, and heart rate information from the user in relation to the user's sleeping environment.

According to an embodiment of the present disclosure, the sensor unit 130 includes one or more transmitting modules for transmitting radio waves of a specific frequency and a receiving module for receiving a reflected wave generated in response to radio waves (eg, microwaves) of a specific frequency may be provided, including In this case, the sensor unit 130 may acquire sleep sensing data from the user in a non-contact manner by detecting a phase difference or frequency change according to a movement distance of a reflected wave corresponding to a radio wave transmitted from the transmitting module. For example, when a radio wave transmitted through the transmission module collides with an object and is reflected, a phase difference or a frequency may be changed. For example, when an object is closer to the transmission module, the frequency of the reflected wave may be shortened, and if it is gradually moving away, the frequency of the reflected wave may be increased. That is, the sensor unit 130 may acquire sleep sensing information related to the user's breathing by detecting the movement of the user's body (eg, abdomen or chest, etc.) based on the phase difference or frequency change state of the reflected wave. For example, as shown in FIG. 3 , the sensor unit 130 may be located in an area of a space in which the user sleeps when the user sleeps. The sensor unit 130 may transmit a radio wave having a specific frequency through the transmitting module, and may receive a reflected wave reflected from the user's body in response to the radio wave through the receiving module. In addition, the sensor unit 130 may acquire at least one of the user's respiration information, motion information, and heart rate information based on the phase difference or frequency change of the reflected wave received through the reception module. However, the position at which the sensor unit of the present disclosure is provided is not limited to the position shown in FIG. 3 , and a plurality of sensors may be provided in various areas in the space where the user sleeps.

In addition, as described above, the sensor unit 130 includes a transmission module and a reception module to acquire sleep sensing data through a radio frequency (RF) sensing method, but the present disclosure is not limited thereto. In an additional embodiment, the sensor unit 130 of the present disclosure is configured to further include a sensor module for transmitting and detecting a wifi radio wave and a sensor module for detecting an airflow related to the user's breathing, such as sleep sensing data can be obtained.

According to an embodiment of the present disclosure, the sensor unit 130 includes information on at least one of the user's body temperature, room temperature, indoor airflow, indoor humidity, indoor sound, and indoor illuminance in relation to the user's sleeping environment. It may include one or more environment sensing modules for acquiring indoor environment information. The indoor environment information is information related to the user's sleep environment, and may be information serving as a reference for considering the influence of external factors on the user's sleep through the sleep state related to the change in the user's sleep stage. The one or more environment sensing modules may include, for example, at least one sensor module selected from a temperature sensor, an airflow sensor, a humidity sensor, an acoustic sensor, and an illuminance sensor. However, the present invention is not limited thereto, and various sensors that may affect the user's sleep may be further included.

According to an embodiment of the present disclosure, the computing device 100 may adjust the user's sleeping environment through the environment creation unit 140 . Specifically, the environment builder 140 may include one or more driving modules, and operates a driving module related to at least one of temperature, wind direction, humidity, sound, and illuminance based on an environment control signal received from the processor 150 . By doing so, the user's sleeping environment can be adjusted. The environment control signal may be a signal generated by the processor 150 based on a sleep state determination according to a change in the user's sleep stage, for example, lowering the temperature or raising the humidity in relation to the user's sleep environment. Or, it may be a signal to lower the illuminance or lower the sound. The detailed description of the above-described environment control signal is only an example, and the present disclosure is not limited thereto.

The one or more driving modules may include, for example, at least one of a temperature control module, a wind direction control module, a humidity control module, a sound control module, and an illuminance control module. However, the present invention is not limited thereto, and the one or more driving modules may further include various driving modules capable of bringing a change to the user's sleeping environment. That is, the environment builder 140 may adjust the user's sleep environment by driving one or more driving modules based on the environment control signal of the processor 150 .

According to another embodiment of the present disclosure, the environment creation unit 140 may be implemented through connection through the Internet of Things (IOT). Specifically, the environment builder 140 may be implemented through connection with various devices that can change the indoor environment in relation to the space where the user is located for sleeping. For example, the environment builder 140 may be implemented as a smart air conditioner, a smart heater, a smart boiler, a smart window, a smart humidifier, a smart dehumidifier, and a smart lighting based on connection through the Internet of Things. The detailed description of the above-described environment composition unit is only an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 150 may include one or more cores, and a central processing unit (CPU) of a computing device, a general purpose graphics processing unit (GPGPU) , data analysis such as a tensor processing unit (TPU), and a processor for deep learning.

The processor 150 may read a computer program stored in the memory 120 to perform data processing for machine learning according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, the processor 150 may perform an operation for learning the neural network. The processor 150 is for learning of a neural network, such as processing input data for learning in deep learning (DL), extracting features from input data, calculating an error, and updating the weight of the neural network using backpropagation. calculations can be performed.

In addition, at least one of a CPU, a GPGPU, and a TPU of the processor 150 may process learning of a network function. For example, the CPU and the GPGPU can process learning of a network function and data classification using the network function. Also, in an embodiment of the present disclosure, learning of a network function and data classification using the network function may be processed by using the processors of a plurality of computing devices together. In addition, the computer program executed in the computing device according to an embodiment of the present disclosure may be a CPU, GPGPU or TPU executable program.

In the present specification, a network function may be used interchangeably with an artificial neural network and a neural network. In the present specification, a network function may include one or more neural networks, and in this case, an output of the network function may be an ensemble of outputs of one or more neural networks.

In this specification, a model may include a network function. The model may include one or more network functions, in which case the output of the model may be an ensemble of outputs of the one or more network functions.

The processor 150 may read a computer program stored in the memory 120 to provide a sleep evaluation model according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, the processor 150 may perform a calculation for calculating sleep analysis information based on the sleep sensing data. According to an embodiment of the present disclosure, the processor 150 may perform a calculation for learning the sleep evaluation model.

According to an embodiment of the present disclosure, the processor 150 may typically process the overall operation of the computing device 100 . The processor 150 processes signals, data, information, etc. input or output through the above-described components or by driving an application program stored in the memory 120 to provide or process appropriate information or functions to the user terminal. there is.

According to an embodiment of the present disclosure, the processor 150 may receive sleep sensing data related to the user's sleep environment from the sensor unit 130 . The sleep sensing data is information acquired through the sensor unit 130 and may include user's respiration information for a predetermined time period in relation to the user's sleep environment. For example, the sleep sensing data may be user's respiration information acquired for 60 minutes. The sleep sensing data may be user's respiration information for 60 minutes measured at a 10 Hz cycle, and may be time series information having a matrix size of 3600 x 10. Description of specific numerical values for sleep sensing data is only an example, and the present disclosure is not limited thereto. The sleep sensing data may be input data processed as an input to the sleep evaluation model to calculate a result value (ie, sleep analysis information) through the sleep evaluation model, which is a neural network model.

In an additional embodiment, the sleep sensing data may further include movement information and heart rate information of the user obtained through the sensor unit 130 for a predetermined time period in relation to the user's sleep environment. That is, according to an implementation aspect of the sensor unit 130 , the sleep sensing data may include at least one of user's respiration information, motion information, and heart rate information. In this case, the sleep sensing data may include, for example, user's respiration information for 60 minutes measured at a 10 Hz cycle, user movement information and heart rate information for 60 minutes measured at a 1 Hz cycle. Specific numerical description of the above-described sleep sensing data is only an example, and the present disclosure is not limited thereto.

According to an embodiment, in the present disclosure, the sleep analysis information calculated by the sleep evaluation model based on the sleep sensing data includes three elements (ie, respiration information, motion information, and heart rate information) obtained through the sensor unit 130 . ), it can be calculated with higher accuracy. However, three elements (ie, respiration information, motion information, and heart rate information) obtained through the sensor unit 130 as an input of a sleep evaluation model for calculating sleep analysis information in the present disclosure are not necessarily required. .

According to an embodiment of the present disclosure, the processor 150 may calculate sleep analysis information. Specifically, the processor 150 may calculate the user's sleep analysis information by inputting the user's sleep sensing data to the sleep evaluation model. In this case, the sleep evaluation model is a model for predicting the user's sleep state and disease occurrence, and may provide sleep analysis information based on data obtained from the user's sleep.

The sleep evaluation model is a model for calculating information on at least one of a user's sleep state and the possibility of disease occurrence based on sleep sensing data, and may include one or more network functions.

A sleep evaluation model is composed of one or more network functions, which may be composed of a set of interconnected computational units, which may generally be referred to as 'nodes'. These 'nodes' may also be referred to as 'neurons'. The one or more network functions are configured by including at least one or more nodes. Nodes (or neurons) constituting one or more network functions may be interconnected by one or more 'links'.

In the neural network, one or more nodes connected through a link may relatively form a relationship between an input node and an output node. The concepts of an input node and an output node are relative, and any node in an output node relationship with respect to one node may be in an input node relationship in a relationship with another node, and vice versa. As described above, an input node to output node relationship may be created around a link. One or more output nodes may be connected to one input node through a link, and vice versa.

In the relationship between the input node and the output node connected through one link, the value of the output node may be determined based on data input to the input node. Here, a node interconnecting the input node and the output node may have a weight. The weight may be variable, and may be changed by a user or an algorithm in order for the neural network to perform a desired function. For example, when one or more input nodes are interconnected to one output node by respective links, the output node sets values input to input nodes connected to the output node and links corresponding to the respective input nodes. An output node value may be determined based on the weight.

As described above, in a neural network, one or more nodes are interconnected through one or more links to form an input node and an output node relationship in the neural network. The characteristics of the neural network may be determined according to the number of nodes and links in the neural network, the correlation between the nodes and the links, and the value of a weight assigned to each of the links. For example, when the same number of nodes and links exist and there are two neural networks having different weight values between the links, the two neural networks may be recognized as different from each other.

Some of the nodes constituting the neural network may configure one layer based on distances from the initial input node. For example, a set of nodes having a distance of n from the initial input node may constitute n layers. The distance from the initial input node may be defined by the minimum number of links that must be passed to reach the corresponding node from the initial input node. However, the definition of such a layer is arbitrary for description, and the order of the layer in the neural network may be defined in a different way from the above. For example, a layer of nodes may be defined by a distance from the final output node.

The initial input node may mean one or more nodes to which data is directly input without going through a link in a relationship with other nodes among nodes in the neural network. Alternatively, in a relationship between nodes based on a link in a neural network, it may mean nodes that do not have other input nodes connected by a link. Similarly, the final output node may refer to one or more nodes that do not have an output node in relation to other nodes among nodes in the neural network. In addition, the hidden node may mean nodes constituting the neural network other than the first input node and the last output node. The neural network according to an embodiment of the present disclosure may have more nodes in the input layer than nodes in the hidden layer close to the output layer, and may be a neural network in which the number of nodes decreases as the input layer progresses to the hidden layer.

A neural network may include one or more hidden layers. The hidden node of the hidden layer may have the output of the previous layer and the output of the neighboring hidden nodes as inputs. The number of hidden nodes for each hidden layer may be the same or different. The number of nodes of the input layer may be determined based on the number of data fields of the input data and may be the same as or different from the number of hidden nodes. Input data input to the input layer may be calculated by a hidden node of the hidden layer and may be output by a fully connected layer (FCL) that is an output layer.

In more detail, the processor 150 may build a training data set for training the sleep evaluation model. The processor 150 may receive a sleep diagnosis data set including each of a plurality of sleep diagnosis data corresponding to each of a plurality of users. These sleep diagnosis data may be information received from the external server 20 . The sleep diagnosis data may include information related to a sleep examination of a specific user, for example, information related to polysomnography. The user's polysomnography-related information may include information on EEG, safety level, jaw EMG, respiration, electrocardiogram, arterial blood oxygen saturation, anterior peroneal EMG, and other physiological and physical variables for several hours.

In addition, the information related to the polysomnography may include information on the sleep analysis result analyzed through the above information. As a specific example, as a result of diagnosis of information acquired during the user's sleep period, the information that the Apnea-Hypopnea Index (AHI) related to the sleep apnea index is 21, the user's sleep stage at a specific point in time is REM sleep information may be included. The apnea-hypopnea index is information that is the basis for judging sleep apnea. In general, an AHI of less than 5 is normal, an AHI of 5 or more but less than 15 is mild sleep apnea, and an AHI of 15 or more is less than 30. Moderate sleep apnea, with an AHI of 30 or higher, can be classified as severe sleep apnea.

The processor 150 may generate a learning input data set by extracting information about the user's respiration, heart rate, and movement for a predetermined time period from each of the plurality of sleep diagnosis data. For example, the processor 150 may extract information related to the user's respiration for 1 hour from the sleep diagnosis data. In this case, the learning input data may be the user's respiration information for 60 minutes measured at a 10 Hz cycle, and may be time series information having a matrix size of 3600 x 10.

That is, the learning input data set may include information on at least one of information about respiration, heart rate, and movement for a predetermined time extracted from each of a plurality of sleep diagnosis data.

In addition, the processor 150 may generate learning output data by extracting at least one of information on an apnea index, information on a sleep state, and information on whether or not a disease occurs from each of a plurality of sleep diagnosis data.

As a specific example, the processor 150 may extract information about the apnea index of user A (eg, AHI of 19) from the sleep diagnosis data of user A. As another example, the processor 150 may extract information that user B has a heart disease related to arrhythmia from user B's sleep diagnosis data. As another example, the processor 150 may extract information on the sleep phase of user C at each point in time from the sleep diagnosis data of user C. The detailed description of the above-described learning output data is only an example, and the present disclosure is not limited thereto.

That is, the processor 150 may generate a learning output data set based on each of the plurality of sleep diagnosis data. In other words, the learning output data set may be information related to a sleep measurement result (or sleep diagnosis result) extracted from each of a plurality of diagnostic data.

In addition, the processor 150 may generate a labeled training data set by matching the training input data set and the training output data set. For example, extracting information about respiration measured for a predetermined time period (eg, time series data related to respiration measured for 1 hour) from the sleep diagnosis data of the first user to generate first learning input data, When the first learning output data is generated by extracting the information that the sleep diagnosis result corresponding to the information measured during the predetermined time period from the sleep diagnosis data of 1 user has an AHI of 19, the processor 150 performs the same operation at the same time. The labeled training data set may be generated by matching the first learning input data and the first learning output data obtained from the first user who is the user. That is, the processor 150 may generate a labeling training data set by matching each of the training input data and each of the corresponding training output data.

According to an embodiment of the present disclosure, the processor 150 may train a sleep evaluation model. Specifically, the processor 150 may perform learning on one or more network functions constituting the life evaluation model by using the labeled training data set. Specifically, the processor 150 inputs each of the training input data sets to one or more network functions, and compares each of the output data calculated by the one or more network functions with each of the training output data sets corresponding to the labels of each of the training input data sets. This can lead to an error. That is, in training of a neural network, learning input data may be input to an input layer of one or more network functions, and learning output data may be compared with outputs of one or more network functions. The processor 150 may train the neural network based on an error between an operation result of one or more network functions on the learning input data and an error of the learning output data (label).

Also, the processor 150 may adjust the weights of one or more network functions in a backpropagation method based on the error. That is, the processor 150 may adjust the weight so that the output of the one or more network functions approaches the learning output data based on an error between the operation result of the one or more network functions on the learning input data and the learning output data.

When the learning of one or more network functions is performed for more than a predetermined epoch, the processor 150 may determine whether to stop the learning by using the verification data. The predetermined epoch may be part of an overall learning objective epoch. The validation data may consist of at least a portion of the labeled training data set. That is, the processor 150 performs learning of the neural network through the learning data set, and after the learning of the neural network is repeated more than a predetermined epoch, it is determined whether the learning effect of the neural network is above a predetermined level using the verification data. can For example, when the processor 150 performs learning with a target repetition learning number of 10 times using 100 training data, after performing 10 repetition learning, which is a predetermined epoch, 3 using 10 verification data By performing repeated learning, if the change in the neural network output during three repetitions of learning is less than or equal to a predetermined level, it is determined that further learning is meaningless and learning can be terminated. That is, the verification data may be used to determine the completion of learning based on whether the effect of learning for each epoch in the iterative learning of the neural network is greater than or equal to a certain level. The above-described number of training data and verification data and the number of repetitions are merely examples, and the present disclosure is not limited thereto.

The processor 150 may generate a sleep evaluation model by testing the performance of one or more network functions using the test data set and determining whether to activate the one or more network functions. The test data may be used to verify the performance of the neural network, and may be composed of at least a part of the training data set. For example, 70% of the training data set can be utilized for training the neural network (i.e., learning to adjust the weights to output a result similar to a label), and 30% is a test to validate the performance of the neural network. It can be used as data. The processor 150 may determine whether to activate the neural network according to whether it is above a predetermined performance by inputting a test data set to the trained neural network and measuring an error. The processor 150 verifies the performance of the trained neural network using test data on the trained neural network, and when the learned neural network performance is greater than or equal to a predetermined criterion, the processor 150 may activate the neural network to be used in other applications. In addition, when the performance of the trained neural network is less than or equal to a predetermined criterion, the processor 150 may deactivate and discard the neural network. For example, the processor 150 may determine the performance of the generated neural network model based on factors such as accuracy, precision, and recall. The above-described performance evaluation criteria are merely examples, and the present disclosure is not limited thereto. According to an embodiment of the present disclosure, the processor 150 may generate a plurality of neural network models by independently learning each neural network, and only neural networks with a certain performance or higher may be used to calculate sleep analysis information by evaluating the performance.

Hereinafter, a data operation process of one or more network functions (or neural networks) will be described in more detail with reference to FIG. 4 .

In the present disclosure, the sleep environment sensing data 300 used as an input of the neural network may include the user's respiration information 320 obtained for a predetermined time period in relation to the user's sleep environment. For example, the sleep sensing data 300 may be the user's respiration information 320 acquired for 60 minutes. In this case, the sleep sensing data 300 may be the user's respiration information 320 for 60 minutes measured at a 10Hz cycle, and may be time series information having a matrix size of 3600 (60 minutes converted to seconds) x 10. That is, in the present disclosure, data used as an input of a neural network may be data having a relatively long length. Since it is a long time series of data used as an input to the neural network, it may be difficult to calculate the neural network.

For example, in the case of a general convolutional neural network, a receptive field may be extended by increasing a kernel size or by stacking it as deep as possible (depth = number of filters). The receptive field may relate to an area of a filter for processing data related to an input of the neural network. For example, as the number of receptive fields increases, the amount of information used in the process of calculating the output may increase, and as the number of receptive fields decreases, the amount of information used in the process of calculating the output may decrease.

However, as described above, when the kernel size is increased or the depth is increased, overfitting may occur as the number of parameters increases. Also, in the operation of a general convolutional neural network, the length of the output terminal corresponding to the input may be reduced according to the stride related to the position interval to which the position of the filter is applied. For example, if the stride is 2, the output length may be 1/2 of the input length as the interval of the filter is 2. In addition, data loss may occur during the pooling process of a general CNN. For example, in the case of max pooling, since only a record with a maximum value among records included in a specific kernel size is extracted as a feature, a loss of feature values corresponding to the remaining records may occur. In the time series data, the positions of input points must be maintained, and lengths of the data of the input terminal and the output terminal may be different during the calculation process.

That is, when the sleep environment sensing data (ie, long time series data) related to the input of the neural network of the present disclosure is to be calculated through a general convolutional neural network, there is a risk of overfitting as the parameter increases, Also, it may be difficult to learn the temporal relationship of time series data due to the change in the length of the input and output.

In an additional embodiment, the sleep sensing data 300 may further include the user's movement information 310 and heart rate information 330 for a predetermined time. In this case, the sleep sensing data includes the user's respiration information 320 for 60 minutes measured at a 10 Hz cycle, the user's movement information 310 and heart rate information 330 for 60 minutes measured at a 1 Hz cycle, ( It may be time series information having a matrix size of 3600 x 10) + (3600 x 1) + (3600 x 1). In this case, as longer data than sleep sensing data including only respiration information 320 is utilized as an input of the neural network, the efficiency of calculation and learning through the convolutional neural network may be further reduced.

Accordingly, one or more network functions constituting the sleep evaluation model of the present disclosure may include a Dilated-Convolutional Neural Network 430 for strengthening long-term associations without loss of input data. can The extended convolutional neural network 430 strengthens long-term associations by expanding the receptive fields of filters with respect to inputs of one or more network functions, and adds zero padding to filters with respect to inputs of one or more network functions. It may be characterized by maintaining the length of the input and output. Specifically, the extended convolutional neural network may extend the receptive field by considering the dilation rate, which is the interval between kernels, as a parameter. For example, a 3x3 kernel having a dilation rate of 2 may have the same field of view as a 5x5 kernel, and thus the receptive field may be extended. That is, by using the dilation operation to strengthen the long-term correlation by increasing the receptive field while maintaining the number of parameters, processing of long time series data can be made possible. In addition, the extended convolutional neural network 430 may prevent loss by adding zero padding to a loss record (ie, a skipped record) that may occur during a dilation operation process. Accordingly. It is possible to reduce the loss of features and at the same time maintain the length of the input and the output (temporal correlation is maintained).

Also, one or more network functions may include 1D-CNN 440 . 1D-CNN 440 may be a network function for reducing the dimension of long time series data. The 1D-CNN 440 may be a network function for reducing (or compressing) the dimension of data transmitted from the extended convolutional neural network 430 located in the latter part of the neural network.

That is, the input data input to the input layer (eg, sleep environment sensing data and learning input data) undergoes an operation process through the extended convolutional neural network 430, and the dimension is reduced through the 1D-CNN 440, The output may be output by a fully connected layer (FCL) 450 that is an output layer.

The sleep evaluation model of the present disclosure can strengthen the long-term correlation by increasing the receptive field while maintaining the number of parameters required for calculation through the extended convolutional neural network, and it is possible to reduce the loss of features that occur in the calculation process. At the same time, it is possible to maintain the length of the input and output (temporal correlation). Accordingly, the processor 150 performs learning on the neural network model configured including the convolutional neural network extended through the training data set, thereby generating the sleep analysis information 500 based on the sleep sensing data. can create

The sleep evaluation model may output sleep analysis information 500 by receiving sleep-related sensing data as an input. The sleep analysis information 500 may include at least one of apnea severity information on the user's sleep-related apnea occurrence degree, sleep stage information on a change in sleep state, and disease prediction information on the possibility of disease occurrence.

The apnea severity information may include information about an Apnea-Hypopnea Index (AHI) measured for a predetermined time in relation to the user's sleep environment. For example, the apnea severity information may include information that the apnea-hypopnea index for 90 minutes is 8, and based on the index, it may be determined that the user corresponds to mild sleep apnea. The sleep stage information may include information on changes in one or more sleep states according to time changes in relation to the user's sleep environment. For example, the sleep stage information may include information on a sleep stage for each point of time of a user for a predetermined time or information on a ratio of a sleep stage. The disease prediction information may include prediction information on at least one of a sleep disease, a mental disease, a brain disease, and a cardiovascular disease. For example, the disease prediction information may include information that a user has a 40% chance of developing a brain disease within 3 years. Specific description of the above-described apnea severity information, sleep stage information, and disease prediction information is only an example, and the present disclosure is not limited thereto.

That is, the sleep evaluation model of the present disclosure outputs raw data to output (or, End to End deep learning) output data (ie, sleep analysis information) without a separate pre-processing process for input data (sleep sensing data), which is time series information. ) can be a model. In other words, in the data operation process using the neural network, the data pre-processing process may be omitted.

In addition, the sleep sensing data processed as an input of the sleep evaluation model may include the user's respiration information measured for a predetermined time period. That is, the sleep evaluation model of the present disclosure may be a neural network model that calculates the sleep analysis information as described above based on only a minimum element (ie, user's respiration information). In general, various information (e.g., EEG, safety level, jaw EMG, respiration, electrocardiogram, arterial blood oxygen saturation, anterior peroneus EMG, other physiological and physical parameters, etc.) may be required for sleep evaluation (or analysis). . The present disclosure is intended to provide a user with the convenience of obtaining sleep evaluation information, and may provide a neural network model (ie, a sleep evaluation model) that utilizes only information about respiration during a user's sleep as input data. Accordingly, a sensing process for acquiring a plurality of input data may be reduced. This may provide convenience for continuous health management by facilitating sleep evaluation (or analysis) of the user in real life.

In a further embodiment, the sleep sensing data may further include movement information and heart rate information of the user measured for a predetermined time period. In this case, the sleep evaluation model may be a neural network model trained to provide sleep analysis information based on three factors (ie, user's respiration information, movement information, and heart rate information). In this case, as the sleep sensing data includes the user's respiration information, motion information, and heart rate information, the input data of the neural network may include three elements. That is, as three factors are considered as input data, the accuracy of sleep analysis information calculated through the sleep evaluation model may be further improved.

According to an embodiment of the present disclosure, the processor 150 may generate health management information for improving the user's sleep environment and health based on the sleep analysis information. The health management information may include at least one of information on eating habits, information on exercise amount, and information on an optimal sleeping environment.

As a specific example, when the sleep analysis information includes information that corresponds to mild sleep apnea as the AHI is calculated as 13, the processor 150 is 'information recommending weight loss through exercise or a drug that relaxes muscles It is possible to generate health management information including 'information to refrain from

As another example, when the sleep analysis information includes information that corresponds to severe sleep apnea as the AHI is calculated as 32, the processor 150 provides health management information including 'information recommending CPAP treatment'. can create

For another example, if the sleep analysis information includes information that the probability of occurrence of arrhythmia within the next 3 years is 70%, the processor 150 'uses vegetable oils such as soybean oil, perilla oil, and sesame oil rather than animal oil and eggs, Health that includes information on eating habits to consume foods with a high cholesterol content, such as fish, 2-3 times a week or less, or information on the amount of exercise that recommends avoiding excessive strength training and recommending 30 minutes of aerobic exercise a day You can create management information. Specific description of information included in the aforementioned sleep analysis information and health care information is merely an example, and the present disclosure is not limited thereto.

Also, the processor 150 may determine to transmit the health management information generated based on the sleep analysis information to the user terminal 10 . Accordingly, the user can easily recognize information for improving apnea according to his or her sleep or reducing the probability of developing a comorbidity.

According to an embodiment of the present disclosure, the processor 150 sleeps based on the amount of change in the disease prediction information output by changing the respiration information for a predetermined time period included in the user's sleep sensing data input to the sleep evaluation model. A correlation between the sensing data and the disease prediction information may be derived.

For example, the processor 150 may adjust respiration information for a predetermined time period included in the sleep sensing data related to the user's sleep environment. In this case, as the input of the neural network model (ie, the sleep evaluation model) is changed, the disease prediction information may be changed and output. That is, the processor 150 may derive a correlation between sleep analysis information that is changed and output according to a change in sleep sensing data related to the input of the neural network. For example, the correlation between the sleep sensing data and the disease prediction information is information indicating that the probability of developing cardiovascular disease increases as the user's apnea cycle during sleep is shortened (that is, even if the apnea index is the same, the Depending on the disease, the probability of developing a disease varies, etc.). The detailed description of the correlation between the above-described sleep sensing data and disease prediction information is only an example, and the present disclosure is not limited thereto.

In other words, by deriving a correlation between respiration information and disease prediction information by a neural network in addition to commonly known respiration during sleep and a disease predicted in response to the respiration, meaningful insight into the relationship between respiration and disease can be provided.

According to an embodiment of the present disclosure, the processor 150 may generate sleep degrading factor information based on sleep stage information and indoor environment information. The indoor environment information is information about the user's sleeping environment obtained by the sensor unit 130, and includes information about at least one of the user's body temperature, the indoor temperature of the room where the user is located, indoor humidity, indoor sound, and indoor illuminance. may include

The sleep degrading factor information may be information about an external factor that impairs the user's sleep quality in the user's sleep process. For example, the sleep deterioration factor information may include information indicating that the quality of the user's sleep is impaired as the indoor temperature is lowered at a specific time. As another example, as the illuminance increases at a specific time, information indicating that the user's sleep quality is impaired may be included. The detailed description of the above-described sleep-decreasing factor information is only an example, and the present disclosure is not limited thereto.

More specifically, the processor 150 may identify a first time point at which the quality of the user's sleep deteriorates based on the sleep stage information. In general, the sleep stage may be divided into one or more stages according to the user's deep sleep level. For example, the sleep stage may be divided into stages 1 to 4, and as the stage increases, the user may be in deep sleep. These sleep phases may be repeated several times based on a certain time period (eg, 1 hour and 30 minutes) during the user's daily sleep. In this case, the processor 150 may identify a first time point at which the quality of the user's sleep deteriorates through the sleep stage information included in the sleep analysis information calculated through the sleep evaluation model. As a specific example, when the user's sleep stage is changed from the third stage to the first stage at 3:10 am (that is, when the user's sleep stage is changed from deep sleep to light sleep regardless of the repeat cycle of sleep), the processor 150 may identify the corresponding time point (ie, 3:10 pm) as the first time point at which the quality of the user's sleep is deteriorated. The detailed description of the above-described first time point is only an example, and the present disclosure is not limited thereto.

In addition, the processor 150 may identify a singularity related to the amount of change in the indoor environment information corresponding to the first time point. The indoor environment information is information related to the environment of the space in which the user sleeps, and may include information on at least one of the user's body temperature, indoor temperature, indoor airflow, indoor humidity, indoor sound, and indoor illuminance, and a sensor. It may be obtained through the unit 130 .

More specifically, the processor 150 identifies a singularity related to the variability of the indoor environment information based on whether the amount of change of each of the one or more pieces of information included in the indoor environment information exceeds a predetermined threshold change amount in response to the first time point. can

As a specific example, when the first time point at which the quality of the user's sleep deteriorates is 2:40 am, the processor 150 corresponds to the corresponding time point (ie, 2:40) at least one or more items included in the indoor environment information. Changes in information can be identified. In this case, the amount of change in temperature can be identified as +3°C, and as the predetermined threshold amount of change in response to the temperature is ±2°C, the processor 150 determines that the indoor temperature is a singularity related to the variability of indoor environment information. can be identified as That is, the processor 150 may identify that the factor that lowered the quality of the user's sleep during sleep is the temperature. Also, the processor 150 may generate sleep degrading factor information based on the identified singularity. For example, the processor 150 may generate sleep deterioration factor information indicating that the quality of the user's sleep is deteriorated due to a 2:40 change in the room temperature. That is, the sleep deterioration factor information generated by the processor 150 may include information on when the quality of the user's sleep deteriorates and external factor information related to the deterioration of the sleep quality. Also, for example, a change in temperature as described above does not affect changes in user A's sleep stage (i.e., does not degrade user A's sleep quality), but may affect changes in user B's sleep stage. (ie, lowering user B's sleep quality). That is, the sleep degrading factor information may reflect individual sleep-related characteristics. The above-described first time point, the amount of change of the indoor environment information, and the specific description of the predetermined threshold change amount are only examples, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 150 may generate an environment control signal for controlling the environment builder 140 based on the sleep degrading factor information. For example, when the sleep degrading factor information includes information indicating that the quality of the user's sleep has deteriorated due to a change in room temperature at 2:40, the processor 150 responds to the corresponding time point based on the sleep degrading factor information. Thus, an environment control signal for allowing the environment builder 140 to adjust the temperature may be generated. That is, the processor 150 generates an environment control signal for controlling the environment builder 140 in order to change the indoor environment of the space in which the user sleeps based on the specific time point and the sleep degrading factor included in the sleep degrading factor information. can do. Accordingly, the environment builder 140 may adjust the indoor environment in which the user is located by operating one or more driving modules based on the environment control signal. In this case, the sleep degrading factor information is generated based on a change in an individual's sleep stage, and since the environmental control signal is information generated based on the sleep degrading factor information as described above, the sleep environment adjustment operation in the present disclosure is The characteristics of each user related to the sleeping environment may be reflected.

That is, the processor 150 provides an optimal sleep environment corresponding to each of a plurality of users by generating an environment control signal for driving one or more driving modules included in the environment builder 140 based on the sleep degrading factor information. can do. Accordingly, the sleep efficiency of the user may be increased.

5 is a schematic diagram illustrating a network function according to an embodiment of the present disclosure.

A deep neural network (DNN) may refer to a neural network including a plurality of hidden layers in addition to an input layer and an output layer. Deep neural networks can be used to identify the latent structures of data. In other words, it can identify the potential structure of photos, texts, videos, voices, and music (e.g., what objects are in the photos, what the text and emotions are, what the texts and emotions are, etc.) . Deep neural networks include convolutional neural networks (CNNs), recurrent neural networks (RNNs), auto encoders, generative adversarial networks (GANs), and restricted Boltzmann machines (RBMs). boltzmann machine), a deep belief network (DBN), a Q network, a U network, a Siamese network, and the like. The description of the deep neural network described above is only an example, and the present disclosure is not limited thereto.

In an embodiment of the present disclosure, the network function may include an auto-encoder. The auto-encoder may be a kind of artificial neural network for outputting output data similar to input data. The auto encoder may include at least one hidden layer, and an odd number of hidden layers may be disposed between the input/output layers. The number of nodes in each layer may be reduced from the number of nodes in the input layer to an intermediate layer called the bottleneck layer (encoding), and then expanded symmetrically with reduction from the bottleneck layer to the output layer (symmetrical to the input layer). The nodes of the dimensionality reduction layer and the dimension restoration layer may or may not be symmetrical. The auto-encoder can perform non-linear dimensionality reduction. The number of input layers and output layers may correspond to the number of sensors remaining after pre-processing of input data. In the auto-encoder structure, the number of nodes of the hidden layer included in the encoder may have a structure that decreases as the distance from the input layer increases. If the number of nodes in the bottleneck layer (the layer with the fewest nodes located between the encoder and decoder) is too small, a sufficient amount of information may not be conveyed, so a certain number or more (e.g. more than half of the input layer, etc.) ) may be maintained.

The neural network may be trained by at least one of supervised learning, unsupervised learning, and semi-supervised learning. The training of the neural network is to minimize the error in the output. In the training of a neural network, iteratively input the training data into the neural network, calculate the output of the neural network and the target error for the training data, and calculate the error of the neural network from the output layer of the neural network to the input layer in the direction to reduce the error. It is a process of updating the weight of each node in the neural network by backpropagation in the direction. In the case of teacher learning, learning data in which the correct answer is labeled in each learning data is used (ie, labeled learning data), and in the case of comparative learning, the correct answer may not be labeled in each learning data. That is, for example, learning data in the case of teacher learning related to data classification may be data in which categories are labeled in each of the learning data. Labeled training data is input to the neural network, and an error can be calculated by comparing the output (category) of the neural network with the label of the training data. As another example, in the case of comparison learning related to data classification, an error may be calculated by comparing the input training data with the neural network output. The calculated error is backpropagated in the reverse direction (ie, from the output layer to the input layer) in the neural network, and the connection weight of each node of each layer of the neural network may be updated according to the backpropagation. The change amount of the connection weight of each node to be updated may be determined according to a learning rate. The computation of the neural network on the input data and the backpropagation of errors can constitute a learning cycle (epoch). The learning rate may be applied differently according to the number of repetitions of the learning cycle of the neural network. For example, in the early stage of learning of a neural network, a high learning rate can be used to enable the neural network to quickly obtain a certain level of performance to increase efficiency, and in the late learning period, a low learning rate can be used to increase accuracy.

In the training of neural networks, in general, the training data may be a subset of real data (that is, data to be processed using the trained neural network), and thus, errors on the training data are reduced, but errors on the real data are reduced. There may be increasing learning cycles. Overfitting is a phenomenon in which errors on actual data increase by over-learning on training data as described above. For example, a phenomenon in which a neural network that has learned a cat by seeing a yellow cat does not recognize that it is a cat when it sees a cat other than yellow may be a type of overfitting. Overfitting can act as a cause of increasing errors in machine learning algorithms. In order to prevent such overfitting, various optimization methods can be used. In order to prevent overfitting, methods such as increasing training data, regularization, or dropout in which a part of nodes in the network are omitted in the process of learning, may be applied.

Throughout this specification, computational model, neural network, network function, and neural network may be used interchangeably. (Hereinafter, the neural network is unified and described.) The data structure may include a neural network. And the data structure including the neural network may be stored in a computer-readable medium. Data structures, including neural networks, may also include data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, and loss functions for learning the neural network. there is. A data structure comprising a neural network may include any of the components disclosed above. That is, the data structure including the neural network includes all or all of the data input to the neural network, the weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, the activation function associated with each node or layer of the neural network, and the loss function for training the neural network. may be configured including any combination of In addition to the above-described configurations, a data structure including a neural network may include any other information that determines a characteristic of a neural network. In addition, the data structure may include all types of data used or generated in the operation process of the neural network, and is not limited to the above. Computer-readable media may include computer-readable recording media and/or computer-readable transmission media. A neural network may be composed of a set of interconnected computational units, which may generally be referred to as nodes. These nodes may also be referred to as neurons. A neural network is configured by including at least one or more nodes.

6 is a flowchart exemplarily illustrating a method of providing sleep analysis information based on sleep sensing data related to a user's sleep environment related to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, the computing device 100 may obtain the user's sleep sensing data ( 610 ).

According to an embodiment of the present disclosure, the computing device 100 may calculate the user's sleep analysis information by using the sleep sensing data as an input of the sleep evaluation model ( 620 ).

According to an embodiment of the present disclosure, the computing device 100 may transmit sleep analysis information to the user terminal ( 630 ).

The order of the steps illustrated in FIG. 6 described above may be changed if necessary, and at least one or more steps may be omitted or added. That is, the above-described steps are merely an embodiment of the present disclosure, and the scope of the present disclosure is not limited thereto.

The steps of a method or algorithm described in relation to an embodiment of the present disclosure may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. A software module may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present disclosure pertains.

Components of the present disclosure may be implemented as a program (or application) to be executed in combination with a computer, which is hardware, and stored in a medium. Components of the present disclosure may be implemented as software programming or software components, and similarly, embodiments may include various algorithms implemented as data structures, processes, routines, or combinations of other programming constructs, including C, C++ , may be implemented in a programming or scripting language such as Java, assembler, or the like. Functional aspects may be implemented in an algorithm running on one or more processors.

Those of ordinary skill in the art of the present disclosure will recognize that the various illustrative logical blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein include electronic hardware, (convenience For this purpose, it will be understood that it may be implemented by various forms of program or design code (referred to herein as "software") or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. A person skilled in the art of the present disclosure may implement the described functionality in various ways for each specific application, but such implementation decisions should not be interpreted as a departure from the scope of the present disclosure.

The various embodiments presented herein may be implemented as methods, apparatus, or articles of manufacture using standard programming and/or engineering techniques. The term “article of manufacture” includes a computer program, carrier, or media accessible from any computer-readable device. For example, computer-readable media include magnetic storage devices (eg, hard disks, floppy disks, magnetic strips, etc.), optical disks (eg, CDs, DVDs, etc.), smart cards, and flash memory. devices (eg, EEPROMs, cards, sticks, key drives, etc.). Also, various storage media presented herein include one or more devices and/or other machine-readable media for storing information. The term “machine-readable medium” includes, but is not limited to, wireless channels and various other media that can store, hold, and/or convey instruction(s) and/or data.

It is to be understood that the specific order or hierarchy of steps in the presented processes is an example of exemplary approaches. Based on design priorities, it is to be understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of the present disclosure. The appended method claims present elements of the various steps in a sample order, but are not meant to be limited to the specific order or hierarchy presented.

The description of the presented embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not intended to be limited to the embodiments presented herein, but is to be construed in the widest scope consistent with the principles and novel features presented herein.

Claims (3)

a processor that receives the user's sleep sensing data and calculates the user's sleep analysis information;
a memory storing program codes executable by the processor; and
a sensor unit including one or more environment sensing modules for obtaining indoor environment information;
including,
The sleep analysis information,
Includes sleep stage information on the change of one or more sleep states according to time change in relation to the user's sleep environment,
The processor is
Identifies a first time point at which the quality of the user's sleep deteriorates based on the sleep phase information, identifies a singularity related to the amount of change in the indoor environment information in response to the first time point, and based on the identified singularity to generate sleep-decreasing factor information,
Sleep state determination device through non-contact data collection.
A method performed on one or more processors of a computing device, comprising:
acquiring the user's sleep sensing data;
Calculating the sleep analysis information of the user using the sleep sensing data, wherein the sleep analysis information includes sleep stage information on changes in one or more sleep states according to time changes in relation to the user's sleep environment. step;
identifying a first time point at which the quality of the user's sleep deteriorates based on the sleep phase information;
identifying a singularity related to a change amount of indoor environment information obtained through an environment sensing module in response to the first time point; and
generating sleep-decreasing factor information based on the identified singularity;
containing,
A method of determining sleep state through non-contact data collection.
A computer program stored in a computer-readable storage medium, wherein the computer program performs the following operations when executed on one or more processors, the operations comprising:
acquiring the user's sleep sensing data;
Calculating the sleep analysis information of the user using the sleep sensing data, wherein the sleep analysis information includes sleep stage information on changes in one or more sleep states according to time changes in relation to the user's sleep environment. movement;
identifying a first time point at which the quality of the user's sleep deteriorates based on the sleep phase information;
identifying a singularity related to a change amount of indoor environment information acquired through an environment sensing module in response to the first time point; and
generating sleep degrading factor information based on the identified singularity;
comprising,
A computer program stored on a computer-readable storage medium.

Images