KR20220082720A - Method and computer device for providing analytical information related to sleep - Google Patents

Abstract

Disclosed is a method for providing sleep-related analysis information performed by a processor of a computing device for realizing the above-described task. The method includes the steps of obtaining the user's sleep sensing data, processing the sleep sensing data as an input of a sleep analysis model to output sleep analysis information, and generating characteristic map information based on the sleep analysis information may include

Description

METHOD AND COMPUTER DEVICE FOR PROVIDING ANALYTICAL INFORMATION RELATED TO SLEEP

The present disclosure provides information on a sleep state based on a user's sleep data obtained in a sleep environment, and more specifically, to provide analysis information corresponding to the user's sleep state by using an artificial neural network.

There are various ways to maintain and improve health, such as exercise and diet, but it is most important to manage sleep, which accounts for more than 30% of the time of the day. However, modern people cannot get a good night's sleep due to irregular eating habits, lifestyle, and stress despite simple replacement of labor and leisure time by machines, and suffer from sleep disorders such as insomnia, hypersomnia, sleep apnea syndrome, nightmares, night terrors, and sleepwalking. are receiving

According to the National Health Insurance Corporation, the number of sleep disorder patients in Korea increased by an average of about 8% per year from 2014 to 2018, and in 2018, about 570,000 patients were treated for sleep disorder in Korea. In particular, in the case of sleep apnea, the most common but dangerous sleep disorder, one in six Korean adults suffers. Sleep apnea is considered to be a major cause of heart disease, mental disease, and brain disease as well as disturbing sleep. In order to diagnose these diseases during sleep, polysomnography may be performed.

Polysomnography, which is used as a standard method for analyzing sleep stages and diagnosing sleep disorders, collects various biosignals from a sleeping patient. These biosignals show different patterns depending on the sleep stage and the presence or absence of sleep disorders. However, depending on the characteristics of the patient and the measurement time, these biosignals can have various patterns even at the same sleep stage. should directly view and analyze biosignals. Usually, since the sleep phase is set based on 30 seconds, it takes 2-3 hours to analyze the entire patient's sleep, which is several hours, and labor costs for analysis can also be burdensome.

Biometric data analysis using artificial intelligence is already widely used in the field of imaging medicine, but research on biometric data analysis made up of time series data is insufficient. In particular, there is insufficient research on techniques that enable specialists to interpret, trust, and use predictions of artificial intelligence models.

In fact, the reason why the prediction of an AI model is difficult to replace the judgment of a specialist is not because the average accuracy of the model is low, but because of the uncertainty of the AI model that works well in some situations and does not work well in other situations. Because.

Accordingly, in order to minimize the risk due to the uncertainty of artificial intelligence, systematic analysis of cases where the predictions of the artificial intelligence model fit well and cases where the predictions of the artificial intelligence models do not fit well through performance by sleep stage as well as correlation analysis of sleep diseases and performance By doing so, there may be a demand for research and development for an auxiliary model for predicting sleep stages that can be reliable and practically helpful.

Republic of Korea Patent Publication 2003-0032529

The present disclosure has been devised in response to the above-described background technology, and is to provide information related to a sleep state based on data measured in an individual user's sleep environment.

The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In an embodiment of the present disclosure for solving the above-described problem, a method for providing sleep-related analysis information performed by a processor of a computing device is disclosed. The method includes the steps of obtaining the user's sleep sensing data, processing the sleep sensing data as an input of a sleep analysis model to output sleep analysis information, and generating characteristic map information based on the sleep analysis information may include

In an alternative embodiment, the sleep sensing data includes one or more sequence data related to a user's bio-signals acquired in time series through one or more channels in relation to the user's sleep environment, and the sleep analysis information includes: and sleep stage information indicating that the user's sleep corresponds to at least one of one or more sleep stages, and the sleep analysis model receives the sleep sensing data as an input to obtain sleep analysis information corresponding to each of the one or more sequence data. It may be characterized by outputting.

In an alternative embodiment, the sleep analysis model includes a first sub-model and a second sub-model, wherein the first sub-model receives each of the one or more sequence data included in the sleep sensing data as an input, Extracting one or more features corresponding to each of one or more channels, and integrating the extracted one or more features to generate one or more integrated features corresponding to each sequence data, wherein the second sub-model comprises: It may be characterized in that the sleep analysis information is output by inputting each of the integrated features.

In an alternative embodiment, the sleep analysis model further includes one or more attention modules, wherein the one or more attention modules calculate an attention weight for sleep analysis information corresponding to each of the one or more integrated features. can do.

In an alternative embodiment, the generating of the characteristic map information based on the sleep analysis information includes generating the characteristic map information by visualizing an attention weight for the sleep analysis information according to each integrated feature through the one or more attention modules. characterized in that, the characteristic map information may be information that visualizes data affecting the prediction for each sequence data output by the sleep analysis model.

In an alternative embodiment, the sleep analysis model is characterized in that transfer learning is performed so that an algorithm trained in a source domain can be applied in another target domain, wherein the transfer learning is , it may include performing learning in a target domain related to a sleep disorder based on an algorithm trained in a source domain related to normal.

In an alternative embodiment, the sleep analysis information includes prediction confidence information corresponding to each of one or more sequence data, and the method includes: The method may further include performing calibration.

In an alternative embodiment, the method may include generating relationship information between the predictive certainty information and the sleep stage information, and updating the sleep analysis information based on the relationship information.

In another embodiment of the present disclosure, a computing device for providing sleep-related analysis information is disclosed. The computing device includes a processor including one or more cores, a memory for storing program codes executable in the processor, and a network unit for transmitting and receiving data to and from a user terminal, wherein the processor obtains the user's sleep sensing data, The sleep sensing data may be processed as an input of the sleep analysis model to output sleep analysis information, and characteristic map information may be generated based on the sleep analysis information.

In another embodiment of the present disclosure, a computer program stored in a computer-readable storage medium is disclosed. The computer program, when executed on one or more processors, causes the one or more processors to perform the following operations for providing sleep-related analysis information, the operations comprising: acquiring user's sleep sensing data; , outputting sleep analysis information by processing the sleep sensing data as an input of a sleep analysis model, and generating characteristic map information based on the sleep analysis information.

Other specific details of the present disclosure are included in the detailed description and drawings.

The present disclosure has been devised in response to the above-described background technology, and may provide information related to sleep disorders based on data measured in an individual user's sleep environment.

Additionally, it is possible to provide a reliable and practically helpful sleep stage prediction auxiliary model by systematically analyzing the cases in which the predictions of the artificial intelligence model fit well and the cases where they do not fit well.

Effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

Various aspects are now described with reference to the drawings, wherein like reference numbers are used to refer to like elements throughout. In the following example, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. It will be evident, however, that such aspect(s) may be practiced without these specific details.
1 is a conceptual diagram illustrating a system in which various aspects of a computing device for providing sleep-related analysis information related to an embodiment of the present disclosure may be implemented.
2 is a block diagram of a computing device for providing sleep-related analysis information according to an embodiment of the present disclosure.
3 is a diagram exemplarily illustrating sleep sensing data related to an embodiment of the present disclosure.
4 is a diagram exemplarily illustrating a process in which sleep sensing data related to an embodiment of the present disclosure is input to a sleep analysis model.
5 is a diagram exemplarily illustrating an output process of a sleep analysis model related to an embodiment of the present disclosure.
6 is a flowchart exemplarily illustrating a method of providing sleep-related analysis information related to an embodiment of the present disclosure.
7 is a schematic diagram illustrating one or more network functions related to an embodiment of the present disclosure.

Advantages and features of the present disclosure, and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments are provided so that the disclosure of the present disclosure is complete, and those of ordinary skill in the art to which the present disclosure belongs It is provided to fully inform those skilled in the art of the scope of the present disclosure, which is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present disclosure. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present disclosure.

Unless otherwise defined, all terms (including technical and scientific terms) used herein may have the meaning commonly understood by those of ordinary skill in the art to which this disclosure belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

As used herein, the term “unit” or “module” refers to a hardware component such as software, FPGA, or ASIC, and “unit” or “module” performs certain roles. However, “part” or “module” is not meant to be limited to software or hardware. A “unit” or “module” may be configured to reside on an addressable storage medium or to reproduce one or more processors. Thus, as an example, “part” or “module” refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, Includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. Components and functionality provided within “parts” or “modules” may be combined into a smaller number of components and “parts” or “modules” or as additional components and “parts” or “modules”. can be further separated.

In this specification, a computer refers to all types of hardware devices including at least one processor, and may be understood as encompassing software configurations operating in the corresponding hardware device according to embodiments. For example, a computer may be understood to include, but is not limited to, smart phones, tablet PCs, desktops, notebooks, and user clients and applications running on each device.

Those skilled in the art will further appreciate that the various illustrative logical blocks, configurations, modules, circuits, means, logics, and algorithm steps described in connection with the embodiments disclosed herein may be implemented in electronic hardware, computer software, or combinations of both. It should be recognized that they can be implemented with To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, configurations, means, logics, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application. However, such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

Each step described in this specification is described as being performed by a computer, but the subject of each step is not limited thereto, and at least a portion of each step may be performed in different devices according to embodiments.

1 is a conceptual diagram illustrating a system in which various aspects of a computing device for providing sleep-related analysis information related to an embodiment of the present disclosure may be implemented.

A system according to embodiments of the present disclosure may include a computing device 100 , a user terminal 10 , an external server 20 , and a network. The computing device 100 , the user terminal 10 and the external server 20 according to embodiments of the present disclosure may mutually transmit/receive data for the system according to embodiments of the present disclosure through a network.

Networks according to embodiments of the present disclosure include Public Switched Telephone Network (PSTN), x Digital Subscriber Line (xDSL), Rate Adaptive DSL (RADSL), Multi Rate DSL (MDSL), Very High Speed DSL (VDSL). ), a variety of wired communication systems such as Universal Asymmetric DSL (UADSL), High Bit Rate DSL (HDSL), and Local Area Network (LAN) can be used.

In addition, the networks presented herein include Code Division Multi Access (CDMA), Time Division Multi Access (TDMA), Frequency Division Multi Access (FDMA), Orthogonal Frequency Division Multi Access (OFDMA), Single Carrier-FDMA (SC-FDMA) and Various wireless communication systems may be used, such as other systems.

The network according to the embodiments of the present disclosure may be configured regardless of its communication mode, such as wired and wireless, and is composed of various communication networks such as a personal area network (PAN) and a wide area network (WAN). can be In addition, the network may be a well-known World Wide Web (WWW), and may use a wireless transmission technology used for short-range communication, such as infrared (IrDA) or Bluetooth (Bluetooth). The techniques described herein may be used in the networks mentioned above, as well as in other networks.

According to an embodiment of the present disclosure, the user terminal 10 is a terminal capable of receiving information related to a user's sleep through information exchange with the computing device 100 , and may refer to a terminal possessed by the user. For example, the user terminal 10 may be a terminal related to a user who wants to improve health through information related to his or her sleeping habits. Also, the user terminal 10 may be a terminal related to an examiner (eg, a specialist) that provides a checkup result to a user (eg, an examinee). When the user terminal 10 is a terminal related to an examiner that provides examination results (eg, examination results for polysomnography) to the examinee, through the analysis information obtained from the computing device 100 , for reading the examination result of the examiner It can be used as a medical assistant terminal.

The user may receive, through the user terminal 10 , sleep phase information corresponding to each time point in the sleep environment, prediction confidence information corresponding to each sleep phase information for each time point, and the like. The user terminal 10 is provided with a display, and can receive a user's input and provide an output of any type to the user.

The user terminal 10 is a customer terminal (UE), a mobile, a PC capable of wireless communication, a mobile phone, a kiosk, a cellular phone, a cellular, a cellular terminal, a subscriber unit, a subscriber station, a mobile station, a terminal, a remote station, a PDA, a remote terminal Any that can use a wireless mechanism, such as an access terminal, user agent, cellular phone, wireless phone, session initiation protocol (SIP) phone, wireless local loop (WLL) station, portable device with wireless access capability, wireless model It may be referred to as a device of, but is not limited to. In addition, the user terminal 10 may be referred to as any device capable of using a wired connection mechanism, such as a wired fax machine, a PC equipped with a wired model, a wired telephone, a terminal capable of wired communication, etc., but is not limited thereto. does not

According to an embodiment of the present disclosure, the external server 20 may be a server that stores health checkup information or sleep checkup information for a plurality of users. For example, the external server 20 may be at least one of a hospital server and an information server, and may be a server that stores information about a plurality of polysomnography records, electronic health records, electronic medical records, and the like. For example, polysomnography records include information on brain waves, eye movements, muscle movements, respiration, electrocardiogram, etc. during sleep of the sleep examination subject and sleep diagnosis results corresponding to the information (e.g., sleep stages, sleep apnea, sleep disorders, etc.). Information stored in the external server 20 may be utilized as training data, verification data, and test data for learning the neural network in the present disclosure. That is, the external server 20 may be a server storing a data set for learning the sleep analysis model of the present disclosure.

The computing device 100 of the present disclosure may receive health checkup information or sleep checkup information from the external server 20 , and build a learning data set based on the information. The computing device 100 may generate a sleep analysis model for calculating sleep analysis information corresponding to the user's sleep sensing data by learning the sleep analysis model including one or more network functions through the training data set. A configuration for constructing a learning data set for learning a neural network of the present disclosure and a detailed description of a learning method using the learning data set will be described later with reference to FIG. 2 .

The external server 20 is a digital device, and may be a digital device equipped with a processor, such as a laptop computer, a notebook computer, a desktop computer, a web pad, and a mobile phone, and having a computing capability with a memory. The external server 20 may be a web server that processes a service. The above-described types of servers are merely examples, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the computing device 100 may obtain sleep sensing data related to the user's sleep environment, and generate sleep analysis information related to the user's sleep state based on the sleep sensing data. Specifically, the computing device 100 may generate a sleep analysis model that learns one or more network functions using the labeled training data set to output sleep analysis information based on the sleep sensing data. That is, the computing device 100 may obtain sleep sensing data related to the user's sleep environment, process the sleep sensing data as an input of a sleep analysis model that is a learned neural network model, and generate sleep analysis information. In this case, the sleep analysis information provided by the computing device 100 may include prediction information regarding the sleep phase of the user during sleep. For example, the sleep analysis information may include sleep stage information indicating that the user's sleep corresponds to at least one of one or more sleep stages in relation to a specific time point. Here, the one or more sleep stages may include, for example, wake (non-sleep state), N1 (Non REM 1), N2, N3, and REM sleep. As a specific example, the sleep analysis information may include information that the user's sleep stage at the first time point corresponds to REM sleep.

Additionally, the sleep analysis information may include prediction confidence information for each time point related to the user's sleep environment. The prediction confidence information may be information about prediction accuracy with respect to sleep stage information output (or predicted) through the sleep analysis model in response to a specific time point. For example, the sleep analysis information may include prediction information that the user's sleep stage at the first time point corresponds to REM sleep, and information that the prediction confidence information related to reliability in the prediction information is '80'. For example, the greater the prediction confidence information, the higher the accuracy (or reliability) of the sleep stage predicted through the artificial neural network. That is, the computing device 100 may provide prediction information on sleep phases for each time point during the user's sleep and prediction confidence information corresponding to the prediction information in each sleep phase. In this case, the prediction confidence information may be used as a determination index for how reliable the output (ie, the prediction information on the sleep stage) of the artificial neural network (ie, the sleep analysis model) is reliable. In other words, by presenting predictive confidence information related to sleep stage estimation at each time point together with sleep stage information, reliable use in a medical environment may be possible.

According to an embodiment of the present disclosure, the computing device 100 may generate characteristic map information based on sleep analysis information. The characteristic map information may be information displayed by visualizing data influenced in the process of calculating the sleep analysis information output by the sleep analysis model. Specifically, the computing device 100 may acquire an attention weight for sleep analysis information according to sleep sensing data related to a specific time point through one or more attention modules. In this case, the one or more attention modules may be modules to learn a matching relationship between an input and an output of the sleep analysis model. One or more attention modules may emphasize an element to be focused on between input data and output data by assigning an attention weight to each element of time series-related input data (ie, sleep sensing data) in the learning process of the neural network. . In other words, the one or more attention modules may be modules that generate information about which input element the output value of the sleep analysis model has the highest correlation with.

As a specific example, the computing device 100 may determine that the user's sleep stage is N3 for the corresponding one minute through the user's sleep sensing data obtained in response to the first time point (eg, after going to sleep, the first user's sleep for 1 minute). It is possible to generate predictive information corresponding to the sleep stage. In addition, the computing device 100 calculates the attention weight of each input element from one or more attention modules to sleep analysis information related to the first time point (that is, predictive information that the user's sleep at the corresponding time point corresponds to the N3 sleep stage) can be obtained The computing device 100 may generate characteristic map information by visualizing the obtained attention weights for each element.

That is, the computing device 100 may visualize the basis for determination in the process of calculating prediction information for input data related to time series (ie, sleep sensing data) by utilizing one or more attention modules. In other words, which signal (ie, which type of sleep sensing data) played an important role in reading the sleep phase at each time point can be visualized and provided through the attention weight value. For example, the characteristic map information may be generated to provide visualization information in a meaningful form by displaying differently for each pixel based on a signal or information noticed through an attention weight.

In general, it is difficult to understand the internal algorithm of an artificial neural network model. That is, it is difficult to determine on what basis the output related to the input data is generated. This may have a problem that it is difficult to be utilized in an actual medical environment as it acts as a risk due to the uncertainty of the artificial neural network model.

The computing device 100 of the present disclosure may visualize and provide a pattern of a signal that is important when predicting a sleep phase for each time point, thereby visualizing the basis on which the neural network model predicted or judged. Accordingly, it is possible to precisely verify the validity of the prediction information output by the sleep analysis model of the present disclosure, thereby maximizing collaboration synergy with the user (eg, a specialist).

A detailed configuration of the computing device 100 of the present disclosure and effects according to the configuration will be described later in detail with reference to FIG. 2 .

Also, the computing device 100 may be a terminal or a server, and may include any type of device. The computing device 100 is a digital device, and may be a digital device equipped with a processor, such as a laptop computer, a notebook computer, a desktop computer, a web pad, and a mobile phone, and having a computing power having a memory. The computing device 100 may be a web server that processes a service. The above-described types of servers are merely examples, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the computing device 100 may be a server that provides a cloud computing service, according to an embodiment of the present disclosure. More specifically, the computing device 100 is a type of Internet-based computing, and may be a server that provides a cloud computing service that processes information not with a user's computer but with another computer connected to the Internet. The cloud computing service may be a service that stores data on the Internet and allows the user to use it anytime and anywhere through Internet access without installing necessary data or programs on his/her computer. Easy to share and deliver with a click. In addition, cloud computing service not only stores data on a server on the Internet, but also allows users to perform desired tasks using the functions of applications provided on the web without installing a separate program, and multiple people can simultaneously view documents. It may be a service that allows you to work while sharing. In addition, the cloud computing service may be implemented in the form of at least one of Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Software as a Service (SaaS), a virtual machine-based cloud server, and a container-based cloud server. . That is, the computing device 100 of the present disclosure may be implemented in the form of at least one of the above-described cloud computing services. The detailed description of the above-described cloud computing service is merely an example, and may include any platform for building the cloud computing environment of the present disclosure.

Hereinafter, a method in which the computing device 100 provides sleep analysis information based on sleep sensing data will be described in more detail below with reference to FIG. 2 .

2 is a block diagram of a computing device for providing sleep-related analysis information according to an embodiment of the present disclosure.

As shown in FIG. 2 , the computing device 100 may include a network unit 110 , a memory 120 , and a processor 130 . Components included in the aforementioned computing device 100 are exemplary and the scope of the present disclosure is not limited to the aforementioned components. That is, additional components may be included or some of the above-described components may be omitted depending on implementation aspects for the embodiments of the present disclosure.

According to an embodiment of the present disclosure, the computing device 100 may include the user terminal 10 and the network unit 110 for transmitting and receiving data to and from the external server 20 . The network unit 110 may transmit/receive data for performing a method of providing sleep analysis information based on the sleep environment sensing data according to an embodiment of the present disclosure to other computing devices, servers, and the like. That is, the network unit 110 may provide a communication function between the computing device 100 , the user terminal 10 , and the external server 20 . For example, the network unit 110 may receive sleep examination records and electronic health records for a plurality of users from a hospital server. Additionally, the network unit 110 may allow information transfer between the computing device 100 and the user terminal 10 and the external server 20 by calling a procedure to the computing device 100 .

The network unit 110 according to an embodiment of the present disclosure includes a Public Switched Telephone Network (PSTN), x Digital Subscriber Line (xDSL), Rate Adaptive DSL (RADSL), Multi Rate DSL (MDSL), VDSL ( A variety of wired communication systems such as Very High Speed DSL), Universal Asymmetric DSL (UADSL), High Bit Rate DSL (HDSL), and Local Area Network (LAN) can be used.

In addition, the network unit 110 presented herein is CDMA (Code Division Multi Access), TDMA (Time Division Multi Access), FDMA (Frequency Division Multi Access), OFDMA (Orthogonal Frequency Division Multi Access), SC-FDMA ( A variety of wireless communication systems can be used, such as Single Carrier-FDMA) and other systems.

In the present disclosure, the network unit 110 may be configured regardless of its communication mode, such as wired and wireless, and may be composed of various communication networks such as a short-range network (PAN: Personal Area Network) and a local area network (WAN: Wide Area Network). can In addition, the network may be a well-known World Wide Web (WWW), and may use a wireless transmission technology used for short-range communication, such as infrared (IrDA) or Bluetooth (Bluetooth). The techniques described herein may be used in the networks mentioned above, as well as in other networks.

According to an embodiment of the present disclosure, the memory 120 may store a computer program for performing the method for providing sleep analysis information according to an embodiment of the present disclosure, and the stored computer program is executed by the processor 130 . It can be read and driven. In addition, the memory 120 may store any type of information generated or determined by the processor 130 and any type of information received by the network unit 110 . Also, the memory 120 may store data related to the user's sleep. For example, the memory 120 may temporarily or permanently store input/output data (eg, sleep sensing data related to the user's sleep environment, ie, sensing data related to polysomnography).

According to an embodiment of the present disclosure, the memory 120 is a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (eg, a SD or XD memory, etc.), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read (PROM) -Only Memory), a magnetic memory, a magnetic disk, and an optical disk may include at least one type of storage medium. The computing device 100 may operate in relation to a web storage that performs a storage function of the memory 120 on the Internet. The description of the above-described memory is only an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 130 may include one or more cores, and a central processing unit (CPU) of a computing device, a general purpose graphics processing unit (GPGPU), and a general purpose graphics processing unit (GPGPU). , data analysis such as a tensor processing unit (TPU), and a processor for deep learning.

The processor 130 may read a computer program stored in the memory 120 to perform data processing for machine learning according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, the processor 130 may perform an operation for learning the neural network. The processor 130 for learning of the neural network, such as processing input data for learning in deep learning (DL), extracting features from input data, calculating an error, updating the weight of the neural network using backpropagation calculations can be performed.

In addition, at least one of a CPU, a GPGPU, and a TPU of the processor 130 may process learning of a network function. For example, the CPU and the GPGPU can process learning of a network function and data classification using the network function. Also, in an embodiment of the present disclosure, learning of a network function and data classification using the network function may be processed by using the processors of a plurality of computing devices together. In addition, the computer program executed in the computing device according to an embodiment of the present disclosure may be a CPU, GPGPU or TPU executable program.

In the present specification, a network function may be used interchangeably with an artificial neural network and a neural network. In the present specification, a network function may include one or more neural networks, and in this case, an output of the network function may be an ensemble of outputs of one or more neural networks.

In this specification, a model may include a network function. The model may include one or more network functions, in which case the output of the model may be an ensemble of outputs of the one or more network functions.

The processor 130 may read a computer program stored in the memory 120 to provide a sleep analysis model according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, the processor 130 may perform a calculation for calculating sleep analysis information based on the sleep sensing data. According to an embodiment of the present disclosure, the processor 130 may perform a calculation for learning the sleep analysis model.

According to an embodiment of the present disclosure, the processor 130 may typically process the overall operation of the computing device 100 . The processor 130 processes signals, data, information, etc. input or output through the above-described components or by driving an application program stored in the memory 120 to provide or process appropriate information or functions to the user terminal. have.

According to an embodiment of the present disclosure, the processor 130 may acquire the user's sleep sensing data. According to an embodiment of the present disclosure, the acquisition of the sleep sensing data may be receiving or loading the sleep sensing data stored in the memory 120 . Acquisition of the sleep sensing data may be receiving or loading the sleep sensing data from another computing device or a separate processing module within the same computing device in another storage medium based on a wired/wireless communication means.

The sleep sensing data may be a biosignal obtained during the user's sleep. For example, the sleep sensing data may be a biosignal measured from a user in relation to a polysomnography examination. The biosignal related to polysomnography may include information on EEG, safety level, jaw electromyography, respiration, electrocardiogram, arterial blood oxygen saturation, anterior peroneal EMG, and other physiological and physical variables. According to an additional embodiment, the biosignal related to the polysomnography may further include information about the user's breathing and movement during sleep. Such sleep sensing data may be acquired through one or more channels. Each of the one or more channels may be associated with a respective electrode pair configured to acquire a biosignal.

For example, the sleep sensing data may be a biosignal measured in relation to polysomnography, and may include information on electroencephalography (EEG), safety level (EOG, Electrooculography), and electromyography (EMG). have. In this case, information on EEG, safety level, and EMG may be obtained through different channels, respectively. As a specific example, as shown in FIG. 3 , information on EEG is transmitted through six channels (ie, F3-A2, F4-A1, C3-A2, C4-A1, O1-A2 and O1-A1). may be obtained, information on the safety level may be obtained through two channels (LEFT, RIGHT), and information on the EMG may be obtained through one channel. That is, the sleep sensing data of the present disclosure means biosignals related to electroencephalogram, safety level, and electromyography measured from the user's body during the sleep period for polysomnography examination, and the biosignals are 6 related to electroencephalography as described above. It can be obtained through two channels, two channels related to safety, and one channel related to EMG, that is, a total of 9 channels. As described above, the sleep sensing data that is the basis for generating sleep analysis information of the present disclosure includes various bio-signals obtained through a plurality of channels, so that the output accuracy of the artificial neural network may be improved through the corresponding data.

In addition, the sleep sensing data of the present disclosure may be time series data obtained during the user's sleep period. In this case, the sleep sensing data may be a combination of one or more sequence data divided by a predetermined unit time. The predetermined unit time may be, for example, 30 seconds as a reference for detecting a change in sleep. That is, the sleep sensing data may include one or more sequence data related to the user's bio-signals obtained in time series through one or more channels in relation to the user's sleep environment. For example, the one or more sequence data may include information about an EEG, a safety level, and an EMG obtained through 9 channels based on a predetermined unit time (eg, 30 seconds).

For a specific example, referring to FIG. 3 , one or more sequence data included in the sleep sensing data may include first sequence data 201 and second sequence data 202 . Here, the first sequence data 201 may be a biosignal related to an electroencephalogram, a safety level, and an EMG obtained through nine channels for 0 to 30 seconds. The second sequence data 202 may be biosignals related to electroencephalogram, safety level, and electromyography obtained through nine channels for 31 seconds to 60 seconds, which is a time point after the first sequence data 201 . The description described above with reference to FIG. 3 is only an example for helping understanding of the present disclosure, and one or more sequence data included in the sleep sensing data of the present disclosure is not limited to the first and second sequence data. That is, as the sleep sensing data of the present disclosure is data related to biosignals measured during the user's sleep period (eg, for 8 hours), the sleep sensing data includes a larger number of sequence data. It will be apparent to one of ordinary skill in the art.

According to an embodiment of the present disclosure, the processor 130 may perform pre-processing on the sleep sensing data. The pre-processing of the sleep sensing data according to an embodiment may include performing down sampling on the sleep sensing data. For example, sleep sensing data acquired through multiple channels (eg, 9 channels) in relation to a polysomnography test are acquired at different sampling rates, respectively, but may have a high sampling rate that is unnecessary for sleep stage measurement. . Accordingly, the processor 130 may down-sample data acquired through each channel at a sampling rate suitable for the neural network model of the present disclosure. As a specific example, the period of each signal may be reduced by down-sampling data acquired through each channel at a sampling rate of 25 Hz. This may provide an effect of preventing aliasing by reducing distortion of data obtained in time series. Accordingly, classification of data acquired through each channel may be made clear.

In an additional embodiment, the pre-processing of the sleep sensing data may include pre-processing of removing noise of each data acquired through each channel. In detail, the processor 130 may normalize the magnitude of the signal included in each data based on the comparison between the magnitude of the signal included in each data acquired through each channel and the magnitude of a predetermined reference signal. For example, when the magnitude of a signal included in each of the sleep sensing data acquired through a plurality of channels is less than a predetermined reference signal, the processor 130 greatly adjusts the magnitude of the signal, and sleep sensing acquired through each channel When the magnitude of the signal included in each data is greater than or equal to the predetermined reference signal, the magnitude of the corresponding signal may be adjusted to be small (ie, not clipping). The detailed description of the above-described pre-processing of the sleep sensing data is only an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 130 may output sleep analysis information by processing the sleep sensing data as an input of the sleep analysis model. The sleep analysis information may include sleep stage information indicating that the user's sleep corresponds to at least one of one or more sleep stages. In other words, the sleep analysis information may include prediction information on a sleep phase of the user during sleep. For example, the sleep analysis information may include sleep stage information indicating that the user's sleep corresponds to at least one of one or more sleep stages in relation to a specific time point. Here, the one or more sleep stages may include, for example, wake (non-sleep state), N1 (Non REM 1), N2, N3, and REM sleep. As a specific example, the sleep analysis information may include information that the user's sleep stage at the first time point corresponds to REM sleep. The detailed description of the above-described sleep analysis information is only an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 130 may learn about one or more network functions through a training data set. To this end, the processor 130 may receive a training data set from the external server 20 . Here, the external server 20 may be a server related to at least one of a hospital server and a government server, and a plurality of users related to polysomnography from at least one of an electronic health record, an electronic medical record, and a health examination DB of each server Each of the examination data may be received. The processor 130 may build a learning data set including a plurality of learning data based on the examination data of each of the plurality of users received from the external server 20 . The training data set may include a training input data set relating to an input of the neural network and a training output data set to be compared with the output.

More specifically, the learning data set includes a learning input data set related to bio-signals measured from the user's body through polysomnography in relation to the sleep environment of each of a plurality of users, and sleep stage determination information for each time point of each bio-signal. It may include an associated training output data set. The processor 130 may generate the sleep analysis model of the present disclosure by performing learning on one or more network functions through the training data set.

The sleep analysis model may include a first sub-model 310 and a second sub-model 330 . Hereinafter, the first sub-model 310 will be described as the dimensionality reduction sub-model 310 and the second sub-model 330 will be described as the dimensional restoration sub-model 330 , but the present invention is not limited thereto. The processor 130 may use the training input data as an input of the dimension reduction sub-model 310 to train the dimension restoration sub-model 330 to output the training data associated with the label of the corresponding training input data.

The dimension reduction sub-model 310 may be a model that extracts features (ie, embeddings) by inputting learning input data related to biosignals acquired in time series during the user's sleep. That is, the dimension reduction sub-model 310 may receive training input data from the processor 130 and designate a feature vector column of the training input data as an output to learn an intermediate process in which the input data is converted into features.

In addition, the processor 130 may transmit embeddings (ie, features) related to the output of the dimensionality reduction submodel 310 to the dimensionality restoration submodel 330 . The dimension reduction sub-model 310 may receive a feature as an input and output sleep analysis information related to the feature. The processor 130 may derive an error by comparing the sleep analysis information that is the output of the dimension restoration sub-model with the learning output data, and adjust the weight of each model in a backpropagation method based on the derived error.

The processor 130 makes the sleep analysis information that is the output of the dimension restoration sub-model 330 closer to the learning output data based on the error between the operation result of the dimensional restoration sub-model 330 for the learning input data and the learning output data. The weights of the above network functions can be adjusted.

In other words, the dimension reduction sub-model may be trained to extract features for the training input data, and the dimension restoration sub-model may be trained to output analysis information corresponding to the extracted features.

Also, the sleep analysis model may further include one or more attention modules. The one or more attention modules may calculate an attention weight for the analysis information corresponding to the feature. Specifically, the processor 130 may cause one or more attention modules to learn a matching relationship between an input (ie, learning input data) and an output (ie, learning output data) through the learning input data and the learning output data. . Accordingly, the one or more attention modules 320 may generate association information regarding the association between the feature and the time step of the dimensional reconstruction sub-model 330 . In other words, the one or more attention modules may emphasize an element to be focused between the input data and the output data by giving an attention weight to each element of the input data related to the time series in the learning process of the neural network. In other words, the one or more attention modules may be modules that are trained to assign an attention weight related to a correlation between an output value and an input value of the sleep analysis model.

As a specific example, the dimension reconstruction sub-model 330 includes one or more RNNs, and may be a model that outputs prediction information of a second time stamp by inputting prediction information of a first time stamp through one or more RNNs. . In this case, the first time stamp may be earlier than the second time stamp. Specifically, the prediction information predicted through the first RNN of the dimensional reconstruction submodel is transmitted to the second RNN of the next time stamp, and the prediction information predicted through the second RNN is transmitted to the third RNN of the next time stamp. have. In this process, one or more attention modules may determine a point of view of a feature to be focused on through association information between time steps. That is, the dimensional reconstruction sub-model 330 determines the time of the change factor to focus on through one or more attention modules in the process of repeatedly predicting the prediction information through one or more RNNs, thereby predicting information on the correlation between the viewpoints of various factors. can be learned to output

In other words, the processor 130 uses the learning input data as an input of the dimensional sensing sub-model to output a feature corresponding to the learning input data, and uses the corresponding feature as an input of the dimension restoration sub-model 330 through one or more attention modules. can be processed with In this case, the dimension restoration sub-model 330 may output sleep analysis information by taking the feature as an input, and the processor 130 compares the output sleep analysis information with the learning output data that is a label of the learning input data to reduce the error. It can be derived and the weight of each model can be adjusted based on the derived error. Through the above-described learning process, the processor 130 may generate a sleep analysis model by learning one or more network functions.

Accordingly, the sleep analysis model may output sleep analysis information by receiving the sleep sensing data as an input.

In more detail, the sleep analysis model may receive sleep sensing data including one or more sequence data as input, and output sleep analysis information corresponding to each of the one or more sequence data. That is, the sleep analysis model may output sleep analysis information corresponding to each of one or more sequence data by receiving sleep sensing data as an input. The sleep analysis model may include a dimension reduction sub-model (eg, an encoder) and a dimensionality restoration sub-model (eg, a decoder). The dimension reduction sub-model and the dimension restoration sub-model described below may refer to models learned through the above-described learning process. A detailed description of a process in which the sleep analysis model outputs sleep analysis information corresponding to each sequence data by inputting sleep sensing data including one or more sequence data will be described below with reference to FIGS. 4 and 5 .

The dimension reduction sub-model 310 receives each of one or more sequence data included in the sleep sensing data as an input, extracts one or more features corresponding to each of one or more channels, and integrates the extracted one or more features to each sequence data and generating one or more corresponding integrated features.

More specifically, referring to FIG. 4 , the learning input sleep sensing data of the present disclosure may relate to a user's bio-signals acquired in time series through one or more channels during the user's sleep. The sleep sensing data may include one or more sequence data. The one or more sequence data may include first sequence data 201 and second sequence data 202 . Here, the first sequence data 201 may be a biosignal related to an electroencephalogram, a safety level, and an EMG obtained through nine channels for 0 to 30 seconds. The second sequence data 202 may be biosignals related to electroencephalogram, safety level, and electromyography obtained through nine channels for 31 seconds to 60 seconds, which is a time point after the first sequence data 201 . Furthermore, sleep sensing data used for analysis may include 10 sequence data in units of 30 seconds, but is not limited thereto.

In this case, the dimension reduction sub-model 310 may extract one or more features corresponding to each channel by receiving the first sequence data 201 as an input. Specifically, the dimensionality reduction sub-model can generate 6 features corresponding to each of the 6 channels related to the EEG for 0 to 30 seconds, and two channels corresponding to each of the 2 channels related to the safety level for the corresponding time period. A feature may be created, and one feature may be generated corresponding to one channel related to the EMG for the corresponding time period. That is, when the first sequence data 201 included in the sleep sensing data is input, the dimension reduction sub-model 310 may generate nine features corresponding to each channel. In addition, the dimensionality reduction sub-model 310 may integrate one or more features, that is, nine features to generate an integrated feature. The integrated feature integrates various bio-signals acquired through a plurality of channels for the same time, and may refer to a feature representing a sleep environment related to a specific time point. That is, one integrated feature may mean a feature (ie, embedding) corresponding to one sequence data. In addition, the dimension reduction sub-model 310 generates an integrated feature corresponding to the second sequence data 202 by inputting the second sequence data 202 related to the sleeping environment at a later time point of the first sequence data 201 as an input. can do.

For example, a model for extracting features for each channel in the dimension reduction submodel 310 may be Res1DCNN, and a model for extracting integrated features by integrating features for each channel may also be Res1DCNN. That is, the dimension reduction sub-model 310 may be MultiRes1DCNN, but is not limited thereto.

In an embodiment, a plurality of dimensionality reduction sub-models of the present disclosure may be provided to perform an integrated feature extraction operation corresponding to one or more sequence data in parallel. When a plurality of dimensionality reduction sub-models are provided and one or more integrated feature extraction operations are performed in parallel with each sequence data as an input, the processing speed may be further improved.

As described above, the dimension restoration sub-model 330 generates one or more integrated features corresponding to each sequence data when the sleep sensing data including one or more sequence data divided by a predetermined unit time is input. can For example, 10 integrated features may be obtained when analysis is performed on 10 sequences.

In addition, the dimension restoration sub-model 330 may be characterized in that it outputs sleep analysis information with the integrated feature as an input.

As a specific example, the dimensional reconstruction sub-model 330 may include an RNN, input a plurality of integrated features obtained as an analysis result for a plurality of sequences, and analyze the integrated features for a plurality of sequences together. Sleep analysis information for each of a plurality of sequences may be output. The output sleep analysis information may include, but is not limited to, sleep phase information for each sequence and certainty information therefor.

For example, when sleep analysis information is obtained using integrated features extracted from 10 sequences, integrated features extracted from 10 MultiRes1DCNN operations are input to the RNN, and from the output, sleep phase information for each of the 10 sequences and It is possible to obtain certainty information about this, but is not limited thereto.

As another example, the dimension reconstruction sub-model 330 may include one or more RNNs, and may be a model that outputs prediction information of a second timestamp by inputting prediction information of a first time stamp through one or more RNNs. In this case, the first time stamp may be earlier than the second time stamp. Specifically, the prediction information predicted through the first RNN of the dimensional reconstruction submodel is transmitted to the second RNN of the next time stamp, and the prediction information predicted through the second RNN is transmitted to the third RNN of the next time stamp. have. In this process, one or more attention modules may determine a point of view of a feature to be focused on through association information between time steps. That is, the dimension restoration sub-model 330 determines the timing of the change factor to focus on through one or more attention modules 320 in the process of repeatedly predicting prediction information through one or more RNNs, thereby providing a correlation between the viewpoints of various factors. Forecast information can be output.

That is, the integrated feature output through the dimension reduction sub-model 310 may be transmitted to one or more attention modules 320 . In this case, the one or more attention modules 320 may calculate an attention weight for sleep analysis information corresponding to each integrated feature. The one or more attention modules 320 may emphasize an element to be focused between the input data and the output data by giving an attention weight to each feature of the time series-related input data (ie, sleep sensing data).

That is, the sleep analysis model including the dimension reduction submodel 310 of the present disclosure, one or more attention modules 320 and the dimension restoration submodel 330 of the present disclosure simply considers a viewpoint related to one sequence data to provide sleep analysis information Instead of outputting (that is, predictive information related to the sleep stage), the sleep analysis information may be output by reflecting the sequence data at the time point.

For example, when a specialist or sleep technician actually reads the polysomnography test result, the reading may proceed by considering the previous sequence, rather than looking at only one sequence corresponding to a single 30 seconds. In other words, when predictive information on a sleep phase is calculated based on a single sequence without considering a correlation with a previous sequence, there is a risk of lack of accuracy.

The sleep analysis model of the present disclosure may perform sleep phase prediction in consideration of characteristics of before and after sequence data in the process of calculating sleep analysis information corresponding to one sequence data through the above-described process. Accordingly, the accuracy and reliability of the calculated sleep analysis information can be guaranteed.

According to an embodiment of the present disclosure, the sleep analysis model may be characterized by transfer learning. For example, the accuracy of sleep stage prediction for patients suffering from a sleep disorder may be lower than for normal people. For example, when performing sleep stage prediction using a neural network based on sleep sensing data obtained from patients suffering from sleep apnea, the accuracy of sleep stage prediction for users without sleep disorders is about 10 It can be as low as ~15%.

In general, it is difficult to clearly classify whether the user has a sleep disorder before proceeding with the polysomnography test. Accordingly, in order to create a sleep analysis model that provides sleep analysis information corresponding to the sleep sensing data of a user suffering from a sleep disorder and a user without a sleep disorder, sleep sensing data related to a user suffering from a sleep disorder may be additionally required. have. In other words, a plurality of sleep sensing data of a plurality of users suffering from a sleep disorder must be built as training data for training a sleep analysis model (that is, a neural network model), which takes a lot of time and There is a risk of incurring costs.

Accordingly, the sleep analysis model of the present disclosure may be characterized by transfer learning to output sleep analysis information corresponding to sleep sensing data corresponding to different domains (eg, users suffering from sleep disorders) through transfer learning. . Specifically, the sleep analysis model may be characterized in that it is transfer-learned so that an algorithm trained in a source domain can be applied in another target domain. Here, the source domain may mean related to sleep sensing data related to a user who does not suffer from a sleep disorder, and the target domain may mean related to sleep sensing data related to a user suffering from a sleep disorder. Transfer learning may refer to, for example, recalibrating and using weights of a model trained in a source domain to fit a target domain. According to an additional embodiment, the sleep analysis model of the present disclosure may perform learning on data in a situation where there is no label related to the correct answer through domain adaptation. That is, through transfer learning and domain adaptation, the sleep analysis model can improve learning speed and performance while using a small amount of learning data for various domains (eg, users suffering from or not suffering from sleep disorders).

Accordingly, the sleep analysis model of the present disclosure may work well not only for users who do not suffer from sleep disorders, but also for patients with sleep disorders. That is, the sleep analysis model may output more robust sleep analysis information in response to the sleep sensing data of a user suffering from a sleep disorder. In other words, it is possible to provide robust and automated sleep analysis information for users with sleep disorders.

According to an embodiment of the present disclosure, the sleep analysis information output by the sleep analysis model may include prediction confidence information. The sleep analysis information may include prediction confidence information corresponding to each of one or more sequence data.

Specifically, the sleep analysis model may receive sleep sensing data including one or more sequence data as input, and output sleep stage information corresponding to each sequence data. In this case, the sleep analysis model may calculate a score (ie, softmax) corresponding to each of one or more sleep stages by receiving one sequence data as an input, and a sleep stage based on the score calculated in response to each sleep stage information can be created. When the prediction information on the sleep stage is calculated through the sleep analysis model, the processor 130 may generate prediction confidence information through a score value contributing to the calculation of the prediction information.

For example, when the sleep analysis model receives the first sequence data 201 obtained through 0 channels for 0 to 30 seconds in relation to the user's sleep as an input, in response to the first sequence data 201 A score corresponding to each of one or more sleep stages may be calculated. As a specific example, the sleep analysis model may calculate wake - 2 points, N1 - 10 points, N2 - 80 points, N3 - 7 points, and REM - 1 points in response to the first sequence data 201 , and the largest Based on the score, 'N2' may be determined as sleep stage information corresponding to the first sequence data 201 . That is, the user for 0 to 30 seconds can calculate prediction information corresponding to the N2 sleep stage. In this case, the processor 130 may determine the prediction confidence information as '80' based on the score contributing to the N2 sleep stage prediction. For example, the greater the prediction confidence information, the higher the accuracy (or reliability) of the sleep stage predicted through the artificial neural network. Specific description of the above sequence data, score value, sleep stage, and predictive confidence information is merely an example, and the present disclosure is not limited thereto.

That is, the processor 130 may provide prediction information on sleep stages for each time point during the user's sleep and prediction confidence information corresponding to the prediction information for each sleep stage. In this case, the prediction confidence information may be used as a determination index for how reliable the output (ie, the prediction information on the sleep stage) of the artificial neural network (ie, the sleep analysis model) is reliable. In other words, by presenting predictive confidence information related to sleep stage estimation at each time point together with sleep stage information, reliable use in a medical environment may be possible.

According to an embodiment of the present disclosure, the processor 130 may correct the prediction confidence information output by the sleep analysis model through temperature scaling. For example, the use of artificial neural networks in a medical environment may be at risk not because of low average accuracy, but due to uncertainty in operation that works well in some situations but does not work well in other situations. Accordingly, when the prediction reliability information is generated using only the softmax value, which is the output of the final output layer of the artificial neural network, it may be difficult to guarantee the reliability of the prediction reliability information. For example, when the neural network is overconfirmed in its prediction, there is a risk of delivering inaccurate information to a user referring to the corresponding prediction certainty information.

Accordingly, the processor 130 may perform an operation of matching the degree of confidence in the model with respect to the prediction and the actual accuracy to the same level through the temperature scaling process. Specifically, when a logit vector Z is given using a single scalar parameter T, the confidence prediction can be as follows.

는 최대 불확실성을 나타내는 1/K에 근접할 수 있다. 즉, 본 개시는, validation set에 대해 NLL(Negative log likelihood)을 최소화하는 식으로 optimization하는 temperature scaling을 통해 예측 확신도에 대한 보정을 수행할 수 있다. 이 경우, temperature scaling은 모델의 정확도에는 영향을 주지 않고, Calibration에만 영향을 줄 수 있다. As T increases, the probability value

can be close to 1/K, representing the maximum uncertainty. That is, in the present disclosure, it is possible to correct the prediction reliability through temperature scaling that optimizes the validation set in a way to minimize negative log likelihood (NLL). In this case, temperature scaling does not affect the accuracy of the model, but only the calibration.

In other words, it is possible to provide the effect of securing the reliability of the prediction confidence information itself, which is used as a user's judgment index, through the confidence correction using temperature scaling.

According to an embodiment of the present disclosure, the processor 130 may generate relationship information between the prediction certainty information and the sleep stage information. Also, the processor 130 may update the sleep analysis information based on the relationship information.

In more detail, sleep analysis information including sleep stage information and prediction confidence information may be characterized in that it is generated for each time point. As a specific example, as shown in FIG. 5 , the sleep analysis information 400 may include sleep stage information and prediction confidence information generated in response to each time point. For example, the first sleep analysis information 401 may be generated in response to the first sequence data 201 related to biosignals acquired through 9 channels during 0 to 30 seconds of sleep. The first sleep analysis information 401 may include information that the user's sleep corresponds to N1 at a time point corresponding to the first sequence data, and the prediction confidence information is 90%.

In this case, the processor 130 may generate the relationship information in consideration of the prediction certainty information and the sleep stage information for the entire sleep period. For example, if the overall average of the prediction confidence information is calculated relatively low, and when the change in sleep stage information during the entire sleep is more than a predetermined change threshold, the processor 130 determines the change in the sleep stage when the prediction confidence is low. It is possible to create relationship information that is frequent. As another example, if the user is interpreted as having a disease related to sleep apnea through the sleep phase ratio during the user's sleep, and the overall average of the accumulated predictive confidence information is relatively low, the processor 130 may determine that sleep apnea is low. It is possible to generate relational information that is related to the predictive certainty information. In other words, in the case of a user having a low prediction confidence, relationship information indicating a high probability of having a sleep disorder may be generated. The above-described detailed description of the relationship information is only an example, and the present disclosure is not limited thereto.

That is, the processor 130 may generate relationship information between the prediction certainty information and the sleep stage information, and update the sleep analysis information based on the generated relationship information. This may enable provision of sleep analysis information through new relationship information between sleep patterns and prediction accuracy according to changes in sleep stages. Accordingly, the processor 130 may provide the user (eg, a specialist) with insight into new associations other than sleep analysis of generally known patterns.

According to one embodiment of the present disclosure, the processor 130 may generate characteristic map information based on the sleep analysis information. Specifically, the processor 130 may generate characteristic map information by visualizing an attention weight for sleep analysis information according to each integrated feature through one or more attention modules.

As a specific example, the computing device 100 may determine that the user's sleep stage is N3 for the corresponding one minute through the user's sleep sensing data obtained in response to the first time point (eg, after going to sleep, the first user's sleep for 1 minute). It is possible to generate predictive information corresponding to the sleep stage. In addition, the computing device 100 calculates the attention weight of each input element from one or more attention modules to sleep analysis information related to the first time point (that is, predictive information that the user's sleep at the corresponding time point corresponds to the N3 sleep stage) can be obtained The computing device 100 may generate characteristic map information by visualizing the obtained attention weights for each element.

That is, the computing device 100 may visualize the basis for determination in the process of calculating prediction information for input data related to time series (ie, sleep sensing data) by utilizing one or more attention modules. In other words, which signal (ie, which sleep sensing data) played an important role in reading the sleep phase for each time point can be visualized and provided through the attention weight value. For example, the characteristic map information may be generated to provide visualization information in a meaningful form by displaying differently for each pixel based on a signal or information noticed through an attention weight.

In general, it is difficult to understand the internal algorithm of an artificial neural network model. That is, it is difficult to determine on what basis the output related to the input data is generated. This may have a problem that it is difficult to be utilized in an actual medical environment as it acts as a risk due to the uncertainty of the artificial neural network model.

The processor 130 of the present disclosure may visualize and provide a pattern of a signal that is important when predicting a sleep phase for each time point, thereby visualizing the basis on which the neural network model predicted or made a decision. Accordingly, it is possible to precisely verify the validity of the prediction information output by the sleep analysis model of the present disclosure, thereby maximizing collaboration synergy with the user (eg, a specialist).

6 is a flowchart exemplarily illustrating a method of providing sleep-related analysis information related to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, the method may include acquiring ( 510 ) the user's sleep sensing data.

According to an embodiment of the present disclosure, the method may include processing sleep sensing data as an input of a sleep analysis model and outputting sleep analysis information ( 520 ).

According to an embodiment of the present disclosure, the method may include generating 530 characteristic map information based on the sleep analysis information.

The order of the steps illustrated in FIG. 6 described above may be changed if necessary, and at least one or more steps may be omitted or added. That is, the above-described steps are merely an embodiment of the present disclosure, and the scope of the present disclosure is not limited thereto.

7 is a schematic diagram illustrating one or more network functions related to an embodiment of the present disclosure.

Throughout this specification, computational model, neural network, network function, and neural network may be used interchangeably. A neural network may be composed of a set of interconnected computational units, which may generally be referred to as “nodes”. These “nodes” may also be referred to as “neurons”. A neural network is configured by including at least one or more nodes. Nodes (or neurons) constituting neural networks may be interconnected by one or more “links”.

In the neural network, one or more nodes connected through a link may relatively form a relationship between an input node and an output node. The concepts of an input node and an output node are relative, and any node in an output node relationship with respect to one node may be in an input node relationship in a relationship with another node, and vice versa. As described above, an input node-to-output node relationship may be created around a link. One or more output nodes may be connected to one input node through a link, and vice versa.

In the relationship between the input node and the output node connected through one link, the value of the output node may be determined based on data input to the input node. Here, a node interconnecting the input node and the output node may have a weight. The weight may be variable, and may be changed by a user or an algorithm in order for the neural network to perform a desired function. For example, when one or more input nodes are interconnected to one output node by respective links, the output node sets values input to input nodes connected to the output node and links corresponding to the respective input nodes. An output node value may be determined based on the weight.

As described above, in a neural network, one or more nodes are interconnected through one or more links to form an input node and an output node relationship in the neural network. The characteristics of the neural network may be determined according to the number of nodes and links in the neural network, the correlation between the nodes and the links, and the value of a weight assigned to each of the links. For example, when the same number of nodes and links exist and there are two neural networks having different weight values between the links, the two neural networks may be recognized as different from each other.

A neural network may include one or more nodes. Some of the nodes constituting the neural network may configure one layer based on the distances from the initial input node. For example, a set of nodes having a distance n from the initial input node is You can configure n layers. The distance from the initial input node may be defined by the minimum number of links that must be passed to reach the corresponding node from the initial input node. However, the definition of such a layer is arbitrary for description, and the order of the layer in the neural network may be defined in a different way from the above. For example, a layer of nodes may be defined by a distance from the final output node.

The initial input node may mean one or more nodes to which data is directly input without going through a link in a relationship with other nodes among nodes in the neural network. Alternatively, in a relationship between nodes based on a link in a neural network, it may mean nodes that do not have other input nodes connected by a link. Similarly, the final output node may refer to one or more nodes that do not have an output node in relation to other nodes among nodes in the neural network. In addition, the hidden node may mean nodes constituting the neural network other than the first input node and the last output node. The neural network according to an embodiment of the present disclosure may be a neural network in which the number of nodes in the input layer may be the same as the number of nodes in the output layer, and the number of nodes decreases and then increases again as progresses from the input layer to the hidden layer. can Also, in the neural network according to another embodiment of the present disclosure, the number of nodes in the input layer may be less than the number of nodes in the output layer, and may be a neural network in which the number of nodes decreases as the number of nodes progresses from the input layer to the hidden layer. have. In addition, the neural network according to another embodiment of the present disclosure may be a neural network in which the number of nodes in the input layer may be greater than the number of nodes in the output layer, and the number of nodes increases as the number of nodes progresses from the input layer to the hidden layer. can The neural network according to another embodiment of the present disclosure may be a neural network in a combined form of the aforementioned neural networks.

A deep neural network (DNN) may refer to a neural network including a plurality of hidden layers in addition to an input layer and an output layer. Deep neural networks can be used to identify the latent structures of data. In other words, it can understand the potential structure of photos, texts, videos, voices, and music (e.g., what objects are in the photos, what the text and emotions are, what the texts and emotions are, etc.) . Deep neural networks include convolutional neural networks (CNNs), recurrent neural networks (RNNs), auto encoders, generative adversarial networks (GANs), and restricted boltzmann machines (RBMs). machine), a deep trust network (DBN), a Q network, a U network, a Siamese network, and the like. The description of the deep neural network described above is only an example, and the present disclosure is not limited thereto.

The neural network may be trained by at least one of supervised learning, unsupervised learning, and semi-supervised learning. The training of the neural network is to minimize the error in the output. In the training of a neural network, iteratively input the training data into the neural network, calculate the output of the neural network and the target error for the training data, and calculate the error of the neural network from the output layer of the neural network to the input layer in the direction to reduce the error. It is a process of updating the weights of each node of the neural network by backpropagation in the direction. In the case of teacher learning, learning data in which the correct answer is labeled in each learning data is used (ie, labeled learning data), and in the case of comparative learning, the correct answer may not be labeled in each learning data. That is, for example, learning data in the case of teacher learning related to data classification may be data in which categories are labeled in each of the learning data. Labeled training data is input to the neural network, and an error can be calculated by comparing the output (category) of the neural network with the label of the training data. As another example, in the case of comparison learning related to data classification, an error may be calculated by comparing the input training data with the neural network output. The calculated error is backpropagated in the reverse direction (ie, from the output layer to the input layer) in the neural network, and the connection weight of each node of each layer of the neural network may be updated according to the backpropagation. The change amount of the connection weight of each node to be updated may be determined according to a learning rate. The computation of the neural network on the input data and the backpropagation of errors can constitute a learning cycle (epoch). The learning rate may be applied differently according to the number of repetitions of the learning cycle of the neural network. For example, in the early stage of learning of a neural network, a high learning rate can be used to enable the neural network to quickly obtain a certain level of performance to increase efficiency, and in the late learning period, a low learning rate can be used to increase accuracy.

In the training of neural networks, in general, the training data may be a subset of real data (that is, data to be processed using the trained neural network), and thus, errors on the training data are reduced, but errors on the real data are reduced. There may be increasing learning cycles. Overfitting is a phenomenon in which errors on actual data increase by over-learning on training data as described above. For example, a phenomenon in which a neural network that has learned a cat by seeing a yellow cat does not recognize that it is a cat when it sees a cat other than yellow may be a type of overfitting. Overfitting can act as a cause of increasing errors in machine learning algorithms. In order to prevent such overfitting, various optimization methods can be used. In order to prevent overfitting, methods such as increasing training data, regularization, or dropout in which a part of nodes in the network are omitted in the process of learning, may be applied.

The steps of a method or algorithm described in relation to an embodiment of the present disclosure may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. A software module may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present disclosure pertains.

Components of the present disclosure may be implemented as a program (or application) to be executed in combination with a computer, which is hardware, and stored in a medium. Components of the present disclosure may be implemented as software programming or software components, and similarly, embodiments may include various algorithms implemented as data structures, processes, routines, or combinations of other programming constructs, including C, C++ , may be implemented in a programming or scripting language such as Java, assembler, or the like. Functional aspects may be implemented in an algorithm running on one or more processors.

Those of ordinary skill in the art of the present disclosure will recognize that the various illustrative logical blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein include electronic hardware, (convenience For this purpose, it will be understood that it may be implemented by various forms of program or design code (referred to herein as "software") or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. A person skilled in the art of the present disclosure may implement the described functionality in various ways for each specific application, but such implementation decisions should not be interpreted as a departure from the scope of the present disclosure.

The various embodiments presented herein may be implemented as methods, apparatus, or articles of manufacture using standard programming and/or engineering techniques. The term “article of manufacture” includes a computer program, carrier, or media accessible from any computer-readable device. For example, computer-readable media include magnetic storage devices (eg, hard disks, floppy disks, magnetic strips, etc.), optical disks (eg, CDs, DVDs, etc.), smart cards, and flash memory. devices (eg, EEPROMs, cards, sticks, key drives, etc.). Also, various storage media presented herein include one or more devices and/or other machine-readable media for storing information. The term “machine-readable medium” includes, but is not limited to, wireless channels and various other media that can store, hold, and/or convey instruction(s) and/or data.

It is to be understood that the specific order or hierarchy of steps in the presented processes is an example of exemplary approaches. Based on design priorities, it is understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of the present disclosure. The appended method claims present elements of the various steps in a sample order, but are not meant to be limited to the specific order or hierarchy presented.

The description of the presented embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not intended to be limited to the embodiments presented herein, but is to be construed in the widest scope consistent with the principles and novel features presented herein.

Claims (11)

A method performed on a processor of a computing device, comprising:
acquiring the user's sleep sensing data;
outputting sleep analysis information by processing the sleep sensing data as an input of a sleep analysis model; and
generating characteristic map information based on the sleep analysis information;
containing,
A method for providing analysis information related to sleep performed on a processor of a computing device.
The method of claim 1,
The sleep sensing data is
It includes one or more sequence data related to the user's bio-signals obtained in time series through one or more channels in relation to the user's sleeping environment,
The sleep analysis information,
and sleep stage information indicating that the user's sleep corresponds to at least one of one or more sleep stages,
The sleep analysis model is,
Characterized in that by receiving the sleep sensing data as an input, sleep analysis information corresponding to each of the one or more sequence data is output,
A method for providing analysis information related to sleep performed on a processor of a computing device.
3. The method of claim 2,
The sleep analysis model is,
a first sub-model and a second sub-model;
The first sub-model is
Taking each of the one or more sequence data included in the sleep sensing data as an input, extracting one or more features corresponding to each of the one or more channels, and integrating the extracted one or more features to one or more integrated features corresponding to each sequence data characterized in that it creates
The second sub-model is
Characterized in outputting the sleep analysis information by inputting each of the one or more integrated features,
A method for providing analysis information related to sleep performed on a processor of a computing device.
4. The method of claim 3,
The sleep analysis model is,
It further comprises one or more attention modules,
The one or more attention modules,
Characterized in calculating an attention weight for sleep analysis information corresponding to each of the one or more integrated features,
A method for providing analysis information related to sleep performed on a processor of a computing device.
5. The method of claim 4,
The step of generating characteristic map information based on the sleep analysis information includes:
Visualizing the attention weight for sleep analysis information according to each integrated feature through the one or more attention modules to generate the characteristic map information,
The characteristic map information is
Information displayed by visualizing the data that affected the prediction for each sequence data output by the sleep analysis model,
A method for providing analysis information related to sleep performed on a processor of a computing device.
The method of claim 1,
The sleep analysis model is,
It is characterized in that transfer learning is performed so that an algorithm trained in a source domain can be applied in another target domain,
The transfer learning is
Including performing learning in a target domain related to sleep disorders based on an algorithm trained in a source domain related to normal,
A method for providing analysis information related to sleep performed on a processor of a computing device.
The method of claim 1,
The sleep analysis information,
and predictive certainty information corresponding to each of the one or more sequence data,
The method is
performing correction on the prediction confidence information output by the sleep analysis model through temperature scaling;
further comprising,
A method for providing analysis information related to sleep performed on a processor of a computing device.
8. The method of claim 7,
The method is
generating relationship information between the prediction certainty information and the sleep stage information; and
updating the sleep analysis information based on the relationship information;
containing,
A method for providing analysis information related to sleep performed on a processor of a computing device.
The method of claim 1,
performing pre-processing on the sleep sensing data;
further comprising,
The pretreatment is
Including performing down-sampling on the sleep sensing data,
A method for providing analysis information related to sleep performed on a processor of a computing device.
A computing device comprising:
a processor including one or more cores;
a memory storing program codes executable by the processor; and
a network unit for transmitting and receiving data to and from the user terminal;
including,
The processor is
Obtaining the user's sleep sensing data, processing the sleep sensing data as an input of a sleep analysis model to output sleep analysis information, and generating characteristic map information based on the sleep analysis information,
A computing device for providing sleep-related analytical information.
A computer program stored in a computer-readable storage medium, wherein the computer program, when executed on one or more processors, causes the one or more processors to perform the following operations for providing sleep-related analysis information, the operations comprising: :
acquiring the user's sleep sensing data;
outputting sleep analysis information by processing the sleep sensing data as an input of a sleep analysis model; and
generating characteristic map information based on the sleep analysis information;
containing,
A computer program stored on a computer-readable storage medium.

Images