KR102653940B1 - Method and system for adjusting humidification device using neural network based on user's sleep state - Google Patents

Abstract

Embodiments present a method and system for adjusting a humidification device using a neural network based on the user's sleep state. The system according to one embodiment includes a sensing device that senses user state information about a user and space state information about the space in which the user is located; The user state information includes information about the user's body temperature, information about the user's movement, and information about the user's breathing sound, and the space state information includes information about the illuminance, information about humidity, and temperature. a sleep determination device that includes information about the user and air quality and is linked with the sensing device to determine whether the user is in a sleeping state; A humidifying device provided in the space where the user is located and spraying water vapor; A central control device that controls the sensing device, the sleep determination device, and the humidification device by transmitting and receiving information with the sensing device, the sleep determination device, and the humidification device through short-distance wireless communication; Predicting the detailed sleep stage of the user based on information received through long-distance wireless communication from the central control device, and transmitting operation information about the humidification device determined according to the detailed sleep stage of the user to the central control device May include servers. For example, the server receives basic information about the user, user state information, space state information, and setting information about the humidification device from the central control device, and configures the basic information about the user and the space First prediction information is determined through a basic prediction model using a first neural network based on state information, and the first prediction information is information predicting the user's detailed sleep stage by time using basic information about the user. And, operation information for the humidifying device is provided through a humidifying device adjustment model using a second neural network based on the basic information about the user, the first prediction information, the space state information, and the setting information for the humidifying device. determines the operation information on the humidifying device and transmits it to the central control device, and the operation of the humidifying device can be adjusted based on the operation information on the humidifying device transmitted from the central control device.

Description

Method and system for adjusting a humidification device using a neural network based on the user's sleep state {METHOD AND SYSTEM FOR ADJUSTING HUMIDIFICATION DEVICE USING NEURAL NETWORK BASED ON USER'S SLEEP STATE}

Embodiments of the present disclosure relate to technology for adjusting a humidifying device using a neural network, and to a method and system for adjusting a humidifying device using a neural network based on a user's sleep state.

Meanwhile, modern people are unable to get a good night's sleep due to irregular eating habits, lifestyle habits, and stress. As a result, the number of people suffering from sleep disorders such as insomnia, hypersomnia, and sleep apnea syndrome is increasing.

Good sleep is an important factor that affects physical or mental health, but there is a lack of devices and methods to help improve sleep disorders. Existing methods and systems receive the user's body information and output vibration and/or ultrasonic waves in the frequency band detected through repetitive learning according to the user's physical condition during sleep to induce optimal sleep. However, it is inconvenient to wear equipment on the body to determine the user's sleep state, and it is difficult to improve the user's sleep state simply by outputting ultrasonic waves.

Additionally, modern people use humidifying devices to create a comfortable living environment. At this time, the humidity suitable for the user may vary depending on the user's sleep cycle, but it is impossible to directly control the humidity with a humidifying device while the user is sleeping, and it is difficult for the user to determine the sleeping state by himself. Furthermore, the appropriate sleeping environment is different for each user using the humidifying device, and the amount of humidification required to maintain appropriate humidity may vary depending on the surrounding environment of the space where the user is located (e.g., illuminance, temperature, humidity, and air quality).

Accordingly, a plurality of sensors are installed in the space where the user is located to sense the user's condition and the state of the space where the user is located, and a central control device that controls the sensed information is used to determine the user's sleeping condition and appropriate humidity. There is a need for a method and system that can operate a humidifying device and adjust a humidifying device using a neural network based on the user's sleeping state.

Embodiments of the present disclosure may provide a method and system for adjusting a humidification device using a neural network based on the user's sleep state.

The technical challenges to be achieved in the embodiments are not limited to the matters mentioned above, and other technical challenges not mentioned may be considered by those skilled in the art from the various embodiments described below. You can.

A system for adjusting a humidifying device using a neural network according to an embodiment includes a sensing device that senses user state information about a user and space state information about the space in which the user is located; The user state information includes information about the user's body temperature, information about the user's movement, and information about the user's breathing sound, and the space state information includes information about the illuminance, information about humidity, and temperature. a sleep determination device that includes information about the user and air quality and is linked with the sensing device to determine whether the user is in a sleeping state; A humidifying device provided in the space where the user is located and spraying water vapor; A central control device that controls the sensing device, the sleep determination device, and the humidification device by transmitting and receiving information with the sensing device, the sleep determination device, and the humidification device through short-distance wireless communication; Predicting the detailed sleep stage of the user based on information received through long-distance wireless communication from the central control device, and transmitting operation information about the humidification device determined according to the detailed sleep stage of the user to the central control device May include servers. For example, the server receives basic information about the user, user state information, space state information, and setting information about the humidification device from the central control device, and configures the basic information about the user and the space First prediction information is determined through a basic prediction model using a first neural network based on state information, and the first prediction information is information predicting the user's detailed sleep stage by time using basic information about the user. And, operation information for the humidifying device is provided through a humidifying device adjustment model using a second neural network based on the basic information about the user, the first prediction information, the space state information, and the setting information for the humidifying device. determines the operation information on the humidifying device and transmits it to the central control device, and the operation of the humidifying device can be adjusted based on the operation information on the humidifying device transmitted from the central control device.

According to embodiments, the server determines the first prediction information through a basic prediction model using a first neural network based on basic information about the user and spatial state information, thereby You can predict detailed sleep stages by hour by considering illumination, temperature, humidity, and air quality.

According to embodiments, the server provides operation information about the humidifying device through a humidifying device adjustment model using a second neural network based on basic information about the user, first prediction information, space state information, and setting information about the humidifying device. By determining and adjusting the humidifying device through the central control device, the humidifying device can be operated at a humidity suitable for the user according to the user's detailed sleep stage by time, taking into account the state of the space where the user is located.

The effects that can be obtained from the examples are not limited to the effects mentioned above, and other effects not mentioned can be clearly derived and understood by those skilled in the art based on the detailed description below. It can be.

The accompanying drawings, which are included as part of the detailed description to aid understanding of the embodiments, provide various embodiments and together with the detailed description describe technical features of the various embodiments.
1 is a diagram showing the configuration of an electronic device according to an embodiment.
Figure 2 is a diagram showing the configuration of a program according to one embodiment.
Figure 3 shows a system for adjusting a humidification device using a neural network based on the user's sleep state according to one embodiment.
Figure 4 is a signal exchange diagram for a method of adjusting a humidification device using a neural network based on the user's sleep state according to an embodiment.
Figure 5 is a diagram showing an example of a basic prediction model according to an embodiment.
Figure 6 is a block diagram showing the configuration of a server according to one embodiment.

The following embodiments combine elements and features of the embodiments in a predetermined form. Each component or feature may be considered optional unless explicitly stated otherwise. Each component or feature may be implemented in a form that is not combined with other components or features. Additionally, various embodiments may be configured by combining some components and/or features. The order of operations described in various embodiments may change. Some features or features of one embodiment may be included in other embodiments or may be replaced with corresponding features or features of other embodiments.

In the description of the drawings, procedures or steps that may obscure the gist of the various embodiments are not described, and procedures or steps that can be understood at the level of a person with ordinary knowledge in the relevant technical field are not described. did.

Throughout the specification, when a part is said to “comprise or include” a certain element, this means that it does not exclude other elements but may further include other elements, unless specifically stated to the contrary. do. In addition, terms such as "... unit", "... unit", and "module" used in the specification refer to a unit that processes at least one function or operation, which refers to hardware, software, or a combination of hardware and software. It can be implemented as: Additionally, the terms “a or an,” “one,” “the,” and similar related terms are used herein in the context of describing various embodiments (particularly in the context of the claims below). Unless otherwise indicated or clearly contradicted by context, it may be used in both singular and plural terms.

Hereinafter, embodiments according to various embodiments will be described in detail with reference to the attached drawings. The detailed description set forth below in conjunction with the accompanying drawings is intended to describe exemplary embodiments of various embodiments and is not intended to represent the only embodiment.

In addition, specific terms used in various embodiments are provided to aid understanding of the various embodiments, and the use of such specific terms may be changed to other forms without departing from the technical spirit of the various embodiments. .

1 is a diagram showing the configuration of an electronic device according to an embodiment.

1 is a block diagram of an electronic device 101 in a network environment 100, according to various embodiments. Referring to FIG. 1, in the network environment 100, the electronic device 101 communicates with the electronic device 102 through a first network 198 (e.g., a short-range wireless communication network) or a second network 199. It is possible to communicate with at least one of the electronic device 104 or the server 108 through (e.g., a long-distance wireless communication network). According to one embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108. According to one embodiment, the electronic device 101 includes a processor 120, a memory 130, an input module 150, an audio output module 155, a display module 160, an audio module 170, and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or may include an antenna module 197. In some embodiments, at least one of these components (eg, the connection terminal 178) may be omitted or one or more other components may be added to the electronic device 101. In some embodiments, some of these components (e.g., sensor module 176, camera module 180, or antenna module 197) are integrated into one component (e.g., display module 160). It can be. The electronic device 101 may also be referred to as a client, terminal, or peer.

The processor 120, for example, executes software (e.g., program 140) to operate at least one other component (e.g., hardware or software component) of the electronic device 101 connected to the processor 120. It can be controlled and various data processing or calculations can be performed. According to one embodiment, as at least part of data processing or computation, the processor 120 stores commands or data received from another component (e.g., sensor module 176 or communication module 190) in volatile memory 132. The commands or data stored in the volatile memory 132 can be processed, and the resulting data can be stored in the non-volatile memory 134. According to one embodiment, the processor 120 includes a main processor 121 (e.g., a central processing unit or an application processor) or an auxiliary processor 123 that can operate independently or together (e.g., a graphics processing unit, a neural network processing unit ( It may include a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device 101 includes a main processor 121 and a secondary processor 123, the secondary processor 123 may be set to use lower power than the main processor 121 or be specialized for a designated function. You can. The auxiliary processor 123 may be implemented separately from the main processor 121 or as part of it.

The auxiliary processor 123 may, for example, act on behalf of the main processor 121 while the main processor 121 is in an inactive (e.g., sleep) state, or while the main processor 121 is in an active (e.g., application execution) state. ), together with the main processor 121, at least one of the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190) At least some of the functions or states related to can be controlled. According to one embodiment, co-processor 123 (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module 180 or communication module 190). there is. According to one embodiment, the auxiliary processor 123 (eg, neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models.

Artificial intelligence models can be created through machine learning. For example, such learning may be performed in the electronic device 101 itself on which the artificial intelligence model is performed, or may be performed through a separate server (e.g., server 108). Learning algorithms may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but It is not limited. An artificial intelligence model may include multiple artificial neural network layers. Artificial neural networks include deep neural network (DNN), convolutional neural network (CNN), recurrent neural network (RNN), restricted boltzmann machine (RBM), belief deep network (DBN), bidirectional recurrent deep neural network (BRDNN), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the examples described above. In addition to hardware structures, artificial intelligence models may additionally or alternatively include software structures.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176) of the electronic device 101. Data may include, for example, input data or output data for software (e.g., program 140) and instructions related thereto. Memory 130 may include volatile memory 132 or non-volatile memory 134.

The program 140 may be stored as software in the memory 130 and may include, for example, an operating system 142, middleware 144, or application 146.

The input module 150 may receive commands or data to be used in a component of the electronic device 101 (e.g., the processor 120) from outside the electronic device 101 (e.g., a user). The input module 150 may include, for example, a microphone, mouse, keyboard, keys (eg, buttons), or digital pen (eg, stylus pen).

The sound output module 155 may output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. Speakers can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.

The display module 160 can visually provide information to the outside of the electronic device 101 (eg, a user). The display module 160 may include, for example, a display, a hologram device, or a projector, and a control circuit for controlling the device. According to one embodiment, the display module 160 may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of force generated by the touch.

The audio module 170 can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device (e.g., directly or wirelessly connected to the electronic device 101). Sound may be output through the electronic device 102 (e.g., speaker or headphone).

The sensor module 176 detects the operating state (e.g., power or temperature) of the electronic device 101 or the external environmental state (e.g., user state) and generates an electrical signal or data value corresponding to the detected state. can do. According to one embodiment, the sensor module 176 includes, for example, a motion sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared sensor, a biometric sensor, It may include an acoustic sensor, temperature sensor, humidity sensor, or light sensor.

The interface 177 may support one or more designated protocols that can be used to connect the electronic device 101 directly or wirelessly with an external electronic device (eg, the electronic device 102). According to one embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102). According to one embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 can convert electrical signals into mechanical stimulation (e.g., vibration or movement) or electrical stimulation that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 can capture still images and videos. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 can manage power supplied to the electronic device 101. According to one embodiment, the power management module 188 may be implemented as at least a part of, for example, a power management integrated circuit (PMIC).

Battery 189 may supply power to at least one component of electronic device 101. According to one embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

Communication module 190 is configured to provide a direct (e.g., wired) communication channel or wireless communication channel between electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108). It can support establishment and communication through established communication channels. Communication module 190 operates independently of processor 120 (e.g., an application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., : LAN (local area network) communication module, or power line communication module) may be included. Among these communication modules, the corresponding communication module is a first network 198 (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (e.g., legacy It may communicate with an external electronic device 104 through a telecommunication network such as a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., LAN or WAN). These various types of communication modules may be integrated into one component (e.g., a single chip) or may be implemented as a plurality of separate components (e.g., multiple chips). The wireless communication module 192 uses subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199. The electronic device 101 can be confirmed or authenticated.

The wireless communication module 192 may support 5G networks after 4G networks and next-generation communication technologies, for example, NR access technology (new radio access technology). NR access technology provides high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low latency). -latency communications)) can be supported. The wireless communication module 192 may support high frequency bands (eg, mmWave bands), for example, to achieve high data rates. The wireless communication module 192 uses various technologies to secure performance in high frequency bands, for example, beamforming, massive array multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. It can support technologies such as input/output (FD-MIMO: full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module 192 may support various requirements specified in the electronic device 101, an external electronic device (e.g., electronic device 104), or a network system (e.g., second network 199). According to one embodiment, the wireless communication module 192 supports Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mmTC, or U-plane latency (e.g., 164 dB or less) for realizing URLLC. Example: Downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) can be supported.

The antenna module 197 may transmit or receive signals or power to or from the outside (eg, an external electronic device). According to one embodiment, the antenna module 197 may include an antenna including a radiator made of a conductor or a conductive pattern formed on a substrate (eg, PCB). According to one embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected to the plurality of antennas by, for example, the communication module 190. can be selected. Signals or power may be transmitted or received between the communication module 190 and an external electronic device through the at least one selected antenna. According to some embodiments, in addition to the radiator, other components (eg, radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module 197.

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, a mmWave antenna module includes: a printed circuit board, an RFIC disposed on or adjacent to a first side (e.g., bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., mmWave band); And a plurality of antennas (e.g., array antennas) disposed on or adjacent to the second side (e.g., top or side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (e.g., bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and signal ( (e.g. commands or data) can be exchanged with each other.

According to one embodiment, commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199. Each of the external electronic devices 102 or 104 may be of the same or different type as the electronic device 101. According to one embodiment, all or part of the operations performed in the electronic device 101 may be executed in one or more of the external electronic devices 102, 104, or 108. For example, when the electronic device 101 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 may perform the function or service instead of executing the function or service on its own. Alternatively, or additionally, one or more external electronic devices may be requested to perform at least part of the function or service. One or more external electronic devices that have received the request may execute at least part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may process the result as is or additionally and provide it as at least part of a response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology can be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to one embodiment, the external electronic device 104 or server 108 may be included in the second network 199. The electronic device 101 may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

The server 108 is connected to the electronic device 101 and can provide services to the connected electronic device 101. In addition, the server 108 may perform a membership registration process, store and manage various information of users who have registered as members, and provide various purchase and payment functions related to the service. Additionally, the server 108 may share execution data of service applications running on each of the plurality of electronic devices 101 in real time so that services can be shared between users. This server 108 may have the same hardware configuration as a typical web server or service server. However, in terms of software, it may be implemented through any language such as C, C++, Java, Python, Golang, and Kotlin and may include program modules that perform various functions. In addition, the server 108 is generally connected to an unspecified number of clients and/or other servers through an open computer network such as the Internet, and receives work performance requests from clients or other servers and derives and provides work results in response. It refers to a computer system and the computer software (server program) installed for it. In addition, in addition to the server program described above, the server 108 includes a series of application programs running on the server 108 and, in some cases, various databases (DBs) built internally or externally, hereinafter " It should be understood as a broad concept including “DB”). Accordingly, the server 108 classifies membership registration information and various information and data about games, stores them in a DB, and manages this DB, which may be implemented inside or outside the server 108. In addition, the server 108 can be implemented using a variety of server programs provided on general server hardware and operating systems such as Windows, Linux, UNIX, and Macintosh. , Representative examples include IIS (Internet Information Server) used in a Windows environment and CERN, NCSA, APPACH, and TOMCAT used in a Unix environment, etc., to implement web services. Additionally, the server 108 may be linked with an authentication system and payment system for user authentication of the service or payment for purchases related to the service.

The first network 198 and the second network 199 are a connection structure that allows information exchange between each node, such as terminals and servers, or a network connecting the server 108 and the electronic devices 101 and 104. It means (Network). The first network 198 and the second network 199 are the Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), and 3G. , 4G, LTE, 5G, Wi-Fi, etc., but are not limited to these. The first network 198 and the second network 199 may be closed, such as a LAN or WAN, but are preferably open, such as the Internet. The Internet includes protocols such as TCP/IP protocol, TCP, and UDP (user datagram protocol), as well as various services that exist at the upper layer, such as HTTP (HyperText Transfer Protocol), Telnet, FTP (File Transfer Protocol), and DNS (Domain Name System). ), a worldwide open computer primary network (198) and secondary network (199) that provides Simple Mail Transfer Protocol (SMTP), Simple Network Management Protocol (SNMP), Network File Service (NFS), and Network Information Service (NIS). ) refers to the structure.

A database can have a general data structure implemented in the storage space (hard disk or memory) of a computer system using a database management program (DBMS). A database may have a data storage format that allows for free search (extraction), deletion, editing, addition, etc. of data. Databases are relational database management systems (RDBMS) such as Oracle, Infomix, Sybase, and DB2, or object-oriented database management such as Gemston, Orion, and O2. It can be implemented according to the purpose of an embodiment of the present disclosure using a system (OODBMS) and an XML native database such as Excelon, Tamino, and Sekaiju, and has its own functions. To achieve this, you can have appropriate fields or elements.

Figure 2 is a diagram showing the configuration of a program according to one embodiment.

Figure 2 is a block diagram 200 illustrating program 140 according to various embodiments. According to one embodiment, the program 140 includes an operating system 142, middleware 144, or an application 146 executable on the operating system 142 for controlling one or more resources of the electronic device 101. It can be included. Operating system 142 may include, for example, AndroidTM, iOSTM, WindowsTM, SymbianTM, TizenTM, or BadaTM. At least some of the programs 140 are preloaded into the electronic device 101, for example, at the time of manufacture, or are stored in an external electronic device (e.g., the electronic device 102 or 104, or a server) when used by a user. It can be downloaded or updated from 108)). All or part of the program 140 may include a neural network.

The operating system 142 may control management (eg, allocation or retrieval) of one or more system resources (eg, process, memory, or power) of the electronic device 101 . Operating system 142 may additionally or alternatively operate on other hardware devices of electronic device 101, such as input module 150, audio output module 155, display module 160, and audio module 170. , sensor module 176, interface 177, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196, or It may include one or more driver programs for driving the antenna module 197.

The middleware 144 may provide various functions to the application 146 so that functions or information provided from one or more resources of the electronic device 101 can be used by the application 146. The middleware 144 includes, for example, an application manager 201, a window manager 203, a multimedia manager 205, a resource manager 207, a power manager 209, a database manager 211, and a package manager 213. ), connectivity manager (215), notification manager (217), location manager (219), graphics manager (221), security manager (223), call manager (225), or voice recognition manager (227). You can.

The application manager 201 may, for example, manage the life cycle of the application 146. The window manager 203 may, for example, manage one or more GUI resources used on the screen. For example, the multimedia manager 205 identifies one or more formats required for playing media files, and encodes or decodes the corresponding media file using a codec suitable for the selected format. It can be done. The resource manager 207 may, for example, manage the source code of the application 146 or the memory space of the memory 130. The power manager 209 manages, for example, the capacity, temperature, or power of the battery 189, and may use this information to determine or provide related information necessary for the operation of the electronic device 101. . According to one embodiment, the power manager 209 may interface with a basic input/output system (BIOS) (not shown) of the electronic device 101.

Database manager 211 may create, search, or change a database to be used by application 146, for example. The package manager 213 may, for example, manage the installation or update of applications distributed in the form of package files. The connectivity manager 215 may manage, for example, a wireless connection or direct connection between the electronic device 101 and an external electronic device. For example, the notification manager 217 may provide a function for notifying the user of the occurrence of a designated event (eg, an incoming call, message, or alarm). The location manager 219 may, for example, manage location information of the electronic device 101. The graphics manager 221 may, for example, manage one or more graphic effects to be provided to the user or a user interface related thereto.

Security manager 223 may provide, for example, system security or user authentication. The telephony manager 225 may manage, for example, a voice call function or a video call function provided by the electronic device 101. For example, the voice recognition manager 227 transmits the user's voice data to the server 108 and provides a command corresponding to a function to be performed in the electronic device 101 based at least in part on the voice data, Alternatively, text data converted based at least in part on the voice data may be received from the server 108. According to one embodiment, the middleware 244 may dynamically delete some existing components or add new components. According to one embodiment, at least a portion of the middleware 144 may be included as part of the operating system 142 or may be implemented as separate software different from the operating system 142.

The application 146 includes, for example, home 251, dialer 253, SMS/MMS (255), instant message (IM) 257, browser 259, camera 261, and alarm 263. , Contacts (265), Voice Recognition (267), Email (269), Calendar (271), Media Player (273), Album (275), Watch (277), Health (279) (such as exercise amount or blood sugar) It may include applications that measure biometric information) or environmental information 281 (e.g., measure atmospheric pressure, humidity, or temperature information). According to one embodiment, the application 146 may further include an information exchange application (not shown) that can support information exchange between the electronic device 101 and an external electronic device. The information exchange application may include, for example, a notification relay application configured to deliver designated information (e.g., calls, messages, or alarms) to an external electronic device, or a device management application configured to manage the external electronic device. there is. The notification relay application, for example, transmits notification information corresponding to a specified event (e.g., mail reception) generated in another application (e.g., email application 269) of the electronic device 101 to an external electronic device. You can. Additionally or alternatively, the notification relay application may receive notification information from an external electronic device and provide it to the user of the electronic device 101.

The device management application, for example, controls the power (e.g., turn-on or turn-off) of an external electronic device or some component thereof (e.g., a display module or camera module of the external electronic device) that communicates with the electronic device 101. ) or functions (such as brightness, resolution, or focus). A device management application may additionally or alternatively support installation, deletion, or update of applications running on external electronic devices.

Throughout this specification, neural network, neural network, and network function may be used with the same meaning. A neural network may consist of a set of interconnected computational units, which may generally be referred to as “nodes.” These “nodes” may also be referred to as “neurons.” A neural network is composed of at least two or more nodes. Nodes (or neurons) that make up neural networks may be interconnected by one or more “links.”

Within a neural network, two or more nodes connected through a link can relatively form a relationship as an input node and an output node. The concepts of input node and output node are relative, and any node in an output node relationship with one node may be in an input node relationship with another node, and vice versa. As described above, input node to output node relationships can be created around links. One or more output nodes can be connected to one input node through a link, and vice versa.

In a relationship between an input node and an output node connected through one link, the value of the output node may be determined based on data input to the input node. Here, nodes connecting the input node and the output node may have weights. Weights may be variable and may be varied by a user or algorithm in order for the neural network to perform a desired function. Here, the edges or links that interconnect the input nodes and output nodes have weights that can be variably applied by the user or algorithm to perform the function desired by the neural network. For example, when one or more input nodes are connected to one output node by respective links, the output node is set to the values input to the input nodes connected to the output node and the links corresponding to each input node. The output node value can be determined based on the weight.

As described above, in a neural network, two or more nodes are interconnected through one or more links to form an input node and output node relationship within the neural network. The characteristics of the neural network may be determined according to the number of nodes and links in the neural network, the correlation between the nodes and links, and the value of the weight assigned to each link. For example, if there are two neural networks with the same number of nodes and links and different weight values between the links, the two neural networks may be recognized as different from each other.

Figure 3 shows a system for adjusting a humidification device using a neural network based on the user's sleep state according to one embodiment. The embodiments of FIG. 3 can be combined with various embodiments of the present disclosure.

Referring to Figure 3, the system 300 that adjusts the humidification device using a neural network based on the user's sleep state includes a sensing device 310, a sleep determination device 320, a humidifying device 330, and a central control device. It may include 340 and server 350.

The sensing device 310 may sense the user's body temperature, movement, and sound using a plurality of sensors, and may be a device that senses the illuminance, humidity, temperature, and air quality of the space where the user is located. The sensing device 310 may detect the user's state and the state of the space in which the user is located, and generate an electrical signal or data value corresponding to the detected state. For example, the sensing device 310 may sense user state information about the user and spatial state information about the space where the user is located. The sensing device 310 includes a processor (e.g., processor 120 in FIG. 1), a memory (e.g., memory 130 in FIG. 1), a communication module (e.g., communication module 190 in FIG. 1), and a plurality of sensors. It may include a module (e.g., sensor module 176 in FIG. 1). The plurality of sensor modules may include at least one infrared sensor, at least one acoustic sensor, at least one illuminance sensor, at least one humidity sensor, at least one temperature sensor, and at least one air quality sensor. For example, when each of the plurality of sensor modules is installed in a different location, the sensing device 310 may receive a sensed value from each of the plurality of sensor modules.

User status information may be sensing information to determine the user's status. For example, user status information may include information about the user's body temperature, information about the user's movement, and information about the user's breathing sound. Information about the user's movement may be information that senses the user's movement. For example, information about the user's movement may include the time when the user's movement was detected and the location where the user's movement was detected. Information about the user's breathing sound may be information recorded by sensing the user's breathing sound. For example, the information about the user's breathing sound may be information in which the user's breathing sound is recorded based on the level of the surrounding sound of the sensing device 310 being detected as being below a preset decibel. Space state information may include information about illuminance, information about humidity, information about temperature, and information about air quality. Information about illuminance may be information that measures the intensity of light in the space where the user is located. For example, information about illuminance may be expressed in units of lux. Information about humidity may be information that measures the amount of water vapor in the air in the space where the user is located. For example, information about humidity can be expressed as a percentage. Information about temperature may be information that measures the temperature of the space where the user is located. For example, information about temperature may be expressed in Celsius or Fahrenheit. Information about air quality may be information that measures the concentration of fine dust, carbon monoxide, carbon dioxide, nitrogen dioxide, and volatile organic compounds in the air in the space where the user is located. For example, information about air quality may include carbon monoxide concentration, carbon dioxide concentration, carbon monoxide concentration, fine dust concentration of 2.5 micrometers or less, fine dust concentration of 10 micrometers or less, concentration of volatile organic compounds, and PPM (parts It can be expressed in units of per million).

The sleep determination device 320 may be a device that is linked with the sensing device 310 to determine whether the user is in a sleeping state. For example, the sleep determination device 320 includes a processor (e.g., processor 120 in FIG. 1), memory (e.g., memory 130 in FIG. 1), and a communication module (e.g., communication module 190 in FIG. 1). ) and a camera module. The camera module can capture still images and moving images of the user. A camera module may include one or more lenses, image sensors, image signal processors, or flashes. For example, the sleep determination device 320 may determine whether the user is in a sleep state based on information received from the sensing device 310 through the communication module.

The humidifying device 330 may be a device provided in the space where the user is located to control humidity by spraying water vapor. For example, the humidifying device 330 may include an air purifying function in addition to the function of spraying water vapor. The humidifying device 330 includes a processor (e.g., processor 120 in FIG. 1), memory (e.g., memory 130 in FIG. 1), a communication module (e.g., communication module 190 in FIG. 1), and a water vapor generation module. , may include a sensor module (eg, sensor module 176 in FIG. 1) including a water vapor injection module, a light emitting module, a wind speed sensor, and a humidity sensor. The water vapor generation module may include a container for storing water and a water vapor conversion module that converts water into water vapor in various ways. The water vapor injection module can spray the generated water vapor according to wind speed, and can spray water vapor using various devices such as a blowing fan. The light-emitting module can operate in conjunction with the discharge of water vapor and blink in a specific color or pattern depending on the wind speed of the water vapor. For example, the light emitting module may be implemented with LED. The wind speed sensor can detect the wind speed of water vapor. The humidity sensor can sense the humidity around the humidifying device 330.

The central control device 340 transmits and receives information to the sensing device 310, the sleep determination device 320, and the humidification device 330 through short-distance wireless communication to communicate with the sensing device 310, the sleep determination device 320, and the humidification device. (330) can be controlled. The central control device 340 may include a processor (e.g., processor 120 in FIG. 1), memory (e.g., memory 130 in FIG. 1), and a communication module (e.g., communication module 190 in FIG. 1). You can.

The central control device 340 may be pre-connected to the sensing device 310, the sleep determination device 320, and the humidifying device 330 through short-distance wireless communication. For example, the central control device 340 may receive user state information and space state information from the sensing device 310 through short-distance wireless communication. For example, the central control device 340 may receive a message related to the sleep state, such as a sleep start message or a sleep end message, from the sleep determination device 320 through short-distance wireless communication. For example, the central control device 340 may transmit control information to the humidifying device 330 through short-distance wireless communication, and the humidifying device 330 may operate according to the control information.

The server 350 may be a server that provides a service that determines the user's detailed sleep state and controls the humidifying device 330 provided in the space where the user is located according to the user's detailed sleep state. For example, the server 350 predicts the user's detailed sleep stage based on information received through long-distance wireless communication from the central control device, and centrally controls the operation information for the humidifying device determined according to the user's detailed sleep stage. It can be transmitted to the device. Detailed sleep stages may include stages 1 to 5. The first stage is the stage where the transition to sleep begins, and may be a very light sleep stage. For example, the first stage may be a stage in which brain waves corresponding to alpha waves of 8 Hz to 10 Hz are generated. The second stage is a light sleep stage, and may be a stage in which brain waves corresponding to theta waves of 4Hz to 7Hz are generated. In the first and second stages, the intensity of breathing may change periodically and the respiratory rate may decrease compared to the normal state. The third stage is a deep sleep stage, and may be a stage in which brain waves corresponding to delta waves of 0.5 Hz to 0.7 Hz occur less than 50%. The fourth stage is a deep sleep stage before moving to the fifth stage, and may be a stage in which brain waves corresponding to delta waves of 0.5 Hz to 0.7 Hz occur at more than 50%. The proportion of sleep time occupied by stages 3 to 4 increases to a maximum during childhood, and may thereafter decrease with age. Additionally, the third to fourth stages show regular breathing, and the amount of breathing may decrease overall. The fifth stage may be a REM (rapid eye movement sleep) sleep stage, and the fifth stage may be a stage in which brain waves corresponding to gamma waves of 30Hz to 40Hz may occur and breathing becomes irregular as the respiratory rate increases. For example, stage 5 may account for 50% of sleep time in infancy.

The server 350 may receive basic information about the user, user state information, space state information, and setting information about the humidifying device 330 from the central control device 340 through long-distance wireless communication.

Basic information about the user may include the user's age, the user's gender, the user's weight, and the user's height. For example, basic information about a user may be transmitted from the user terminal to the central control device. Alternatively, basic information about the user may be input to the server 350 through a user interface installed on the user terminal. For example, the user terminal may be the electronic device 101 of FIG. 1 .

Setting information about the humidifying device 330 may include the model name of the humidifying device 330, the manufacturing year and month of the humidifying device 330, and information about the location of the humidifying device 330 in the space where the user is located.

The server 350 may determine first prediction information through a basic prediction model using a first neural network based on basic information about the user and spatial state information from the central control device 340. The first prediction information may be information that predicts the detailed sleep stage of the user by time using basic information about the user.

The server 350 provides information to the humidifying device 330 through a humidifying device adjustment model using a second neural network based on basic information about the user, first prediction information, space state information, and setting information about the humidifying device 330. Operation information can be determined. The server 350 may transmit operation information about the humidifying device 330 to the central control device 340. At this time, the operation of the humidifying device 330 may be adjusted based on the operation information about the humidifying device 330 transmitted from the central control device 340. For example, the humidifying device 330 may spray water vapor into the space where the user is located according to operation information about the humidifying device 330. For example, server 350 may include server 108 of FIG. 1 .

However, not all of the components shown in FIG. 3 are essential components of the system. The system may be implemented with more components than the components shown in FIG. 3, or the system may be implemented with fewer components than the components shown in FIG. 3.

Figure 4 is a signal exchange diagram for a method of adjusting a humidification device using a neural network based on the user's sleep state according to an embodiment. The embodiment of FIG. 4 can be combined with various embodiments of the present disclosure.

Referring to FIG. 4, in step S401, the central control device may establish a preliminary connection with the sleep determination device, the sensing device, and the humidifying device.

For example, the central control device may establish a prior connection with the sleep determination device, the sensing device, and the humidifying device through short-distance wireless communication. For example, the central control device may transmit a discovery message requesting pre-connection to the sleep determination device, the sensing device, and the humidification device. For example, the central control device may receive a message accepting the pre-connection from the sleep determination device, the sensing device, and the humidification device, thereby establishing a prior connection between the central control device and the sleep determination device, the sensing device, and the humidification device. . For example, a message accepting pre-connection may include identification information between a sleep determination device, a sensing device, and a humidifying device. Identification information may include at least one of an identifier (ID) or an identification number. Here, ID may be an ID for identifying the device. The identification number may consist of at least one of the model name or serial number of the device.

For example, the central control device may transmit basic information about the user, user state information, space state information, and setting information about the humidification device to the server after the pre-connection is established. Basic information about a user may be information transmitted from the user terminal to the central control device. Or, for example, basic information about the user may be input to the server through a user interface installed on the user terminal. The central control device can receive user state information and space state information from the sensing device after a prior connection is established. The central control device may receive setting information about the humidifying device from the humidifying device after the prior connection is established.

For example, the central control device can transmit user status information and space status information to the server in real time after a prior connection is established.

In step S402, the sensing device determines the sleep coordinate value for the area corresponding to the user's eyes based on the elapse of a first preset time from the first time when the user's movement is detected through at least one infrared sensor. You can request it from .

For example, it may be determined that the user's movement has stopped based on the fact that no new movement of the user is detected before a first preset time elapses after the first time the user's movement is detected. For example, if the user's movement is newly detected before a first preset time has elapsed from the first time the user's movement is detected through at least one infrared sensor, the sensing device detects the user's movement newly. The viewpoint may be determined as the first viewpoint, and coordinate values for the area corresponding to the user's eyes may be requested from the sleep determination device after a first preset time has elapsed from the point at which the user's movement is newly detected.

For example, the sensing device may detect the user's movement in the space where the user is located through at least one infrared sensor. At this time, the preset first time may be a value received in the process of establishing a pre-connection with the central control device. For example, the preset first time may be a value input to the central control device from a user terminal or server. The preset first time may be set differently depending on the user's age, gender, weight, and height.

For example, the sensing device may request coordinate values for the area corresponding to the user's eyes from the sleep determination device whenever a first preset time elapses after the user's movement is detected through at least one infrared sensor. there is.

For example, the sensing device may transmit a coordinate value for the location where the user's movement was detected to the sleep determination device based on the elapse of a first preset time after the user's movement was detected through at least one infrared sensor. You can. Here, the coordinate value for the location where the user's movement is detected may be determined as the coordinate value for the user's location.

For example, the sleep determination device may acquire an image of the user through a camera provided in the sleep determination device based on a request for coordinate values for the area corresponding to the user's eyes from the sensing device. For example, the sleep determination device may acquire an image of the user through a camera provided in the sleep determination device based on the coordinate value of the location where the user's movement was detected.

In step S403, the sleep determination device may acquire an image of the user through a camera module and determine the area corresponding to the user's eyes.

According to one embodiment, the sleep determination device may determine the user's shape according to the RGB values of the user's image and determine a shape with a high degree of similarity to the user's shape among a plurality of preset shapes. For example, the area corresponding to the user's eyes may be matched for each of a plurality of preset shapes and pre-stored in the sleep determination device. At this time, the area corresponding to the user's eyes may include a first area that is a skin area around the eyes and a second area that is a pupil area.

According to one embodiment, the sleep determination device may extract a region corresponding to the user's eyes from the user's image using a faster R-CNN (faster regions with convolutional neural network) model. R-CNN (Regions with Convolutional Neural Network) refers to a neural network that performs object detection by using a set region as an input value for CNN. R-CNN can perform two operations: region estimation and region classification. For example, R-CNN uses region proposals to explore the object's region regardless of category by inputting data and labels in the image, and warping/cropping a fixed-sized feature vector from the proposed region. /crop) can be used as input to CNN. Here, a pre-trained network is used in CNN, and classification is performed through SVM (Support Vector Machine), a linear supervised learning model, using feature maps generated through CNN, and bounding box regression through a regressor. box regression) can be performed. At this time, R-CNN has the disadvantage of being very slow because it performs 2,000 CNNs per image. To solve this, Fast R-CNN does not extract (crop) the features of the region from the original image, By extracting from the feature map, it can be passed through one CNN. In other words, Fast R-CNN performs pooling on the region of interest (ROI), converts each region into a vector of a fixed size, and passes each vector through a fully connected (FC) layer to speed up the speed. can be improved. However, there was still a problem of slow speed due to the bottleneck of the algorithm used to propose a region in Fast R-CNN, and Faster R-CNN uses a region proposal network (RPN) instead of the algorithm for region proposal. Proposal speed was improved.

For example, a sleep determination device may generate an input vector consisting of a plurality of regions of interest through data preprocessing of the user's image. The plurality of regions of interest may include regions of interest related to the user's eyes. For example, the region of interest associated with the user's eyes may be the top region when the user's face is classified into top and bottom in the user's image. For example, the sleep determination device can determine the coordinate value for the area corresponding to the user's eyes by inputting the input vector into the Faster R-CNN model. For example, the coordinate value for the area corresponding to the user's eyes may include the coordinate value for the first area, which is the skin area around the eyes, and the coordinate value for the second area, which is the pupil area, for each of both eyes. You can. For example, the Faster R-CNN model may be learned based on a plurality of input vectors, coordinate values for a plurality of first regions with a plurality of correct answers, and coordinate values for a plurality of second regions with a plurality of correct answers.

In step S404, the sleep determination device may transmit coordinate values for the area corresponding to the user's eyes to the sensing device.

In step S405, the sensing device may transmit information about the intensity of infrared rays reflected from the area corresponding to the user's eyes to the sleep determination device through at least one infrared sensor.

For example, information about the intensity of infrared rays may include the intensity of a first area, which is the skin area around the eyes, and the intensity of the second area, which is the pupil area, for each of both eyes.

For example, the sleep determination device may determine whether the user's eyes are closed based on information about the intensity of infrared rays. Additionally, for example, the sleep determination device determines the difference between the temperature of the first area and the temperature of the second area based on the intensity of the first area, which is the skin area around the eyes, and the intensity of the second area, which is the pupil area. can be decided. That is, the sleep determination device can determine whether the eyes are closed by using the fact that the temperature of the eyes is about 0.5 degrees lower than the temperature of the skin around the eyes. For example, if the difference between the temperature of the first area and the temperature of the second area is greater than or equal to a preset difference value, the sleep determination device may determine the sleep determination device to be in an eyes-open state. For example, when the difference between the temperature of the first area and the temperature of the second area is less than a preset difference value, the sleep determination device may determine that the eyes are closed. Here, the preset difference value may be set to a value that reflects the error value in the temperature difference value between the temperature of the eye and the skin around the eye.

In step S406, the sleep determination device may transmit a sleep start message indicating that the user's sleep state has begun to the central control device when a preset second time has elapsed from the second time point at which the user's eyes are determined to be closed.

For example, the preset second time may be a value received in the process of establishing a pre-connection with the central control device. For example, the preset second time may be a value input to the central control device from a user terminal or server. The preset second time may be set differently depending on the user's age, gender, weight, and height.

For example, the sleep start message may include coordinate values for the user's location and a value for the sleep start time.

For example, the sleep determination device may transmit a sleep mode switch message to the sensing device based on the elapse of a second preset time from the second time point at which the user's eyes are determined to be closed. The sleep mode switch message may be a message instructing to switch to a motion sensing mode assuming that the user is in a sleeping state.

For example, the sensing device may sense the user's movement in sleep mode through at least one infrared sensor based on receiving a sleep mode change message. At this time, in the case of sleep mode, the sensing device may set the value of the sensing range for detecting the user's movement to be smaller than that of the normal mode.

In step S407, the central control device may transmit a first request message requesting first operation information about the humidification device to the server based on receiving the sleep start message.

The first request message may include coordinate values for the user's location and a value for when the user's sleep state began.

The first operation information for the humidifying device may be control information instructing the operation of the humidifying device according to the user's sleeping state.

In step S408, the server may determine first prediction information through a basic prediction model using a first neural network based on basic information about the user and space state information.

For example, the server performs data preprocessing on basic information about the user, such that the basic information includes a value for the user's age, a value for the user's gender, a value for the user's weight, and a value for the user's height. Vectors can be created. The value for the user's age is a value representing the user's age and can be expressed in units of years or months. For example, if the user's age is less than 1 year, it may be expressed in months. The value for the user's gender can be set to a value of 1 if the user is a man, and a value of 2 if the user is a woman.

For example, by performing data preprocessing on spatial state information, the server may generate a state vector including values related to illuminance, values related to temperature, values related to humidity, and values related to air quality.

The value related to illuminance may include an illuminance value for each time section for a preset time before the start of the user's sleep state and a predicted illuminance value for each time section after the start of the user's sleep state. For example, if the preset time is 10 minutes, it may include illuminance values at 30-second intervals for the 10 minutes before the user's sleep state begins. The predicted illuminance value for each time section is a preset value for the start of sleep, and may be an average of the illuminance after the start of sleep each year based on past meteorological data. For example, the predicted illuminance value for each time section may be stored in the server for each of a plurality of sleep start times.

Values related to temperature may include a temperature value for each time section for a preset time before the start of the user's sleep state and a predicted temperature value for each time section after the start of the user's sleep state. For example, if the preset time is 10 minutes, temperature values at 30-second intervals may be included for the 10 minutes before the user's sleep state begins. The predicted temperature value for each time section is a preset value for the start of sleep, and may be an average of the temperature after the start of sleep each year based on past meteorological data. For example, the predicted temperature value for each time section may be stored in the server for each of a plurality of sleep start times.

The value related to humidity may include a humidity value for each time section for a preset time before the start of the user's sleep state and a predicted humidity value for each time section after the start of the user's sleep state. For example, if the preset time is 10 minutes, it may include humidity values at 30-second intervals for the 10 minutes before the user's sleep state begins. The predicted humidity value for each time section is a preset value for the start of sleep, and may be an average of the humidity after the start of sleep each year based on past meteorological data. For example, the predicted humidity value for each time section may be stored in the server for each of a plurality of sleep start times.

Values related to air quality include carbon monoxide concentration by time section, carbon dioxide concentration by time section, carbon monoxide concentration by time section, concentration of fine dust below 2.5 micrometers by time section, and time section for a preset time before the start of the user's sleep state. It may include the concentration of fine dust below 10 micrometers for each section and the concentration of volatile organic compounds for each time section. For example, if the preset time is 10 minutes, then for the 10 minutes before the user's sleep state begins, carbon monoxide concentration in 30-second intervals, carbon dioxide concentration in 30-second intervals, carbon monoxide concentration in 30-second intervals, 2.5 in 30-second intervals It may include the concentration of fine dust below a micrometer, the concentration of fine dust below 10 micrometers at 30-second intervals, and the concentration of volatile organic compounds at intervals of 30 seconds.

For example, the server may generate a start vector containing a value for when the user's sleep state began. The value for the start of the sleep state may be determined as a value representing the year, month, day, and time. For example, if the start time of the sleep state is 2:30 PM on September 9, 2023, the value of the start time of the sleep state can be expressed as 202309091430.

For example, the server may determine first prediction information including the user's detailed sleep stage by time by inputting the base vector, state vector, and start vector into the base prediction model. By inputting the basic vector, state vector, and start vector into the basic prediction model, the user's detailed sleep stage by time can be output. The detailed sleep stage by time may be a value representing the detailed sleep stage according to sleep time based on the sleep start time. For example, sleep time may be determined differently depending on a combination of the user's age and sleep start time. Multiple combinations of the user's age and sleep start time and sleep times according to the multiple combinations may be pre-stored in the server. A plurality of combinations of the user's age and sleep start time and sleep time according to the plurality of combinations may be an average value of statistically surveyed values for a plurality of users. For example, if the sleep time is 1 hour, the hourly detailed sleep stage may include a value representing the detailed sleep stage at 5-minute intervals for 1 hour after the start of sleep. The value representing the detailed sleep stage may be a value representing one detailed sleep stage among a plurality of detailed sleep stages. For example, the plurality of detailed sleep stages may include stages 1 to 5. The first stage can be expressed as a value of 1, the second stage as a value of 2, the third stage as a value of 3, the fourth stage as a value of 4, and the fifth stage as a value of 5. For example, if the sleep time is 1 hour and the time interval is set to 10 minutes, the first prediction information including detailed sleep stages by hour is [1, 2, 3, 4, 5, 1], [60, 10] may include. Here, the front parentheses may indicate detailed sleep stages at 10-minute intervals, and the rear parentheses may indicate sleep times and time intervals expressed in minutes.

For example, a basic prediction model may be learned based on a plurality of base vectors, a plurality of state vectors, a plurality of start vectors, and a plurality of detailed sleep stages for each correct answer time. Here, the detailed sleep stages by multiple correct answers may be the detailed sleep stages by hour for a plurality of users measured based on age, gender, body type, sleep start time, and sleep environment (e.g., illuminance, temperature, humidity, air quality). there is.

In step S409, the server provides first operation information about the humidifying device through a humidifying device adjustment model using a second neural network based on basic information about the user, first prediction information, space state information, and setting information about the humidifying device. can be decided.

For example, the server may generate a sleep vector including values for detailed sleep stages by time through data preprocessing on the first prediction information. For example, if the sleep time is 2 hours and the time interval is set to 20 minutes, the sleep vector may include [1, 3, 4, 5, 5, 5], [120, 20]. Here, the front parentheses may indicate detailed sleep stages at 20-minute intervals, and the rear parentheses may indicate sleep times and time intervals expressed in minutes.

For example, the server preprocesses data about the setting information for the humidifying device to generate values for the humidification method, values for the water tank capacity, values for the remaining water capacity, values for the positions of the user and the humidifying device, and values related to the amount of humidification. You can create a settings vector containing . For example, the server can determine the value for the humidification method, the value for the water tank capacity, the value for the remaining water capacity, and the value related to the humidification amount based on the model name and manufacturing date of the humidification device included in the setting information for the humidification device. there is. For example, a combination of the model name and manufacturing year of a plurality of humidifying devices may be pre-stored in the server, and a value for the humidifying method matching each combination of the model name and manufacturing year of a plurality of humidifying devices, and the water tank capacity. Values for, values for remaining water capacity, and values related to humidification amount may also be pre-stored in the server.

The value for the humidification method may be a value representing one humidification method among a plurality of humidification methods. For example, the plurality of humidification methods may include an upper ultrasonic method, a lower ultrasonic method, a floating ultrasonic method, a heater heating method, a barrel heating method, a composite method, a vaporization method using a fiber filter, and a vaporization method using a plastic filter. . For example, values representing each of a plurality of humidification methods may be pre-stored in the server. Additionally, when a humidifying device with a new humidification method is released, the value representing each of the plurality of humidification methods may be updated. The ultrasonic method may be a method of converting water into water vapor using ultrasonic vibration. At this time, the top ultrasonic method may be a method in which a vibrator is located at the top of a water tank that stores water. The bottom ultrasonic method may be a method in which a vibrator is located at the bottom of a water tank that stores water. The floating ultrasonic method may be a method in which a vibrator is located on the surface of a water tank that stores water. The heating method may be a method of heating water and converting it into steam. The heater heating method may be a method of heating water using a heater below the water tank. The tub heating method may be a method of heating the water tank itself, such as a rice cooker. The combined method may be a method of converting water into steam by combining an ultrasonic method and a heating method. The evaporation method may be a method of evaporating water into steam using wind generated by a fan. The vaporization method using a fiber filter may be a vaporization method that uses a filter made of fiber and requires replacement of the filter. The vaporization method using a plastic filter may be a vaporization method that does not require replacement of the filter because it uses a filter made of plastic.

The value for the water tank capacity may be a value representing the total capacity of the water tank included in the humidifying device. The value for the remaining water capacity is a value representing the remaining capacity of the water tank included in the humidifying device and may include the value for the remaining water capacity at the start of sleep. The values for the positions of the user and the humidifying device may include a coordinate value for the user's position at the start of sleep and a coordinate value for the position of the humidifying device. Values related to the humidification amount may include a value for the maximum humidification amount, a value for the minimum humidification amount, and a control value for adjusting the humidification amount according to the humidification method. For example, in the case of the ultrasonic method, the control value for adjusting the humidification amount may include the frequency of the vibrator per hour for each humidification amount. For example, in the case of a heating method, the control value for adjusting the humidification amount may include at least one of a voltage value or a current value for heating for each humidification amount. For example, in the case of a complex method, the control value for adjusting the humidification amount may include the frequency of vibration per hour of the vibrator for each humidification amount and a voltage value or current value for heating for each humidification amount. For example, in the case of the vaporization method, the control value for adjusting the humidification amount may include the wind speed per hour for each humidification amount.

For example, the server may determine a value associated with the hourly first operation of the humidification device by inputting the base vector, sleep vector, state vector, and settings vector into the humidification device adjustment model. A value related to the hourly first operation of the humidifying device may be output based on the basic vector, sleep vector, state vector, and settings vector being input to the humidifying device adjustment model.

For example, the value related to the first hourly operation of the humidifying device may include a control value for adjusting the amount of humidification at a specific time interval during sleep time. For example, the value related to the first operation per time of the humidifying device may include a control value for adjusting the humidification amount for each specific time interval, a value for the operation time expressed in minutes, and a value for the time interval. If the operation time is 1 hour and the time interval is set to 10 minutes, the value related to the first operation per hour of the humidifying device is the control value for adjusting the humidification amount at 10-minute intervals and the value and time for 1 hour expressed in minutes Can contain values for intervals. Here, the operation time may correspond to the sleep time included in the sleep vector, and the time interval may correspond to the time interval included in the sleep vector.

For example, the value related to the first operation of the humidifying device over time may be different depending on the humidifying method of the humidifying device.

For example, a humidification device adjustment model may be learned based on values associated with a plurality of basis vectors, a plurality of sleep vectors, a plurality of state vectors, a plurality of settings vectors, and a plurality of correct hourly operations.

For example, the first operation information for the humidifying device may include a value related to the first operation of the humidifying device by time.

In step S410, the server may transmit first operation information about the humidifying device to the central control device.

In step S411, the central control device may transmit first operation information about the humidifying device to the humidifying device.

The humidifying device may operate based on a value related to the first operation of the humidifying device by time included in the first operation information about the humidifying device.

In step S412, the sleep determination device may transmit a sleep end message indicating the end of the user's sleep state to the central control device when a preset third time has elapsed from the third time determined to be in a state in which the user's eyes are open. .

For example, the sleep end message may include coordinate values for the user's location and a value for the time when sleep ended.

At this time, the specific process by which the sleep determination device determines whether the user's eyes are open is as follows. The sensing device may detect the user's movement at a first preset time interval. When the user's movement exceeding a preset movement-related value is detected through at least one infrared sensor, the sensing device may request the sleep determination device for coordinate values for the area corresponding to the user's eyes from the sleep determination device. . The value related to the preset movement may be a value for the distance the object has moved, measured according to the intensity of reflected infrared rays. At this time, for example, the sensing device sends a coordinate value for the location where the user's movement was detected to the sleep determination device based on the user's movement exceeding a preset movement-related value being detected through at least one infrared sensor. Can be transmitted.

For example, when coordinate values for an area corresponding to the user's eyes are requested from a sensing device switched to sleep mode, the sleep determination device may acquire an image of the user through a camera provided in the sleep determination device. For example, the sleep determination device may acquire an image of the user through a camera provided in the sleep determination device based on the coordinate value of the location where the user's movement was detected. The sleep determination device may obtain an image of the user through a camera module and determine the area corresponding to the user's eyes.

For example, the sleep determination device may transmit coordinate values for the area corresponding to the user's eyes to the sensing device. The sensing device may transmit information about the intensity of infrared rays reflected from the area corresponding to the user's eyes to the sleep determination device through at least one infrared sensor. For example, if the difference between the temperature of the first area and the temperature of the second area is greater than or equal to a preset difference value, the sleep determination device may determine that the user is in a state with his eyes open.

For example, the preset third time may be a value received in the process of establishing a pre-connection with the central control device. For example, the preset third time may be a value input to the central control device from a user terminal or server. The preset third time may be set differently depending on the user's age, gender, weight, and height.

In step S413, the central control device may transmit a second request message requesting second operation information about the humidification device to the server based on receiving the sleep end message.

For example, the second operation information for the humidifying device may be control information instructing the operation of the humidifying device according to the user's waking state.

In step S414, the server may determine second operation information about the humidifying device through a humidifying device adjustment model using a second neural network based on basic information about the user, space state information, and setting information about the humidifying device.

For example, the server may determine values associated with the hourly second operation of the humidification device by inputting the basis vector, state vector, and configuration vector into the humidification device adjustment model. At this time, the sleep vector has a value of 0, and if the sleep vector includes a value of 0, the humidification device adjustment model can output a value related to the hourly second operation of the humidification device based on the base vector, state vector, and settings vector. there is.

For example, the value related to the second time-dependent operation of the humidifying device may include a control value for adjusting the amount of humidification at a specific time interval during the time after the user wakes up. For example, the value related to the second operation by time of the humidifying device may include a control value for adjusting the amount of humidification at a specific time interval and a value for the time interval. When the time interval is set to 10 minutes, the value related to the second operation by time of the humidifying device may include a control value for adjusting the amount of humidification at 10-minute intervals and a value for the time interval. At this time, the time interval may be different from the value related to the second operation by time of the humidifying device.

For example, the value related to the second operation per time of the humidifying device may be different depending on the humidification method of the humidifying device.

In step S415, the server may transmit second operation information about the humidifying device to the central control device.

In step S416, the central control device may transmit second operation information about the humidifying device to the humidifying device.

According to one embodiment, the server may generate a sleep vector input to the humidification device adjustment model based on the second prediction information based on the fact that the number of times a request message has been received from the central control device is more than a preset number. The second prediction information may be information that adjusts the user's detailed sleep stage by time through a fine prediction model using a third neural network.

For example, when a sleep vector generated through data preprocessing of the first prediction information is input to a fine prediction model, the user's detailed sleep stage adjusted by time may be output.

For example, the fine prediction model may be learned based on a plurality of user state information, a plurality of spatial state information, and a plurality of first prediction information acquired during the user's sleeping state.

Additionally, for example, the preset number of times may be determined by Equation 1 below.

In Equation 1, N _th is the preset number of times, n _v is the number of detailed sleep stages for each hour, n _c is the number of clusters set in the fine prediction model, and b _s is the fine is a value for the batch size set in the prediction model, e _p is a value for the learning cycle set in the fine prediction model, and S _ij is the similarity between the ith hourly detailed sleep stage and the jth hourly detailed sleep stage, The N _d may be a basic value for the preset number of times, and the N _max may be a maximum value for the preset number of times.

For example, a plurality of detailed sleep stages by time may be collected for a plurality of users corresponding to the user's age group. For example, the number of clusters may be the number of clusters in which a plurality of time-specific detailed sleep stages obtained for the user are classified according to similarity. For example, the similarity between the ith hourly detailed sleep stage and the jth hourly detailed sleep stage may be calculated as cosine similarity. At this time, the similarity may be a value greater than 0 and less than or equal to 1. The default value for the preset number of times and the maximum value for the preset number of times may be pre-stored on the server.

For example, the default value for the preset number of times and the maximum value for the preset number of times may be set differently for the learning performance of a fine prediction model. Batch size is the size of data calculated at one time in the fine prediction model. The learning cycle is the number of times training data passes through a fine prediction model. For example, the min function may be a function that determines the smaller of the two values in parentheses.

For example, as the similarity is small, the number of clusters is large, the batch size is large, and the learning cycle is large, the preset number of times may be determined to be a large value. Or, for example, as the similarity is large, the number of clusters is small, the batch size is small, and the learning cycle is small, the preset number of times may be determined to be a small value.

Through this, the server can adjust the amount of learning data required to use the fine prediction model based on the performance of the fine prediction model and the similarity to the user's detailed sleep stages by time. Accordingly, the server can set the preset number of times for each user flexibly, rather than setting it to a fixed value.

Figure 5 is a diagram showing an example of a basic prediction model according to an embodiment. The embodiment of FIG. 5 can be combined with various embodiments of the present disclosure.

Referring to FIG. 5, the first neural network used in the basic prediction model may be a gated recurrent unit (GRU)-based neural network 500. Here, GRU may be a modified model of RNN (recurrent neural network).

For example, the first neural network 500 used in the basic prediction model may include a first input layer 510, one or more first hidden layers 520, and a first output layer 530. Learning data consisting of a plurality of base vectors, a plurality of state vectors, a plurality of start vectors, and a plurality of detailed sleep stages for each correct answer time are input to the first input layer 510 to produce one or more first hidden layers 520 and a first output. It passes through the layer 530 and is output as a first output vector, and the first output vector is input to the first loss function layer connected to the first output layer 530, and the first loss function layer is each connected to the first output vector. The first loss value is output using the first loss function that compares the correct answer vector to the learning data, and the parameters of the first neural network used in the basic prediction model can be learned in the direction of decreasing the first loss value. there is.

One base vector, one state vector, and one start vector used as learning data may be composed of one correct answer time-specific detailed sleep stage and one set. For example, multiple sets may be pre-stored on a server.

The detailed sleep stage by correct answer time may be a value representing the detailed sleep stage measured according to the actual sleep time based on the start of sleep.

One or more first hidden layers 510 may include one or more GRU blocks, and one GRU block may include a reset gate and an update gate. Here, the reset gate and update gate may include a sigmoid layer. For example, a sigmoid layer uses a sigmoid function ( ) may be a layer that is an activation function. For example, a hidden state may be controlled through a reset gate and an update gate, and weights may exist for each gate and input.

The reset gate resets past information, and the weight r(t) derived through the previous hidden layer can be determined by Equation 2.

For example, a plurality of base vectors, a plurality of state vectors, and a plurality of start vectors are input to the input layer, and the reset gate is a current generated based on the plurality of base vectors, a plurality of state vectors, and a plurality of start vectors. When the input value (x _t ) of the time point is input, it is inner producted with the weight W _r of the current time point, and the hidden state (h _{(t -1)} ) is dot producted with the weight U _r of the previous point in time, and finally, the two values are added together and input into the sigmoid function, and the result can be output as a value between 0 and 1. Through these values between 0 and 1, it can be determined how much of the hidden state value from the previous point will be utilized.

The update gate determines the update rate for past and present information, and z(t) is the amount of information at the current time and can be determined by Equation 3.

For example, when the input value (x _t ) at the current time is input, it is dot producted with the weight W _z at the current time point, and the hidden state (h _(t-1) ) at the previous time point is the dot product with the weight U _z at the previous time point. And finally, the two values are added together and input into the sigmoid function, so the result can be output as a value between 0 and 1. And 1-z(t) can be multiplied by the information (h _(t-1) ) of the hidden layer at the previous time.

Through this, z(t) can reflect how much current information will be used and 1-z(t) can reflect how much past information will be used.

The information candidate group at the current time t can be determined by multiplying the result of the reset gate. For example, when the input value (x _t ) at the current time is input, the first value is the inner product of the weight W _h at the current time, and the weight U _h at the previous time is added to the hidden state (h _(t-1) ) at the previous time. The sum of the inner product value and the second value multiplied by r(t) may be input to the tanh function. For example, tanh stands for nonlinear activation function (hyperbolic tangent function).

By combining the results of the update gate and the candidate group, the weight of the hidden layer at the current time can be determined. For example, the output value z(t) of the update gate multiplied by the current hidden state (h(t)), the value discarded from the update gate 1-z(t) and the hidden state at the previous time (h( The weight of the hidden layer at the current time can be determined by the sum of the values multiplied by t-1)).

Through this, the server can train a basic prediction model to predict detailed sleep stages by time, considering the sleep start time as well as the ambient illumination, temperature, humidity, and air quality in the user's sleeping state based on the user's basic information.

For example, the third neural network used in the fine prediction model may also be a GRU-based neural network.

For example, the server may generate a plurality of user vectors for each sleep time through data preprocessing on a plurality of user state information acquired during the user's sleep state.

The user vector for each sleep time may include a value related to the user's body temperature, a value related to the user's respiratory rate, and a value related to the user's movement. The value for the user's body temperature may include a value for the user's body temperature at a specific time interval relative to sleep time. For example, values related to the user's respiratory rate may include the user's respiratory rate at specific time intervals relative to sleep time and the frequency of occurrence of hyperventilation, sleep apnea, and special breathing. Additionally, for example, the user's breathing rate may be determined through data preprocessing of information about the user's breathing sound. For example, the server can analyze the breathing pattern by converting information about the user's breathing sound into two axes: frequency and time. The server can analyze when characteristics corresponding to preset sleep apnea or hyperventilation events occur based on information converted into two axes: frequency and time. The value related to the user's movement may include a value for the user's movement at a specific time interval relative to sleep time. For example, the value for movement may be the number of times the user tosses and turns. The server may receive user state information acquired during the user's sleep state along with a sleep end message from the central control device and store it for each user.

For example, a third neural network used in a fine prediction model may include a third input layer, one or more third hidden layers, and a third output layer. Learning data consisting of a plurality of base vectors, a plurality of user vectors for each sleep time, a plurality of state vectors, a plurality of detailed sleep stages for each time, and a plurality of detailed sleep stages adjusted for each correct answer time are input to the third input layer and are input to one or more third hidden layers. layer and the third output layer are output as a third output vector, and the third output vector is input to the third loss function layer connected to the third output layer, and the third loss function layer is connected to the third output vector and each The third loss value is output using the third loss function that compares the correct answer vector to the learning data, and the parameters of the third neural network used in the fine prediction model can be learned in the direction of decreasing the third loss value. .

One base vector used as training data, one sleep-hourly user vector corresponding to that basis vector, one sleep-hourly state vector, and one hourly detailed sleep stage correspond to one correct hourly adjusted detailed sleep stage and one set. It can be composed of: For example, multiple sets may be pre-stored on a server.

The detailed sleep stages adjusted by multiple correct answers times may be modified from the hourly detailed sleep stages predicted through the basic prediction model to the actually measured hourly detailed sleep stages.

Through this, when data related to the user's sleep state is accumulated as needed to predict the user's sleep state, the server fine-tunes the user's detailed sleep stage by hour to suit the user based on the accumulated data related to the user's sleep state. A fine prediction model can be trained to do this.

For example, the second neural network used in the humidification device adjustment model may be a CNN-based neural network.

For example, a second neural network used in a humidifier tuning model may include a second input layer, one or more second hidden layers, and a second output layer. Learning data consisting of values related to a plurality of base vectors, a plurality of sleep vectors, a plurality of state vectors, a plurality of setting vectors, and a plurality of correct answer time-specific actions are input to a second input layer and are input to one or more second hidden layers and a second output. It passes through the layer and is output as a second output vector, and the second output vector is input to the second loss function layer connected to the second output layer, and the second loss function layer is the correct answer for the second output vector and each learning data. A second loss value can be output using a second loss function that compares vectors, and the parameters of the second neural network used in the humidification device adjustment model can be learned in the direction of decreasing the second loss value.

One or more second hidden layers may include one or more convolutional layers and one or more pooling layers.

For example, in a convolutional layer, a plurality of base vectors, a plurality of sleep vectors, a plurality of state vectors, and a plurality of setting vectors may be filtered, and a feature map may be formed through the convolutional layer.

For example, in a pooling layer, a fixed vector related to a feature is selected for dimensionality reduction based on the formed feature map, and sub-sampling is performed on the formed feature map, so that the value related to the operation over time is obtained from the vectorized time series data. Features can be extracted. For example, the pooling layer may be a max pooling layer that extracts the largest value. For example, the pooling layer may be an average pooling layer that extracts average values. For example, at this time, the parameters of the second neural network may include parameters (size of feature map, size of filter, depth, stride, zero padding) related to the convolutional layer and the pooling layer.

One basic vector, one sleep vector, one state vector, and one setup vector used as learning data may be composed of a first value and a first set related to one correct time-specific operation. For example, a plurality of first sets may be pre-stored on a server.

One basic vector, one state vector, and one setting vector used as learning data may be composed of a second value and a second set related to one correct time-specific operation. For example, a plurality of second sets may be pre-stored on the server.

That is, the humidification device adjustment model may be learned so that the output values related to the operation by time are different depending on whether the sleep vector is input.

The first value related to the correct answer time operation may include a control value for adjusting to the first target humidity. The first target humidity can be determined by applying weighting according to the illuminance, humidity, temperature, and air quality of the current space to the humidity required for each detailed sleep stage of the user according to the user's age, gender, height, and weight. The humidity required for the user's detailed sleep stage according to multiple combinations of the user's age, gender, height, and weight may be pre-stored in the server. For example, the humidity required for each user's detailed sleep stage may be set based on the humidity received positive feedback from a plurality of users. The control value is a control value for adjusting the humidification amount and may have different units depending on the humidification method described above.

Additionally, for example, the first value related to the correct answer time operation may be determined by Equation 4 below.

In Equation 4, C is the first value related to the operation at that point in time, and is a value for converting the value obtained by subtracting the humidity at that point from the first target humidity at that point of time into a control value, Temp _o is a value for the temperature at that point in time, and Temp _ref is the value set at the first target humidity. is a value for the standard temperature, L _o is a value for the illuminance at the relevant point in time, L _ref is a value for the standard illuminance set for the first target humidity, and n _a is the type of object for measuring air quality is the number of, where A _k is the concentration of the kth object, Ar _k is the reference concentration set in the first target humidity of the kth object, and the H _stage is the user's age, gender, height, and weight at that point in time. It is the humidity required for the user's current detailed sleep stage according to a combination of , and H _o may be a value for the humidity at that time.

For example, may have different values depending on the model name of the humidifying device. For example, α may be pre-stored in the server for each model name of the humidifying device.

Through this, the server can learn a humidifying device adjustment model to determine information on how to operate the humidifying device appropriately for the user's detailed sleep stage by considering the space status and humidifying device setting information in addition to the user's basic information.

The second value related to the correct answer time operation may include a control value for adjusting to the second target humidity. The second target humidity may be determined by applying weights according to the illuminance, humidity, temperature, and air quality of the current space to the humidity according to the user's age, gender, height, and weight. Humidity according to multiple combinations of the user's age, gender, height, and weight may be pre-stored on the server. For example, the humidity according to a plurality of combinations of the user's age, gender, height, and weight may be set based on the humidity received positive feedback from a plurality of users. The control value is a control value for adjusting the humidification amount and may have different units depending on the humidification method described above.

According to one embodiment, based on the fact that the number of times the server has received a request message from the central control device is more than a preset number, the server uses a fine tuning model using the fourth neural network instead of the humidifying device tuning model to adjust the humidification device. Operation information can be determined.

For example, the server may generate a plurality of state vectors for each sleep time through data preprocessing on a plurality of spatial state information acquired during the user's sleep state. The state vector for each sleep time may be a value related to the sleep time, an illuminance value at a specific time interval, a temperature value at a specific time interval, a humidity value at a specific time interval, and air quality at a specific time interval. The server may receive space state information acquired during the user's sleep state along with a sleep end message from the central control device and store it for each user.

For example, when the server receives a sleep start message, the server can determine first operation information for the humidification device by inputting the base vector, sleep vector, state vector, and setup vector into the fine-tuning model.

For example, when the server receives the termination message, the server can determine second operation information for the humidification device by inputting the basis vector, state vector, and setup vector into the fine-tuning model.

For example, the fourth neural network used in the fine-tuning model may be a CNN-based neural network. For example, it can be learned in the same way as the second neural network described above.

For example, a fourth neural network used in a fine-tuning model may include a fourth input layer, one or more fourth hidden layers, and a fourth output layer. Learning data consisting of a plurality of base vectors, a plurality of sleep vectors, a plurality of state vectors for each sleep time, a plurality of setting vectors, and a plurality of values related to the behavior for each correct answer time are input to the fourth input layer and one or more fourth hidden layers and the first It passes through the 4th output layer and is output as a 4th output vector, and the 4th output vector is input to the 4th loss function layer connected to the 4th output layer, and the 4th loss function layer is connected to the 4th output vector and each training data. The fourth loss value is output using the fourth loss function that compares the correct answer vector, and the parameters of the fourth neural network used in the fine-tuning model can be learned in the direction of decreasing the fourth loss value.

One basic vector, one sleep vector, one state vector for each sleep time, and one setting vector used as learning data may be composed of a first value and a first set related to one correct time-specific operation. For example, a plurality of first sets may be pre-stored on a server. At this time, the sleep vector used as learning data may be a sleep vector generated by measuring the actual user's sleep state.

One basic vector, one state vector, and one setting vector used as learning data may be composed of a second value and a second set related to one correct time-specific operation. For example, a plurality of second sets may be pre-stored on the server.

In other words, the fine-tuned model may be trained so that the output values related to hourly motion are different depending on whether the sleep vector is input.

The first value related to the correct answer time operation may include a control value for adjusting the humidification amount at a specific time interval in the user's sleeping state. For example, the amount of humidification at a specific time interval may be a value performed by the humidifying device during the user's actual sleep time, and may be a value for which positive feedback is received from the user terminal.

Through this, the server can learn a fine-tuning model to operate the humidification device to create an environment optimized for the user's sleep state using the state vector for each actual sleep time accumulated for the user.

The second value related to the correct answer time operation may include a control value for adjusting the humidification amount at a specific time interval when the user is awake. For example, the amount of humidification at a specific time interval may be a value performed by the humidifying device when the user is actually awake, and may be a value for which positive feedback is received from the user terminal.

Through this, the server can use the accumulated actual state vector for the user to learn a fine-tuning model so that the humidification device operates to create an optimized environment even when the user is awake.

Figure 6 is a block diagram showing the configuration of a server according to one embodiment. One embodiment of FIG. 6 may be combined with various embodiments of the present disclosure.

As shown in FIG. 6, the server 600 may include a processor 610, a communication unit 620, and a memory 630. However, not all of the components shown in FIG. 6 are essential components of the server 600. The server 600 may be implemented with more components than those shown in FIG. 6, or the server 600 may be implemented with fewer components than those shown in FIG. 6. For example, the server 600 according to some embodiments may further include a user input interface (not shown), an output unit (not shown), etc. in addition to the processor 610, the communication unit 620, and the memory 630. .

The processor 610 typically controls the overall operation of the server 600. The processor 610 may include one or more processors and control other components included in the server 600. For example, the processor 610 can generally control the communication unit 620 and the memory 630 by executing programs stored in the memory 630. Additionally, the processor 610 may perform the functions of the server 600 shown in FIGS. 3 to 5 by executing programs stored in the memory 630.

The communication unit 620 may include one or more components that allow the server 600 to communicate with other devices (not shown) and servers (not shown). The other device (not shown) may be a computing device such as the server 600 or a sensing device, but is not limited thereto. The communication unit 620 may receive a user input from another electronic device or receive data stored in an external device from an external device through a network.

For example, the communication unit 620 may transmit and receive a message to establish a connection with at least one device. The communication unit 620 may transmit information generated by the processor 610 to at least one device connected to the server. The communication unit 620 may receive information from at least one device connected to the server. The communication unit 620 may transmit information related to the received information in response to information received from at least one device.

The memory 630 may store programs for processing and control of the processor 610. For example, the memory 630 may store information input to a server or information received from another device through a network. Additionally, the memory 630 may store data generated by the processor 610. The memory 630 may store information input to or output from the server 600.

The memory 630 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory, etc.), and RAM. (RAM, Random Access Memory) SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk , and may include at least one type of storage medium among optical disks.

The embodiments described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, and a field programmable gate (FPGA). It may be implemented using one or more general-purpose or special-purpose computers, such as an array, programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed over networked computer systems and thus stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described with limited drawings as described above, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the claims described below.

Claims (5)

In a system for controlling a humidification device using a neural network,
A sensing device that senses user state information about a user and space state information about the space in which the user is located;
The user status information includes information about the user's body temperature, information about the user's movement, and information about the user's breathing sound,
The space state information includes information about illuminance, information about humidity, information about temperature, and information about air quality,
a sleep determination device that is linked with the sensing device to determine whether the user is in a sleeping state;
A humidifying device provided in the space where the user is located and spraying water vapor;
A central control device that controls the sensing device, the sleep determination device, and the humidification device by transmitting and receiving information with the sensing device, the sleep determination device, and the humidification device through short-distance wireless communication;
Predicting the detailed sleep stage of the user based on information received through long-distance wireless communication from the central control device, and transmitting operation information about the humidification device determined according to the detailed sleep stage of the user to the central control device Including servers,
The server is,
Receiving basic information about the user, the user state information, the space state information, and setting information about the humidification device from the central control device,
Determining first prediction information through a basic prediction model using a first neural network based on the basic information about the user and the space state information,
The first prediction information is information that predicts the detailed sleep stage of the user by time using basic information about the user,
Determine operation information for the humidifying device through a humidifying device adjustment model using a second neural network based on the basic information about the user, the first prediction information, the space state information, and setting information for the humidifying device, and ,
Transmitting operation information about the humidification device to the central control device,
The operation of the humidifying device is adjusted based on operation information about the humidifying device transmitted from the central control device,
system.
According to clause 1,
The sleep determination device,
Based on the elapse of a first preset time from the first time when the user's movement is detected through at least one infrared sensor included in the sensing device, the user's sleep is detected through a camera module provided in the sleep determination device. acquire the image,
Determine a region corresponding to the user's eyes in the user's image,
Transmitting coordinate values for the area corresponding to the user's eyes to the sensing device,
The sensing device is,
Transmitting information about the intensity of infrared rays reflected from the area corresponding to the user's eyes to the sleep determination device through the at least one infrared sensor,
The sleep determination device,
Determine whether the user's eyes are closed based on information about the intensity of the infrared rays,
Transmitting a sleep start message indicating that the user's sleep state has begun to the central control device when a preset second time has elapsed from the second time point at which the user's eyes are determined to be closed,
system.
According to clause 2,
The central control device is,
Based on receiving the sleep start message, transmitting a request message requesting operation information about the humidification device to the server,
The request message includes coordinate values for the user's location and a value for when the user's sleep state began,
The server is,
By performing data preprocessing on the basic information about the user, a basic vector is generated including a value for the user's age, a value for the user's gender, a value for the user's weight, and a value for the user's height,
By performing data preprocessing on the space state information, a state vector including a value related to illuminance, a value related to temperature, a value related to humidity, and a value related to air quality is generated,
Generating a start vector containing a value for when the user's sleep state began,
The first prediction information including the user's detailed sleep stage by time is output based on the basic vector, the state vector, and the start vector being input to the basic prediction model,
The basic prediction model is learned based on a plurality of base vectors, a plurality of state vectors, a plurality of start vectors, and a plurality of detailed sleep stages for each correct answer time.
system.
According to clause 3,
The server is,
Generating a sleep vector containing values for detailed sleep stages by time through data preprocessing of the first prediction information,
Through data preprocessing of the setting information for the humidifying device, a setting vector including a value for the humidification method, a value for the water tank capacity, a value for the remaining water capacity, a value for the location of the user and the humidifying device, and a value related to the humidification amount Create a ,
A value related to the hourly operation of the humidifying device is output based on the basic vector, the sleep vector, the state vector, and the setting vector being input to the humidifying device adjustment model,
Values related to the operation of the humidifying device are different depending on the humidifying method of the humidifying device,
The humidification device adjustment model is learned based on values associated with a plurality of base vectors, a plurality of sleep vectors, a plurality of state vectors, a plurality of setting vectors, and a plurality of correct time-based operations,
The operation information for the humidifying device includes values related to hourly operation of the humidifying device,
system.
According to clause 3,
The server is,
Based on the fact that the number of times the request message has been received from the central control device is more than a preset number, generate a sleep vector input to the humidification device adjustment model based on second prediction information,
The second prediction information is information that adjusts the user's detailed sleep stage by time through a fine prediction model using a third neural network,
The sleep vector generated through data preprocessing for the first prediction information is input to the fine prediction model, and the user's detailed sleep stage adjusted by time is output,
The fine prediction model is learned based on a plurality of user state information, a plurality of spatial state information, and a plurality of first prediction information acquired during the user's sleeping state.
system.

Images