JP2021023689A - Determination device, light irradiation apparatus and program - Google Patents

Abstract

To provide a determination device which can improve the convenience.SOLUTION: The determination device determines the sleep state of the brain of a subject on the basis of a machine learning model including a neural network. The model has learned a correspondence between a first input information including a first brain waveform information and a first time information and a first sleep state information indicating a first sleep state, estimates the sleep state associated with a second input information when the second input information including a second brain waveform information and a second time information is input, and outputs the information indicating the estimated result as an output information. The determination device inputs the second input information to the model and determines the sleep state in accordance with the output information output from the model to which the second input information is input.SELECTED DRAWING: Figure 3

Description

The present invention relates to a determination device, a light irradiation device, and a program.

Research and development are being conducted on techniques for determining the sleep state of a subject's brain.

Here, the state of the subject's brain is a state characterized by the waveform of the subject's electroencephalogram. The state of the subject's brain can be classified into a plurality of states according to the waveform of the subject's electroencephalogram under predetermined conditions. For example, a subject's brain state can be classified into a plurality of states according to the waveform of the subject's brain wave during sleep. The sleep state of the subject's brain described above is any one of the plurality of states when the state of the subject's brain is classified into a plurality of states according to the waveform of the subject's electroencephalogram during sleep. Hereinafter, for convenience of explanation, each of the plurality of states will be referred to as a sleep classification state. That is, determining the sleep state of the subject's brain means determining which of the plurality of sleep classification states the subject's brain state is. The electroencephalogram of the subject is a wave indicating a temporal change in the electric potential detected from the brain of the subject.

Here, Patent Document 1 describes a method of using a plurality of channel electroencephalogram sensors and a method of using one or more myoelectric sensors together with one or more channel electroencephalogram sensors as a method of determining the sleep state of the subject's brain. It is described (see Patent Document 1).

U.S. Patent Application Publication No. 2019/0126033

The two methods described in Patent Document 1 (that is, a method using a plurality of channel electroencephalogram sensors and a method using one or more myoelectric sensors together with one or more channel electroencephalogram sensors) have a plurality of channels. The sleep state of the subject's brain can be determined with high accuracy by reducing noise by using a myoelectric sensor together with an electroencephalogram sensor (that is, using a plurality of sensors). However, the two methods are inconvenient because the number of channels is large and the myoelectric sensor is used together with the electroencephalogram sensor, which makes it complicated to attach the sensor to the subject.

Therefore, the present invention has been made in view of the above-mentioned problems of the prior art, and provides a determination device, a light irradiation device, and a program capable of improving convenience.

(1) The determination device according to one aspect of the present invention is a determination device that determines the sleep state of a subject's brain based on a machine learning model including a neural network, and the model is the first of the subject. The first input information including the first brain waveform information indicating the waveform of one brain wave and the first time information indicating the first time associated with the first brain waveform information is associated with the first input information. The correspondence relationship with the first sleep state information indicating the first sleep state, which is the sleep state, has been learned, and the second brain waveform information showing the waveform of the second brain wave of the subject and the second brain waveform information. When the second input information including the second time information indicating the second time associated with is input, the sleep state associated with the input second input information is estimated, and the estimated result is obtained. The indicated information is output as output information, the determination device includes a control unit, the control unit inputs the second input information to the model, and the second input information is output from the input model. It is a determination device that determines the sleep state according to the output information.

(2) In the determination device according to (1) above, the sleep state is one of a plurality of states classified by the subject's brain waves during sleep among the states of the subject's brain. The control unit has the sleep state in any of the plurality of states according to the output information output from the model to which the second input information is input according to the output information. It may have a configuration for determining whether or not.

(3) In the determination device according to (2) above, the plurality of states may have a configuration including a REM sleep state, a non-REM sleep state, and an awake state.

(4) In the determination device according to (2) or (3) above, it is possible that the model is in the sleeping state associated with the second input information when the second input information is input. The likelihood indicating the likelihood is estimated for each of the plurality of states, and information including each of the estimated likelihoods for each of the plurality of states is output as the output information, and the control unit outputs the output information. Based on this, the state having the highest likelihood estimated by the model among the plurality of states may be determined as the sleeping state.

(5) In the determination device according to any one of (1) to (4) above, the first input information includes first frequency domain information indicating the waveform of the first electroencephalogram in the frequency domain. The second input information may have a configuration in which the second frequency domain information indicating the waveform of the second electroencephalogram in the frequency domain is included.

(6) In the determination device according to any one of (1) to (5) above, the model is a convolutional neural network into which the second brain waveform information included in the second input information is input. Includes the neural network in which the output from the convolutional neural network and the fully connected neural network in which information other than the second brain waveform information among the information included in the second input information is input are combined. The fully connected neural network outputs the output information, the control unit inputs the second brain waveform information included in the second input information to the convolutional neural network, and outputs from the convolutional neural network. And information other than the second brain waveform information among the information included in the second input information is input to the fully connected neural network, and the information is described according to the output information output from the fully connected neural network. It may have a configuration for determining a sleep state.

(7) In the determination device according to any one of (1) to (6) above, the control unit performs a state transition between the past sleep state and the sleep state from the past to the present. The output information output from the model may be corrected based on the probability, and the sleep state may be determined according to the corrected output information.

(8) In the determination device according to any one of (1) to (7) above, the kernel size of the neural network has a configuration of a value determined according to a predetermined frequency band. You may.

(9) The light irradiation device according to one aspect of the present invention is a light irradiation device including the determination device according to any one of (1) to (8) above and an endoscope. The endoscope includes an irradiation unit that irradiates the subject's brain with light having a predetermined wavelength, an imaging unit that images the subject's brain, and a light irradiation control unit, and the light irradiation control unit. Is a light irradiation device that irradiates the irradiation unit with the light according to the determination result by the determination device.

(10) The program according to one aspect of the present invention is a program that causes a computer to determine a sleep state of a subject's brain based on a machine learning model including a neural network, and the model is a program of the subject. The first input information including the first brain waveform information indicating the waveform of the first brain wave and the first time information indicating the first time associated with the first brain waveform information is associated with the first input information. The correspondence relationship with the first sleep state information indicating the first sleep state, which is the sleep state, has been learned, and the second brain waveform information showing the waveform of the second brain wave of the subject and the second brain waveform When the second input information including the second time information indicating the second time associated with the information is input, the sleep state associated with the input second input information is estimated and the estimated result. The program outputs the information indicating the above as output information, and the program inputs the second input information to the model, and the sleep is in response to the output information output from the model to which the second input information is input. It is a program that judges the state.

The determination device, the light irradiation device, and the program according to one aspect of the present invention can improve convenience.

It is a figure which shows an example of the structure of the determination device 1. It is a figure which shows an example of the hardware composition of the determination device 1. It is a figure which shows an example of the functional structure of the determination device 1. It is a figure which shows an example of the flow of the process of determining a sleep state among the processes performed by the determination apparatus 1. It is a figure which shows an example of the machine learning model which a determination part 164 has. It is an image diagram which shows how the sleep state changes with the passage of time.

<Embodiment>
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Overview of judgment device>
First, an outline of the determination device according to the embodiment will be described. The determination device determines the sleep state of the subject's brain.

The subject may be any animal as long as it has a brain, for example, a human. The subject may be an animal other than a human such as a mouse. In the following, as an example, a case where the subject is a person will be described.

Further, in the present embodiment, the state of the subject's brain is a state characterized by the waveform of the subject's electroencephalogram. The state of the subject's brain can be classified into a plurality of states according to the waveform of the subject's electroencephalogram under predetermined conditions. For example, a subject's brain state can be classified into a plurality of states according to the waveform of the subject's brain wave during sleep. The sleep state of the subject's brain described above is any one of the plurality of states when the state of the subject's brain is classified into a plurality of states according to the waveform of the subject's electroencephalogram during sleep. Hereinafter, for convenience of explanation, each of the plurality of states will be referred to as a sleep classification state. That is, determining the sleep state of the subject's brain means determining which of the plurality of sleep classification states the subject's brain state is. Further, in the following, for convenience of explanation, the electroencephalogram of the subject during sleep will be described simply as a sleep electroencephalogram. The electroencephalogram of the subject is a wave indicating a temporal change in the electric potential detected from the brain of the subject.

More specifically, the state of the subject's brain can be classified into the state of the subject's brain during sleep and the state of the subject's brain during awakening according to the waveform of the sleep EEG. Are known. Further, it is known that the state of the subject's brain during sleep is classified into a plurality of states according to the waveform. For example, the state of the subject's brain during sleep can be one of two states, a REM (REM; Rapid Eye Movement) sleep state and a non-REM (NREM; Non Rapid Eye Movement) state, depending on the waveform. being classified. In this case, the sleep state of the subject's brain is one of three sleep classification states: REM sleep state, non-REM sleep state, and awake state. Therefore, in the embodiment, as an example, the sleep state of the subject's brain is three sleep states: REM (REM; Rapid Eye Movement) sleep state, non-REM (NREM; Non Rapid Eye Movement) state, and awakening (Wake) state. The case where it is one of the classification states will be described. Therefore, in the embodiment, determining the sleep state of the subject's brain means that the subject's brain state is one of the three sleep classification states according to the waveform of the sleep electroencephalogram. It means to judge. In the following, for convenience of explanation, the three sleep classification states of the REM sleep state, the non-REM state, and the awake state will be referred to simply as the three sleep classification states. Further, in the following, for convenience of explanation, the sleep state of the subject's brain will be described simply as a sleep state.

Here, as a conventional device for determining a sleep state, a device using a plurality of channel electroencephalogram sensors, a device using one or more myoelectric sensors together with one or more channel electroencephalogram sensors, and the like are known. These devices determine the sleep state with high accuracy by reducing noise by using a plurality of channels, using an electromyographic sensor together with an electroencephalogram sensor (that is, using a plurality of sensors), and the like. be able to. However, since these devices have a plurality of channels and use an electromyographic sensor together with an electroencephalogram sensor, the attachment of the sensor to the subject, the setting of the device, the calibration of the device, etc. become complicated. In some cases, it was inconvenient. Further, in these devices, it may be difficult to suppress an increase in manufacturing cost due to a plurality of channels, the use of an electromyographic sensor together with an electroencephalogram sensor, and the like. This leads to an increase in the cost required for determining the sleep state, which is not desirable.

Therefore, the determination device according to the embodiment determines the sleep state based on a machine learning model including a neural network. In the model, the correspondence between the first input information and the first sleep state information has been learned. The first input information is information including the first brain waveform information and the first time information. The first brain waveform information is information indicating the waveform of the first brain wave of the subject. The first time information is information indicating the first time associated with the first brain waveform information. The first sleep state information is information indicating the first sleep state. The first sleep state is a sleep state associated with the first input information. Further, when the second input information is input, the model estimates the sleep state associated with the input second input information, and outputs information indicating the estimated result as output information. The second input information is information including the second brain waveform information and the second time information. The second brain waveform information is information indicating the waveform of the second brain wave of the subject. The second time information is information indicating the second time associated with the second brain waveform information. In addition, the determination device includes a control unit. Then, the control unit inputs the second input information to the model, and determines the sleep state according to the output information output from the model to which the second input information is input.

As a result, the determination device according to the embodiment determines the sleep state according to the difference in the waveform of the subject's brain wave to the extent that it cannot be discriminated by the human eye and the diurnal variation of the waveform of the subject's brain wave. It can be carried out. As a result, the determination device can accurately determine the sleep state without using a plurality of channels, using an electromyographic sensor together with the electroencephalogram sensor, and the like. In other words, the determination device can accurately determine the sleep state based on the brain wave of the subject detected by the one-channel brain wave sensor. This means that the determination device can improve convenience. Further, this means that the determination device can suppress an increase in the manufacturing cost, and as a result, the cost required for determining the sleep state can be reduced. Hereinafter, the configuration of the determination device and the processing performed by the determination device will be described in detail.

<Configuration of judgment device>
Hereinafter, the configuration of the determination device according to the embodiment will be described. Further, in the following, as an example of the determination device, the determination device 1 will be described as an example.

FIG. 1 is a diagram showing an example of the configuration of the determination device 1. The determination device 1 is connected to an electroencephalogram detection unit S attached to the head of the subject. In the example shown in FIG. 1, the electroencephalogram detection unit S is attached to the head of the subject H. The subject H may be any person as long as it can detect an electroencephalogram.

Here, the electroencephalogram detection unit S is a device capable of detecting the electric potential of the subject's brain. For example, the electroencephalogram detection unit S is a one-channel electroencephalograph (electroencephalograph). The electroencephalogram detection unit S outputs brain potential information indicating the detected potential to the determination device 1. The electroencephalogram detection unit S may be an electroencephalogram sensor of a plurality of channels instead of the electroencephalogram sensor of one channel. Further, the electroencephalogram detection unit S may be any device capable of detecting the electric potential of the subject's brain to which the electroencephalogram detection unit S is attached, instead of the electroencephalogram sensor.

The determination device 1 is communicably connected to the electroencephalogram detection unit S by wire or wirelessly. In the example shown in FIG. 1, the determination device 1 is wirelessly connected to the electroencephalogram detection unit S in a communicable manner. Here, the determination device 1 may be connected to the brain wave detection unit S via some interface connecting the brain wave detection unit S and the determination device 1, and is directly connected to the brain wave detection unit S. It may be a configuration. Hereinafter, in order to simplify the description, a case where the determination device 1 is directly connected to the electroencephalogram detection unit S will be described. Further, the determination device 1 may be connected to the electroencephalogram detection unit S and also to another sensor such as an electromyographic sensor. In this case, the other sensor is also attached to the subject.

The determination device 1 determines the sleep state based on the machine learning model. The determination device 1 is, for example, a notebook PC (Personal Computer). The determination device 1 may be another information processing device such as a desktop PC, a tablet PC, a multifunctional mobile phone terminal (smartphone), or a PDA (Personal Digital Assistant) instead of the notebook PC.

In the machine learning model used by the determination device 1, the correspondence between the first input information and the first sleep state information has already been learned. Here, the first input information is the input information input to the model at the stage where the model is trained. The first input information is information including first brain waveform information indicating the waveform of the first brain wave of the subject and first time information indicating the first time associated with the first brain waveform information. .. The first electroencephalogram is an electroencephalogram detected in the brain of a subject whose sleep state is known at a certain timing before determining the sleep state. The first time is, for example, the time when the first electroencephalogram begins to be detected. The first time may be a time when the first brain wave has been detected instead of the time when the first brain wave has started to be detected, or another time during the period when the first brain wave has been detected. May be good. The first sleep state information is information indicating the first sleep state. The first sleep state is the sleep state of the subject's brain that is known at that timing. That is, the first sleep state is a sleep state that is known to be associated with the first input information.

In other words, in the machine learning model used by the determination device 1, when the first input information is input, the first sleep state indicated by the first sleep state information is set as the sleep state associated with the first input information. The correspondence between the first input information and the first sleep state information is learned so as to presume that it is the most plausible. In other words, the model is included in the model so that in this case, the first sleep state indicated by the first sleep state information is presumed to be the most probable as the sleep state associated with the first input information. Each weight of the neural network is updated by backpropagation. That is, the first input information is training data in the model. The first sleep state information is teacher data in the model. The model learns such correspondence for each of a plurality of brain waves of a subject. The association of the first sleep state with the first input information is performed by a person in advance. That is, when the first electroencephalogram is detected at a certain timing, the sleep state at that timing is determined by a person. In other words, the sleep state associated with the first brain wave is determined by a person based on the first brain wave. Then, the combination of the sleep state determined by a person and the brain wave associated with the sleep state has already been obtained by a large number of experiments conducted in the past. By using the combination already obtained in this way, a plurality of correspondences between the first input information and the first sleep state information can be easily generated. The noise in the determination of the sleep state by the determination device 1 can be statistically reduced by learning a large number of the correspondence relationships with the model. The association of the first sleep state with the first input information may be performed by any method.

Here, the reason why the first input information includes the first time information is that it is known that the brain waves of the subject cause diurnal fluctuations. That is, since the first input information includes the first time information, the determination device 1 uses a machine learning model to determine the difference in the electroencephalogram waveform of the subject that cannot be discriminated by the human eye and the test. It becomes possible to determine the sleep state of the subject's brain according to the diurnal variation of the waveform of the body's brain waves.

Further, the machine learning model used by the determination device 1 estimates the sleep state of the subject's brain associated with the input second input information when the second input information is input. Then, the model outputs information indicating the estimated result as output information. The model may have a configuration in which such estimation is performed by online estimation or an offline estimation. In the following, as an example, a case where the model makes such an estimation by online estimation will be described. Here, the second input information is the input information input to the model at the stage of causing the model to perform estimation. The second input information is information including the second brain waveform information and the second time information indicating the second time associated with the second brain waveform information. The second brain waveform information is information indicating the waveform of the second brain wave of the subject. In this example, since the model performs online estimation, the second electroencephalogram is an electroencephalogram detected during the process of determining the sleep state among the subject's electroencephalograms. When the model performs offline estimation, the second electroencephalogram is an electroencephalogram detected from the subject's brain at a certain timing before determining the sleep state. The second time is, for example, the time when the second electroencephalogram begins to be detected. The second time may be the time when the second brain wave has been detected instead of the time when the second brain wave has started to be detected, and may be another time during the period when the second brain wave has been detected. May be good. However, the method of determining the second time is the same as the method of determining the first time. That is, when the time when the first brain wave starts to be detected is determined as the first time, the second time is determined as the time when the second brain wave starts to be detected.

When the determination device 1 receives an operation for starting the determination of the sleep state, the determination device 1 outputs brain potential information indicating the potential detected by the electroencephalogram detection unit S from the timing at which the operation is accepted until a predetermined measurement time elapses. Continue to acquire at a predetermined sampling cycle. The predetermined sampling period is, for example, 128 [Hz]. The predetermined sampling period may be shorter than 128 [Hz] or longer than 128 [Hz].

The determination device 1 sets the elapsed predetermined time as the target time each time a predetermined time elapses from the timing of receiving the operation to start the determination of the sleep state, and the brain potential acquired from the electroencephalogram detection unit S within the target time. The brain waveform information based on the information is generated as the second brain waveform information within the target time. Here, the second brain waveform information within a certain target time is information composed of a plurality of brain potential information acquired by the brain wave detection unit S within the target time. More specifically, the second brain waveform information within the target time is a wave waveform showing a temporal change of the potential when the potentials indicated by each of the plurality of brain potential information are arranged in chronological order. It is the information shown as the waveform of the second brain wave of the subject within the target time. Further, the second brain waveform information within the target time is associated with the second time information indicating the time when the brain potential information is first acquired by the electroencephalogram detection unit S within the target time as the second time. After generating the second brain waveform information within a certain target time, the determination device 1 includes information including the second brain waveform information within the target time and the second time information associated with the second brain waveform information. Is generated as the second input information within the target time. The determination device 1 generates the second input information within the target time, and then inputs the generated second input information to the machine learning model.

The determination device 1 inputs the second input information within a certain target time to the machine learning model, and then the target time according to the output information output from the machine learning model to which the second input information is input. Determine the sleep state of the subject's brain within. More specifically, the determination device 1 determines which of the three sleep classification states is the sleep state of the subject's brain within the target time according to the output information. Here, the output information includes information indicating three likelihoods estimated for each of the three sleep classification states. The three likelihoods indicate the likelihood of the subject's brain sleeping state within the target time for each of the three sleep classification states. That is, the determination device 1 determines the sleep classification state having the highest likelihood among the three likelihoods indicated by the information included in the output information as the sleep state of the subject's brain within the target time.

By making a judgment using such a machine learning model, the judgment device 1 can determine the difference in the electroencephalogram waveform of the subject to the extent that it cannot be discriminated by the human eye, and the diurnal frequency of the electroencephalogram waveform of the subject. The sleep state can be determined according to the fluctuation. As a result, the determination device 1 can accurately determine the sleep state without increasing the number of channels, using the myoelectric sensor together with the electroencephalogram sensor, and the like. In other words, the determination device 1 can accurately determine the sleep state based on the brain wave of the subject detected by the brain wave detection unit S, which is a one-channel brain wave sensor. That is, the determination device 1 can improve convenience. In addition, the determination device 1 can suppress an increase in the manufacturing cost, and can reduce the cost required for determining the sleep state of the brain of the subject.

The determination device 1 may be configured to include an electroencephalogram detecting unit S, or may be configured not to include an electroencephalogram detecting unit S. Further, when the determination device 1 is provided with the electroencephalogram detection unit S, the determination device 1 may be integrally configured with the electroencephalogram detection unit S, or may be configured separately from the electroencephalogram detection unit S. Hereinafter, as an example, a case where the determination device 1 does not include the electroencephalogram detection unit S will be described.

<Hardware configuration of judgment device>
Hereinafter, the hardware configuration of the determination device 1 will be described with reference to FIG. FIG. 2 is a diagram showing an example of the hardware configuration of the determination device 1.

The determination device 1 includes, for example, a CPU (Central Processing Unit) 11, a storage unit 12, an input reception unit 13, a communication unit 14, and a display unit 15. These components are communicably connected to each other via a bus. Further, the determination device 1 communicates with the electroencephalogram detection unit S and other devices via the communication unit 14.

The CPU 11 executes various processes by executing various programs stored in the storage unit 12. The CPU 11 receives an operation from the user via the input receiving unit 13. The CPU 11 executes a process according to the received operation.

The storage unit 12 includes, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a ROM (Read-Only Memory), a RAM (Random Access Memory), and the like. The storage unit 12 may be an external storage device connected by a digital input / output port such as USB, instead of the one built in the determination device 1. The storage unit 12 stores various information, various programs, and the like processed by the determination device 1.

The input receiving unit 13 is an input device such as a keyboard, a mouse, and a touch pad. The input receiving unit 13 may be a touch panel integrally configured with the display unit 15.

The communication unit 14 includes, for example, a digital input / output port such as USB, an Ethernet (registered trademark) port, and the like.

The display unit 15 is, for example, a liquid crystal display panel or an organic EL (ElectroLuminescence) display panel.

<Functional configuration of judgment device>
Hereinafter, the functional configuration of the determination device 1 will be described with reference to FIG. FIG. 3 is a diagram showing an example of the functional configuration of the determination device 1.

The determination device 1 includes, for example, a storage unit 12, an input reception unit 13, a communication unit 14, a display unit 15, and a control unit 16.

The control unit 16 controls the entire determination device 1. The control unit 16 includes an acquisition unit 161, a generation unit 162, a timekeeping unit 163, a determination unit 164, and a display control unit 165. These functional units included in the control unit 16 are realized, for example, by the CPU 11 executing various programs stored in the storage unit 22. Further, a part or all of the functional parts may be hardware functional parts such as LSI (Large Scale Integration) and ASIC (Application Specific Integrated Circuit).

The acquisition unit 161 acquires the brain potential information indicating the potential detected by the brain wave detection unit S from the brain wave detection unit S.

The generation unit 162 generates various information used by the determination device 1. For example, the generation unit 162 generates the second brain waveform information based on the brain potential information acquired by the acquisition unit 161 within the predetermined time described above. Further, for example, the generation unit 162 generates the second input information.

The timekeeping unit 163 clocks various times.

The determination unit 164 determines the sleep state based on the second input information generated by the generation unit 162 and the machine learning model.

The display control unit 165 generates various images according to the operation received from the user via the input reception unit 13. The display control unit 165 causes the display unit 15 to display the generated image.

<Process to determine sleep state>
Hereinafter, among the processes performed by the determination device 1, the process of determining the sleep state will be described with reference to FIG. FIG. 4 is a diagram showing an example of the flow of the process of determining the sleep state among the processes performed by the determination device 1. In the following, as an example, a case where the determination device 1 accepts an operation of starting the process of determining the sleep state at a timing before the process of step S110 shown in FIG. 4 is performed will be described. Further, in the following, as an example, a case where the second frequency domain information is included in the second input information together with the second brain waveform information and the second time information will be described. The second frequency domain information is information indicating the waveform of the second electroencephalogram in the frequency domain. The waveform of the second brain wave in the frequency domain is a waveform obtained by performing frequency domain conversion to the waveform indicated by the second brain waveform information. The frequency domain transform is, in the embodiment, a process of converting a time domain waveform signal into a frequency domain waveform signal, such as Fourier transform, discrete cosine transform, Hilbert transform, and wavelet transform. As a result, the determination device 1 has a difference in the waveform of the subject's brain wave that cannot be discriminated by the human eye, a difference in the waveform of the subject's brain wave in the frequency domain, and a diurnal frequency of the waveform of the subject's brain wave. The sleep state can be determined according to the fluctuation. When the second input information includes the second frequency domain information, the first input information includes the first frequency domain information. The first frequency domain information is information indicating the waveform of the first electroencephalogram in the frequency domain. The waveform of the first brain wave in the frequency domain is a waveform obtained by performing frequency domain conversion to the waveform indicated by the first brain waveform information. Hereinafter, a case where the second frequency domain information is the second spectrum information will be described. In this case, the second spectrum information is information indicating the frequency spectrum obtained after Fourier transforming the waveform indicated by the second brain waveform information, that is, information indicating the frequency spectrum of the second brain wave. As a result, the determination device 1 determines the difference in the waveform of the subject's brain wave that cannot be discriminated by the human eye, the difference in the frequency spectrum of the subject's brain wave, and the diurnal variation of the subject's brain wave waveform. The sleep state can be determined according to the above. When the second input information includes the second frequency spectrum information, the first input information includes the first frequency spectrum information. The first frequency spectrum information is information indicating the first frequency spectrum. The first frequency spectrum is a frequency spectrum for the first electroencephalogram.

The acquisition unit 161 starts acquiring the brain potential information from the electroencephalogram detection unit S (step S110). Here, as described above, the acquisition unit 161 acquires the brain potential information from the electroencephalogram detection unit S at a sampling frequency of 128 [Hz].

Next, each time a predetermined time elapses from the timing at which the process of step S110 is performed, the control unit 16 repeatedly executes the processes of steps S130 to S180, setting the elapsed predetermined time as the target time (step). S120). Here, the predetermined time is, for example, 10 seconds. The predetermined time may be shorter than 10 seconds or longer than 10 seconds.

The generation unit 162 generates the second input information within the target time based on all the brain potential information acquired by the acquisition unit 161 within the target time (step S130). Here, the process of step S130 will be described.

The generation unit 162 generates brain waveform information based on the brain potential information acquired by the acquisition unit 161 from the brain wave detection unit S within the target time as the second brain waveform information within the target time. When the predetermined time is 10 seconds and the predetermined sampling frequency is 128 [Hz], the second brain waveform information is the potential of the subject's brain at each time within the target time. It is a 1280-dimensional vector having as a component. In addition, the generation unit 162 generates the second time information indicating the time when the electroencephalogram detection unit S first acquires the brain potential information from the electroencephalogram detection unit S within the target time as the second time. The generation unit 162 associates the generated second time information with the generated second brain waveform information. Further, the generation unit 162 calculates the frequency spectrum of the second brain wave of the waveform indicated by the generated second brain waveform information as the second frequency spectrum based on the Fourier transform. The method for calculating the frequency spectrum by the generation unit 162 may be a known method or a method to be developed in the future. Therefore, a detailed description of the calculation method will be omitted. The generation unit 162 uses information including the generated second brain waveform information, the generated second time information, and the second frequency spectrum information indicating the calculated second frequency spectrum as the second input information within the target time. Generate. In this way, the generation unit 162 generates the second input information within the target time in step S130. The second input information may include other information in addition to the second brain waveform information, the second time information, and the second frequency spectrum information.

Next, the determination unit 164 estimates the sleep state based on the second input information generated in step S130 and the machine learning model (step S140). Here, the process of step S140 will be described with reference to FIG.

FIG. 5 is a diagram showing an example of a machine learning model included in the determination unit 164. The model M shown in FIG. 5 shows an example of a machine learning model included in the determination unit 164. Further, the second input information D shown in FIG. 5 shows an example of the second input information input by the determination unit 164 to the machine learning model. The second input information D includes the second brain waveform information D1, the second frequency spectrum information D2, and the second time information D3. The second brain waveform information D1 shows an example of the second brain waveform information. The second frequency spectrum information D2 shows an example of the second frequency spectrum information. The second time information D3 shows an example of the second time information. That is, in the example shown in FIG. 5, the determination unit 164 models the three pieces of information of the second brain waveform information D1, the second frequency spectrum information D2, and the second time information D3 as the second input information D. Enter in M. As a result, the determination unit 164 acquires the output information output from the model M. The output information D4 shown in FIG. 5 shows an example of such output information. As described above, the output information D4 has a likelihood of indicating the likelihood that the sleep state is in the REM sleep state within the target time and a likelihood of indicating the likelihood that the sleep state is in the non-REM sleep state. It contains information indicating three likelihoods, that is, and the likelihood that the sleeping state is awake.

Here, in the example shown in FIG. 5, the model M is a convolutional neural network (CNN) N1 in which the second brain waveform information D1 included in the second input information D is input, and a convolutional neural network N1. Of the information contained in the output from and the second input information D, information other than the second brain waveform information D1 (in the example shown in FIG. 5, the second frequency spectrum information D2 and the second time information D3) is input. Includes a neural network that is combined with a fully connected neural network (FCN) N2. The configuration of the model M may be another configuration instead of the configuration shown in FIG. For example, the model M may be configured to include other neural networks, other functions, and the like in addition to the convolutional neural network N1 and the fully connected neural network N2.

The second brain waveform information D1 included in the second input information D input to the model M by the determination unit 164 is input to the convolutional neural network N1. The convolutional neural network N1 generates a predetermined number of features based on the input second brain waveform information D1. The predetermined number is determined so that the determination accuracy is high by prior trial and error. The predetermined number is, for example, 8. The predetermined number may be less than 8 or more than 8. The method of generating a predetermined number of features by the convolutional neural network N1 may be a known method or a method to be developed in the future. Further, the number of layers, the number of nodes, the number of filters, the kernel size, etc. of the convolutional neural network N1 may be any number. Here, in the embodiment, the number of filters indicates the number of filters (ie, kernels) used. Also, in embodiments, the kernel size indicates the dimension of each filter (ie, the kernel), which is the vector used for convolution.

A predetermined number of features output from the convolutional neural network N1 in this way are input to the fully coupled neural network N2 together with the second frequency spectrum information D2 and the second time information D3 included in the second input information D. Will be done. Then, the fully coupled neural network N2 outputs the above-mentioned output information D4 based on the input predetermined number of feature quantities, the second frequency spectrum information D2, and the second time information D3. Here, the number of layers, the number of nodes, and the like of the fully connected neural network N2 may be any number.

The determination unit 164 acquires the output information D4 from the model M by inputting the second input information D into the model M having such a configuration. Here, the output information D4 is an example of output information showing the result of estimating the sleep state by the machine learning model.

After the processing of step S140 is performed, the determination unit 164 acquired from the machine learning model in step S140 based on the past sleep state and the state transition probability for the sleep state from the past to the present. Correct the output information (step S150). Here, the process of step S150 will be described.

The sleep state transitions to one of the three sleep classification states with the passage of time. Here, FIG. 6 is an image diagram showing how the sleep state changes with the passage of time. The time t shown in FIG. 6 indicates an example of the target time. The time t-1 shown in FIG. 6 indicates a predetermined time determined to have elapsed one time before the target time. The time t-2 shown in FIG. 6 indicates a predetermined time determined to have elapsed two times before the target time. Further, the state St shown in FIG. 6 indicates a sleep state at time t. The state St-1 shown in FIG. 6 indicates a sleep state at time t-1. The state St-2 shown in FIG. 6 indicates the sleep state at time t-2. In FIG. 6, for the sake of simplification of the figure, the time before the time t-2 is omitted.

For example, the number of state transitions that can occur in a state transition from state St-2 to state St-1 is nine. That is, when the state St-2 is the REM sleep state, the state St-1 transitions to any of the REM sleep state, the non-REM sleep state, and the awake state. Further, even when the state St-2 is in the non-REM sleep state, the state St-1 transitions to any of the REM sleep state, the non-REM sleep state, and the awake state. Even when the state St-2 is in the awake state, the state St-1 transitions to any of a REM sleep state, a non-REM sleep state, and an awake state.

Further, for example, the number of state transitions that can occur in the state transition from the state St-1 to the state St is also 9. That is, when the state St-1 is the REM sleep state, the state St transitions to any one of the REM sleep state, the non-REM sleep state, and the awake state. Further, even when the state St-1 is in the non-REM sleep state, the state St transitions to any one of the REM sleep state, the non-REM sleep state, and the awake state. Even when the state St-1 is in the awake state, the state St transitions to any of a REM sleep state, a non-REM sleep state, and an awake state.

And, such a state transition occurs in the same time before the time t-2. That is, the number of state transitions that can occur in the state transition from the state Stn, which is the sleep state at time tn, to the state St is 3 ⁿ . Here, n may be any integer as long as it is an integer of 1 or more.

^{Here, the state transition probabilities in which each of the 3 n} state transitions that can occur in the state transition from the state St-n to the state St occurs can be calculated as a conditional probability by Bayesian estimation based on the Markov chain. The above-mentioned first input information can be used as the sample data required for calculating the conditional probability by Bayesian estimation based on the Markov chain. The determination unit 164 calculates the state transition probability in advance based on the plurality of first input information at the timing before the process of step S110 shown in FIG. 4 is performed. Hereinafter, for convenience of explanation, the state transition probability calculated in advance by the determination unit 164 will be referred to as a pre-state transition probability.

Then, the determination unit 164 refers to the past determination history in step S150, and determines that the predetermined time determined to have elapsed n times before the target time has elapsed one time before the target time. The determination result for the n sleep states determined by the determination unit 164 within the range up to the predetermined time is specified. The determination unit 164 is a state transition in which the sleep state within the target time transitions to each of the REM sleep state, the non-REM sleep state, and the awake state via these n determination results (that is, n sleep states) specified. The probability is specified as the target state transition probability based on the prior state transition probability. The determination unit 164 corrects the three likelihood values indicated by the information included in the output information output from the machine learning model in step S140 based on the specified target state transition probability. That is, the determination unit 164 has specified the probability distribution for the sleep state within the target time as the probability distribution for the sleep state determined at each of the n predetermined times elapsed before the target time. By multiplying the target state transition probability and normalizing (that is, standardizing for converting the likelihood into a probability), the probability distribution for the sleep state within the target time is corrected. Here, the probability distribution for the sleep state is the three likelihood values estimated in step S140. However, the total value of the three likelihood values included in the probability distribution for the sleep state determined in the past is standardized so as to be 1. Therefore, these three likelihood values are referred to as the probability distribution for the sleep state. Further, the probability distribution corrected in step S150 and the probability distribution for the sleep state within the target time is determined in each of the n predetermined times elapsed before the target time in the next process of step S150. It is used as one of the probability distributions for the determined sleep state. In this sense as well, in the embodiment, these three likelihood values are referred to as a probability distribution for the sleep state.

More specifically, with respect to step S150, the determination unit 164 determines n predetermined values that have passed before the target time with respect to the likelihood indicating the likelihood that the sleep state within the target time is the REM sleep state. Multiply the probability distribution for the sleep state determined at each of the times and the state transition probability that the sleep state within the target time transitions to the REM sleep state among the specified target state transition probabilities, and calculate the likelihood. to correct. Further, the determination unit 164 determined at each of the n predetermined times elapsed before the target time with respect to the likelihood indicating the likelihood that the sleep state within the target time is the non-REM sleep state. The likelihood distribution for the sleep state is multiplied by the state transition probability that the sleep state in the target time transitions to the non-REM sleep state among the specified target state transition probabilities, and the likelihood is corrected. Further, the determination unit 164 determines the sleep at each of the n predetermined times elapsed before the target time with respect to the likelihood indicating the likelihood that the sleep state within the target time is the awake state. The likelihood distribution for the state is multiplied by the state transition probability that the sleep state transitions to the awake state within the target time among the specified target state transition probabilities, and the likelihood is corrected. Then, the determination unit 164 specifies the information indicating these three likelihoods after the correction as the output information after the correction (in other words, the probability distribution of the sleep state within the target time). In this way, the determination unit 164 outputs the output information output from the machine learning model in step S150 based on the past sleep state and the state transition probability for the sleep state from the past to the present. to correct. The method of correcting the likelihood based on the target state transition probability may be another method based on the target state transition probability instead of the method described above.

In addition, in the process of step S150 (for example, the process of the first step S150) when there is no past determination history, the determination unit 164 performs the process of step S150 by using the information substitute for the past determination history. .. The information is, for example, information indicating a sleep classification state having the highest probability of being realized at the time indicated by the second time information included in the second input information among the three sleep classification states. The determination unit 164 calculates such information using the target state transition probability. The method for calculating such information may be a known method or a method to be developed in the future. The determination unit 164 may be configured to calculate such information using the first input information used in the learning stage.

After the processing of step S150 is performed, the determination unit 164 determines the sleep state within the target time based on the output information corrected in step S150 (step S160). Specifically, the determination unit 164 determines the sleep classification state having the highest likelihood among the three likelihoods after the target state transition probability is multiplied, based on the output information, as the sleep state within the target time. Judge as.

Next, the display control unit 165 causes the display unit 15 to display information indicating the determination result in step S160 (step S170). That is, the display control unit 165 displays information indicating the sleep classification state having the highest likelihood among the three likelihoods after the target state transition probability is multiplied as information indicating the sleep state within the target time. Display on 15.

Next, the control unit 16 determines whether or not a predetermined measurement time has elapsed from the timing when the process of step S110 is performed (step S180). The predetermined measurement time is, for example, one hour. The predetermined measurement time may be shorter than 1 hour or longer than 1 hour.

When the control unit 16 determines that the predetermined measurement time has not elapsed from the timing when the process of step S110 is performed (step S180-NO), the control unit 16 transitions to step S120 and starts from the latest time included in the target time. Wait until the specified time has elapsed.

On the other hand, when it is determined that the predetermined measurement time has elapsed from the timing when the process of step S110 is performed (step S180-YES), the control unit 16 ends the process.

In the example described above, the process of step S150 may be omitted. In this case, in step S160, the determination device 1 determines the sleep classification state having the highest likelihood among the three likelihoods not corrected by the target state transition probability as the sleep state within the target time.

<Guidelines for determining the structure of convolutional neural networks>
Hereinafter, a guideline for determining the structure of the convolutional neural network N1 described above will be described. Here, the structure of the convolutional neural network N1 is represented by the number of layers, the number of nodes, the number of filters, the kernel size, and the like of the convolutional neural network N1. That is, the guideline is a guideline for determining the number of layers, the number of nodes, the number of filters, the kernel size, etc. of the convolutional neural network N1. In the following, a guideline for determining the kernel size among the number of layers, the number of nodes, the number of filters, the kernel size, etc. of the convolutional neural network N1 will be described.

In other words, the kernel size in the convolutional neural network N1 is a parameter indicating a range to be combined into one numerical value in the signal input to the convolutional neural network N1 (in the example shown in FIG. 5, the second brain waveform information D1). Combining the desired range of the signal input to the convolutional neural network N1 into one numerical value corresponds to extracting the features included in the range by one layer of the convolutional neural network N1. That is, by adjusting the kernel size of the convolutional neural network N1, the machine learning model M can estimate the sleep state based on the one numerical value as a feature extracted by the convolutional neural network N1, and as a result, , The estimation accuracy of the sleeping state can be improved.

Here, of the range of the second brain waveform information D1, the range including the half wavelength of the wave in the frequency band known to be effective for identifying the sleep state is input to the convolutional neural network N1. By selecting it as a desired range in the brain waveform information D1, the machine learning model M can improve the estimation accuracy of the sleep state. The frequency band known to be effective in identifying the sleep state is, for example, the wavelength band of theta wave (that is, 4 to 8 Hz). The reason for selecting the range including one half wavelength when the wavelength of the wave is divided into two half wavelengths as the desired range is not the range including the wavelength of the wave, in the convolution neural network N1. This is because the range including the other half wavelength of the two half wavelengths is obtained by performing code inversion in the layer next to the layer closest to the input in the convolutional neural network N1. As a result, the determination device 1 can reduce the kernel size of the convolutional neural network N1. As a result, the determination device 1 can suppress the occurrence of overfitting (overfitting, that is, a phenomenon in which the accuracy at the time of actual prediction is reduced due to overfitting to the training data).

Specifically, in the range of the second brain waveform information D1, the frequency band known to be effective for identifying the sleep state is Ω, the individual frequencies included in the frequency band Ω are f, and the sampling frequency is the sampling frequency. Assuming that r and the kernel size are s, the half wavelength of the wave in the frequency band is (1 / (2f)), so the kernel size s is calculated by the following equation (1).

As described above, the sampling frequency r is 128 [Hz] in this example. In this case, assuming that the frequency band Ω is the frequency band of theta wave, the kernel size s is calculated as 16 based on the above equation (1). That is, the machine learning model M can improve the estimation accuracy of the sleep state by setting the kernel size s to 16. As a result, the determination device 1 can improve the accuracy of determining the sleep state.

Here, the approximate number of weights of the convolutional neural network N1 is obtained by multiplying the kernel size by the number of filters and the number of layers of the convolutional neural network N1. That is, when the kernel size increases, the number of parameters of the convolutional neural network N1 also increases. In this case, overfitting is more likely to occur.

However, in the experiments we conducted, it was found that the accuracy of estimating the sleep state by the machine learning model M did not change even if the number of filters was reduced. Therefore, in order to avoid such an increase in the possibility of overfitting, in the machine learning model M, the increase in the number of weights of the convolutional neural network N1 is suppressed by reducing the number of filters of the convolutional neural network N1. To do. As a result, the determination device 1 can improve the accuracy of determining the sleep state while suppressing the occurrence of overfitting.

As described above, the determination device 1 determines the sleep state by using the machine learning model. As a result, the determination device 1 has a difference in the waveform of the subject's brain wave that cannot be discriminated by the human eye, a difference in the frequency spectrum of the subject's brain wave, and a diurnal variation in the waveform of the subject's brain wave. The sleep state can be determined accordingly. As a result, the determination device 1 can accurately determine the sleep state without increasing the number of channels, using the myoelectric sensor together with the electroencephalogram sensor, and the like. In other words, the determination device 1 can accurately determine the sleep state based on the brain wave of the subject detected by the brain wave detection unit S, which is a one-channel brain wave sensor. That is, the determination device 1 can improve convenience. Further, the determination device 1 can suppress an increase in the manufacturing cost, and can reduce the cost required for determining the sleep state.

Here, when a person determines a sleep state, the person determines based on the temporal change of the brain wave of the subject regardless of whether the person is conscious or unconscious. That is, the determination of the sleep state by a person is based on the temporal change of the brain wave of the subject. By performing the process of step S150 described above, the determination device 1 makes a determination close to the determination of the sleep state by a person. As a result, the determination device 1 can determine the sleep state with higher accuracy while improving convenience.

Although the machine learning model described above includes the convolutional neural network N1, it may be configured to include a recursive neural network instead of the convolutional neural network N1. However, the data length of the second input information does not change with the passage of time. Therefore, it is preferable that the model includes a convolutional neural network N1 rather than a recursive neural network.

Further, the determination of the sleep state of the brain by the determination device 1 described above can be applied to an animal whose criteria for determining the sleep state of the brain are unknown. This is because even if the subject is the animal, the determination device 1 is used by the determination device 1 as long as the person can determine the sleep state of the animal's brain by looking at the brain waves of the animal. This is because it is possible to learn the correspondence between the first input information and the first sleep state information for the machine learning model.

Here, the determination device 1 extracts information indicating a waveform known to be effective for determining a sleep state in a person (for example, sleep spindle wave, slow wave, etc.) from the second brain waveform information D1. It does not determine the sleep state based on the extracted information. If the determination device 1 determines the sleep state based on the information, the determination device 1 cannot determine the sleep state of an animal other than a human. This is because it is not always possible to find a waveform that is effective in determining the sleep state in animals other than humans. However, as described above, the determination device 1 does not use the information in determining the sleep state. This means that, as described above, the determination of the sleep state of the brain by the determination device 1 described above can be applied to an animal whose criteria for determining the sleep state of the brain are unknown.

<Application example of judgment device>
Hereinafter, an application example of the determination device 1 will be described. The determination device 1 may be provided in the light irradiation device together with the endoscope, for example. Here, in the present embodiment, when referred to as an endoscope, an irradiation unit that irradiates the brain of the subject with light having a predetermined wavelength, an imaging unit that images the brain of the subject, and a light irradiation control unit are used. Shows the endoscope to be equipped with. The light irradiation control unit irradiates the irradiation unit with the light according to the determination result by the determination device 1. This allows the light irradiator to control at least part of the subject's brain activity, eg, based on optogenetics. As a result, the light irradiation device can perform some treatment based on optogenetics according to the sleep state of the subject's brain, for example.

Here, the light irradiation control unit includes a light source that irradiates light having a wavelength corresponding to the operation accepted by the light irradiation device. The light irradiation control unit is connected to the irradiation unit by an optical fiber. The irradiation unit irradiates the light guided from the optical fiber from the irradiation port. As a result, the light irradiation device can irradiate light having a wavelength desired by the user from the irradiation unit. As a result, the light irradiation device can easily irradiate the brain in the sleeping state desired by the user with light having a wavelength desired by the user.

Further, the determination device 1 is replaced with such a light irradiation device as another medical device such as a sound output device that outputs a sound, an odor output device that outputs an odor, and a breathing assist device that assists the subject's breathing. It may be provided. In this case, the determination device 1 is a device that gives a trigger to start some processing to these medical devices. That is, for example, when the determination device 1 is provided in the sound output device, the sound output device outputs a predetermined sound according to the sleep state of the subject's brain determined by the determination device 1. More specifically, the sound output device outputs a predetermined sound, for example, when it is determined that the sleep state of the subject's brain is the REM sleep state. Further, for example, when the determination device 1 is provided in the odor output device, the odor output device outputs a predetermined odor according to the sleep state of the subject's brain determined by the determination device 1. More specifically, the odor output device outputs a predetermined odor when, for example, it is determined that the sleep state of the subject's brain is the REM sleep state. Further, for example, when the determination device 1 is provided in the breathing assist device, the respiratory assist device assists the subject's breathing according to the sleep state of the subject's brain determined by the determination device 1. More specifically, the respiratory assist device assists the subject's breathing, for example, when it is determined that the sleep state of the subject's brain is the REM sleep state.

As described above, the determination device (determination device 1 in the example described above) is a subject (described above) based on a machine learning model including a neural network (model M in the example described above). In the example, it is a determination device for determining the sleep state of the brain of the subject H), and the model is associated with the first brain waveform information indicating the waveform of the first brain wave of the subject and the first brain waveform information. Learn the correspondence between the first input information including the first time information indicating the first time and the first sleep state information indicating the first sleep state which is the sleep state associated with the first input information. The second input information including the second brain waveform information indicating the waveform of the second brain wave of the subject and the second time information indicating the second time associated with the second brain waveform information has been input. In this case, the sleep state associated with the input second input information is estimated, and information indicating the estimated result is output as output information, and the determination device is a control unit (in the example described above, the control unit 16). ) Is provided, the control unit inputs the second input information to the model, and determines the sleep state according to the output information output from the model to which the second input information is input. Thereby, the determination device can improve the convenience.

Further, in the determination device, the sleep state of the subject's brain is one of a plurality of states classified by the subject's electroencephalogram at the time of sleep among the states of the subject's brain. According to the output information, according to the output information output from the machine learning model to which the second input information is input, it is determined which of the plurality of states the sleep state is. It may be used.

Further, in the determination device, a configuration may be used in which the plurality of states include a REM sleep state, a non-REM sleep state, and an awake state.

Further, in the determination device, the machine learning model has a plurality of likelihoods indicating the likelihood of being in the sleeping state of the subject's brain associated with the second input information when the second input information is input. Estimates for each of the states, outputs information including each of the estimated likelihoods for each of the plurality of states as output information, and the control unit estimates from the model among the plurality of states based on the output information. A configuration may be used in which the state with the highest likelihood is determined as the sleeping state.

Further, in the determination device, the first input information includes the first frequency domain information indicating the waveform of the first brain wave in the frequency domain, and the second input information includes the waveform of the second brain wave in the frequency domain. A configuration may be used that includes the indicated second frequency domain information.

Further, in the determination device, the machine learning model is derived from the convolutional neural network (convolutional neural network N1 in the example described above) in which the second brain waveform information included in the second input information is input, and the convolutional neural network. Is combined with a fully connected neural network (in the example described above, a fully connected neural network N2) in which information other than the second brain waveform information among the information contained in the second input information is input. The fully connected neural network outputs the output information, and the control unit inputs the second brain waveform information included in the second input information to the convolution neural network and outputs it from the convolution neural network. Of the information contained in the second input information, information other than the second brain waveform information is input to the fully connected neural network, and the sleeping state of the subject's brain is adjusted according to the output information output from the fully connected neural network. A configuration may be used to determine.

Further, in the determination device, the control unit is based on a machine learning model based on the sleep state of the subject's brain in the past and the state transition probability of the sleep state of the subject's brain from the past to the present. A configuration may be used in which the output information is corrected and the sleep state of the subject's brain is determined according to the corrected output information.

Further, in the determination device, a configuration may be used in which the kernel size of the neural network is a value determined according to a predetermined frequency band.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and changes, substitutions, deletions, etc., are made as long as the gist of the invention is not deviated. May be done.

Further, a program for realizing the function of an arbitrary component in the device (for example, the determination device 1) described above is recorded on a computer-readable recording medium, and the program is read into a computer system and executed. You may try to do it. The term "computer system" as used herein includes hardware such as an OS (Operating System) and peripheral devices. The "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD (Compact Disk) -ROM, or a storage device such as a hard disk built in a computer system. Say. Furthermore, a "computer-readable recording medium" is a volatile memory (RAM) inside a computer system that serves as a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. As such, it shall include those that hold the program for a certain period of time.

Further, the above program may be transmitted from a computer system in which this program is stored in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the "transmission medium" for transmitting a program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
Further, the above program may be for realizing a part of the above-mentioned functions. Further, the above program may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

1 ... Judgment device, 12 ... Storage unit, 13 ... Input reception unit, 14 ... Communication unit, 15 ... Display unit, 16 ... Control unit, 22 ... Storage unit, 161 ... Acquisition unit, 162 ... Generation unit, 163 ... Time counting unit , 164 ... Judgment unit, 165 ... Display control unit, D ... Second input information, D1 ... Second brain waveform information, D2 ... Second frequency spectrum information, D3 ... Second time information, D4 ... Output information, H ... Test Body, M ... model, N1 ... convolutional neural network, N2 ... fully connected neural network, S ... brain wave detector

Claims (10)

A determination device that determines the sleep state of a subject's brain based on a machine learning model including a neural network.
The model is
The first input information including the first brain waveform information indicating the waveform of the first brain wave of the subject and the first time information indicating the first time associated with the first brain waveform information, and the first input. The correspondence relationship with the first sleep state information indicating the first sleep state, which is the sleep state associated with the information, has been learned, and the second brain waveform information indicating the waveform of the second brain wave of the subject and the said When the second input information including the second time information indicating the second time associated with the second brain waveform information is input, the sleep state associated with the input second input information is estimated. , Outputs the information indicating the estimated result as output information,
The determination device is
Equipped with a control unit
The control unit
The second input information is input to the model, and the sleep state is determined according to the output information output from the model to which the second input information is input.
Judgment device. The sleep state is one of a plurality of states of the subject's brain states classified by the subject's brain waves during sleep.
The control unit
Depending on the output information
It is determined which of the plurality of states the sleep state is, according to the output information output from the model to which the second input information is input.
The determination device according to claim 1. The plurality of states include a REM sleep state, a non-REM sleep state, and an awake state.
The determination device according to claim 2. The model is
When the second input information is input, the likelihood indicating the likelihood of being in the sleeping state associated with the second input information is estimated for each of the plurality of states, and each of the plurality of states is estimated. The information including each of the estimated likelihoods is output as the output information.
Based on the output information, the control unit determines a state having the highest likelihood estimated by the model among the plurality of states as the sleep state.
The determination device according to claim 2 or 3. The first input information includes first frequency domain information indicating the waveform of the first electroencephalogram in the frequency domain.
The second input information includes second frequency domain information indicating the waveform of the second electroencephalogram in the frequency domain.
The determination device according to any one of claims 1 to 4. The model includes a convolutional neural network in which the second brain waveform information included in the second input information is input, an output from the convolutional neural network, and the second brain among the information included in the second input information. Includes the neural network combined with a fully connected neural network into which information other than waveform information is input.
The fully connected neural network outputs the output information and outputs the output information.
The control unit inputs the second brain waveform information included in the second input information to the convolutional neural network, and the second of the output from the convolutional neural network and the information included in the second input information. Information other than the brain waveform information is input to the fully connected neural network, and the sleeping state is determined according to the output information output from the fully connected neural network.
The determination device according to any one of claims 1 to 5. The control unit corrects the output information output from the model based on the past sleep state and the state transition probability of the sleep state from the past to the present, and after the correction. The sleep state is determined according to the output information.
The determination device according to any one of claims 1 to 6. The kernel size of the neural network is a value determined according to a predetermined frequency band.
The determination device according to any one of claims 1 to 7. A light irradiation device including the determination device according to any one of claims 1 to 8 and an endoscope.
The endoscope is
An irradiation unit that irradiates the subject's brain with light of a predetermined wavelength,
An imaging unit that images the subject's brain,
Light irradiation control unit and
With
The light irradiation control unit
The irradiation unit is irradiated with the light according to the determination result by the determination device.
Light irradiation device. A program that allows a computer to determine the sleep state of a subject's brain based on a machine learning model that includes a neural network.
The model is
The first input information including the first brain waveform information indicating the waveform of the first brain wave of the subject and the first time information indicating the first time associated with the first brain waveform information, and the first input. The correspondence relationship with the first sleep state information indicating the first sleep state, which is the sleep state associated with the information, has been learned, and the second brain waveform information indicating the waveform of the second brain wave of the subject and the said When the second input information including the second time information indicating the second time associated with the second brain waveform information is input, the sleep state associated with the input second input information is estimated. , Outputs the information indicating the estimated result as output information,
The program
The second input information is input to the model,
The sleep state is determined according to the output information output from the model to which the second input information is input.
program.

Images