JP6952299B2 - Sleep depth judgment system, sleep depth judgment device and sleep depth judgment method - Google Patents

Description

The present invention relates to a sleep depth determination system, a sleep depth determination device, and a sleep depth determination method for determining sleep depth based on a moving image of a person during sleep.

Sleep is essential for humans and is important for maintaining brain and physical function. In recent years, as health consciousness has increased, there is a need to improve sleep and strive for health management by grasping one's daily sleep state and managing daily sleep in ordinary households.

A person's sleep state is not uniform throughout the night, and is broadly divided into REM sleep and non-REM sleep. Non-rem sleep has four stages of stages 1 to 4, and among non-rem sleep, light sleep (Shallow sleep) in which sleep is light in stages 1 and 2 and slow wave in which sleep is deep in stages 3 to 4 It is divided into sleep (Slow wave sleep).

When a person falls asleep from awake state, they generally transition to deeper REM sleep, light sleep, slow-wave sleep and sleep, and then to lighter sleep, REM sleep and lighter sleep. This cycle is usually repeated in about 90 minutes. However, immediately after falling asleep from the awake state, the REM sleep stage is generally skipped, and the sleep shifts to light sleep and then to slow-wave sleep. In addition, slow-wave sleep is more common at the beginning of sleep, while REM sleep is more frequent as the morning awakens. By examining the change in sleep depth during sleep, it is possible to grasp the sleep state of a person.

Conventionally, a polysomnography (PSG) method is known as a method for estimating sleep depth. The sleep polygraph method is a method of evaluating the state of sleep by simultaneously measuring, for example, brain waves, eye movements, jaw myoelectricity, electrocardiography, etc., and analyzing these measurement data to determine the sleep depth. Since there are many types of data and the measurement accuracy is high, the reliability of the evaluation is high. However, the polysomnography method is not suitable for daily use at home like a health device because the device is large-scale and can be used only in a place equipped with measuring equipment such as a hospital. In addition, in the polysomnography method, since multiple electrodes must be attached to a person, the burden and stress on the person are large, and the feeling of restraint and contact often interferes with sleep, so accurate sleep depth is measured. There is also the problem that it is difficult to do. Furthermore, since the polysomnography method cannot accurately evaluate the sleep depth without the help of a specialist such as a doctor, it cannot be said that only a device is required.

Therefore, a technique has been proposed that can accurately determine the sleep depth while being unrestrained and non-contact (see, for example, Patent Document 1). In Patent Document 1, a moving image of a person during sleep is photographed by a video camera, and data on body movement (body movement) during sleeping is analyzed based on the captured moving image to determine the sleep depth. It is disclosed to do. In the technique described in Patent Document 1, since the person is not restrained by the sensors and the sensors do not come into contact with each other, the sleep depth can be determined without burden or stress. Moreover, since the device is not large-scale, it can be used in ordinary households.

On the other hand, in Patent Document 2, the body movement of a sleeping person is detected by a Doppler radar sensor, a camera, or the like, and if the number of body movements is less than a predetermined value for a specified period, it is determined that the person has fallen asleep, or the number of times of body movements. It is disclosed that it is determined that a person has fallen asleep when the number of times is less than or equal to a predetermined value and is counted a predetermined number of times.

Further, in Patent Document 3, the depth information of the infrared projection position with respect to the user (sleeping person) is detected by the infrared camera, and the posture information (static) such as the sleeping phase and position of the user during sleep is detected based on the depth information. Information) and information on the amount of activity (dynamic information) such as head movement and turning over are calculated and judged. When calculating the activity amount information (dynamic information), it is disclosed that the determination is made by comparing with the posture information (static information) calculated in the past image frame.

Further, in Patent Document 4, the pulse interval detecting means for detecting the pulse interval of the subject and the imaging means for capturing the image of the subject are provided, and the pulse wave interval is detected based on the time change of the image captured by the imaging means. .. A camera is used as the imaging means, an image on the futon is taken, and the image difference calculation unit obtains a time difference image of the image. Since the futon is slightly displaced in synchronization with the subject's respiration and heartbeat, the value corresponding to the amount of displacement can be obtained by taking the difference for each frame in the time direction and calculating the total for each pixel. Similarly, if body movement occurs, a corresponding change can be obtained. The difference image is a superposition of these respirations and body movements. Therefore, it is disclosed that the body movement measuring unit separates and extracts the body movement data by a filter in a band suitable for the body movement measurement.

Japanese Unexamined Patent Publication No. 2012-375 Japanese Unexamined Patent Publication No. 2012-159250 Japanese Unexamined Patent Publication No. 2014-151120 Japanese Unexamined Patent Publication No. 2008-6005

However, the technique described in Patent Document 1 can determine the sleep depth in three stages of wakefulness, light sleep (including REM sleep), and slow wave sleep, but cannot determine REM sleep. There is.

Further, in the technique described in Patent Document 2, it is only estimated that the sleep state is close to awakening when there is a lot of body movement, and deep sleep when there is little body movement, and REM sleep or non-REM sleep (shallow). It cannot be determined in detail, such as (divided into sleep and slow-wave sleep).

In addition, the technique described in Patent Document 3 monitors the movement of the head, the presence or absence of turning over, the sleeping phase / posture, the sleeping position, snoring, the presence or absence of sleep-talking, and the sleep state is REM sleep or non-REM sleep. It is not possible to make a detailed judgment such as (divided into light sleep and slow wave sleep).

Further, in the technique described in Patent Document 4, the sleep state is detected only by the pulse interval detecting means, and the moving image of a person during sleep is not used.

The present invention has been made by paying attention to the above-mentioned problems, and it is possible to determine the sleep depth without restraint and non-contact, and the sleep depth can be determined as wakefulness, REM sleep, light sleep and slow wave sleep. It is an object of the present invention to provide a sleep depth determination system, a sleep depth determination device, and a sleep depth determination method capable of determining in four stages.

The object of the present invention includes a photographing means capable of capturing a moving image of a person during sleep, and a sleep depth determination device for determining the sleep depth based on the moving image of the person during sleep. The device includes a processor and a computer including a memory under the control of the processor, and the computer converts the moving image into a plurality of frame intervals constituting the moving image, and the plurality of frames. A density value between a gray conversion unit that converts an image into a grayscale image when the image is not a grayscale image and a frame image that has undergone difference processing by performing difference processing on the plurality of grayscale frame images at predetermined frame intervals. The difference processing unit that detects the changed pixel and the number of pixels whose density value has changed detected by the difference processing are counted, and the amount of body movement, the duration of body movement, and the stillness are counted for each epoch consisting of a predetermined time from the counting result. It functions as a body movement data calculation unit that calculates the duration, and a determination unit that determines the sleep depth of a person for each epoch based on the calculated body movement amount, the body movement duration, and the rest duration. Achieved by the sleep depth determination system.

In the sleep depth determination system of the present invention, the body movement data calculation unit performs moving averaging processing on the body movement amount, the body movement duration, and the resting duration for each epoch, and the determination unit performs the moving averaging. It is preferable to determine the sleep depth of a person for each epoch based on the processed amount of body movement, the duration of body movement, and the duration of rest.

Further, in the sleep depth determination system of the present invention, the frame interval of the difference processing is not particularly limited, and a frame interval capable of accurately detecting a fast-moving thin body movement and a slow-moving coarse body movement can be obtained. Any of the frame intervals that can be detected with high accuracy can be set arbitrarily. For example, the frame interval of the difference processing is set from a range of 0.2 seconds or more and 4.0 seconds or less, and the frame interval is set to 0.2 seconds or more and 1.0 for accurately detecting a fast-moving thin body movement. It is preferable to set from a range of seconds or less, and it is preferable to set from a range of 2.0 seconds or more and 4.0 seconds or less in order to accurately detect slow-moving rough body movements.

Further, in the sleep depth determination system of the present invention, it is preferable that the determination unit determines the sleep depth of a person for each epoch by machine learning.

Further, in the sleep depth determination system of the present invention, the determination unit classifies the sleep depth of a person for each epoch into either a wakefulness state or a sleep state, and then classifies the state into a sleep state. It is preferable to classify it as either REM sleep, light sleep or slow wave sleep.

Further, in the sleep depth determination system of the present invention, it is preferable that the determination unit uses linear discriminant analysis for classification of either the awake state or the sleep state. Further, it is preferable that the determination unit uses a non-linear discriminant analysis such as a support vector machine for classification of any of REM sleep, light sleep and slow wave sleep. However, the determination unit can also use linear discriminant analysis for any of the classifications of REM sleep, light sleep and slow wave sleep.

Further, in the sleep depth determination system of the present invention, the noise that the computer performs at least one expansion process and a contraction process more than the number of expansion processes on the pixel whose density value is changed detected by the difference process. It is preferable that it further functions as a processing unit.

Further, in the sleep depth determination system of the present invention, the difference processing unit performs difference processing at predetermined first frame intervals by performing difference processing on the plurality of frame images at predetermined first frame intervals. By detecting pixels whose density values have changed between images and performing difference processing at a predetermined second frame interval longer than the first frame interval, the difference processing is performed between the frame images at the second frame interval. The pixel whose density value has changed is detected, and the body movement data calculation unit determines the number of pixels whose density value has changed, which is detected by the difference processing at the first frame interval and the difference processing at the second frame interval. Each count is performed, and the amount of body movement, the duration of body movement, and the duration of rest are calculated for each epoch consisting of a predetermined time from each counting result, and the determination unit counts the pixels by the difference processing at the first frame interval. The sleep depth of a person for each epoch is determined based on the result and the pixel count result obtained by the difference processing at the second frame interval, respectively, based on the amount of body movement, the duration of body movement, and the duration of rest. Is preferable.

The object of the present invention is a sleep depth determination device that determines the sleep depth based on a moving image of a person during sleep, the computer comprising a processor and a computer including a memory under the control of the processor. An image conversion unit that converts the moving image into a plurality of frame intervals constituting the moving image, a gray conversion unit that converts the moving image into a grayscale image when the plurality of frame images are not grayscale images, and a grayscale unit. By performing difference processing on the plurality of frame images at predetermined frame intervals, the difference processing unit that detects pixels whose density values have changed between the frame images that have undergone difference processing and the density values detected by the difference processing change. A body movement data calculation unit that counts the number of pixels and calculates the amount of body movement, the duration of body movement, and the duration of rest for each epoch consisting of a predetermined time from the counting result, and the calculated amount of body movement and the duration of body movement. It is also achieved by a sleep depth determination device that functions as a determination unit that determines a person's sleep depth for each epoch based on time and the rest duration.

The object of the present invention is a sleep depth determination method for determining a sleep depth based on a moving image of a person during sleep, and an image for converting the moving image into a plurality of frame intervals constituting the moving image. A difference is obtained by performing a conversion step, a gray conversion step of converting the plurality of frame images into a grayscale image when the plurality of frame images are not grayscale images, and a difference processing of the plurality of grayscale frame images at predetermined frame intervals. The difference processing step of detecting the pixels whose density value has changed between the processed frame images and the number of pixels whose density value has changed detected by the difference processing are counted, and each epoch consisting of a predetermined time from the counting result is counted. , The sleep depth of the person for each epoch based on the body movement data calculation step for calculating the body movement amount, the body movement duration and the rest duration, and the calculated body movement amount, the body movement duration and the rest duration. It is also achieved by the determination step of determining and the sleep depth determination method of executing.

An object of the present invention is an image conversion process for converting a moving image into a plurality of frame intervals constituting the moving image on a computer including a processor and a computer under the control of the processor, and the plurality of frame images. By performing a gray conversion process of converting to a grayscale image when is not a grayscale image and a difference process of the plurality of grayscale frame images at predetermined frame intervals, the density between the frame images subjected to the difference process is performed. The difference processing that detects the pixel whose value has changed and the number of pixels whose density value has changed detected by the difference processing are counted, and the amount of body movement, the duration of body movement, and the body movement duration are counted for each epoch consisting of a predetermined time from the counting result. A body movement data calculation process for calculating the rest duration and a determination process for determining a person's sleep depth for each epoch based on the calculated body movement amount, the body movement duration, and the rest duration are executed. It is also achieved by a computer program that lets you.

According to the present invention, the sleep depth of a person can be determined unrestrained and non-contact, and can be determined in four stages of wakefulness, REM sleep, light sleep, and slow wave sleep.

It is explanatory drawing which shows the schematic structure of the sleep depth determination system of one Embodiment of this invention. It is a block diagram which shows the schematic structure of the sleep depth determination system of one Embodiment of this invention. It is a flow chart which shows the flow of the sleep depth determination method of one Embodiment of this invention. It is explanatory drawing about the difference processing of a frame image. It is a graph which compared the body movement data at the frame interval t2 at the time of sleep of a person, and the actual measurement data of the sleep depth by the polysomnography method. The amount of body movement, body movement duration, and rest duration of each epoch during human sleep calculated from the time-series data of body movement in FIG. 5 are wakefulness (WAKE), REM sleep (REM), and light sleep (LIGHT). ) And slow wave sleep (DEEP). Using the time-series data of the body movement of FIG. 5, the concordance rate between the sleep depth determination result determined by the sleep depth determination device according to the embodiment of the present invention and the measured sleep depth data by the polysomnography method is shown. It is a table. It is a graph which compared the body movement data at the frame interval t3 at the time of sleep of a person, and the actual measurement data of the sleep depth by the polysomnography method. The amount of body movement, body movement duration, and rest duration of each epoch during human sleep calculated from the time-series data of body movement in FIG. 8 are wakefulness (WAKE), REM sleep (REM), and light sleep (LIGHT). ) And slow wave sleep (DEEP). Using the time-series data of the body movement of FIG. 8, the concordance rate between the sleep depth determination result determined by the sleep depth determination device according to the embodiment of the present invention and the measured sleep depth data by the polysomnography method is shown. It is a table. It is a flow chart which shows the flow of the sleep depth determination method of another embodiment of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. The sleep depth determination system according to the present invention captures a moving image of a person during sleep by a photographing means such as an infrared camera capable of photographing even in a dark bedroom, analyzes the moving image, and moves the head, turns over, and limbs. The time-series data of the body movement during sleep such as the movement of the body is acquired, and the sleep depth of the person is determined based on the time-series data of the body movement.

FIG. 1 shows a schematic configuration of a sleep depth determination system 10 according to an embodiment of the present invention. As shown in FIG. 1, the sleep depth determination system 10 includes a photographing means 1 capable of capturing a moving image of a person during sleep, and a sleep depth determination device that determines the sleep depth based on the moving image of a person during sleep. 2 and.

The photographing means 1 is connected to the sleep depth determination device 2 (computer 3) via a communication interface (not shown), captures a moving image of a person during sleep, and captures the captured moving image of the sleep depth determining device 2. Output to computer 3. The output moving image is stored in the computer 3 of the sleep depth determination device 2. As the photographing means 1, various means such as a video camera, a webcam, and a mobile phone having a camera function (for example, a smartphone) can be used as long as they can capture a moving image of a person during sleep. It is preferable to have an infrared photographing function capable of photographing even at night or in the dark. The frame rate of the moving image is preferably at least 1 fps or more, preferably 2 fps or more, but during sleep, there is little movement due to human body movement, and in order to detect human body movement, 1 fps or more and 3 fps or more. The following frame rates are sufficient, and in this embodiment, they are 2 fps (0.5 seconds at frame intervals).

As shown in FIG. 2, the sleep depth determination device 2 includes a computer 3 that executes various arithmetic processes, an input means 4 such as a keyboard, a mouse, and a touch panel device, and a display means 5 such as a liquid crystal display. The input means 4 and the display means 5 may be devices separate from the computer 3, but may be one unit. The sleep depth determination device 2 can be configured by, for example, a personal computer (desktop, notebook, smartphone, portable information terminal (for example, tablet PC), etc.) and is not particularly limited.

The computer 3 has a processor (CPU), a memory (ROM, RAM) under the control of the CPU, an auxiliary storage device (HDD), and other devices (input means 4, display means 5, shooting means 1) as a hardware configuration. It has each interface (input / output interface, image output interface, communication interface) with and, and an interface for connecting to the Internet. The HDD may be included in the computer 3 or may be placed as an external device of the computer 3. The CPU functions as an arithmetic processing unit and a control device, and executes various arithmetic processing by reading various computer programs stored in ROM or HDD into RAM and executing them. The moving image taken by the photographing means 1 is transmitted to the computer 3 of the sleep depth determination device 2 via a computer-readable portable recording medium (flash memory, SD card, CD-ROM, etc.) on which the moving image is recorded. May be supplied to.

In addition to the storage unit 30, the computer 3 has a moving image acquisition unit 31, an image conversion unit 32, a gray conversion unit 33, a difference processing unit 34, a noise processing unit 35, and a body movement data calculation unit as functional blocks. 36 and a determination unit 37 are provided. Each functional block is realized by software when the CPU executes a computer program.

Hereinafter, the sleep depth determination method executed by each functional block of the computer 3 based on the computer program related to the sleep depth determination will be described with reference to FIGS. 2 and 3.

The moving image acquisition unit 31 acquires a moving image of a person during sleep output from the photographing means 1 (moving image acquisition step ST1). The image acquisition unit 31 may store the acquired moving images in the storage unit 30 in chronological order.

Next, the image conversion unit 32 performs a process of converting the acquired moving image into a plurality of frame images (still images) constituting the moving image (image conversion step ST2). When the frame rate of the moving image captured by the photographing means 1 is larger than 3 fps, resampling is performed at 1 fps or more and 3 fps or less, preferably 2 fps (frame interval t1 is 0.5 seconds). As a result, the amount of information stored in the computer 3 of the sleep depth determination device 2 can be reduced.

The gray conversion unit 33 performs a process of converting all frame images into grayscale and converting them into grayscale images (gray conversion step ST3). As a result, it is possible to detect a person's body movement during sleep from a moving image of the person during sleep without being affected by a background object such as a bed.

Specifically, in the present invention, which will be described in detail later, the body movement during sleep of a person is detected by differential processing of a plurality of frame images (still images) constituting the moving image. That is, the image size of the frame image is composed of n pixels in the x direction (horizontal) and m pixels in the y direction (vertical), for a total of n × m pixels (for example, 640 horizontal × 480 vertical). When a body movement occurs, the density value (brightness) of the pixel at the original coordinate position (x1, y1) of the movement decreases between the frame image before the movement and the frame image after the movement, and the coordinates of the destination of the movement. The density value (brightness) of the pixel at the position (x2, y2) increases. Therefore, the body movement can be detected by taking the difference between a plurality of consecutive frame images and detecting the pixels whose density value (luminance) has changed. Here, in each frame image, when a person and a background object such as a bed appear in the same color, the body movement is detected because the pixel density does not change even if the body movement occurs in the portion overlapping the background object. Can not do it. However, by converting the moving image into a grayscale image and making the person and the background object appear in different colors (density), it is possible to detect the change in the pixel density value due to body movement even in the part that overlaps the background object. Therefore, it is possible to detect body movement.

The method for converting to a grayscale image is not particularly limited, and a known method can be used. For conversion to a grayscale image, it is preferable to set 256 gradations. The gray conversion unit 33 may store the grayscaled moving images in the storage unit 30 in chronological order.

If a device having a function of shooting a moving image in gray scale is used as the shooting means 1, the gray conversion unit 33 and the gray conversion step ST3 can be omitted.

Next, the difference processing unit 34 performs a difference process on all the frame images, and performs a process of detecting pixels whose density value has changed with the frame image from which the difference has been taken (difference processing step ST4). Here, the frame interval between the two frame images for which the difference is taken is not particularly limited, but in the present embodiment, as shown in FIG. 4, two consecutive frame images (frames) adjacent to each other are used. Instead of a frame image having an interval t1 (0.5 seconds in this embodiment), two frame images having a frame interval t2 longer than the frame interval t1 are used. That is, for example, the frame image that takes the difference from the Nth frame image is not the N-1th frame image of the previous one, but the NAth frame of a (= t2 / t1) before. It becomes an image. The frame interval t2 can be set, for example, from a range of 2.0 seconds or more and 4.0 seconds or less, preferably from a range of 2.5 seconds or more and 3.5 seconds or less. Then it is 3.0 seconds. Therefore, in the present embodiment, the Nth frame image takes a difference from the N-6th frame image 6 sheets before, and the N + 1th frame image is the N-5th frame. A difference is taken from the image, and the N + 2nd frame image is made a difference from the N-4th frame image.

In this way, the reason why the difference processing of the frame image is performed at the frame interval t2 with a long time interval is that the body movement during sleep of a person is mainly a large rough body movement with slow movement such as turning over and a movement of the limbs. There are small body movements that move quickly. Here, for slow-moving coarse body movements, if the frame interval is short, the difference due to body movement is less likely to appear in the comparison of the two frame images (Nth and N-1th) to be subjected to difference processing, and the frame image. Since the number of pixels whose density value changes between them is small, this can be obtained, but the detection accuracy is not good. On the other hand, when the frame interval is long, a large difference due to body movement appears in the comparison of the two frame images (Nth and NA) to be subjected to the difference processing, and the density value changes between the frame images. Since the number of pixels to be processed is large, it is possible to detect this with high accuracy. It should be noted that, for fast-moving body movements, the shorter the frame interval between the two frame images to be subjected to the difference processing, the more accurately the detection can be performed, but the frame interval t2 is 2.0 seconds or more and 4.0 seconds or less. Within the range of, it is possible to detect the slender movement even at the frame interval t2.

Next, the noise processing unit 35 performs expansion processing and contraction processing after expansion processing on the pixels whose density value is changed, which is detected by the difference processing (noise processing step ST5). In the present embodiment, after the expansion treatment is performed at least once, the contraction treatment is performed a number of times larger than the number of expansion treatments. For example, the expansion treatment is performed once and the contraction treatment is performed twice. If the contraction process is performed only once, for example, the body movement detected by the difference process may be removed together with the noise. However, after the expansion process, by performing the contraction process more times than the expansion process, the density value change of each continuous / combined pixel detected as a body movement by the difference process remains as it is even after the noise process, and the difference process results in the change. The change in the density value of each discontinuous / dispersed pixel detected as noise can be removed after the noise processing. Therefore, only noise can be effectively removed. The expansion treatment and the contraction treatment can be performed by a known method. The noise processing unit 35 may store the image of the pixel whose density value has changed detected by the noise processing difference processing in the storage unit 30 in chronological order.

When the noise generated in the captured moving image is small, such as when a high-performance camera is used as the photographing means 1, the noise processing unit 35 and the noise processing step ST5 can be omitted.

Next, the body movement data calculation unit 36 counts the number of pixels whose density values have changed between the frame images for which the differences have been taken and the frame images for which the difference processing has been performed by the above method. Performs processing to measure time-series data of body movements during sleep. The body movement data calculation unit 36 may store the measured time-series data of the body movement in the storage unit 30.

FIG. 5 shows body movement data (lower row) at a frame interval t2 during sleep of a person. In FIG. 5, the horizontal axis is the time [sec] and the vertical axis is the number of pixels [pixel] counted as the body movement. The larger the number of pixels, the larger the body movement. Note that FIG. 5 also shows actual measurement data (upper row) of the sleep depth determined by the polysomnography method for comparison with the sleep depth.

As described above, in the present invention, in addition to the slender movements that appear during sleep, the coarse body movements are accurately measured, and the details will be described later. It is possible to discriminate into four stages of slow-wave sleep.

Then, the body movement data calculation unit 36 performs a process of calculating the body movement amount, the body movement duration, and the stationary duration for each epoch consisting of the predetermined time T from the measured time series data of the body movement (body movement). Data calculation process ST6).

Specifically, the amount of body movement is the total number of pixels whose density value has changed during one epoch. For example, assuming that the predetermined time T of one epoch is 30 seconds, if the moving image obtained by the photographing means 1 is, for example, 2 fps (0.5 seconds / frame), 60 images (T / t1) are included in one epoch. Frame images are included, and each frame image has a total of n × m (for example, 640 horizontal × 480 vertical) pixels. Since the pixel whose density value has changed between the two frame images for which the difference is taken is counted as the size of the body movement, the amount of body movement is 60 (T / t1) included in one epoch. It is represented by the total number of pixels counted in the frame image difference processing.

The body movement duration is the total number of frame images in which body movement is detected in one epoch, and this is converted into time. That is, for example, assuming that the predetermined time T of one epoch is 30 seconds, if the moving image obtained by the photographing means 1 is, for example, 2 fps (0.5 seconds / frame), 60 images (T /) per epoch. Since the frame image of t1) is included, the body movement duration is the result of the difference processing of the 60 frames (T / t1) included in one epoch, and the body in one epoch (60 frame images). The number of frame images in which motion is detected (the number of pixels whose density value has changed is not 0) is calculated, and it is expressed by the time obtained by multiplying the calculated number of frame images by the frame interval t1 (for example, 0.5 seconds). Is done.

The rest duration is the duration of how long the rest has continued since the last detection of body movement in any epoch. When calculating the duration of rest in an epoch, the number of epochs before the epoch and the last body movement detected (the number of pixels whose density value has changed is not 0) is before the epoch. Detects whether it is an epoch. For example, if body movement is detected in b (for example, 6) previous epochs, no body movement occurs in b-1 (for example, 5) epochs in between. Therefore, the stationary duration is expressed as (b-1) × T (predetermined time of one epoch).

Based on the above-mentioned calculation method, the amount of body movement, the duration of body movement, and the duration of rest are calculated from the body movement data for all epochs.

Further, in the present embodiment, the body movement data calculation unit 36 has the body movement amount, body movement duration, and rest duration of all the epochs, respectively, regarding the body movement amount, body movement duration, and rest of the latest c epochs. A moving average process for calculating a simple average value with the duration is performed (moving average processing step ST7). For example, in the M-th epoch, the average value of the Mc-th epoch to the M-th epoch is obtained, and in the M + 1-th epoch, the average value of the M + 1-c-th epoch to the M + 1-th epoch is obtained. In this embodiment, a moving average process of 15 minutes (30 epochs) is performed. In this way, the moving average values of the body movement amount, body movement duration, and rest duration of each epoch were calculated based on the finding that the sleep depth in each epoch depends on the body movement information in the previous epochs. By considering the body movement information of the previous epochs, it is possible to accurately determine the sleep depth in each epoch.

For example, taking the amount of body movement as an example, in case A, the amount of body movement of 3000 was measured in a certain epoch, but there was no body movement until 15 minutes before the epoch, whereas in case B, there was no body movement. If 3000 body movements are measured in a certain epoch and 27,000 body movements are in the epoch 15 minutes before the epoch, the body movements (without moving average processing) are the same in the epochs of cases A and B. Since it is 3000, the sleep depth looks the same. However, in the amount of body movement (with moving average processing), 3000/60 epoch (15 minutes) = 50 in case A, whereas (3000 + 27000) / 60 = 500 in case B. Therefore, the epochs of cases A and B can be classified as different sleep depths. In this way, in the epoch that had body movement in the previous epoch, the body movement amount without moving average processing can be used for the classification of sleep depth, and in the epoch without body movement in the previous epoch, the body movement amount with moving average processing. By including the body movement information of the previous epoch, the accuracy of sleep depth classification can be improved.

The target range of the moving average processing is not necessarily limited to 15 minutes, and can be preferably set, for example, between 10 minutes and 20 minutes.

Next, the determination unit 37 determines the amount of body movement for each epoch calculated from the body movement data (without moving averaging processing), the body movement duration (without moving averaging processing), the resting duration (without moving averaging processing), and further. , Body movement amount (with moving average processing), body movement duration (with moving average processing), and rest duration (with moving average processing), the sleep depth during sleep of a person is in the awake state (with moving average processing). WAKE), REM sleep (REM), light sleep (LIGHT), and slow wave sleep (DEEP) are classified (determination step ST8).

Here, in a person's sleep, in the wakefulness state (WAKE), large-movement rough body movements frequently occur, whereas as the sleep becomes deeper, that is, REM sleep, light sleep, and slow-wave sleep continue. As the body movements appear less frequently. Therefore, there is a remarkable difference in the amount of body movement between the awake state (WAKE) and the sleep state (SLEEP), and the amount of body movement in the awake state (WAKE) is larger than that in the sleep state (SLEEP). Therefore, among the parameters calculated above, by paying attention to the amount of body movement, the sleep depth can be first classified into a wakefulness state (WAKE) and a sleep state (SLEEP).

Further, among the sleep states (SLEEP), in REM sleep (REM), small body movements are likely to occur, while coarse body movements hardly occur. In addition, since REM sleep (REM) itself does not continue for a long time, the rest duration does not take a long value. In light sleep (LIGHT), both slender and coarse body movements occur. In slow-wave sleep (DEEP), neither thin body movement nor coarse body movement occurs. Therefore, first, slow wave sleep (DEEP) has a shorter body movement duration and a smaller body movement amount than REM sleep (REM) and light sleep (LIGHT). Therefore, among the parameters calculated above, the body movement duration By paying attention to the amount of body movement and the amount of body movement, it can be classified into REM sleep (REM) and light sleep (LIGHT). In addition, since light sleep (LIGHT) has a larger amount of body movement than REM sleep (REM) and slow wave sleep (DEEP), by focusing on the amount of body movement, REM sleep (REM) and slow wave sleep (DEEP) can be defined. Can be categorized. Finally, REM sleep (REM) has a shorter rest duration than light sleep (LIGHT) and slow wave sleep (DEEP), and has a shorter body movement duration than light sleep (LIGHT). By focusing on the duration of body movement and body movement, it can be classified into light sleep (LIGHT) and slow wave sleep (DEEP).

6 shows, (a) amount of body movement (without moving average processing), (b) duration of body movement (without moving average processing), and (c) duration of rest during sleep of a person calculated from the body movement data of FIG. Each parameter of time (without moving average processing), (d) body movement amount (with moving average processing), (e) body movement duration (with moving average processing) and (f) rest duration (with moving average processing) , WAKE, REM sleep (REM), light sleep (LIGHT) and slow wave sleep (DEEP), respectively.

In FIG. 6, it can be seen that the amount of body movement (with moving average processing) is distributed more in the awake state (WAKE) than in the others, and contributes to the classification of the awake state (WAKE) and the others. Light sleep (LIGHT) is more distributed than REM sleep (REM) and slow wave sleep (DEEP), and light sleep (LIGHT), REM sleep (REM), and slow wave sleep are distributed in the amount of body movement (without moving average treatment). It can be read that it contributes to the classification of (DEEP). The duration of body movement (without moving averaging treatment) and the duration of body movement (with moving averaging treatment) are distributed smaller in slow-wave sleep (DEEP) than in REM sleep (REM) and light sleep (LIGHT). It can be seen that it contributes to the classification of sleep (DEEP), REM sleep (REM) and light sleep (LIGHT). Rest duration (without moving average treatment) and rest duration (with moving average treatment) are distributed smaller in REM sleep (REM) than in slow wave sleep (DEEP) and light sleep (LIGHT), and REM sleep (REM). ) And slow wave sleep (DEEP) and light sleep (LIGHT). In addition, regarding the stationary duration (without moving average processing) and the stationary duration (with moving average processing), the awake state (WAKE) is distributed smaller than the others, which contributes to the classification of the awake state (WAKE) and others. You can also read what you are doing.

As described above, in the present embodiment, the amount of body movement (without moving averaging processing), the duration of body movement (without moving averaging processing), the duration of rest (without moving averaging processing), the amount of body movement (with moving averaging processing), and the body movement. Considering that the duration (with moving average processing) and the resting duration (with moving average processing) are greatly related to the sleep depth, from the body movement data during human sleep, each epoch is related to the sleep depth. By extracting the above parameters related to strong sexual movement, it is possible to discriminate between four stages of wakefulness (WAKE), REM sleep (REM), light sleep (LIGHT), and slow wave sleep (DEEP) for each epoch. It has become. Hereinafter, a method for determining the sleep depth will be specifically described.

In the present embodiment, the determination unit 37 first determines whether the sleep depth of a person during sleep is classified into an awake state (WAKE) or a sleep state (SLEEP), and the determination unit 37 may be in the sleep state (SLEEP). For example, it is determined whether the sleep is classified into REM sleep (REM), light sleep (LIGHT), or slow wave sleep (DEEP).

To determine the wakefulness state (WAKE) and the sleep state (SLEEP), the amount of body movement (with moving average processing) is used as a parameter, and for example, whether or not the amount of body movement (with moving average processing) exceeds a predetermined threshold. Each epoch is classified into either a wakefulness state (WAKE) or a sleep state (SLEEP) (ST80). As shown in FIG. 6 (d), this threshold has the parameters of the amount of body movement (with moving average processing) confirmed in advance: wakefulness (WAKE), REM sleep (REM), light sleep (LIGHT), and gradual sleep (LIGHT). It can be set from teacher data showing how it is distributed in wave sleep (DEEP). If the amount of body movement (with moving average processing) exceeds the threshold value, it is determined to be in the awake state (WAKE) (ST81), and if it is below the threshold value, it is determined to be in the sleep state (SLEEP) (ST82). In this embodiment, linear discrimination, which is one of machine learning, is used, and each epoch is classified into two discrimination groups, awake state (WAKE) and a sleep state (SLEEP), with the amount of body movement (with moving average processing) as a parameter. The threshold is calculated and classified. Regarding the classification of awake state (WAKE) and sleep state (SLEEP), in addition to the amount of body movement (with moving average processing), the rest duration (without moving average processing) or rest duration (with moving average processing) is a parameter. Even so, each epoch can be classified according to whether or not it exceeds a predetermined threshold. Furthermore, each epoch is also performed by machine learning (linear discrimination, etc.) using at least two parameters of body movement amount (with moving average processing), rest duration (without moving average processing), and rest duration (with moving average processing). It can be classified into two discriminant groups, awake state (WAKE) and sleep state (SLEEP).

Next, the determination unit 37 uses each epoch classified into the sleep state (SLEEP) as parameters of the amount of body movement, the duration of body movement, and the duration of rest, and uses REM sleep (REM), light sleep (LIGHT), and light sleep (LIGHT). It is classified as one of slow-wave sleep (DEEP) (ST83). The parameters include body movement amount (without moving averaging processing), body movement duration (without moving averaging processing), resting duration (without moving averaging processing), body movement amount (with moving averaging processing), and body movement duration (without moving averaging processing). It is preferable to use all six of (with moving averaging processing) and static duration (with moving averaging processing) because it is possible to classify accurately, but body movement amount (without moving averaging processing) and body movement duration (moving averaging). Only three may be used: no treatment) and stationary duration (without moving averaging), body movement (with moving averaging), body movement duration (with moving averaging) and quiescent duration (moving averaging). Only three of (with processing) may be used.

In this embodiment, the amount of body movement (without moving average processing), the duration of body movement (without moving average processing), the duration of rest (without moving average processing), the amount of body movement (with moving average processing), and the duration of body movement (movement). Three discriminant groups of slow-wave sleep (DEEP), light sleep (LIGHT), and REM sleep (REM) for each epoch by machine learning using 6 parameters of averaging (with averaging) and static duration (with moving average). It is classified into (ST84 to ST86). SVM (support vector machine) can be preferably used for machine learning, but existing classification methods such as linear discrimination, Mahalanobis distance, and quadratic discrimination can also be used.

For each epoch classified into the sleep state (SLEEP), the amount of body movement (without moving average processing), the duration of body movement (without moving average processing), the duration of rest (without moving average processing), and the amount of body movement (moving average). Using all or part of the six parameters of body movement duration (moving average treatment) and resting duration (moving average treatment), first, REM sleep (REM) and light sleep (LIGHT). After classifying into two discriminant groups, slow wave sleep (DEEP) and slow wave sleep (DEEP) by machine learning etc., each epoch classified into REM sleep (REM) and light sleep (LIGHT) is classified into REM sleep (REM) and light sleep (REM) and light sleep. (LIGHT) may be classified into two discrimination groups by machine learning or the like.

The sleep depth determination data thus obtained is, for example, generated as a locus of the time transition of the sleep depth and displayed on the display means 5.

7 (a) and 7 (b) show a concordance table between the determination result of the sleep depth of the person determined from the body movement data of FIG. 5 and the measured data of the sleep depth measured by the polysomnography method. Referring to FIG. 7 (a), 875 (99.8%) of the 877 epochs determined to be in sleep state (SLEEP) by the sleep polygraph method are in sleep state (SLEEP) by the sleep depth determination device 2 of the present invention. ), And 29 (65.9%) of the 44 epochs determined to be wakeful (WAKE) by the sleep polygraph method are also wakeful (65.9%) by the sleep depth determination device 2 of the present invention. WAKE). In the present embodiment, 438 epochs, which is half of the 875 epochs determined to be in the sleep state (SLEEP) by the sleep depth determination device 2 of the present invention, are used as teacher data for machine learning, and the other half, 437 epochs. Was used as the pull judgment data. Referring to FIG. 7B, the 437 epochs determined to be in a sleep state (SLEEP) by the sleep depth determination device 2 of the present invention were further determined to be slow wave sleep (DEEP) by the sleep polygraph method 110. Of the epochs, 73 (66.4%) were determined to be slow-wave sleep (DEEP) by the sleep depth determination device 2 of the present invention, and were determined to be light sleep (LIGHT) by the sleep polygraph method. Of the 262 epochs, 226 (86.3%) were determined to be light sleep (LIGHT) by the sleep depth determination device 2 of the present invention, and were determined to be REM sleep (REM) by the sleep polygraph method. Of the 67 epochs, 53 (79.1%) were determined to be REM sleep (REM) by the sleep depth determination device 2 of the present invention. As described above, according to the sleep depth determination device 2 of the present invention, the sleep depth is set to four stages of wakefulness (WAKE), REM sleep (REM), light sleep (LIGHT) and slow wave sleep (DEEP) with generally good accuracy. It can be said that it can be classified into.

As described above, according to the sleep depth determination system 10, the sleep depth determination device 2, and the sleep depth determination method of the present invention, body movement data is measured from a moving image of a person during sleep taken by the photographing means 1, and the body movement data is measured. Since the sleep depth is determined, the device does not become large-scale and is suitable as a sleep depth determination device in a general household. Further, since the sleep depth can be determined in a completely unrestrained and non-contact state without restraining or contacting the person with the sensors, the sleep depth can be determined without any burden on the person. Furthermore, since REM sleep and light sleep can be classified into four stages of determination, the sleep depth can be accurately determined and the sleep cycle can be derived. This makes it possible to control the environment (air conditioning control, lighting control, acoustic control, temperature control, etc.) for a good night's sleep based on the sleep cycle.

In addition, since it is possible to classify a person's sleep depth into four stages: wakefulness, REM sleep, light sleep, and slow-wave sleep, various information related to sleep, such as time to sleep, sleep efficiency, and REM sleep, can be classified. The occupancy rate, the occupancy rate of slow-wave sleep, etc. can be displayed on the display means 5, and the report can be output from a printer (not shown) or the like.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the spirit of the present invention.

For example, in the present embodiment, when the difference processing unit 34 performs difference processing on all frame images, the difference processing is performed at a frame interval t2 (2.0 seconds or more and 4.0 seconds or less) having a long time interval, and then The body movement data calculation unit 36 detects the body movement (the number of pixels whose density value has changed) based on the difference processing result at the frame interval t2, but the difference processing unit 34 has a frame interval shorter than the frame interval t2. Difference processing of all frame images is performed at t3, and after noise processing by the noise processing unit 34, the body movement data calculation unit 36 performs body movement (of pixels whose density value has changed) based on the difference processing result at the frame interval t2. Number) may be detected. The frame interval t3 depends on the fps of the moving image obtained by the photographing means 1, but can be set from, for example, 0.2 seconds or more and 1.0 seconds or less, preferably 0.5 seconds. ..

Assuming that the frame interval t3 is 0.5 seconds, the difference processing of all the frame images by the difference processing unit 34 is performed by the difference processing unit 34 when the moving image obtained by the photographing means 1 is 2 fps, as shown in FIG. It is done between frame images. That is, for example, the frame image for which the difference is taken from the Nth frame image is the previous N-1th frame image, and the N + 1th frame image is between the N + 1th frame image and the Nth frame image. The difference is taken, and the difference is taken between the N + 2nd frame image and the N + 1th frame image.

As described above, when the frame image difference processing is performed at the frame interval t3 shorter than the frame interval t2, it is possible to detect the fast-moving thin body motion that could not be detected at the frame interval t2. Due to the short frame interval, when comparing the two frame images (Nth and N-1th) to be subjected to the difference processing, the difference due to the body movement appears large, and the pixel whose density value changes between the frame images appears. Since the number is large, it is possible to detect this with high accuracy.

FIG. 8 shows the result (lower row) of measuring the time series data of the body movement at the frame interval t3 during sleep of a person by the same method. In FIG. 8, the horizontal axis is time [sec] and the vertical axis is the number of pixels [pixel] detected as body movement. The larger the number of pixels, the larger the body movement. Note that FIG. 8 also shows actual measurement data (upper row) of the sleep depth determined by the polysomnography method for comparison with the sleep depth.

In this embodiment, in addition to the coarse body movement that appears during sleep, the sleep depth during sleep is discriminated into four stages of wakefulness, REM sleep, light sleep, and slow wave sleep by accurately measuring fine body movement. It is possible.

FIG. 9 shows (a) body movement amount (without moving average processing), (b) body movement duration (without moving average processing) during sleep of a person calculated from the body movement data of FIG. 8 by the same method. (C) Resting duration (without moving average processing), (d) Body movement amount (with moving average processing), (e) Body movement duration (with moving average processing) and (f) Resting duration (with moving average processing) ) Is a graph showing how each parameter of) is distributed in the wakefulness state (WAKE), the REM sleep (REM), the light sleep (LIGHT), and the slow wave sleep (DEEP).

Also in FIG. 9, it can be seen that the amount of body movement (with moving average processing) is distributed more in the awake state (WAKE) than in the others, and contributes to the classification of the awake state (WAKE) and the others. Light sleep (LIGHT) is more distributed than REM sleep (REM) and slow wave sleep (DEEP), and light sleep (LIGHT), REM sleep (REM), and slow wave sleep are distributed in the amount of body movement (without moving average treatment). It can be read that it contributes to the classification of (DEEP). The duration of body movement (without moving averaging treatment) and the duration of body movement (with moving averaging treatment) are distributed smaller in slow-wave sleep (DEEP) than in REM sleep (REM) and light sleep (LIGHT). It can be seen that it contributes to the classification of sleep (DEEP), REM sleep (REM) and light sleep (LIGHT). Rest duration (without moving average treatment) and rest duration (with moving average treatment) are distributed smaller in REM sleep (REM) than in slow wave sleep (DEEP) and light sleep (LIGHT), and REM sleep (REM). ) And slow wave sleep (DEEP) and light sleep (LIGHT). In addition, regarding the stationary duration (without moving average processing) and the stationary duration (with moving average processing), the awake state (WAKE) is distributed smaller than the others, which contributes to the classification of the awake state (WAKE) and others. You can also read what you are doing.

10 (a) and 10 (b) show comparison of the determination result of the sleep depth of the person determined from the body movement data of FIG. 8 with the measured data of the sleep depth measured by the polysomnography method in the same manner. ing. Referring to FIG. 10 (a), 875 (99.8%) of the 877 epochs determined to be in sleep state (SLEEP) by the sleep polygraph method are in sleep state (SLEEP) by the sleep depth determination device 2 of the present invention. ), And 29 (65.9%) of the 44 epochs determined to be wakeful (WAKE) by the sleep polygraph method are also wakeful (65.9%) by the sleep depth determination device 2 of the present invention. WAKE). In the present embodiment, 438 epochs, which is half of the 875 epochs determined to be in the sleep state (SLEEP) by the sleep depth determination device 2 of the present invention, are used as teacher data for machine learning, and the other half, 437 epochs. Was used as the pull judgment data. Referring to FIG. 10B, the 437 epochs determined to be in a sleep state (SLEEP) by the sleep depth determination device 2 of the present invention were further determined to be slow wave sleep (DEEP) by the sleep polygraph method 110. Of the epochs, 65 (59.1%) were determined to be slow-wave sleep (DEEP) by the sleep depth determination device 2 of the present invention, and were determined to be light sleep (LIGHT) by the sleep polygraph method. Of the 262 epochs, 243 (89.3%) were determined to be light sleep (LIGHT) by the sleep depth determination device 2 of the present invention, and were determined to be REM sleep (REM) by the sleep polygraph method. Of the 67 epochs that were made, 50 (74.6%) were determined to be REM sleep (REM) by the sleep depth determination device 2 of the present invention. As described above, according to the sleep depth determination device 2 of the present invention, the sleep depth is set to four stages of wakefulness (WAKE), REM sleep (REM), light sleep (LIGHT) and slow wave sleep (DEEP) with generally good accuracy. It can be said that it can be classified into.

Further, in the above embodiment, when the difference processing unit 34 performs the difference processing at the frame interval t2 or the frame interval t3, the difference processing unit 34 performs the difference processing at the frame interval t2 or the frame interval t3, and the difference processing result at the frame interval t2 or the frame interval t3. The body movement data calculation unit 36 detects the body movement (the number of pixels whose density value has changed), but the difference processing unit 34 has two different frame intervals t2 (first frame interval t2) and frames. Difference processing of all frame images is performed at the interval t3 (second frame interval t3), and after noise processing by the noise processing unit 34, the body motion data calculation unit 36 performs the difference processing at the first frame interval t2 and the second frame interval t3, respectively. The body movement (the number of pixels whose density value has changed) may be detected based on the difference processing result of.

Specifically, in FIG. 11, the difference processing unit 34 sets the first frame interval t2 for all the frame images acquired by the same method as in the moving image acquisition step ST1 to gray conversion step ST3 in FIG. In addition to performing processing to detect pixels whose density value has changed with the frame image from which the difference was taken by performing difference processing (image processing step ST4-1), difference processing is performed at the second frame interval t3 to obtain the difference. A process is performed to detect a pixel whose density value has changed between the captured frame image and the captured frame image (image processing step ST4-2). As a result, two types of data are measured: the time-series data of the body movement at the first frame interval t2 and the time-series data of the body movement at the second frame interval t3.

Then, after the noise processing unit 35 performs noise processing on the pixels whose density values are changed from each of the difference processing at the frame interval t2 and the difference processing at the frame interval t3 (noise processing step ST5-1). , ST5-2), the body movement data calculation unit 36 counts the number of the pixels, and calculates the body movement amount, the body movement duration, and the stationary duration for each epoch consisting of the predetermined time T from each counting result. (Body movement data calculation steps ST6-1, ST6-2). In addition, the body movement data calculation unit 36 performs moving average processing for each of the body movement amount, body movement duration, and rest duration of all epochs (moving average processing steps ST7-1 and ST7-2). The method of difference processing, the method of noise processing, the amount of body movement, the method of calculating the body movement duration and the stationary duration, and the method of moving average processing are the same as those of the difference processing step ST4 to the moving average processing step ST7 in FIG. It is done in a way.

Finally, the determination unit 37 determines the amount of body movement (moving average processing) for each epoch calculated from each of the time-series data of body movements based on the difference processing at the first frame interval t2 and the difference processing at the second frame interval t3. (None), body movement duration (without moving average processing) and rest duration (without moving average processing), as well as body movement amount (with moving average processing), body movement duration (with moving average processing) and rest duration Based on the 12 parameters (with moving average processing), the sleep depth of a person during sleep is one of wakefulness (WAKE), REM sleep (REM), light sleep (LIGHT), and slow wave sleep (DEEP). It is determined whether or not it is classified into (ST8). The determination is made by using machine learning or the like as in ST8 of FIG.

In this embodiment, by accurately measuring the coarse and fine movements that appear during sleep, the sleep depth during sleep can be discriminated into four stages: wakefulness, REM sleep, light sleep, and slow wave sleep. It is possible.

Further, it is possible to create a computer program for making the hardware such as the CPU, ROM, and RAM built in the computer 3 of the sleep depth determination device 2 exhibit the same functions as the above-mentioned configurations of the sleep depth determination device 2. be. A storage medium for storing the computer program is also provided.

1 Shooting means 2 Sleep depth determination device 3 Computer 32 Image conversion unit 33 Gray conversion unit 34 Difference processing unit 35 Noise processing unit 36 Body motion data calculation unit 37 Judgment unit

Claims (8)

Shooting means that can take moving images of people during sleep,
It is equipped with a sleep depth determination device that determines the sleep depth based on a moving image of a person during sleep.
The sleep depth determination device includes a processor and a computer including a memory under the control of the processor.
The computer
An image conversion unit that converts the moving image into a plurality of frame images constituting the moving image, and
A gray conversion unit that converts the plurality of frame images into grayscale images when they are not grayscale images, and
A difference processing unit that detects pixels whose density values have changed between the frame images for which the difference processing has been performed by performing the difference processing on the plurality of grayscale frame images at predetermined frame intervals.
A body movement data calculation unit that counts the number of pixels whose density value has changed detected by the difference processing and calculates the amount of body movement, the duration of body movement, and the duration of rest for each epoch consisting of a predetermined time from the counting result.
It functions as a determination unit that determines the sleep depth of a person for each epoch by machine learning based on the calculated amount of body movement, the duration of body movement, and the duration of rest.
The determination unit classifies the sleep depth of a person for each epoch into either a wakefulness state or a sleep state, and then classifies the state classified into a sleep state into any one of REM sleep, light sleep, and slow wave sleep. A sleep depth judgment system that classifies into. The body movement data calculation unit performs moving average processing on the body movement amount, the body movement duration, and the rest duration for each epoch.
The sleep depth determination system according to claim 1, wherein the determination unit determines the sleep depth of a person for each epoch based on the moving average processed body movement amount, body movement duration, and rest duration. .. The sleep depth determination system according to claim 1 or 2, wherein the frame interval of the difference processing is set from a range of 2.0 seconds or more and 4.0 seconds or less. Any of claims 1 to 3, wherein the determination unit uses linear discrimination for classification of either wakefulness or sleep state, and uses a support vector machine for classification of REM sleep, light sleep, or slow-wave sleep. The sleep depth determination system described in. Claims 1 to 4 further function as a noise processing unit in which the computer further functions as a noise processing unit that performs at least one expansion process and a contraction process more than the number of expansion processes on the pixel whose density value has changed detected by the difference process. The sleep depth determination system described in any of the above. The difference processing unit detects pixels whose density values have changed between the frame images for which the difference processing has been performed at the first frame interval by performing the difference processing on the plurality of frame images at a predetermined first frame interval. At the same time, by performing the difference processing at a predetermined second frame interval longer than the first frame interval, the pixels whose density values have changed between the frame images subjected to the difference processing at the second frame interval are detected.
The body movement data calculation unit counts the number of pixels whose density values have changed, which are detected by the difference processing at the first frame interval and the difference processing at the second frame interval, respectively, and a predetermined time is obtained from each counting result. For each epoch consisting of, the amount of body movement, the duration of body movement, and the duration of rest are calculated, respectively.
The determination unit includes the body movement amount, the body movement duration, and the body movement duration calculated from the pixel counting result by the difference processing at the first frame interval and the pixel counting result by the difference processing at the second frame interval, respectively. The sleep depth determination system according to claim 1, wherein the sleep depth of a person for each epoch is determined based on the duration of rest. It is a sleep depth determination device that determines the sleep depth based on a moving image of a person during sleep.
A computer including a processor and a memory under the control of the processor.
The computer
An image conversion unit that converts the moving image into a plurality of frame images constituting the moving image, and
A gray conversion unit that converts the plurality of frame images into grayscale images when they are not grayscale images, and
A difference processing unit that detects pixels whose density values have changed between the frame images for which the difference processing has been performed by performing the difference processing on the plurality of grayscale frame images at predetermined frame intervals.
A body movement data calculation unit that counts the number of pixels whose density value has changed detected by the difference processing and calculates the amount of body movement, the duration of body movement, and the duration of rest for each epoch consisting of a predetermined time from the counting result.
It functions as a determination unit that determines the sleep depth of a person for each epoch by machine learning based on the calculated amount of body movement, the duration of body movement, and the duration of rest.
The determination unit classifies the sleep depth of a person for each epoch into either a wakefulness state or a sleep state, and then classifies the state classified into a sleep state into any one of REM sleep, light sleep, and slow wave sleep. A sleep depth determination device that classifies into. It is a sleep depth determination method that determines the sleep depth based on a moving image of a person during sleep.
An image conversion step of converting the moving image into a plurality of frame images constituting the moving image, and
A gray conversion step of converting a plurality of frame images into a grayscale image when the image is not a grayscale image, and
A difference processing step of detecting pixels whose density values have changed between the frame images subjected to the difference processing by performing the difference processing of the plurality of grayscale frame images at predetermined frame intervals, and a difference processing step.
A body movement data calculation process that counts the number of pixels whose density value has changed detected by the difference processing and calculates the amount of body movement, the duration of body movement, and the duration of rest for each epoch consisting of a predetermined time from the counting result.
Based on the calculated amount of body movement, the duration of body movement, and the duration of rest, a determination step of determining the sleep depth of a person for each epoch by machine learning is executed.
The determination step classifies the sleep depth of a person for each epoch into either a wakefulness state or a sleep state, and then classifies the state classified into a sleep state into any one of REM sleep, light sleep, and slow wave sleep. Sleep depth determination method to classify into.

Images