JP7180601B2 - SLEEP STATE DETECTION DEVICE AND METHOD, AND MONITORED PERSON MONITORING SUPPORT SYSTEM - Google Patents

Description

The present invention relates to a sleep state detection device and a sleep state detection method for detecting a sleep state. The present invention also relates to a monitored person monitoring support system using this sleeping state detection device.

2. Description of the Related Art In the medical and nursing care industries, there is a demand for a technique for confirming whether or not a person receiving medical care or nursing care is sleeping soundly. Techniques for detecting such a sleeping state include, for example, the techniques disclosed in Patent Literature 1 and Patent Literature 2.

The sleep depth determination device disclosed in Patent Document 1 uses a sensory threshold, which is the minimum stimulus intensity that can be perceived by a subject in a sleep state at a predetermined sleep depth, to determine the depth of sleep of the subject. A device for determining depth of sleep, wherein a first stimulus having a first stimulus intensity and a second stimulus having a second stimulus intensity stronger than the first stimulus intensity are applied to the subject. The stimulus application means, the perception detection means for detecting whether or not the subject perceives the first stimulus and the second stimulus, and the perception detection means detect that the subject perceives the first stimulus. The subject's sleep depth corresponds to a sensory threshold between the first stimulus intensity and the second stimulus intensity when non-perceived and detected to perceive the second stimulus. and sleep depth determination means for determining that the sleep depth is determined. The stimulus imparting means is a part that imparts a physical stimulus to the subject. For example, there is a speaker that imparts an auditory stimulus with sound, and the stimulus intensity is changed by adjusting the sound pressure (Patent Document 1, paragraph [0023]).

The sleep state management device disclosed in Patent Document 2 is a sleep state management device that manages the sleep state of the person to be measured based on the body movement of the person to be measured. and a body movement determination unit that determines whether or not there is body movement of the subject using a detection signal from the sensor unit, wherein the body movement determination unit uses the detection signal a first period in which a no-body-motion period in which it is determined that there is no body movement is periodically generated, or a period with body movement in which it is determined that there is body movement by using the detection signal, and the period with body-motion continues for a predetermined time or longer When there is at least one of the second period and the body motion period, at least one of the body movement period and the second period in the first period is corrected as a period without body movement.

The sleep depth determination device disclosed in Patent Document 1 is an active type that stimulates the subject, so there is a risk of disturbing the subject's sleep, but the sleep state management device disclosed in Patent Document 2 Since the is a passive type that detects the movement of the bedding on which the subject is sleeping, the risk of disturbing the subject's sleep can be reduced, and in this respect it is superior to the sleep depth determination device disclosed in Patent Document 1. ing. However, both the sleep depth determination device disclosed in Patent Document 1 and the sleep state management device disclosed in Patent Document 2 are based on the premise that the subject (person to be measured) is sleeping. Since the sleep state is measured, when the subject (subject) is not asleep, that is, when the subject is awake, the output signal from the perception detection means or the sensor unit may cause the sleep depth to be poor. There is a risk of erroneous determination.

JP 2010-200956 A JP 2013-198654 A

The present invention has been made in view of the circumstances described above, and aims to provide a sleep state detection device and a sleep state detection method that can further reduce erroneous determinations, and a monitored person using this sleep state detection device. It is to provide a monitoring support system.

In order to achieve the above-described object, a sleep state detection device, a sleep state detection method, and a monitored person monitoring support system that reflect one aspect of the present invention measure a subject with first and second sensors, respectively, and Determining the sleep state of the subject based on the first measurement result of the first sensor, detecting detection timing based on the second measurement result of the second sensor, and determining the sleep state according to the detection result Control processing.

The advantages and features provided by one or more embodiments of the invention will be fully appreciated from the detailed description and accompanying drawings provided below. These detailed descriptions and accompanying drawings are given by way of example only and are not intended as a definition of the limits of the invention.

It is a figure which shows the structure of the monitored person monitoring assistance system in embodiment. 2 is a diagram showing the configuration of a sensor device in which the sleep state detection device of the embodiment is incorporated in the monitored person monitoring support system; FIG. 4 is a flowchart showing the operation of the sensor device in service mode; 4 is a flowchart showing the operation of the sensor device in the sleep state determination process shown in FIG. 3; 4 is a flowchart showing the operation of the sensor device in the fall determination process shown in FIG. 3; 4 is a flow chart showing the operation of the sensor device in the overturn determination process shown in FIG. 3; 4 is a flow chart showing the operation of the sensor device in the bed entry determination process shown in FIG. 3; 4 is a flowchart showing the operation of the sensor device in the bed leaving determination process shown in FIG. 3; FIG. 4 is a flow chart showing the operation of the sensor device in the micro-motion abnormality determination process shown in FIG. 3; FIG. It is a figure which shows an example of the main screen displayed on the said terminal device. It is a figure which shows an example of the 1st setting screen displayed on the said terminal device. It is a figure which shows an example of the 2nd setting screen displayed on the said terminal device. It is a figure which shows an example of the monitoring information screen displayed on the said terminal device. It is a figure which shows another example of the monitoring information screen displayed on the said terminal device. It is a figure which shows another example of the monitoring information screen displayed on the said terminal device.

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment according to the present invention will be described below with reference to the drawings. However, the scope of the invention is not limited to the disclosed embodiments. It should be noted that the configurations denoted by the same reference numerals in each figure indicate the same configurations, and the description thereof will be omitted as appropriate. In the present specification, reference numerals with suffixes omitted are used when referring to generically, and reference numerals with suffixes are used when referring to individual configurations.

A monitored person monitoring support system according to an embodiment includes a sensor device that is provided corresponding to a subject to determine the sleep state of the subject, and a sleep state that is communicably connected to the sensor device and received from the sensor device. and a terminal device that is communicably connected to the central processing unit and receives and displays the sleep state via the central processing unit, wherein the subject is the monitored person. It is a monitored person monitoring support system for supporting the monitoring of. This sensor device includes the sleep state detection device in this embodiment. The sleep state detection device in this embodiment includes first and second sensors for measuring a subject, and sleep state processing for determining the sleep state of the subject based on the first measurement result of the first sensor. A processing unit, a timing detection unit that detects detection timing based on the second measurement result of the second sensor, and a processing control unit that controls the sleep state processing unit according to the detection result of the timing detection unit. . Since such a sleep state detection device controls the sleep state processing unit according to the detection result of the timing detection unit, erroneous determination can be further reduced. Further, in the present embodiment, from the viewpoint of more appropriately monitoring the person being monitored (subject), the sensor device detects a predetermined action of the person being monitored (subject), and the central processing unit manages the detection result of the predetermined action received from the sensor device, and the terminal device receives and displays the detection result via the central processing unit. The terminal device may be one type of device, but in this embodiment, the terminal device is two types of devices, a fixed terminal device and a mobile terminal device. The main difference between these fixed terminal devices and mobile terminal devices is that while fixed terminal devices are used in a fixed manner, mobile terminal devices are carried and operated by supervisors (users) such as nurses and caregivers. In that respect, the fixed terminal device and the mobile terminal device are substantially the same.

FIG. 1 is a diagram showing the configuration of a monitored person monitoring support system according to an embodiment. FIG. 2 is a diagram showing the configuration of a sensor device in which the sleep state detection device of the embodiment is incorporated in the monitored person monitoring support system.

More specifically, the monitored person monitoring support system MS includes, for example, one or more sensor devices SU (SU-1 to SU-4), a management server device SV, a fixed terminal, A device SP, one or a plurality of portable terminal devices TA (TA-1, TA-2), and a private branch exchange (PBX, Private branch exchange) CX, which are wired or wirelessly connected to a LAN (Local Area Network). ), etc., are communicably connected via a network (network, communication line) NW. The network NW may be provided with repeaters such as repeaters, bridges and routers for relaying communication signals. In the example shown in FIG. 1, the plurality of sensor devices SU-1 to SU-4, the management server device SV, the fixed terminal device SP, the plurality of mobile terminal devices TA-1 and TA-2, and the private branch exchange CX are L2 switches. are communicably connected to each other by a mixed wired and wireless LAN (for example, a LAN conforming to the IEEE 802.11 standard) NW including a line concentrator (hub, HUB) LS and an access point AP. More specifically, a plurality of sensor devices SU-1 to SU-4, a management server device SV, a fixed terminal device SP and a private branch exchange CX are connected to a line concentrator LS, and connected to a plurality of mobile terminal devices TA-1 and TA-2. is connected to the concentrator LS via an access point AP. The network NW constitutes a so-called intranet by using Internet protocols such as TCP (Transmission control protocol) and IP (Internet protocol).

The private branch exchange (circuit switch) CX is connected to the network NW, controls extension calls such as outgoing calls, incoming calls, and conversations between the mobile terminal devices TA, implements extension calls between the mobile terminal devices TA, and , for example, is connected to an outside line telephone TL such as a fixed telephone or a mobile telephone via a public telephone network PN such as a fixed telephone network or a mobile telephone network, and originating, receiving and calling between the outside telephone TL and the mobile terminal device TA. It is a device that performs outside line telephone calls between the outside line telephone TL and the mobile terminal device TA by controlling outside line telephones such as the mobile terminal device TA. The private branch exchange CX is, for example, a digital exchange or an IP-PBX (Internet Protocol Private Branch Exchange).

The monitored person monitoring support system MS is arranged at an appropriate place according to the monitored person (target person) Ob. The monitored person (person to be watched over) Ob is, for example, a person who needs nursing care due to illness or injury, a person who needs nursing care due to a decline in physical ability, a solitary person living alone, or the like. In particular, from the viewpoint of enabling early detection and early countermeasures, the monitored person Ob should be a person who needs to be found when a predetermined inconvenient event, such as an abnormal condition, occurs to him or her. preferable. Therefore, the monitored person monitoring support system MS is preferably installed in buildings such as hospitals, welfare facilities for the elderly, and dwelling units according to the type of the monitored person Ob. In the example shown in FIG. 1, the monitored person monitoring support system MS is installed in a building of a nursing care facility having a plurality of rooms RM in which a plurality of monitored persons Ob reside and a plurality of rooms such as a nurse station. .

The sensor device SU has a communication function or the like to communicate with other devices SV, SP, and TA via the network NW, detects a predetermined event related to the person to be monitored Ob, and reports the detected event to the management server device SV. , performs voice communication with the terminal devices SP and TA, generates images including moving images, and distributes the moving images to the terminal devices SP and TA. The predetermined event (phenomenon) preferably includes an event requiring a countermeasure (response). In this embodiment, the predetermined event includes the sleep state of the monitored person Ob, a preset predetermined action of the monitored person Ob, and a nurse call. For example, as shown in FIG. 2, such a sensor device SU includes a sensor unit 11, a sensor-side sound input/output unit (SU sound input/output unit) 12, a nurse call reception operation unit 13, and a sensor-side control processing unit. section (SU control processing section) 14 , sensor side communication interface section (SU communication IF section) 15 , and sensor side storage section (SU storage section) 16 .

The sensor unit 11 is a device that is connected to the SU control processing unit 14 and measures various preset values of the monitored person Ob under the control of the SU control processing unit 14 . In the present embodiment, the sensor unit 11 includes a first sensor that measures the target person in order to determine the sleep state of the monitored person Ob, and measures the target person Ob in order to determine the predetermined behavior of the monitored person Ob. A second sensor is provided for measuring the person.

Since the first sensor determines the sleep state of the monitored person Ob based on the respiratory state in this embodiment, any type of device that can measure the breathing of the monitored person Ob as one of the predetermined quantities. can be The first sensor may be, for example, a pulse meter that is worn on the wrist or the like to measure the pulse. configured with.

For example, in the present embodiment, the predetermined actions include entering the bed when the monitored person Ob enters the bedding BD, leaving the bed when the monitored person Ob leaves the bedding BD, falling when the monitored person Ob falls off the bedding BD, This includes a fall in which the monitored person Ob falls outside the bedding BD, and an abnormality in microbody movement caused by breathing of the monitored person. The second sensor may be any device capable of determining such predetermined behavior. In the present embodiment, the predetermined behavior is divided into a first group including bed entry, bed leaving, falling and falling, and a second group including micromotion abnormality, and the second sensor is divided into the first group A first sub-sensor for detecting a first action (in this embodiment, getting into bed, getting out of bed, falling, and falling), and detecting a second action (in this embodiment, abnormal microbody movement) belonging to the second group. and a second sub-sensor for. The first sub-sensor may be, for example, a thermography device capable of extracting a human area from a temperature distribution, a distance image sensor capable of extracting a human area from a distance distribution, or the like. Since the first action belonging to one group is determined, it is configured with a camera 111 that generates an image as one of the predetermined quantities. In the present embodiment, the second sub-sensor determines the second behavior belonging to the second group based on the respiratory state, so the apparatus can measure the breathing of the monitored person Ob as one of the predetermined quantities. and is configured with a Doppler sensor 112, for example. That is, in this embodiment, the Doppler sensor 112 is used both as the first sensor and the second sub-sensor.

Thus, in this embodiment, the sensor unit 11 includes the camera 111 and the Doppler sensor 112 .

The camera 111 is a device that is connected to the SU control processing unit 14 and generates an image (image data) under the control of the SU control processing unit 14 . The images include still images (still image data) and moving images (moving image data). The camera 111 is arranged so as to be able to monitor the space where the person to be monitored Ob is supposed to be located (located space, in the example shown in FIG. 1, the living room RM of the installation location). It captures an image, generates an image (image data) of a bird's-eye view of the imaging target, and outputs the image of the imaging target (target image) to the SU control processing unit 14 . Preferably, since there is a high probability that the entire monitored person Ob can be imaged, the camera 111 is set so that the head of the monitored person Ob is positioned in the bedding (for example, a bed) BD on which the monitored person Ob lies. It is arranged so that an image of the object to be imaged can be imaged from directly above a predetermined head position (usually, the position where a pillow is arranged). The sensor unit SU acquires an image of the monitored person Ob from above the monitored person Ob, preferably from directly above the expected head position, using the camera 111 .

Such a camera 111 may be a device that generates a visible light image, but in this embodiment, a digital infrared camera that generates an infrared image is used so that the monitored person Ob can be monitored even in relatively dark conditions. be. For example, in this embodiment, the digital infrared camera 111 includes an imaging optical system that forms an infrared optical image of an object to be imaged on a predetermined imaging plane, and a light receiving surface that matches the imaging plane. and an image sensor that converts an infrared optical image of the imaging target into an electrical signal, and an image that is data representing the infrared image of the imaging target by image processing the output of the image sensor. It is configured by including an image processing unit for generating data and the like. In this embodiment, the imaging optical system of the camera 111 is preferably a wide-angle optical system (a so-called wide-angle lens (including a fisheye lens)) capable of imaging the entire room RM in which it is installed.

The Doppler sensor 112 is a device that is connected to the SU control processing unit 14 and measures movement of the body surface of the chest associated with respiration in the monitored person Ob according to the control of the SU control processing unit 14 . The Doppler sensor 112 is a body motion sensor that transmits a transmitted wave, receives a reflected wave of the transmitted wave reflected by an object, and outputs a Doppler signal of a Doppler frequency component based on the transmitted wave and the reflected wave. . When the object is in motion, the frequency of the reflected wave shifts in proportion to the speed at which the object is moving due to the so-called Doppler effect. occur. The Doppler sensor 112 generates a signal of this Doppler frequency component as a Doppler signal at a predetermined sampling rate and outputs it to the SU control processing unit 14 . Upon receiving the Doppler signal from the Doppler sensor 112, the SU control processing unit 14 stores the received Doppler signal in the SU storage unit 16 in chronological order. The transmission waves may be ultrasonic waves, microwaves, or the like, but in this embodiment, they are microwaves. Microwaves can be transmitted through clothes and reflected by the body surface of the monitored person Ob, so even if the monitored person Ob is wearing clothes, the movement of the body surface can be detected, which is preferable.

The SU sound input/output unit 12 is a circuit connected to the SU control processing unit 14 for acquiring external sounds and inputting them to the sensor unit SU. It is a circuit for generating and outputting a sound according to the The SU sound input/output unit 12 includes, for example, a microphone or the like that converts acoustic vibration of sound into an electric signal, and a speaker or the like that converts the electric signal of sound into acoustic vibration of sound. The SU sound input/output unit 12 outputs an electrical signal representing an external sound to the SU control processing unit 14, converts the electrical signal input from the SU control processing unit 14 into an acoustic vibration of sound, and outputs the acoustic vibration.

The nurse call reception operation unit 13 is connected to the SU control processing unit 14 and is a switch circuit such as a push-button switch for inputting a nurse call to the sensor unit SU. The nurse call reception operation unit 13 may be connected to the SU control processing unit 14 by wire, or may be connected to the SU control processing unit 14 by short-range wireless communication such as the Bluetooth (registered trademark) standard.

The SU communication IF unit 15 is a communication circuit that is connected to the SU control processing unit 14 and performs communication under the control of the SU control processing unit 14 . SU communication IF unit 15 generates a communication signal containing data to be transferred input from SU control processing unit 14 according to a communication protocol used in network NW of monitored person monitoring support system MS, and generates a It transmits communication signals to other devices SV, SP, TA via the network NW. The SU communication IF unit 15 receives communication signals from other devices SV, SP, and TA via the network NW, extracts data from the received communication signals, and the SU control processing unit 14 can process the extracted data. format data and output to the SU control processing unit 14 . The SU communication IF unit 15 includes, for example, a communication interface circuit complying with the IEEE802.11 standard.

The SU storage unit 16 is a circuit that is connected to the SU control processing unit 14 and stores various prescribed programs and various prescribed data under the control of the SU control processing unit 14 . The various predetermined programs include, for example, an SU control program for controlling each part of the sensor device SU according to the function of each part, and sleep state processing for the monitored person Ob based on the output of the sensor unit 11. The detection timing is detected based on the sleep state processing program for obtaining the sleep state and the output of the sensor unit 11, and the predetermined behavior of the monitored person Ob is determined based on the output of the sensor unit 11, and sent to the management server device SV. A behavior detection processing program (timing detection program) to be notified, a processing control program for controlling the sleep state processing program according to the detection result of the timing detection program (behavior detection processing program), and a nurse call reception operation unit 13 A nurse call processing program that notifies the management server device SV when a call is accepted and performs a voice call between the terminal devices SP and TA by using the SU sound input/output unit 12 or the like; It includes a control processing program such as a streaming processing program for streaming the generated moving image to the terminal device SP or TA that requested the moving image. The various predetermined data include a sensor device identifier (sensor ID), which is an identifier for specifying and identifying the sensor device SU of the device itself, and a communication address of the management server device SV. It contains the data necessary for the above. The SU storage unit 16 includes, for example, a ROM (Read Only Memory) that is a non-volatile storage element, an EEPROM (Electrically Erasable Programmable Read Only Memory) that is a rewritable non-volatile storage element, and the like. The SU storage unit 16 includes a RAM (Random Access Memory) or the like that serves as a so-called working memory of the SU control processing unit 14 that stores data generated during execution of the predetermined program.

The SU control processing unit 14 controls each unit 11 to 13, 15, and 16 of the sensor unit SU according to the function of each unit, detects a predetermined event related to the monitored person Ob, and detects the detected event. This is a circuit for notifying the management server device SV, making a voice call with the terminal devices SP and TA, generating an image including a moving image, and distributing the moving image to the terminal devices SP and TA. The SU control processing unit 14 is composed of, for example, a CPU (Central Processing Unit) and its peripheral circuits. By executing the control processing program, the SU control processing unit 14 includes a sensor-side control unit (SU control unit) 141, a sleep state processing unit 142, an action detection processing unit (timing detection unit) 143, and a processing control unit 144. , a nurse call processing unit 145 and a streaming processing unit 146 are functionally provided.

The SU control section 141 controls the respective sections 11 to 13, 15, and 16 of the sensor device SU according to the functions of the respective sections, and governs the overall control of the sensor device SU.

As sleep state processing, the sleep state processing unit 142 obtains the sleep state of the monitored person Ob based on the output of the sensor unit 11, that is, the first measurement result of the first sensor, and notifies the management server device SV of the sleep state. is. More specifically, the sleep state processing unit 142 obtains the degree of sleep representing the depth of sleep of the monitored person Ob as the sleep state based on the first measurement result of the first sensor. In this embodiment, the sleep state processing unit 142 obtains the sleep state based on the Doppler signal from the Doppler sensor 112, which is an example of the first sensor, at predetermined time intervals. More specifically, the sleep state processing unit 142 performs, for example, a fast Fourier transform (FFT) on Doppler signals measured within a predetermined time (for example, 1 minute or 2 minutes) from the time of measurement to the past, and with this FFT From the obtained spectrum, the average value of the amplitude in the frequency band corresponding to the general respiration frequency is obtained, and the obtained average value is compared with the threshold for determining the degree of sleep (sleep degree classification threshold). The sleep degree is provisionally obtained as the sleep state, and when the provisionally obtained sleep degree continues for a preset continuation determination time, the provisionally obtained sleep degree is determined as the final sleep degree. Then, the sleep state processing unit 142 updates the state variable with the finally determined degree of sleep. The state variable is a variable that stores the degree of sleep obtained by the sleep state processing unit 142 and a predetermined behavior obtained by the behavior detection processing unit 143, which will be described later. The sleep degree classification threshold is appropriately set using, for example, a plurality of samples. In the present embodiment, the sleep level is during wakefulness, a low sleep level in which sleep is relatively light, and a high sleep level in which sleep is relatively deep. It includes two first and second sleep intensity segmentation thresholds to divide into three stages of high sleep intensity and high sleep intensity. Note that the sleep degree classification threshold may be three or more in order to further divide into multiple stages. The first sleep level threshold for discriminating between awake and low sleep levels is set to a value larger than the second sleep level threshold for discriminating between low sleep levels and high sleep levels (first sleep level threshold> second sleep degree classification threshold).

When the sleep state (sleep degree in this embodiment) is obtained in this way, the sleep state processing unit 142 stores event information (event information) representing the content of a predetermined event (event) related to the monitored person Ob. The SU communication IF unit 15 notifies the management server device SV of a communication signal (first event notification communication signal) for notifying the event. More specifically, the sleep state processing unit 142 transmits the first event notification communication signal containing the sensor ID of its own device and the event information representing the content of the event to the management server device SV via the SU communication IF unit 15. Send to In the present embodiment, the event information is one or more of sleep level (awakening, low sleep level, high sleep level), entering bed, getting out of bed, falling, falling, abnormal micromobility, and nurse call (NC) Here, the sleep state processing unit 142 stores the obtained sleep degree (awake, low sleep degree, high sleep degree) as the event information in the first event notification communication signal. Note that the sleep state processing unit 142 may cause the camera 111 to generate an image when obtaining the degree of sleep, and may further include this image in the first event notification communication signal. The image may be at least one of a still image and a moving image.

In addition, the sleep state processing unit 142 may further obtain a degree of reliability representing the degree of reliability of the degree of sleep, and notify the management server device SV of the degree of reliability. In this case, the sleep state processing unit 142, for example, obtains the duration of the obtained sleep degree, and compares the obtained duration of the sleep degree with a threshold for determining reliability (reliability classification threshold). to find reliability. For example, one or a plurality of the reliability classification thresholds are appropriately set using a plurality of samples. For example, the confidence classification threshold may be a single value that distinguishes between high confidence where the confidence is relatively high and low confidence where the confidence is relatively low. In order to divide into three levels of high confidence with a relatively high level of confidence, medium confidence with a relatively medium level of confidence, and low confidence with a relatively low level of confidence, each of the two values It's okay.

The behavior detection processing unit (timing detection unit) 143 detects detection timing based on the output of the sensor unit 11, determines a predetermined behavior of the monitored person Ob based on the output of the sensor unit 11, and performs the management server device. It notifies the SV.

More specifically, in the detection of the detection timing, in the present embodiment, whether or not the state is suitable for detection of the sleeping state is determined by whether or not the monitored person (subject) Ob is in the bedding. Therefore, the behavior detection processing unit (timing detection unit) 143 detects the detection timing by determining whether or not the monitored person Ob is in the bedding from the image generated by the camera 111 of the sensor unit 11. do. More specifically, the area where the bedding BD is arranged in the target image (area where the bedding BD is located) is stored in advance in the storage unit 16 as one of the various data. The timing detection unit 143 detects when the person area extracted from the target image acquired from the camera 111 by, for example, the background subtraction method completely overlaps the location area of the bedding BD (the person area is completely within the location area of the bedding BD). ), it is determined that the state is suitable for detecting the sleep state and it is the detection timing, and in other cases, it is determined that the state is not suitable for the detection of the sleep state and is not the detection timing, and this detection result is notified (output) to the processing control unit 144 .

In the detection of the predetermined behavior, in the present embodiment, the predetermined behavior is entering the bed, getting out of bed, falling, falling, and abnormal micromotion as described above. , is detected based on the output (target image) of the camera 111 , and the micro-motion abnormality is detected based on the output (Doppler signal) of the Doppler sensor 112 .

More specifically, getting into bed, getting out of bed, falling, tumbling, and abnormal microtremors are determined as follows. Various thresholds and continuation determination times in the following description are appropriately set from a plurality of samples and stored in the storage unit 16 in advance as one of the various data.

In determining whether to enter the bed, the behavior detection processing unit 143 determines that the previous state variable was “leaving bed”, and this time, the human region extracted from the target image acquired from the camera 111 by, for example, the background subtraction method is the location of the bedding BD. When it completely overlaps with the area (when the person area is completely within the location area of the bedding BD), it is tentatively determined to be in bed, and the duration of the completely superimposed state continues beyond the in-bed continuation determination time. If so, it is finally determined that there is bed entry, and the bed entry is detected. Then, the action detection processing unit 143 updates the state variable with "entering bed". The in-bed continuation determination time is a threshold value for finally determining in-bed as in-bed tentatively determined based on the complete overlap of the extracted person area and the location area of the bedding BD.

In the determination of getting out of bed, the action detection processing unit 143 determines that the previous state variable is either “getting into bed” or “awakening” and that the person region extracted from the target image acquired from the camera 111 this time is When the protruding region protruding from the location region of the bedding BD is equal to or greater than the bed leaving determination threshold, it is provisionally determined to be leaving the bed, and the duration of the protruding region being equal to or greater than the bed leaving determination threshold exceeds the bed leaving continuation determination time. In this case, it is finally determined that the person has left the bed, and the person has left the bed. Then, the action detection processing unit 143 updates the state variable with "getting out of bed". The bed-leaving determination threshold is a threshold for determining whether or not the bed-leaving is based on the size of the protruding region. The bed-leaving continuation determination time is a threshold value for finally determining bed-leaving tentatively determined by comparing the protruding area and the bed-leaving determination threshold.

In the fall determination, the action detection processing unit 143 determines that the size of the head region of the human region extracted from the target image acquired from the camera 111 is equal to or less than the lying posture determination threshold, and the size of the head region is equal to or greater than the fall determination speed threshold, and the person area is within the fall determination area set around the location area of the bedding BD, it is determined that there is a fall, and the fall is detected. . The lying posture determination threshold value is a threshold value for determining whether or not it is the size of the head region in the lying posture. The fall determination speed threshold is a threshold for determining whether or not a fall has occurred based on the change speed of the size of the head region.

In the fall determination, the action detection processing unit 143 determines that the size of the head region of the human region extracted from the target image acquired from the camera 111 is equal to or smaller than the lying posture determination threshold, and the size of the head region is equal to or higher than the fall determination speed threshold, and the person area is in an area excluding the location area of the bedding BD and the fall determination area, it is determined that there is a fall, and the fall is detected. The fall determination speed threshold is a threshold for determining whether or not a fall has occurred based on the change speed of the size of the head region.

As for the abnormal minute body movement, the action detection processing unit 143 determines the abnormal minute body movement based on the Doppler signal of the Doppler sensor 112 . More specifically, like the sleep state processing unit 142, the behavior detection processing unit 143 performs, for example, a fast Fourier transform (FFT) on Doppler signals measured within a predetermined time from the time of measurement to the past, and with this FFT From the obtained spectrum, the average value of the amplitude in the frequency band corresponding to the general respiratory frequency is obtained, and the obtained average value and the threshold value for determining whether or not there is an abnormality in microtremor (microscopic movement abnormality determination threshold value), and if the obtained average value is equal to or less than the minute body movement abnormality determination threshold value, it is tentatively determined as minute body movement abnormality, and if the average value is equal to or less than the minute body movement abnormality determination threshold value When the continuation time of a certain state continues beyond the micro-motion abnormality continuation determination time, it is finally determined that there is a micro-motion abnormality, and the micro-motion abnormality is detected. The threshold value for determining abnormality in minute body movement is set to a value smaller than the threshold for second sleep degree classification (threshold for second sleep degree classification > threshold value for abnormality in minute body movement). The micro-motion abnormality continuation determination time is set so that the micro-motion abnormality temporarily determined by comparing the calculated average value and the micro-motion abnormality determination threshold value is finally determined to be the micro-motion abnormality. is the threshold of

When the predetermined behavior is detected in this manner, the behavior detection processing unit 143 stores one or more of entering the bed, leaving the bed, falling, falling, and abnormal micromotion as the sensor ID of the machine itself and the event information. A first event notification communication signal is transmitted to the management server device SV via the SU communication IF section 15 . Note that when any one of entering the bed, leaving the bed, falling, and falling is detected, the action detection processing unit 143 notifies the first event of the target image used in detecting the entering the bed, leaving the bed, falling, and the fall. It may be further included in the communication signal, and upon detection of the micro-motion anomaly, the camera 111 may generate an image, and this image may be further included in the first event notification communication signal. The image may be at least one of a still image and a moving image.

Note that the sensor device SU may be provided with a dedicated sensor used for detecting detection timing by the timing detection unit (behavior detection processing unit) 143, but in the present embodiment, as described above, the camera 111 It is used both for detecting the detection timing and for detecting the predetermined action.

The processing control unit 144 controls the sleep state processing unit 142 according to the detection result of the timing detection unit (behavior detection processing unit) 143 . More specifically, in this embodiment, the processing control unit 144 determines whether or not to execute the sleep state processing in the sleep state processing unit 142 according to the detection result of the timing detection unit (behavior detection processing unit) 143. to control. More specifically, for example, the processing control unit 144 detects that the detection result of the behavior detection processing unit (timing detection unit) 143 is a state suitable for detecting a sleep state, and when it is the detection timing, the sleep state processing unit 142 to execute the sleep state processing, and if the detection result of the behavior detection processing unit (timing detection unit) 143 is a state unsuitable for sleep state detection and it is not the detection timing, the sleep state processing unit 142 The sleep state processing is not executed.

When the nurse call reception operation unit 13 receives a nurse call, the nurse call processing unit 145 notifies the management server device SV of a first event notification communication signal containing the fact as another example of the predetermined event, By using the SU sound input/output unit 12 and the like, voice communication is performed between the terminal devices SP and TA. More specifically, when the nurse call reception operation unit 13 is operated for input, the nurse call processing unit 145 transmits the first event notification communication signal containing the nurse call as the sensor ID of the device itself and the event information by SU communication. It is transmitted to the management server device SV via the IF section 15 . Then, the nurse call processing unit 145 uses the SU sound input/output unit 12 and the like to perform voice communication with the terminal devices SP and TA by, for example, VoIP (Voice over Internet Protocol).

When a video distribution request is received from the fixed terminal SP or the mobile terminal TA via the communication IF section 3, the SU streaming processing section 146 sends a stream to the requesting fixed terminal SP or mobile terminal TA. , a moving image (for example, a live moving image) generated by the camera 11 is distributed via the SU communication IF unit 15 by streaming reproduction.

FIG. 1 shows, as an example, four first to fourth sensor units SU-1 to SU-4 provided corresponding to the monitored person Ob. is installed in a living room RM-1 (not shown) of Mr. A Ob-1 who is one of the monitored persons Ob, and the second sensor unit SU-2 is installed in Mr. B Ob- who is one of the monitored persons Ob. 2, and the third sensor unit SU-3 is installed in the living room RM-3 (not shown) of Mr. C Ob-3, who is one of the monitored persons Ob. , and the fourth sensor unit SU-4 is installed in a living room RM-4 (not shown) of Mr. D Ob-4 who is one of the monitored persons Ob.

In the sensor device SU configured as described above, the sensor unit 11, the sleep state processing unit 142, the behavior detection processing unit (timing detection unit) 143, and the processing control unit 144 constitute an example of a sleep state detection device.

The management server device SV has a communication function for communicating with the other devices SU, TA, and SP via the network NW, and upon receiving notification of the predetermined event by the first event notification communication signal from the sensor device SU, Information (monitoring information) related to monitoring of the observer Ob is managed, the predetermined event is notified (re-notified, notified, transmitted) to the predetermined terminal devices SP and TA by the second event notification communication signal, and the client (this implementation It is a device that provides the client with data according to a request from a terminal device (SP, TA, etc.) and manages the entire monitored person monitoring support system MS. In the present embodiment, the monitoring information includes, for example, the sensor ID of the sensor device SU which is the transmission source of the first event notification communication signal, the predetermined event (sleeping state (sleep degree) obtained by the sensor device SU, the sensor The type of predetermined action detected by the device SU, the nurse call received by the sensor device SU), the image of the monitored person Ob, the time when the notification was received (notification time), etc. These are mutually They are associated and stored (recorded and managed). The monitoring information is contained in the second event notification communication signal. Said second event notification communication signal may be transmitted, for example, by broadcast communication (broadcast or multicast), and for example, one or more sensors associated with the sensor device SU from which said first event notification communication signal was transmitted. terminal devices SP, TA. Such a management server device SV can be configured by, for example, a computer with a communication function.

The fixed terminal device SP has a communication function to communicate with other devices SU, SV, and TA via the network NW, a display function to display predetermined information, and an input function to input predetermined instructions and data. By inputting predetermined instructions and data given to the management server device SV and the portable terminal device TA, and displaying the monitoring information obtained by the sensor device SU in response to a notification from the management server device SV, It is a device that functions as a user interface (UI) of the person monitoring support system MS. Such a fixed terminal device SP can be composed of, for example, a computer with a communication function.

The mobile terminal device TA has a communication function of communicating with other devices SV, SP, and SU via the network NW, a display function of displaying predetermined information, an input function of inputting predetermined instructions and data, and a voice call. inputting predetermined instructions and data given to the management server device SV and the sensor device SU, displaying the monitoring information obtained by the sensor device SU in response to a notification from the management server device SV, It is a device for responding to a nurse call or calling out to the sensor unit SU through voice communication. Such a mobile terminal device TA can be configured by a portable communication terminal device such as a so-called tablet computer, a smart phone, or a mobile phone, for example.

Next, the operation of this embodiment will be described. FIG. 3 is a flow chart showing the operation of the sensor device in service mode. FIG. 4 is a flow chart showing the operation of the sensor device in the sleep state determination process shown in FIG. FIG. 5 is a flow chart showing the operation of the sensor device in the fall determination process shown in FIG. FIG. 6 is a flow chart showing the operation of the sensor device in the overturn determination process shown in FIG. FIG. 7 is a flow chart showing the operation of the sensor device in the bed entry determination process shown in FIG. FIG. 8 is a flow chart showing the operation of the sensor device in the bed leaving determination process shown in FIG. FIG. 9 is a flow chart showing the operation of the sensor device in the micro-motion abnormality determination process shown in FIG. FIG. 10 is a diagram showing an example of a main screen displayed on the terminal device. FIG. 11 is a diagram showing an example of a first setting screen displayed on the terminal device. FIG. 12 is a diagram showing an example of a second setting screen displayed on the terminal device. FIG. 13 is a diagram showing an example of a monitoring information screen displayed on the terminal device. FIG. 14 is a diagram showing another example of the monitoring information screen displayed on the terminal device. FIG. 15 is a diagram showing still another example of the monitoring information screen displayed on the terminal device. FIGS. 13A, 14A, and 15A are diagrams showing monitoring information screens displaying event details in text, respectively, and FIGS. 13B, 14B, and 15B are respectively diagrams showing monitoring information screens displaying event details in icons. It is a figure which shows an information screen.

In the monitored person monitoring support system MS configured as described above, each device SU, SV, SP, and TA initializes each necessary part when the power is turned on, and starts its operation. In the sensor device SU, by executing the control processing program, the SU control processing unit 14 includes an SU control unit 141, a sleep state processing unit 142, an action detection processing unit (timing detection unit) 143, a processing control unit 144, a nurse call A processing unit 145 and a streaming processing unit 146 are functionally configured.

When the power is turned on and operation is started, the terminal devices SP and TA, for example, the fixed terminal device SP, accept a login operation by an observer (user) such as a nurse or caregiver. In the login operation, an ID such as a user name of the supervisor (user) is input and stored in the storage unit. When the fixed terminal device SP logs into the monitored person monitoring support system MS by a login operation, the fixed terminal device SP displays a main screen on its display unit.

The main screen is a screen for accepting the operation mode of the fixed terminal SP. The main screen 21 includes, for example, a "service mode" button 211 and a "setting mode" button 212, as shown in FIG. The "service mode" button 211 is a button for requesting a service mode for monitoring the monitored person Ob as the operation mode of the fixed terminal SP, and is an instruction (command, command) to operate the fixed terminal SP in the service mode. ) is a button for entering The "setting mode" button 212 is a button for requesting a setting mode for setting predetermined parameters as the operation mode of the fixed terminal SP, and is used to issue an instruction to operate the fixed terminal SP in the setting mode. It is a button for input.

When an input operation is received from the input unit of the fixed terminal device SP while the main screen 21 is being displayed, the fixed terminal device SP causes the control processing unit of the fixed terminal device SP to change the received input operation to "service mode." ” button 211 or the “setting mode” button 212 .

As a result of this determination, if the accepted input operation is the input operation of the "service mode" button 211, the fixed terminal device SP starts operating in the service mode so as to monitor the monitored person Ob. The input operation is, for example, an operation of moving the mouse to move the cursor onto the "service mode" button 211 and left-clicking the mouse.

On the other hand, as a result of the determination, if the accepted input operation is the input operation of the "setting mode" button 212, the fixed terminal SP starts operating in the setting mode so as to set the predetermined parameters. .

In the setting mode, as described above, when the input operation of the "setting mode" button 212 is accepted, the fixed terminal device SP displays the setting screen on its display unit. In this embodiment, authentication is performed using an ID such as a user name, and only authorized users (configurers) can enter the setting mode.

The setting screen is a screen for inputting and setting the predetermined parameters. In the present embodiment, the predetermined parameter is a set value used to appropriately determine the predetermined event. , and the like (determination threshold). More specifically, in this embodiment, the predetermined parameters are the frame rate, brightness level, ceiling height, and the area where the bedding BD is located. The setting screen includes two first and second setting screens in this embodiment. The first setting screen is used for inputting (determining) the monitored person Ob (sensor device SU) for which parameter setting is to be performed. , brightness level and ceiling height). The second setting screen is a screen for inputting and setting parameters other than numerical parameters (the location area of the bedding BD in this embodiment) among the predetermined parameters.

More specifically, when the input operation of the "setting mode" button 212 is received from the input unit, the fixed terminal device SP displays the first setting screen on the display unit 4 by the control processing unit.

For example, as shown in FIG. 12, the first setting screen 22 includes a monitored person name input setting area 221 for inputting and setting the name of the monitored person Ob, and an area 221 for inputting and setting the frame rate. A frame rate input setting area 222, a brightness level input setting area 223 for inputting and setting the brightness level, a ceiling height input setting area 224 for inputting and setting the ceiling height, and " It has an area setting button 225 and a return to main screen button 226 . Each of the input setting areas 221 to 224 has an input field for parameter values and an "update" button for storing and setting the parameters with the parameter values input in the input fields. The "area setting" button 225 is a button for requesting the second setting screen, and is a button for inputting an instruction to display the second setting screen on the fixed terminal device SP. The "return to main screen" button 226 is a button for requesting the main screen 21, and is a button for inputting an instruction to display the main screen 21 on the fixed terminal device SP. The "return to main screen" button 226 is also a button for setting each parameter value stored as a parameter in the storage section to the sensor device SU via the management server device SV.

In the example shown in FIG. 11, "Mr. A", "15 fps", "5" and "2.4 m" are entered in the input fields in the input setting areas 221 to 224, respectively. When the "Update" button in the monitored person name input setting area 221 is operated, the control processing section of the fixed terminal SP sets the parameter setting target to "Mr. A", and the storage section of the fixed terminal SP stored in It should be noted that the sensor device SU monitoring "Mr. A" or its sensor ID may be input and set as a parameter setting target. When the "Update" button in the frame rate input setting area 222 is operated, the frame rate is set to "15 fps" by the control processing section of the fixed terminal device SP, and stored in the storage section of the fixed terminal device SP. In this embodiment, the sensor unit SU executes the event detection operation for each frame or every several frames, so setting the frame rate defines the execution interval of the event detection operation. When the "update" button in the brightness level input setting area 223 is operated, the brightness level is set to "5" by the control processing section of the fixed terminal SP and stored in the storage section of the fixed terminal SP. be. If the target image captured by the sensor unit SU is too dark or too bright, it becomes difficult to extract the person area of the monitored person Ob from the target image, and it becomes difficult to identify the person. By setting the brightness level, the target image can be generated with appropriate exposure, the person area can be extracted more appropriately, and the person can be identified more appropriately. When the "update" button in the ceiling height input setting area 224 is operated, the ceiling height is set to "2.4 m" by the control processing section of the fixed terminal device SP, and is stored in the storage section of the fixed terminal device SP. remembered. The lying posture determination threshold values and the like for when the sensor device SU is installed on a ceiling with a reference ceiling height (for example, 2.4 m) are pre-stored in the sensor device SU as default values. By setting the ceiling height, the default lying posture determination threshold value is corrected, and the corrected lying posture determination threshold value is used to detect the predetermined behavior described above. By setting the ceiling height, the lying posture determination threshold value can be adjusted according to the situation of the actually installed sensor device SU.

When the input operation of the "area setting" button 225 is accepted, the fixed terminal device SP displays the second setting screen on its display unit.

The second setting screen 23 includes, for example, a target image display area 231 for displaying a target image, an "update" button 232, and a "return" button 233, as shown in FIG. In the target image display area 231, the camera 111 of the sensor unit SU that monitors the monitored person Ob having the name of the monitored person Ob input in the monitored person name input setting area 221 of the first setting screen 22 is displayed. The generated target image is displayed. The “update” button 232 is a button for requesting parameter setting in the area input from the input unit while referring to the target image displayed in the target image display area 231 . The "return" button 233 is a button for requesting the first setting screen 22, and is a button for inputting an instruction to display the first setting screen 22 on the fixed terminal device SP.

In such a second setting screen 23, when the second setting screen 23 is displayed, the fixed terminal device SP waits for input of the location area of the bedding BD. When the user inputs the four vertices of the location area of the bedding BD from the input unit and operates the "update" button 232, the control processing unit of the fixed terminal device SP controls each of the four vertices of the location area of the bedding BD. A pixel position is input and stored in the storage unit, and the location area of the bedding BD is set. FIG. 12 shows how the user has input the four vertices of the location area of the bedding BD.

Then, when the “return” button 233 is operated for input, the first setting screen 22 is displayed on the display section 4 . Further, when the "return to main screen" button 226 is operated to input, the main screen 21 is displayed. SV to the sensor device SU, and the sensor device SU stores and sets the received parameter values in the SU storage unit 16 of its own device SU. Thereby, each value of the predetermined parameter is set in the sensor device SU.

On the other hand, when the input operation of the "service mode" button 211 is accepted on the main screen 21 described above, the fixed terminal device SP operates in the service mode, and the monitored person monitoring support system MS operates as follows. monitors the monitored person Ob.

In monitoring the monitored person Ob, the sensor unit SU detects the sleep state of the monitored person Ob by operating as follows for each frame or every few frames. Predetermined motions are detected, and whether or not a nurse call has been received is determined.

In FIG. 3, first, the sensor unit SU acquires an image of one frame as a target image from the camera 111 of the sensor unit 11 by the SU control processing unit 14 (S1).

Subsequently, the sensor unit SU extracts a person area from the target image acquired from the camera 111 in the process S1 by, for example, the background subtraction method, by the behavior detection processing unit (timing detection unit) 143 of the SU control processing unit 14 (S2). .

Subsequently, the sensor device SU determines whether or not it is time to detect a sleeping state by the timing detection unit (behavior detection processing unit) 143 (S3). More specifically, in the present embodiment, the timing detection unit 143 determines whether the person area extracted in process S2 completely overlaps the location area of the bedding BD (the person area completely overlaps the bedding BD). is within the location area). As a result of this determination, when the person area completely overlaps the location area of the bedding BD, the timing detection unit 143 determines that the state is suitable for detecting the sleeping state and that it is time to detect the sleeping state ( Yes), this detection result is notified to the process control unit 144, and then process S4 is executed. On the other hand, as a result of the determination, the timing detection unit 143 determines that the person area is not suitable for sleep state detection except when the person area completely overlaps the location area of the bedding BD. It is determined that it is not the timing (No), the detection result is notified to the process control unit 144, and then process S5 is executed. If it is determined that it is not the sleep state detection timing (No), the timing detection unit 143 may not notify the processing control unit 144 of the detection result, and the process S5 may be executed next.

In process S<b>4 , the sensor device SU controls the sleep state processing section 142 with the processing control section 144 of the SU control processing section 14 .

More specifically, when the processing control unit 144 receives from the action detection processing unit (timing detection unit) 143 a detection result indicating that it is not the sleep state detection timing, the sleep state processing unit 142 detects the sleep state. Next, without executing the process, the process S10 is executed, and on the other hand, when the detection result indicating that it is time to detect the sleep state is received from the behavior detection processing unit (timing detection unit) 143, the sleep state After causing the processing unit 142 to execute the sleep state processing, next, processing S10 is executed.

In this sleep state processing, in FIG. 4, first, the sleep state processing unit 142 determines whether or not the sleep state determination condition is satisfied based on the Doppler signal of the Doppler sensor 112 (S41). More specifically, the sleep state processing unit 142 first acquires from the SU storage unit 16 Doppler signals measured within a predetermined period of time, for example, within one minute from the current time to the past. Subsequently, the sleep state processing unit 142 performs, for example, fast Fourier transform (FFT) on the acquired 1-minute Doppler signal. Subsequently, the sleep state processing unit 142 obtains the average value of the amplitude in the frequency band corresponding to the general respiration frequency from the spectrum obtained by this FFT. Subsequently, the sleep state processing unit 142 compares the obtained average value with the sleep degree classification thresholds, which are the first and second sleep degree classification thresholds in the present embodiment. As a result of this comparison, if the calculated average value exceeds the first sleep level threshold, the sleep state processing unit 142 temporarily determines that the user is awake, and the average exceeds the first sleep level threshold. If the duration of the state of being in a state exceeds a preset predetermined continuation determination time (continuation determination time during wakefulness), determine the tentatively obtained wakefulness as the final degree of sleep (Yes), Next, process S42 is executed. As a result of the comparison, the sleep state processing unit 142 temporarily obtains a low sleep degree when the obtained average value is equal to or less than the first sleep degree division threshold and exceeds the second sleep degree division threshold, The duration of the state in which the average value is equal to or less than the first sleep degree division threshold and exceeds the second sleep degree division threshold exceeds a preset predetermined continuation determination time (during low sleep continuation determination time). In this case, the tentatively obtained low degree of sleep is determined as the final degree of sleep (Yes), and then processing S42 is executed. As a result of the comparison, if the calculated average value is equal to or less than the second sleep degree classification threshold, the sleep state processing unit 142 temporarily obtains a high degree of sleep, and if the average value is equal to or less than the second sleep degree classification threshold, If the duration of a certain state exceeds a preset predetermined continuation determination time (during high sleep continuation determination time), the tentatively obtained high sleep level is determined as the final sleep level (Yes). , then the process S42 is executed. Then, as a result of the comparison, in other cases than this, the sleep state processing unit 142 determines that the sleep state is being provisionally determined (No), and terminates this sleep state processing.

In the process S42, the sleep state processing unit 142 determines the sleep state and obtains the sleep state information indicating that the sleep state has been detected (for example, the sleep state detection flag indicating whether or not the sleep state has been detected changes from "0" to " 1") is stored in the SU storage unit 16, and the state variable is updated with "sleep level (either of awake, low sleep level, or high sleep level)" (state variable ← "sleep level" ), ending the sleep state processing.

Returning to FIG. 3, in the process S5, the sensor device SU causes the behavior detection processing section 143 of the control processing section 14 to execute the fall determination process based on the person area extracted in the process S2.

More specifically, in this fall determination process, first, in FIG. 5, the action detection processing unit 143 determines whether or not a fall determination condition is satisfied based on the person area extracted in the process S2 (S51). ). More specifically, the action detection processing unit 143 determines that the size of the head region of the human region extracted from the target image in the process S2 is equal to or smaller than the lying posture determination threshold, and the change rate of the size of the head region is equal to or greater than the fall determination speed threshold and the person area is within the fall determination area set around the location area of the bedding BD, it is determined that there is a fall (Yes), and then the process S52 is executed, and in other cases, it is determined that there is no fall (No), and this fall determination process is terminated. Here, the head region is extracted from the human region by, for example, circular or elliptical Hough transform, pattern matching using a head model (template) prepared in advance, or head extraction. extracted by a neural network trained for In the process S52, the action detection processing unit 143 determines the occurrence of a fall and stores information indicating the presence of a fall (for example, a fall flag indicating the presence or absence of a fall is changed from "0" to "1"). The data is stored in the unit 16, and the fall determination process ends.

Returning to FIG. 3, in the process S6 following the process S5, the sensor device SU causes the action detection processing unit 143 to execute the fall determination process based on the person area extracted in the process S2.

More specifically, in this fall determination process, first, in FIG. 6, the action detection processing unit 143 determines whether or not a fall determination condition is satisfied based on the person area extracted in the process S2 (S61). ). More specifically, the action detection processing unit 143 determines that the size of the head region of the human region extracted from the target image in the process S2 is equal to or smaller than the lying posture determination threshold value, and the change rate of the size of the head region is greater than or equal to the overturn determination speed threshold, and the person area is in the area excluding the location area of the bedding BD and the fall determination area, it is determined that there is overturn (Yes), and then the process S62 is executed. Otherwise, it is determined that there is no overturn (No), and the overturn determination process is terminated. In the process S62, the action detection processing unit 143 determines the occurrence of a fall and stores information indicating the presence of a fall (for example, a fall flag indicating whether or not a fall has occurred is changed from "0" to "1"). The data is stored in the unit 16, and the turnover determination process ends.

Returning to FIG. 3, in the process S7 following the process S6, the sensor device SU causes the behavior detection processing unit 143 to execute bed-in determination processing based on the person area extracted in the process S2.

More specifically, in this bed-going determination process, first, in FIG. 7, the action detection processing unit 143 determines whether or not the state variable (previous state variable) is (S71). As a result of this determination, if the state variable is not "leaving bed" (No), the action detection processing unit 143 terminates the regular bed determination processing. On the other hand, if the result of the determination is that the state variable is "getting out of bed" (Yes), the action detection processing unit 143 next executes processing S72. In this process S72, the action detection processing unit 143 determines whether or not the determination condition for entering the bed is satisfied based on the person area extracted in the process S2. More specifically, the behavior detection processing unit 143 detects the human area extracted from the target image in the process S2 this time when it completely overlaps the location area of the bedding BD (the person area is completely within the location area of the bedding BD). If it is), it is tentatively determined to be in bed, and if the duration of the complete superimposed state continues beyond the determination time of in-bed continuation, it is finally determined to be in bed (Yes), and then, Processing S73 is executed, and in other cases, it is determined that there is no bed admission (No), and the regular bed admission determination processing is terminated. In the process S73, the action detection processing unit 143 determines the occurrence of bed entry and determines whether bed entry information indicating the presence of bed entry (for example, the bed entry flag representing the presence or absence of bed entry is changed from "0" to "1"). ) is stored in the SU storage unit 16 . Subsequently, the action detection processing unit 143 updates the state variable with “entering bed” (state variable←“entering bed”) (S74), and ends the regular entering bed determination process.

Returning to FIG. 3, in the process S8 following the process S7, the sensor device SU causes the behavior detection processing unit 143 to execute the bed leaving determination process based on the person area extracted in the process S2.

More specifically, in this bed leaving determination process, first, in FIG. 8, the action detection processing unit 143 determines whether or not the state variable (previous state variable) is (S81). As a result of this determination, if the state variable is neither "entering bed" nor "awakening" (No), the behavior detection processing unit 143 terminates this bed leaving determination processing. On the other hand, if the result of the determination is that the state variable is either "entering bed" or "awakening" (Yes), the behavior detection processing unit 143 next executes processing S82. In this process S82, the action detection processing unit 143 determines whether or not the condition for determining leaving the bed is satisfied based on the person area extracted in the process S2. More specifically, the behavior detection processing unit 143 temporarily determines that the person is leaving the bed when the protruding region in which the person region extracted from the target image in the process S2 protrudes from the location region of the bedding BD is equal to or larger than the bed leaving determination threshold value. , when the duration of the protruding region in a state equal to or greater than the bed-leaving determination threshold exceeds the bed-leaving continuation determination time, it is finally determined that there is bed-leaving (Yes), and then the process S83 is executed. Otherwise, it is determined that there is no getting out of bed (No), and the process of determining getting out of bed is terminated. In the process S83, the action detection processing unit 143 determines the occurrence of leaving the bed, and stores the presence of bed leaving information indicating presence of leaving the bed (for example, the bed leaving flag indicating presence/absence of leaving the bed is changed from "0" to "1"). Store in section 16 . Subsequently, the action detection processing unit 143 updates the state variable with "getting out of bed" (state variable←"getting out of bed") (S84), and ends this bed-leaving determination process.

Returning to FIG. 3 , in process S9 subsequent to process S8, the action detection processing unit 143 of the sensor unit SU performs a subtle motion abnormality determination process based on the Doppler signal of the Doppler sensor 112 in the sensor unit 11 .

More specifically, in this micro-motion abnormality determination process, first, in FIG. Determine (S91). More specifically, the action detection processing unit 143 first acquires from the SU storage unit 16 Doppler signals measured within a predetermined period of time, for example, within one minute from the current time to the past. Subsequently, the action detection processing unit 143 performs, for example, a fast Fourier transform (FFT) on the acquired Doppler signal for one minute. Subsequently, the action detection processing unit 143 obtains the average value of the amplitude in the frequency band corresponding to the general respiration frequency from the spectrum obtained by this FFT. Subsequently, the action detection processing unit 143 compares the obtained average value with the threshold value for determining abnormality of minute body movement, and if the average value obtained is equal to or less than the threshold value for determining abnormality of minute body movement, provisionally determined, and when the duration of the state in which the average value is equal to or less than the micro-motion abnormality determination threshold continues beyond the micro-motion abnormality continuation determination time, it is finally determined that there is a micro-motion abnormality It is determined (Yes), and then the process S92 is executed. Otherwise, it is determined that there is no abnormality in minute body movement (No), and this abnormality determination process for minute body movement is finished. In the process S92, the action detection processing unit 143 determines the occurrence of an abnormality in minute body movement, and determines the presence of abnormality in minute movement (for example, the abnormality in minute movement flag indicating the presence or absence of an abnormality in minute movement). changed from "0" to "1") is stored in the SU storage unit 16, and the microbody movement abnormality determination processing is terminated.

Returning to FIG. 3, next, the sensor device SU determines whether or not notification of a predetermined event is required by the SU control processing unit 14 (S10). More specifically, the sensor device SU stores the sleep state detected information in the SU storage unit 16 by the sleep state processing unit 142 of the SU control processing unit 14, and the SU storage unit by the action detection processing unit 143 of the SU control processing unit 14. 16, for example, whether or not the result such as the falling information or the falling information is stored, and if the result is stored in the SU storage unit 16, the SU control processing unit 14 After determining that the notification is necessary (Yes), and then executing the processing S11, the present processing is terminated, and on the other hand, if the result is not stored in the SU storage unit 16, the SU control processing unit 14 determines that the notification is unnecessary (No), and terminates this processing.

In this process S11, the sensor device SU sends a first event notification communication signal containing the result as the event information in order to notify the result event to the predetermined terminal devices SP and TA by the SU control processing unit 14. is sent to the management server device SV. In one specific example, when the SU storage unit 16 stores information indicating that there is leaving the bed, the SU control processing unit 14 stores the sensor ID of the machine itself, "getting out of bed" as the event information, and A first event notification communication signal containing the used target image is transmitted to the management server device SV via the SU communication IF section 15 .

Then, while the processes S1 to S11 are being executed as described above, the sensor unit SU determines whether or not a nurse call is being received by the nurse call processing unit 145 of the SU control processing unit 14. Then, when the nurse call is received, the sensor device SU sends a first event notification communication signal containing the received nurse call as the event information in order to notify the reception of the nurse call to the predetermined terminal devices SP and TA. is sent to the management server device SV.

When the management server device SV receives the first event notification communication signal from the sensor device SU via the network NW, the management server device SV converts each information such as the sensor ID and the event information contained in the first event notification communication signal to the sensor ID. is stored (recorded) as monitoring information of the monitored person Ob who is monitored by the sensor device SU. That is, the management server device SV stores each information such as the sensor ID and the event information contained in the first event notification communication signal in association with each other. Then, the management server device SV transmits the second event notification communication signal to the terminal devices SP and TA of the notification destination corresponding to the sensor device SU that is the transmission source (notification source) in the received first event notification communication signal.

When the fixed terminal device SP and the mobile terminal device TA receive the second event notification communication signal from the management server device SV via the network NW, the fixed terminal device SP and the mobile terminal device TA receive the sensor ID and the event information contained in the second event notification communication signal. Each piece of information is stored (recorded) as monitoring information of the monitored person Ob monitored by the sensor device SU having this sensor ID, and the monitoring information is displayed. For example, when the sensor device SU detects "being awake" and is notified of this "being awake" by the second event notification communication signal, the event information shown in FIG. 13A is displayed in text on the terminal devices SP and TA. A monitoring information screen 31a is displayed. Alternatively, the monitoring information screen 31b displaying the event information shown in FIG. 13B with icons is displayed on the terminal devices SP and TA. Further, for example, when the sensor device SU detects the "low sleep level" and is notified of the "low sleep level" by the second event notification communication signal, the event information shown in FIG. 14A is sent to the terminal devices SP and TA. A monitoring information screen 31a displayed in text is displayed. Alternatively, the monitoring information screen 31b displaying the event information shown in FIG. 14B with icons is displayed on the terminal devices SP and TA. Further, for example, when the sensor device SU detects a "high sleep level" and is notified of this "high sleep level" by the second event notification communication signal, the terminal devices SP and TA receive the event information shown in FIG. 14A. A monitoring information screen 31a displayed in text is displayed. Alternatively, the monitoring information screen 31b displaying the event information shown in FIG. 14B with icons is displayed on the terminal devices SP and TA.

As described above, the monitored person monitoring support system MS and the sleep state detection device and sleep state detection method incorporated in the sensor device SU according to the present embodiment are based on the detection by the timing detection unit (behavior detection processing unit) 143. Since the sleeping state processing unit 142 is controlled according to the result, for example, the subject (monitored Person) While obtaining the sleep state of Ob, it is possible to perform control so as not to obtain the sleep state of the subject Ob in a state unsuitable for detection of the sleep state. The sleep state detection device and sleep state detection method can further reduce erroneous determinations.

The monitored person monitoring support system MS, the sleep state detection device, and the sleep state detection method described above, for example, execute sleep state processing in a state suitable for detecting a sleep state, or conversely, perform sleep state processing in a state unsuitable for detection of a sleep state. By controlling the sleep state processing in the sleep state processing unit 142 such that the sleep state processing is not executed in , erroneous determination can be further reduced.

In the above-described embodiment, the processing control unit 144 controls whether or not to execute sleep state processing in the sleep state processing unit 142 according to the detection result of the timing detection unit (behavior detection processing unit) 143. , the processing control unit 144 may control the sleep state processing unit 142 according to the detection result of the timing detection unit (behavior detection processing unit) 143, as in the following modifications.

In the first modification, the processing control unit 144 controls whether or not to output the sleep state obtained by the sleep state processing unit 142 according to the detection result of the timing detection unit 143 . In the first modification, the sleep state processing is executed by the sleep state processing unit 142, but by controlling the output of the result, an effect equivalent to that obtained by controlling the execution can be obtained. More specifically, in the processing S4, when the processing control unit 144 receives a detection result from the timing detection unit 143 indicating that it is not the sleep state detection timing, the sleep state processing unit 142 receives the request. On the other hand, when the timing detection unit 143 receives the detection result indicating that it is time to detect the sleep state, the sleep state processing unit 142 sends the obtained sleep degree output. The monitored person monitoring support system MS, the sleep state detection device, and the sleep state detection method can, for example, output the sleep state obtained by the sleep state processing unit 142 in a state suitable for detecting the sleep state, or Further, by controlling the sleep state processing unit 142 so as not to output the sleep state obtained by the sleep state processing unit 142 in a state unsuitable for detection of the sleep state, erroneous determination can be further reduced.

In the second modification, the processing control unit 144 controls whether or not the sleep state obtained by the sleep state processing unit 142 is supplemented by the detection result of the timing detection unit 143 according to the detection result of the timing detection unit 43. do. More specifically, in the processing S4, when the processing control unit 144 receives from the timing detection unit 143 the detection result indicating that it is not the sleep state detection timing, the sleep state processing unit 142 Additional information indicating that the sleep level is obtained in a state other than the detection timing of the sleep state, such as "low accuracy" or "low reliability", is added to the sleep level, while the timing detection unit 143, When receiving the detection result indicating that it is time to detect the sleep state, the sleep state processing unit 142 is caused to output the obtained degree of sleep as it is. Such a monitored person monitoring support system MS, a sleep state detection device, and a sleep state detection method can, for example, directly output the sleep state obtained by the sleep state processing unit 142 in a state suitable for detection of the sleep state, Conversely, information indicating that the sleep state is unsuitable for detection of the sleep state is added to the sleep state obtained by the sleep state processing unit 142 in a state unsuitable for detection of the sleep state. Since it is possible to assist the sleeping state obtained by the processing unit 142 with the detection result of the timing detection unit 143, etc., erroneous determination can be further reduced.

In the third modification, the sleep state processing determines the degree of sleep representing the degree of depth of sleep of the target person (monitored person) Ob by using a predetermined threshold for determining the degree of sleep and the first measurement result of the first sensor. The processing control unit 144 changes the sleep state determination threshold according to the detection result of the timing detection unit 143 as the control. In the above-described embodiment, the first and second sleep degree classification thresholds correspond to an example of the sleep degree determination threshold, the Doppler sensor 112 corresponds to an example of the first sensor, and more specifically, the process In S4, when the processing control unit 144 receives from the timing detection unit 143 the detection result indicating that it is not the sleep state detection timing, the first and second sleep degree classification thresholds are set to the low sleep degree and the high sleep degree. The sensor device SU is set so that the sleep level is changed to a value that is difficult to determine as a degree of sleep, thereby substantially not requiring a low sleep level or a high sleep level. On the other hand, when the processing control unit 144 receives the detection result indicating that it is time to detect the sleep state from the timing detection unit 143, the first and second sleep degree classification thresholds are set to low sleep degree and high sleep degree. The sensor device SU is set so that the low sleep degree and the high sleep degree can be substantially easily obtained. The monitored person monitoring support system MS, the sleep state detection device, and the sleep state detection method set the sleep state determination threshold value to a value that is easy to determine the sleep state, for example, in the case of a state suitable for detecting the sleep state. Conversely, if the sleep state is unsuitable for detecting the sleep state, it is possible to change the sleep state degree determination threshold value to a value that makes it difficult to determine the sleep state, thereby further reducing erroneous determination. can.

Also, preferably, the sleep state processing unit 142 uses any one of a plurality of different learning models machine-learned so as to determine the sleep state as the sleep state processing, and uses the learning model of the first sensor. The sleep state of the subject (monitored person) Ob is obtained based on the first measurement result, and the processing control unit 144 selects from among the plurality of learning models according to the detection result of the timing detection unit 143 as the control. , the learning model used in the sleep state processing unit 142 may be changed (selected). Also preferably, the sleep state processing includes a plurality of mutually different algorithms for obtaining a sleep state, and the sleep state processing unit 142 uses any one of the plurality of algorithms to determine the first sensor of the first sensor. The sleep state of the subject (monitored person) Ob is obtained based on the measurement result, and the processing control unit 144 determines the sleep state from among the plurality of algorithms according to the detection result of the timing detection unit 143 as the control. The algorithm used by the processing unit 142 may be changed (selected).

In the fourth modification, the sleep state processing is performed at predetermined time intervals by setting the degree of sleep representing the degree of depth of sleep of the subject (monitored person) Ob to a predetermined threshold for determining the degree of sleep. and the first measurement result of the first sensor to temporarily obtain the sleep state, and if the temporarily obtained degree of sleep continues for a preset continuation determination time, the temporarily obtained This is a process of determining the degree of sleep as the final degree of sleep, and the processing control unit 144 changes the continuation determination time according to the detection result of the timing detection unit 143 as the control. In the above-described embodiment, the first and second sleep degree classification thresholds correspond to an example of the sleep degree determination threshold, the Doppler sensor 112 corresponds to an example of the first sensor, and the awake continuation determination time is low. The continuous determination time during sleep and the continuous determination time during high sleep correspond to an example of the continuous determination time. More specifically, in the processing S4, when the processing control unit 144 receives a detection result indicating that it is not the sleep state detection timing from the timing detection unit 143, the awake continuation determination time, low sleep Changed the medium continuous determination time and the high sleep continuous determination time to values that are difficult to determine as being awake, low sleep level, or high sleep level. The sensor unit SU is set so that it cannot be detected. On the other hand, when the processing control unit 144 receives from the timing detection unit 143 the detection result indicating that it is the detection timing of the sleep state, the determination time during continuous awakening, the continuous determination time during low sleep, and the continuous determination time during high sleep Each of the determination times is changed to a value that makes it easy to determine that the user is awake, has a low sleep level, or has a high sleep level. set. The monitored person monitoring support system MS, the sleep state detection device, and the sleep state detection method change the continuation determination time to a value that makes it easy to determine the sleep state, for example, when the state is suitable for detecting the sleep state. Or conversely, in the case of a state unsuitable for detection of the sleeping state, it is possible to change the continuous determination time to a value that makes it difficult to determine the sleeping state, so that erroneous determination can be further reduced.

From the viewpoint of detecting the sleep state with higher accuracy, the processing control unit 144 receives the detection result indicating that it is time to detect the sleep state from the timing detection unit 143. , the continuation determination time during low sleep and the continuation determination time during high sleep are respectively determined by the continuation determination time during awakening and the continuation determination time during low sleep when the detection result that it is not the detection timing of the sleep state is received from the timing detection unit 143 It may be changed to a longer value than each of the determination time and the high sleep continuation determination time.

In the fifth modification, the first sensor includes a plurality of sub-sensors that measure the target person (monitored person) Ob, and the sleep state processing unit 142 performs the sleep state processing on the plurality of sub-sensors of the first sensor. Based on the sub-measurement result of one of the sub-sensors, the processing control unit 144 obtains the sleep state of the subject Ob based on the sub-measurement result of one of the sub-sensors, and the processing control unit 144 performs the above-described control according to the detection result of the timing detection unit 143, among the plurality of sub-sensors , the sub-sensor used by the sleep state processing unit 142 is changed. In such a case, the plurality of sub-sensors may be different types of sensors from each other, or the plurality of sub-sensors may be of the same type but at different locations (locations).

More specifically, for example, the first sensor is placed between the non-contact Doppler sensor 112 and the bedding BD and the target person (monitored person) Ob to detect the weight of the person. When the processing control unit 144 receives from the timing detection unit 143 the detection result indicating that it is not the sleep state detection timing, the sub-sensor used in the sleep state processing unit 142 is set as described above. A mat sensor is used, and the sleep state processing unit 142 obtains the sleep state of the subject Ob based on the output of the mat sensor. Techniques for obtaining a sleep state using such a mat sensor are disclosed, for example, in Japanese Unexamined Patent Application Publication No. 2013-198654 and Japanese Unexamined Patent Application Publication No. 2014-223174. Further, for example, a sleep meter SleepScan SL-504 (manufactured by Tanita Corporation) or a sleep meter HSL-101 (manufactured by Omron Corporation) may be used as sub-sensors. On the other hand, when the processing control unit 144 receives a detection result indicating that it is time to detect a sleep state from the timing detection unit 143, the sub-sensor used in the sleep state processing unit 142 is changed to the Doppler sensor 112, and sleep detection is performed. Based on the output of the Doppler sensor 112, the state processing unit 142 is caused to obtain the sleep state of the subject Ob.

Further, for example, the first sensor includes a first Doppler sensor arranged on the ceiling facing the bedding BD with the detection direction directed toward the bedding BD (with the detection direction directed vertically downward), and a side wall of the bedding DB. and a second Doppler sensor arranged with its direction facing the bedding BD (with its detection direction facing the horizontal direction). When the detection result of the lying posture is received from the timing detection unit 143, the processing control unit 144 changes the sub-sensor used in the sleep state processing unit 142 to the first Doppler sensor, and sleeps. The state processing unit 142 is caused to obtain the sleep state of the subject Ob based on the output of the first Doppler sensor. On the other hand, when the processing control unit 144 receives the detection result of the sideways lying posture from the timing detection unit 143, the sub sensor used in the sleep state processing unit 142 is changed to the second Doppler sensor, The sleep state processing unit 142 is caused to obtain the sleep state of the subject Ob based on the output of the second Doppler sensor. The posture is extracted, for example, by pattern matching using each model (each template) of a lying posture and a sideways lying posture prepared in advance, or by, for example, a neural network trained for extracting these postures.

Further, for example, the first sensor includes a first Doppler sensor with a relatively narrow detection range, which is arranged on the ceiling facing the bedding BD and faces the bedding BD, and a first Doppler sensor with a relatively wide detection range, which is arranged on the ceiling. When the processing control unit 144 receives the detection result indicating that it is time to detect a sleep state from the timing detection unit 143, the sub sensor used in the sleep state processing unit 142 is set to the second Doppler sensor. 1 Doppler sensor is used, and the sleep state processing unit 142 obtains the sleep state of the subject Ob based on the output of the first Doppler sensor. On the other hand, when the processing control unit 144 receives from the timing detection unit 143 the detection result indicating that it is not the sleep state detection timing, the sub sensor used in the sleep state processing unit 142 is changed to the second Doppler sensor. , causing the sleep state processing unit 142 to obtain the sleep state of the subject Ob based on the output of the second Doppler sensor.

The monitored person monitoring support system MS, the sleep state detection device, and the sleep state detection method as described above select the sub-sensor used in the sleep state processing unit 142 from among the plurality of sub-sensors according to the detection result of the timing detection unit 143. Since it is changed, it is possible to select an appropriate sub-sensor to be used in the sleep state processing unit 142 from among a plurality of sub-sensors according to the detection result of the timing detection unit 143, so that erroneous determination can be further reduced.

In the sixth modification, the timing detection unit 143 obtains the posture of the subject Ob based on the second measurement result of the second sensor, and when the obtained posture is a specific posture set in advance, sleep When the state processing unit 142 outputs a first detection result indicating that it is time to detect the sleep state of the subject Ob, and the processing control unit 144 receives the first detection result from the timing detection unit 143, The sleep state processing unit 142 is controlled to obtain the sleep state of the subject Ob. Preferably, the specific posture is a supine lying posture. Also preferably, the specific posture is a lying posture other than lying down. Preferably, the second sensor is the camera 111 that generates an image, and the timing detection unit 143 obtains the posture of the subject Ob based on the image generated by the camera 111 . More specifically, for example, the timing detection unit 143 detects whether or not the subject Ob is in the recumbent posture by the above-described pattern matching or neural network, and if the subject Ob is in the recumbent posture, the above-described The head region of the subject Ob is extracted by Hough transform, pattern matching, or neural network, and whether the person is lying on his or her back is discriminated based on the area ratio of the white region and the black region in the extracted head region. For example, when the area ratio of the white area is high (area of white area > area of black area), it is determined that the posture is lying on the back, and when the area ratio of black area is high (area of white area < area of black area). is determined to be a prone lying posture. Also preferably, the second sensor is a distance image sensor that generates a distance image, and the timing detection unit 143 obtains the posture of the subject Ob based on the distance image generated by the distance image sensor. In extracting the posture from the distance image, each pixel in the distance image is associated with a part of the human body at the level of, for example, the right upper arm or the head, and joint positions are extracted based on this correspondence. The posture of the target person Ob is obtained from the position. For identification of the parts of the human body, for example, a learning model of Random Decision Forests, which is machine-learned for posture extraction, is used. Also preferably, the second sensor is a marker detection sensor that detects a predetermined marker, and the timing detection unit 143 detects the posture of the subject (monitored person) Ob based on the marker detected by the marker detection sensor. Ask for For example, the marker is worn on the covering that hits the front shoulder of the subject Ob, and the timing detection unit 143 determines that the subject Ob has a lying posture when the marker detection sensor detects the marker, and detects the marker. When the sensor does not detect the marker, it is determined that the lying posture is prone.

The monitored person monitoring support system MS, the sleep state detection device, and the sleep state detection method described above determine the sleep state by the sleep state processing unit 142 when the subject Ob is in a posture suitable for detecting the sleep state. By controlling the sleep state processing unit 142, erroneous determination can be further reduced.

In the seventh modification, as indicated by the dashed line in FIG. 2, the sensor device SU functionally further includes a clock unit 147 for clocking the date, time, and minute in the SU control processing unit 14, and at least determines the sleep time zone. The SU storage unit 16 further includes a schedule information storage unit 161 that stores schedule information representing the schedule of the target person Ob, including the schedule of the target person Ob, in association with the target person Ob. , extracts the schedule of the subject Ob corresponding to the acquired current time from the schedule information storage unit 161, and sleep state processing according to the extracted schedule of the subject Ob and the detection result of the timing detection unit 143 control unit 142; For example, if the subject Ob's schedule corresponding to the current time is a sleep time zone, the processing control unit 144 receives the detection result indicating that it is time to detect the sleep state from the timing detection unit 143. changes the first and second sleep degree classification thresholds to values that are more likely to be determined as a low sleep degree or a high sleep degree than the default values. The monitored person monitoring support system MS, the sleep state detection device, and the sleep state detection method described above activate the sleep state processing unit 142 according to the detection result of the schedule and timing detection unit (behavior detection processing unit) 143 of the subject Ob. Since the control is performed, the schedule of the subject Ob can be taken into consideration, and erroneous determination can be further reduced.

Although this specification discloses various aspects of the technology as described above, the main technologies thereof are summarized below.

A sleep state detection device according to one aspect includes first and second sensors for measuring a subject, and as sleep state processing, a sleep state for determining the sleep state of the subject based on a first measurement result of the first sensor. A processing unit, a timing detection unit that detects detection timing based on the second measurement result of the second sensor, and a processing control unit that controls the sleep state processing unit according to the detection result of the timing detection unit. . Preferably, in the sleep state detection device described above, the sleep state processing is processing for obtaining a degree of sleep representing the degree of depth of sleep of the subject as the sleep state. Preferably, in the sleep state detection device described above, the sleep state processing is a process of obtaining a sleep degree representing the degree of depth of sleep of the subject and a reliability degree representing the degree of reliability of the sleep degree. Preferably, in the sleep state detection device described above, the first sensor is a Doppler sensor that measures body surface movement of the chest associated with respiration in the subject, and the second sensor is an image of the subject. is a camera that generates

Since such a sleep state detection device controls the sleep state processing unit according to the detection result of the timing detection unit, for example, based on the first measurement result of the first sensor in a state suitable for detecting the sleep state While obtaining the sleep state of the subject, it is possible to control so as not to obtain the sleep state of the subject in a state unsuitable for detection of the sleep state, so that the sleep state detection device can prevent erroneous determination can be further reduced.

In another aspect, in the sleep state detection device described above, the processing control unit controls whether or not to execute the sleep state processing in the sleep state processing unit according to the detection result of the timing detection unit.

Such a sleep state detection device, for example, executes sleep state processing in a state suitable for detecting a sleep state, or conversely, does not execute sleep state processing in a state unsuitable for detecting a sleep state. Thus, by controlling the sleep state processing in the sleep state processing unit, erroneous determination can be further reduced.

In another aspect, in the sleep state detection device described above, the processing control unit determines whether or not to output the sleep state obtained by the sleep state processing unit according to the detection result of the timing detection unit. Control.

Such a sleep state detection device outputs, for example, the sleep state obtained by the sleep state processing unit in a state suitable for sleep state detection, or conversely, sleep state processing in a state unsuitable for sleep state detection. By controlling the sleep state processing unit so as not to output the sleep state obtained by the unit, erroneous determination can be further reduced.

In another aspect, in the sleep state detection device described above, the processing control unit determines the sleep state obtained by the sleep state processing unit according to the detection result of the timing detection unit. to control whether or not to assist.

For example, such a sleep state detection device outputs the sleep state obtained by the sleep state processing unit as it is in a state suitable for detecting a sleep state, or conversely outputs a sleep state in a state unsuitable for detection of a sleep state. The timing detection unit detects the sleep state obtained by the sleep state processing unit so as to add information indicating that the sleep state is unsuitable for detection of the sleep state to the sleep state obtained by the processing unit. Since it is possible to assist with detection results, etc., it is possible to further reduce erroneous determinations.

In another aspect, in the sleep state detection device described above, the sleep state processing includes determining a degree of sleep representing the degree of depth of sleep of the subject, a predetermined sleep degree determination threshold value, and the first sensor of the first sensor. 1 measurement result to determine the sleep state, and the processing control unit changes the sleep state determination threshold in accordance with the detection result of the timing detection unit as the control.

For example, in the case of a state suitable for detecting a sleep state, such a sleep state detection device changes the sleep state degree determination threshold value to a value that makes it easy to determine the sleep state, or conversely, In such a state, it is possible to change the sleep state degree determination threshold value to a value that makes it difficult to determine the sleep state, so that erroneous determination can be further reduced.

In another aspect, in the sleep state detection device described above, the sleep state processing includes, at predetermined time intervals set in advance, detecting a degree of sleep representing the degree of depth of sleep of the subject to a predetermined level of sleep. By comparing the degree determination threshold and the first measurement result of the first sensor, the sleep state is tentatively obtained, and if the tentatively obtained degree of sleep continues for a preset continuation determination time, the This is a process of determining the tentatively obtained degree of sleep as the final degree of sleep, and as the control, the processing control unit changes the continuation determination time according to the detection result of the timing detection unit.

For example, in the case of a state suitable for detecting a sleep state, such a sleep state detection device changes the continuous determination time to a value that makes it easy to determine the sleep state, In the case of , it is possible to change the continuation determination time to a value that makes it difficult to determine the sleep state, so that erroneous determination can be further reduced.

In another aspect, in the sleep state detection device described above, the first sensor includes a plurality of sub-sensors that measure the subject, and the sleep state processing unit performs the sleep state processing on the plurality of sub-sensors in the first sensor. The sleep state of the subject is obtained based on the sub-measurement result of one of the sub-sensors of the sub-sensors, and the processing control unit, as the control, according to the detection result of the timing detection unit, the plurality of sub-sensors Among them, the sub-sensor used in the sleep state processing unit is changed. Preferably, in the sleep state detection device described above, the plurality of sub-sensors are different types of sensors. Preferably, in the sleep state detection device described above, the plurality of sub-sensors are sensors of the same type and arranged in different locations (positions). Preferably, in the sleep state detection device described above, the plurality of sub-sensors are sensors of the same type but with different detection directions.

In such a sleep state detection device, the sub-sensor used in the sleep state processing unit is changed from among the plurality of sub-sensors according to the detection result of the timing detection unit. Since it becomes possible to select an appropriate sub-sensor to be used in the sleep state processing unit according to the detection result, erroneous determination can be further reduced.

According to another aspect, in the sleep state detection device described above, the timing detection unit obtains the posture of the subject based on the second measurement result of the second sensor, and the obtained posture is set in advance. In the case of a specific posture, the sleep state processing unit outputs a first detection result indicating that it is the detection timing for determining the sleep state of the subject, and the processing control unit outputs the first detection result from the timing detection unit. 1 When the detection result is received, the sleep state processing unit is controlled so as to obtain the sleep state of the subject. Preferably, in the sleep state detection device described above, the specific posture is a supine lying posture. Preferably, in the sleep state detection device described above, the specific posture is a lying posture other than lying down. Preferably, in the sleep state detection device described above, the second sensor is a camera that generates an image, and the timing detection unit obtains the posture of the subject based on the image generated by the camera. Preferably, in the sleep state detection device described above, the second sensor is a distance image sensor that generates a distance image, and the timing detection unit detects the subject based on the distance image generated by the distance image sensor. Seek the attitude of Preferably, in the sleep state detection device described above, the second sensor is a marker detection sensor that detects a predetermined marker, and the timing detection unit detects the target person based on the marker detected by the marker detection sensor. Seek the attitude of

Such a sleep state detection device controls the sleep state processing unit so that the sleep state is obtained by the sleep state processing unit when the subject is in a posture suitable for detecting the sleep state, thereby reducing erroneous determination. can be reduced.

In another aspect, in the sleep state detecting device described above, schedule information representing a schedule of the subject including at least a sleep time period and a clock unit that measures the date and time of the day is associated with the subject. wherein the processing control unit acquires the current time from the clock unit and extracts the subject's schedule corresponding to the acquired current time from the schedule information storage unit. and controls the sleep state processing unit according to the extracted schedule of the subject and the detection result of the timing detection unit.

Since such a sleep state detection device controls the sleep state processing unit according to the schedule of the subject and the detection result of the timing detection unit, the schedule of the subject can be taken into consideration, and erroneous determination can be further reduced.

A sleep state detection method according to another aspect includes a measurement step of measuring a subject with first and second sensors, and determining the sleep state of the subject based on a first measurement result of the first sensor. a sleep state determination step; a timing detection step of detecting detection timing based on the second measurement result of the second sensor; and a processing control step of controlling the sleep state processing step according to the detection result of the timing detection step. Prepare.

Since such a sleep state detection method controls the sleep state processing step according to the detection result of the timing detection step, for example, based on the first measurement result of the first sensor in a state suitable for detecting the sleep state While obtaining the sleep state of the subject by using the method, it is possible not to obtain the sleep state of the subject in a state unsuitable for detection of the sleep state. can be further reduced.

A monitored person monitoring support system according to another aspect includes a sensor device provided corresponding to a subject to determine the sleep state of the subject, sleep data received from the sensor device communicably connected to the sensor device, and sleep information received from the sensor device. A central processing unit that manages the sleep state, and a terminal device that is communicably connected to the central processing unit and receives and displays the sleep state via the central processing unit, and the subject is the monitored person. A monitored person monitoring support system for assisting monitoring of a monitored person, wherein the sensor device includes any one of the sleep state detection devices described above.

According to this, it is possible to provide a monitored person monitoring support system using any one of the sleep state detection devices described above. Since such a monitored person monitoring support system uses any one of the sleep state detection devices described above, erroneous determination can be further reduced.

This application is based on Japanese Patent Application No. 2017-155071 filed on August 10, 2017, the content of which is included in the present application.

While embodiments of the present invention have been illustrated and described in detail, this is done by way of illustration and example only, and not limitation. The scope of the invention should be construed by the language of the appended claims.

Although the present invention has been adequately and fully described above through embodiments with reference to the drawings in order to express the present invention, modifications and/or improvements to the above-described embodiments can easily be made by those skilled in the art. It should be recognized that it is possible. Therefore, to the extent that modifications or improvements made by those skilled in the art do not depart from the scope of the claims set forth in the claims, such modifications or improvements do not fall within the scope of the claims. is interpreted to be subsumed by

ADVANTAGE OF THE INVENTION According to this invention, the sleep state detection apparatus and sleep state detection method which detect a sleep state, and the monitored person monitoring support system using this sleep state detection apparatus can be provided.

Claims (11)

first and second sensors for measuring a subject;
As sleep state processing, a sleep state processing unit that obtains the sleep state of the subject based on the first measurement result of the first sensor;
a timing detection unit that detects detection timing based on the second measurement result of the second sensor;
a processing control unit that controls the sleep state processing unit according to the detection result of the timing detection unit;
The timing detection unit is also used as an action detection processing unit that determines a predetermined action of the subject based on the second measurement result of the second sensor,
the predetermined actions are divided into first and second groups ;
The second sensor includes a first sub-sensor for detecting a first action belonging to the first group and a second sub-sensor for detecting a second action belonging to the second group,
The first sensor is also used as the second sub-sensor ,
Sleep state detection device. The processing control unit controls whether to execute sleep state processing in the sleep state processing unit according to the detection result of the timing detection unit.
The sleep state detection device according to claim 1. The processing control unit controls whether to output the sleep state obtained by the sleep state processing unit according to the detection result of the timing detection unit.
The sleep state detection device according to claim 1. The processing control unit controls whether or not the sleep state obtained by the sleep state processing unit is supplemented by the detection result of the timing detection unit, according to the detection result of the timing detection unit.
The sleep state detection device according to claim 1. In the sleep state processing, a degree of sleep representing the degree of depth of sleep of the subject is obtained as the sleep state by comparing a preset sleep degree determination threshold with a first measurement result of the first sensor. processing,
As the control, the processing control unit changes the sleep degree determination threshold according to the detection result of the timing detection unit.
The sleep state detection device according to claim 1. In the sleep state processing, at predetermined time intervals set in advance, the degree of sleep representing the degree of depth of sleep of the subject is calculated based on a predetermined threshold for determining the degree of sleep and the first measurement result of the first sensor. is tentatively determined as the sleep state by comparing, and if the tentatively determined degree of sleep continues for a preset continuation determination time, the tentatively determined degree of sleep is determined as the final degree of sleep. processing,
The processing control unit changes the continuation determination time according to the detection result of the timing detection unit as the control.
The sleep state detection device according to claim 1. The first sensor includes a plurality of sub-sensors that measure the subject,
The sleep state processing unit, as the sleep state processing, obtains the sleep state of the subject based on a sub-measurement result of one of the plurality of sub-sensors of the first sensor,
As the control, the processing control unit changes the sub-sensor used in the sleep state processing unit from among the plurality of sub-sensors according to the detection result of the timing detection unit.
The sleep state detection device according to claim 1. The timing detection unit obtains the posture of the subject based on the second measurement result of the second sensor, and when the obtained posture is a predetermined specific posture, the sleep state processing unit Outputting a first detection result indicating that it is the detection timing for obtaining the sleep state of the subject,
The processing control unit controls the sleep state processing unit to determine the sleep state of the subject when the first detection result is received from the timing detection unit.
The sleep state detection device according to any one of claims 1 to 7. a clock unit that measures the date, time, and minute;
a schedule information storage unit that stores schedule information representing the subject's schedule including at least sleeping time slots in association with the subject,
The processing control unit acquires the current time from the clock unit, extracts the subject's schedule corresponding to the acquired current time from the schedule information storage unit, and extracts the subject's schedule and controlling the sleep state processing unit according to the detection result of the timing detection unit;
The sleep state detection device according to any one of claims 1 to 8. a measuring step of measuring the subject with each of the first and second sensors;
As sleep state processing , a sleep state processing step of obtaining the sleep state of the subject based on the first measurement result of the first sensor;
A timing detection step of detecting detection timing based on the second measurement result of the second sensor;
a processing control step of controlling the sleep state processing step according to the detection result of the timing detection step;
The timing detection step is also used as an action detection processing step for determining a predetermined action of the subject based on the second measurement result of the second sensor,
the predetermined actions are divided into first and second groups ;
The second sensor includes a first sub-sensor for detecting a first action belonging to the first group and a second sub-sensor for detecting a second action belonging to the second group,
The first sensor is also used as the second sub-sensor ,
Sleep state detection method. A sensor device provided corresponding to a subject to determine the sleep state of the subject, a central processing unit communicably connected to the sensor device and managing the sleep state received from the sensor device, and the central processing unit and a terminal device that receives and displays the sleep state via the central processing unit and is communicably connected to the monitored person monitoring for supporting the monitoring of the monitored person with the subject as the monitored person. A support system,
The sensor device includes the sleep state detection device according to any one of claims 1 to 9,
Monitored person monitoring support system.

Images