JP2023105966A - Method and program executed by computer to detect change in state of resident, and resident state change detection device - Google Patents

Abstract

To provide a technology capable of accurately detecting a change in a state of a resident.SOLUTION: A processing executed by a CPU or a control device comprises: a step(S1010) of reading out data of each index from a database; a step(S1020) of calculating a variation rate (change rate) and a degree of variation (change) of each index; a step(S1030) of determining whether or not a short-term or rapid variation (change) has been detected for each index; a step(S1050) of outputting an alert based on a detection result when the variation is detected (YES in step S1030); and a step(S1040) of determining whether or not the medium-term variation is detected for each index when the variation is not detected (NO in step S1030).SELECTED DRAWING: Figure 10

Description

TECHNICAL FIELD The present disclosure relates to information processing, and more particularly to techniques for detecting changes in the state of a resident.

2. Description of the Related Art Conventionally, a sensor box is arranged on the ceiling of each living room in a facility where elderly people or care recipients (generically referred to as residents) reside. The sensor box is equipped with a Doppler sensor for detecting the sleeping state of each resident and an image sensor for detecting the movement or state of the resident.

Regarding state detection, for example, Japanese Patent Application Laid-Open No. 2017-164580 (Patent Document 1) discloses that "by a simple method, the sleep state of a sleeping person (measured person) is objectively detected, and the sleep state of the person to be measured is detected. A sleep state evaluation device for evaluating the quality of sleep in a person'. This sleep state evaluation device includes "a sleep state storage means for storing at least one of the three states of getting out of bed, sleep, and wakefulness as the state of the person to be measured, and based on the stored sleep state, , calculating means for calculating daytime in bed time and daytime sleeping time, and evaluation of sleep quality of the subject based on daytime in bed time and daytime sleeping time (See [Summary]).

In addition, International Publication No. 2021/215207 (Patent Document 2) discloses “a technique for “visualizing” the state of activities of daily living (ADL) of a care recipient”. The processing executed by the processor to realize the technology includes “a step of accepting selection of an item that constitutes a daily life activity (S710), a step of accessing a database of the item (S720), and A step of calculating a moving average of item data (S730); a step of calculating a change rate of the moving average (S740); a step of accessing a database and reading rehabilitation records (S750); a step of selecting a rehabilitation menu according to the care recipient (S760), a step of displaying a graph including item data and moving averages (S770), and a step of displaying rehabilitation records (S780); and a step (S790) of displaying a rehabilitation menu suitable for the care recipient." (See [Abstract]).

Furthermore, Japanese Patent Application Laid-Open No. 2020-151511 (Patent Document 3) states, "Even when multiple pieces of biological information are displayed, when each piece of biological information is displayed collectively, the tendency of the patient's condition can be easily confirmed. A patient status display device, etc. that can According to the technology disclosed in Patent Document 3, "a first biological information value is received from a state detection device that continuously detects the patient's biological information, and a second biological information value is received from a measuring device that measures the patient's biological information. The biometric information value is received, and the graph based on the first biometric information value and the graph based on the second biometric information value can be displayed side by side.” ([Summary] reference).

JP 2017-164580 A WO2021/215207 JP 2020-151511 A

In recent years, nursing care robots have been introduced to facilities for the elderly. At that time, in addition to monitoring, the behavior and sleep of the elderly are acquired from data acquired by sensors like care and support facilities, and the state of ADL (Activity of Daily Living) of the elderly is grasped. , is considered to contribute to care and rehabilitation.

However, these indices related to the sleep and movement of the elderly change on a daily basis, reflecting the state of the elderly, and in some cases, data on the comings and goings of nursing staff, nursing staff, and other people other than residents are involved as noise. There is a possibility that it will be lost. Therefore, users such as staff and administrators cannot easily grasp whether changes in indices are statistically significant. In addition, short-term fluctuations (for example, temporary poor physical condition due to a cold, etc.) or medium- to long-term fluctuations (recovery due to rehabilitation or deterioration due to aging) were observed in some cases.

Therefore, there is a need for a technology that can easily ascertain whether changes in the state of a resident are statistically significant. There is also a need for a technique that can separately grasp short-term fluctuations and medium- to long-term fluctuations.

The present disclosure has been made in view of the background as described above, and an object in one aspect is to provide a technique for appropriately detecting changes in the state of residents of a facility.

According to one embodiment, a computer-implemented method is provided for detecting a change in a resident's condition. The method includes the steps of accessing time-series data obtained by observing a resident's living room; detecting short-term or long-term changes in the time-series data; Alternatively, it includes a step of determining whether long-term changes fall within a predetermined normal range, and a step of outputting the result of the determination.

In one aspect, the outputting step includes outputting an alert to the resident's attention.

In one aspect, the outputting step includes displaying an acceptable range for the time-series data about the resident.

In one aspect, the step of outputting includes a step of performing statistical processing according to the period for judging the state of the resident, and a step of displaying the result of the statistical processing according to the period.

In one aspect, the method further includes removing, from the time-series data, data representing movements of others that are different from the resident. The detecting step includes detecting short-term or long-term changes from the time-series data after removing data representing movements of others.

In one aspect, removing the data includes removing from the time-series data data in which the daytime in-bed rate is lower than a preset rate.

In one aspect, the outputting step includes outputting a message for the resident.

According to another embodiment, there is provided a program that causes a computer to perform any of the above methods.

According to yet another embodiment, a resident state change detection device is provided. This state change detection device includes a memory that stores the above program and a processor that executes the program.

According to an embodiment, it is possible to appropriately detect changes in the condition of facility residents.
The above and other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description of the invention taken in conjunction with the accompanying drawings.

It is a figure which shows an example of a structure of the watching system 100. FIG. It is a block diagram which shows the outline of a structure of the watching system 100. FIG. It is a figure which shows the outline of the watching system 100 using the sensor box 119. FIG. 3 is a block diagram showing the hardware configuration of computer system 400. FIG. FIG. 4 is a diagram showing the flow of deriving an optimal care program or rehabilitation plan from observation data of residents. FIG. 3 is a block diagram showing the configuration of functions included in change detection device 600 according to an embodiment; FIG. 11 is a diagram (part 1) conceptually showing one mode of data storage in storage unit 620; FIG. 11 is a diagram (part 2) conceptually showing one aspect of data storage in storage unit 620; 3 is a diagram conceptually showing one mode of data storage in storage unit 620 (No. 3). FIG. 6 is a flowchart showing part of the processing executed by CPU 1 of computer system 400 functioning as change detection device 600. FIG. 4 is a diagram illustrating a screen displayed on monitor 8 or display 226. FIG. It is a figure which shows an example which detects and displays a short-term change. FIG. 13 is an example of a screen 1300 displaying a box-and-whisker diagram showing monthly changes in daytime activity hours. FIG. 14 is an example of a screen 1400 displaying a box-and-whisker diagram showing changes in the daytime activity hours of a certain resident on a yearly basis. FIG. 10 is a diagram for explaining an example of noise removal when detecting a change in the trajectory of an action; FIG. 16 is a diagram showing an example of a screen 1600 displayed by monitor 8 or display 226. FIG. 17 is a diagram showing an example of a screen 1700 displayed on monitor 8 or display 226. FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are given the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

The systems or devices described below are used in nursing homes, hospital facilities, and other facilities that can accommodate multiple people on a continuous basis.

<Overview of the system>
With reference to Drawing 1, an outline of a monitoring system which can acquire information about ADL is explained. FIG. 1 is a diagram showing an example of the configuration of the watching system 100. As shown in FIG. An example of a watching target is a resident in each room provided in the room area of the facility 180 . In the monitoring system 100 of FIG. 1, living rooms 110 and 120 are provided in the living room area. A living room 110 is assigned to a resident 111 . A living room 120 is assigned to a resident 121 .

The monitoring system 100 includes a gateway server 130, a switching device 12, an access point 140, a management server 200, a sensor box 119, various devices communicating with the sensor box 119, and mobile terminals 161, 162, 163, and 164. including.

The gateway server 130 connects the internal network (intranet) of the facility 180 and the external network 16 of the facility 180 to each other. External network 16 is, for example, the Internet or a public telephone line network. A cloud server 150 , a push server 160 and a radio base station 15 are also connected to the external network 16 .

The exchange device 135 connects each device of the internal network of the facility 180 to each other. In some aspects, a router or switch may be used as switching device 135 . In the example shown in FIG. 1, the number of switching devices 135 is two, but the number is not limited to this. The internal network of facility 180 may consist of a combination of multiple switching devices 135 .

Access point 140 is used to connect mobile terminals 161 and 162 to the internal network of facility 180 . In one aspect, a Wi-Fi® (Wireless Fidelity) router may be used as the access point 140 .

The management server 200 receives event information from the sensor box 119 in the facility 180 and manages information on residents in each room. The management server 200 also communicates with the mobile terminals 161 and 162, manages the staff holding the mobile terminals, and transmits various notifications to each mobile terminal. Note that the management server 200 may use an external push server 160 when sending notifications to the mobile terminals 161 and 162 . Also, the external cloud server 150 may have some or all of the functions of the management server 200 .

The sensor box 119 cooperates with the cameras and sensors built into the housing and other various sensors in the living rooms 110 and 120 to obtain information on the residents 111 and 121 in the living rooms 110 and 120, respectively. . The information may include an image showing each resident's walking, body temperature, pulse and other vital information, and the like. Further, the sensor box 119 transmits the acquired information on the residents 111 and 121 to the management server 200 via the internal network. Details of the sensor box 119 will be described later.

The mobile terminals 161 and 162 are used by caregivers and other staff engaged in nursing care at the facility 180 . The staff can use the mobile terminals 161 and 162 to input care records and the like. Portable terminals 161 and 162 transmit the care records to management server 200 . Also, when a problem occurs in the residents 111 and 121, the staff uses the portable terminals 161 and 162 to receive notifications from the management server 200. FIG. Mobile terminals 161 and 162 are connected to access point 140 in facility 180 and communicate with management server 200 via an internal network. In our example, caregivers 141, 142, 143, and 144 hold portable terminals 161, 162, 163, and 164, respectively.

The mobile terminals 163 and 164 can communicate with the management server 200 from outside the facility 180 via the wireless base station 15 and the like and the gateway server 11 . When the mobile terminals 163, 164 communicate with the management server 200 from outside the facility 180, some of the services provided from the management server 200 to the mobile terminals 163, 164 are restricted in order to protect the residents' information. Sometimes.

Note that the numbers of other devices such as the mobile terminals 161 and 162, the access point 140, and the switching device 135 are not limited to the numbers illustrated in FIG.

Living rooms 110 and 120 each include furniture 112, bed 113, and toilet 114 as equipment. Door sensors 118 for detecting the opening and closing of the doors are installed on the doors of the living rooms 110 and 120, respectively. A toilet sensor 116 for detecting opening and closing of the toilet 114 is installed on the door of the toilet 114 . An odor sensor 117 that acquires excretion information of the resident 111 is installed on the bed 113 . The resident 111 wears a vital sensor 290 that detects the vital information of the resident 111 . An example of detected vital information is a resident's body temperature. Another example is the resident's breathing. Yet another example is the resident's heart rate. Yet another example is more than one of these types of information. In living room 110 , resident 111 can operate care call slave device 115 . When the resident 111 operates the care call slave device 115 , the care call slave device 115 transmits a call signal, and the sensor box 119 receives the call signal. Sensor box 119 transmits the received call signal to management server 200 . The management server 200 transmits to the mobile terminal 220 a signal notifying that the resident 111 is calling.

The sensor box 119 incorporates sensors for detecting behavior of objects in the living rooms 110 and 120 . One example of a sensor is a Doppler sensor for detecting motion of an object. Another example is a camera. Still other examples are the care call handset 115, the door sensor 118, the toilet sensor 116, the smell sensor 117, or the vital sensor 290. Sensor box 119 may include at least one of these sensors as a sensor.

The constituent elements of the watching system 100 will be described with reference to FIG. 2 . FIG. 2 is a block diagram showing an overview of the configuration of the watching system 100. As shown in FIG.

[Sensor box 119]
The sensor box 119 includes a control device 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a communication interface 104, a camera 105, a Doppler sensor 106, a wireless communication device 107, and a storage device. 108.

The control device 101 controls the sensor box 119 . The control device 101 is composed of, for example, at least one integrated circuit. Integrated circuits include, for example, at least one CPU (Central Processing Unit), MPU (Micro Processing Unit) and other processors, at least one ASIC (Application Specific Integrated Circuit), at least one FPGA (Field Programmable Gate Array), or these It is composed of a combination of

An antenna (not shown) and the like are connected to the communication interface 104 . The sensor box 119 exchanges data with an external communication device via the antenna. External communication devices include, for example, the management server 200, the mobile terminals 161, 162, 163, 164 and other terminals, the access point 140, the cloud server 150, and other communication terminals.

Camera 105, in one implementation, is a near-infrared camera. A near-infrared camera includes an IR (Infrared) projector that projects near-infrared light. By using a near-infrared camera, an image representing the inside of living room 110, 120 can be captured even at night. In other implementations, camera 105 is a surveillance camera that receives only visible light. In still other implementations, camera 105 may be a 3D sensor or a thermographic camera. Sensor box 119 and camera 105 may be configured as an integral unit or as a combination of separate devices.

Doppler sensor 106 functions as a body motion sensor. The Doppler sensor 106 transmits and receives microwaves, ultrasonic waves, and other radio waves to detect behaviors (movements) of objects (eg, the resident 111, care staff, etc.) in the living rooms 110 and 120 . Thereby, the biological information of the residents 111 and 121 in the living rooms 110 and 120 can be detected.

More specifically, the Doppler sensor 106 is connected to the control device 101 and measures the movement of the body surface of the chest associated with breathing of the resident 111 under the control of the control device 101 . The Doppler sensor 106 transmits microwaves and ultrasonic waves (hereinafter also referred to as “transmitted waves”), receives microwaves and ultrasonic waves reflected by objects (that is, “reflected waves”), and and output a Doppler signal of the Doppler frequency component. When an object is in motion, the frequency of the reflected wave shifts in proportion to the speed at which the object is moving due to the so-called Doppler effect. Therefore, a difference (Doppler frequency component) occurs between the frequency of the transmitted wave and the frequency of the reflected wave. The Doppler sensor 106 generates a signal of this Doppler frequency component as a Doppler signal at a predetermined sampling rate and outputs the Doppler signal to the control device 101 . Upon receiving Doppler signals from the Doppler sensor 106 , the control device 101 stores the received Doppler signals in the storage device 108 in chronological order. Note that when microwaves are used as transmission waves, the microwaves pass through clothing and are reflected on the body surface of the resident 111 . Therefore, even if the resident 111 is wearing clothes, the movement of the body surface can be detected.

In one example, each Doppler sensor 106 radiates microwaves in the 24 GHz band toward the beds 113 of the living rooms 110 and 120, and receives reflected waves reflected by the residents 111 and 121 and the like. The reflected waves are Doppler-shifted by the motions of the residents 111 and 121 . The Doppler sensor 106 can detect the respiratory state and heart rate of the residents 111 and 121 from the reflected waves.

Wireless communication device 107 receives signals from care call child device 115 , door sensor 118 , toilet sensor 116 , odor sensor 117 , and vital sensor 290 and transmits the signals to control device 101 . Care call slave device 115 includes care call button 241 . When the care call button 241 is operated, the care call slave device 115 transmits a signal (for example, a call signal) indicating that the operation has been performed. The transmitted signal is received by wireless communication device 107 . When door sensor 118 , toilet sensor 116 , odor sensor 117 , and vital sensor 290 transmit their respective detection results, the transmitted signals are received by wireless communication device 107 .

The storage device 108 is, for example, a fixed storage device such as a flash memory or hard disk, or a recording medium such as an external storage device. The storage device 108 stores programs executed by the control device 101 and various data used for executing the programs. Various data may include the behavior information of residents 111 and 121 . Details of the action information will be described later.

At least one of the programs and data described above can be stored in a storage device other than the storage device 108 (for example, a storage area of the control device 101 (eg, cache memory), a ROM 102, a storage device that can be accessed by the control device 101). It may be stored in the RAM 103, an external device (for example, the management server 200 or the mobile terminals 161, 162, 163, 164, etc.).

[Action information]
Next, behavior information in this embodiment will be described. Action information is, for example, information indicating that residents 111 and 121 have performed a predetermined action. In one example, the predetermined actions are "get up" indicating that the residents 111 and 121 have woken up, "leave the bed" indicating that the residents 111 and 121 have left the bed (bedding) 113, and the residents 111 and 121 It includes four actions of "falling" indicating falling from the bed (bedding) 113 and "falling down" indicating that the residents 111 and 121 have fallen.

Further, behavioral information may include actions taken by the resident to refuse to receive care. For example, the behavioral information may include a motion of shaking off the caregiver with a hand, a motion of kicking the caregiver with a leg, a motion of clinging to the bed 113 and refusing to change clothes, and the like.

In one embodiment, the control device 101 generates each action information of the residents 111 and 121 associated with each living room 110 and 120 based on the images captured by the cameras 105 installed in each living room 110 and 120. do. For example, the control device 101 detects the heads of the residents 111 and 121 from the above images, and determines the " In addition to the usual information of "getting up", "getting out of bed", "falling" and "falling", the information of "refusal of nursing care" is detected. A specific example of generating behavior information will be described in more detail below.

First, the storage device 108 stores the location area of each bed 113 in the rooms 110 and 120, the first threshold Th1, the second threshold Th2, and the third threshold Th3. The first threshold Th1 distinguishes the size of the resident's head between the lying posture and the sitting posture within the location area of the bed 113 . The second threshold Th2 identifies whether or not the resident is in a standing posture in the living rooms 110 and 120 excluding the area where the bed 113 is located, based on the size of the resident's head. The third threshold Th3 identifies whether or not the resident is lying down based on the head size of the resident in the living rooms 110 and 120 excluding the area where the bed 113 is located.

The control device 101 extracts a moving body region as a human region of the residents 111 and 121 from the target image by, for example, a background subtraction method or a frame subtraction method. The control device 101 further derives from the extracted moving object region, for example, a circular or elliptical Hough transform, pattern matching using a head model prepared in advance, and a neural network trained for head detection. The head regions of the residents 111 and 121 are extracted using the threshold values thus obtained. The control device 101 detects "getting up", "getting out of bed", "falling", and "falling" from the extracted position and size of the head.

The control device 101 determines that the position of the head extracted as described above is within the location area of the bed 113 and the size of the head extracted as described above is determined by using the first threshold value Th1. It may be determined that the action "wake up" has occurred when it detects a change from posture magnitude to sitting posture magnitude.

When the position of the head extracted as described above moves from within the location area of the bed 113 to outside the location area of the bed 113, the control device 101 controls the size of the head extracted as described above. By applying the second threshold Th2 to the head position, it may be determined that the action "getting out of bed" has occurred when it is detected that the size of the head has changed from a certain size to the size of the standing posture.

When the position of the head extracted as described above moves from within the location area of the bed 113 to outside the location area of the bed 113, the control device 101 controls the size of the head extracted as described above. By applying the third threshold value Th3 to , it may be determined that the action "fall" has occurred when it is detected that the head has changed from a certain size to the size of the lying posture.

The control device 101 determines that the position of the head extracted as described above is located in the living room 110 or 120 excluding the location area of the bed 113, and the size of the extracted head uses the third threshold value Th3. It may be determined that the action "fall" has occurred if it detects a change from a size to a lying size by .

Furthermore, the control device 101 detects the caregiver in addition to the resident from the image data, and estimates the distance between the resident and the caregiver. When the control device estimates that the interval is shorter than a predetermined normal interval, it can determine that the care refusal action is taking place. Furthermore, when the control device 101 determines that the resident's motion is different from normal, the control device 101 can determine that the resident is showing a nursing care refusal behavior to the caregiver.

As described above, in one specific example, the control device 101 of the sensor box 119 generates each action information of the residents 111 and 121 . In addition, in the monitoring system 100 according to another aspect, an element other than the control device 101 (for example, the cloud server 150) generates the behavior information of the residents 111 and 121 using the images in the living rooms 110 and 120. good too.

[Portable terminal 220]
Portable terminal 220 includes control device 221 , ROM 222 , RAM 223 , communication interface 224 , display 226 , storage device 228 and input device 229 . In one aspect, the mobile terminals 161, 162, 163, and 164 are implemented as, for example, smart phones, tablet terminals, wristwatch-type terminals, and other wearable devices.

The control device 221 controls the mobile terminals 161 , 162 , 163 and 164 . The control device 221 is composed of, for example, at least one integrated circuit. An integrated circuit is composed of, for example, at least one CPU, at least one ASIC, at least one FPGA, or a combination thereof.

An antenna (not shown) and the like are connected to the communication interface 224 . Mobile terminals 161 , 162 , 163 and 164 exchange data with external communication devices via the antenna and access point 140 . External communication equipment includes, for example, the sensor box 119, management server 200, and the like.

The display 226 is implemented by, for example, an organic EL (Electro Luminescence) display, a liquid crystal display, or the like. The input device 229 is implemented by a touch sensor provided on the display 226, for example. The touch sensor receives touch operations on the mobile terminals 161 , 162 , 163 and 164 and outputs signals corresponding to the touch operations to the control device 221 .

The storage device 228 is implemented by, for example, a flash memory, a hard disk or other fixed storage device, or a removable data recording medium.

In a certain aspect, the control device 101 identifies, for example, going to bed, lying down, getting up, getting out of bed, falling, falling, and abnormal micromotion as follows. Note that the area where the bed 113 is arranged (location area of the bed 113) in the target image is pre-stored in the ROM 102 as one of various data. Further, each threshold value and continuation determination time are appropriately set from a plurality of samples and stored in advance in the ROM 102 as one of various data.

[Entering bed]
In determining whether to enter the bed, the control device 101 determines that the previous state variable (variable for storing the behavior determination result) was "getting out of bed", and this time the person extracted from the target image acquired from the camera 105 by, for example, the background subtraction method. When the area completely overlaps with the location area of the bed 113 (when the person area is completely within the location area of the bed 113), it is tentatively determined to be in bed, and the duration of the complete overlapping state is determination of in-bed continuation. If it continues beyond the time, it is finally determined that there is bed entry, and bed entry is detected. The control device 101 updates the state variable as "entering bed". The in-bed continuation determination time is used as a threshold for finally determining in-bed as in-bed tentatively determined based on the complete overlap of the extracted person area and the location area of the bed 113. .

[Lie down]
Lying means a state in which the resident (person to be monitored) is lying on the bed 113 . For example, if the area occupied by the resident in the image is included in the area of the bed 113 and the amount of movement of the resident is equal to or less than a predetermined amount, the control device 101 detects the behavior information of the resident. Detects "bed rest" as

[Wake up]
In determining whether to get up, the control device 101 determines that the previous state variable was "bed", and this time, the area in which the person area extracted from the target image acquired from the camera 105 protrudes from the location area of the bed 113 is determined to be getting up. If it is equal to or greater than the threshold and less than the threshold for determination of getting out of bed, it is tentatively determined to be awake. The control device 101 finally determines that there is a wake-up when the duration of the region in the state where the region is equal to or greater than the wake-up determination threshold and less than the bed leaving determination threshold exceeds the wake-up continuation determination time, Detect waking. The controller 101 updates the state variable with "wake up". The wake-up determination threshold is used to determine whether or not it is a wake-up based on the size of the area. The bed-leaving determination threshold value is used to determine whether or not the bed-leaving is based on the size of the area. The threshold for determination of getting out of bed is set to a value larger than the threshold for determination of getting up. The wake-up continuation determination time is used as a threshold for finally determining that the wake-up tentatively determined by comparing the area and the wake-up determination threshold value is the wake-up.

[Getting out of bed]
In the determination of getting out of bed, the control device 101 determines that the previous state variable is either “entering bed” or “getting up”, and that the person region extracted from the target image acquired from the camera 105 this time is on the bed 113. When the area protruding from the location area is equal to or larger than the threshold value for leaving the bed, it is tentatively determined to be leaving the bed. The control device 101 finally determines that there is leaving the bed and detects leaving the bed when the duration of the region in which the region is equal to or greater than the leaving-bed determination threshold exceeds the leaving-bed continuation determination time. The control device 101 updates the state variable with "get out of bed". The out-of-bed continuation determination time is used as a threshold for finally determining out-of-bed to be out-of-bed, temporarily determined by comparing the area and the out-of-bed determination threshold.

[fall]
In the determination of the fall, the control device 101 determines that the size of the head region of the human region extracted from the target image acquired from the camera 105 is equal to or smaller than the lying posture determination threshold, and that the change rate of the head region size is equal to or greater than the fall determination speed threshold and the person area is within the fall determination area set around the location area of the bed 113, it is determined that there is a fall, and the fall is detected. The lying posture determination threshold is used to determine whether or not the size of the head region is in the lying posture. Whether the posture is lying down or sitting is determined based on the size of the head region. The fall determination speed threshold is used to determine whether or not there is a fall based on the change speed of the size of the head region.

[fall down]
In the fall determination, the control device 101 determines that the size of the head region of the human region extracted from the target image acquired from the camera 105 is equal to or smaller than the lying posture determination threshold value, and the change rate of the size of the head region is greater than or equal to the overturn determination speed threshold, and the person area is in an area excluding the location area of the bed 113 and the fall determination area, it is determined that there is a fall, and the overturn is detected. The fall determination speed threshold is used to determine whether or not a fall has occurred based on the change speed of the size of the head region.

[Abnormity of micromotion]
Micromotion anomaly is detected based on the Doppler signal output from the Doppler sensor 106 . More specifically, the control device 101 performs, for example, a fast Fourier transform (FFT) on the Doppler signals measured within a predetermined time from the time of measurement to the past, and from the spectrum obtained by this FFT, general Find the average value of the amplitude in the frequency band corresponding to the normal breathing frequency. The control device 101 compares the obtained average value with a threshold value for determining whether or not there is an abnormal micromotion, and if the obtained average value is equal to or less than the threshold, the abnormal micromotion and the provisional If the duration of the state in which the average value is equal to or less than the threshold continues beyond a predetermined determination time, it is finally determined that there is an abnormality in fine body movement, and the abnormality in fine body movement is determined. to detect. The control device 101 updates the state variable with "abnormal body motion". The determination time is used as a threshold value for finally determining anomaly in subtle motion that has been tentatively determined by comparing the calculated average value with an anomaly in fine motion.

When the control device 101 detects the predetermined behavior from the behavior of the resident in this way, the control device 101 transmits an event notification signal including event information representing the content of a predetermined event related to the resident from the communication interface 104 to the management server. 200. More specifically, the control device 101 sends an event notification signal including the sensor ID of the sensor box 119, event information representing the content of the event, and the target image used when detecting the entry, the wake-up, the exit, the fall, and the fall. , to the management server-200 via the communication interface 104.

The event information, in one embodiment, includes one or more of getting in bed, lying down, getting up, getting out of bed, falling, falling, dyskinesia, and a care call. Here, the control device 101 stores one or more of the detected bed entry, wake-up, bed exit, fall, fall, and micromotion abnormality as event information in the event notification signal. The images may include still images and/or moving images. In one embodiment, as will be described later, for example, still images are first transmitted from the sensor box 119 to the management server 200, and moving images are delivered in response to requests from users (eg, caregivers, administrators, doctors, and other staff). can be Note that in another aspect, the video may be distributed first. Also, the still image and the moving image are transmitted to the management server 200, and the mobile terminal 220 may display the still image or the moving image in a split screen state on the display 226 based on the signal sent from the management server 200. .

When the care call slave device 115 receives a nurse call, the control device 101 transmits to the management server 200 an event notification signal including that effect as another example of a predetermined event, and uses a speaker (not shown) or the like. , a voice call is made with the mobile terminal 220 . More specifically, when care call button 241 is operated, wireless communication device 107 receives a signal corresponding to the operation. Sensor box 119 generates an event notification signal including a nurse call as the sensor ID and event information, and transmits the event notification signal to management server-200 via communication interface 104 . The control device 101 performs voice communication between the microphone (not shown) and the speaker (not shown) of the care call slave device 115 and the mobile terminal 220 via the wireless communication device 107 by, for example, VoIP (Voice over Internet Protocol). Realize

When the control device 101 receives a video distribution request from an image display device such as the management server 200, the cloud server 150, or the mobile terminal 220 via the communication interface 104, the control device 101 responds to the request by causing the camera 105 to shoot. A moving image (for example, a live moving image) obtained by is stream-reproduced, and the moving image is distributed to the image display device.

[Outline of monitoring]
Watching using the watching system 100 will be described with reference to FIG. FIG. 3 is a diagram showing an outline of the monitoring system 100 using the sensor box 119. As shown in FIG.

The watching system 100 is used to watch over residents 111 and 121 who are watching targets (monitoring targets) and other residents. A sensor box 119 is attached to the ceiling of the living room 110 . Sensor boxes 119 are similarly installed in other living rooms.

Range 31 represents the detection range by sensor box 119 . If the sensor box 119 has a Doppler sensor as described above, the Doppler sensor detects human behavior occurring within the area 31 . If sensor box 119 has a camera as a sensor, the camera takes an image within range 31 .

Sensor box 119 is installed, for example, in nursing homes, medical facilities, homes, and the like. In the example of FIG. 3, the sensor box 119 is attached to the ceiling and photographs the resident 111 and the bed 113 from the ceiling. The mounting location of the sensor box 119 is not limited to the ceiling, and may be mounted on the side wall of the living room 110 .

The watching system 100 detects a danger to the resident 111 based on a series of images (that is, video) obtained from the camera 105 . By way of example, detectable hazards include resident 111 falling, resident 111 being in a hazardous area (eg, bed rails, etc.), and the like.

When the watching system 100 detects that the resident 111 is in danger, the monitoring system 100 notifies the caregivers 141, 142 and the like of the fact. As an example of the notification method, the watching system 100 notifies the mobile terminals 161 and 162 of the caregivers 141 and 142 of the danger of the resident 111 . Upon receiving the notification, the mobile terminals 161 and 162 notify the caregivers 141 and 142 of the danger of the resident 111 by message, voice, vibration, or the like. As a result, the caregivers 141 and 142 can immediately grasp that the resident 111 is in danger, and can quickly rush to the resident 111 .

Furthermore, the watching system 100 can also notify the mobile terminals 163 and 164 of the caregivers 143 and 144 outside the facility of the danger via the wireless base station 15 .

Although FIG. 3 shows an example in which watching system 100 includes one sensor box 119, watching system 100 may include multiple sensor boxes 119 in other situations. Moreover, although FIG. 3 shows an example in which the watching system 100 includes a plurality of mobile terminals 161 and 162, the watching system 100 can be realized by a single mobile terminal in other situations.

[Computer system configuration]
A configuration of a computer system 400, which is one aspect of the information processing apparatus, will be described with reference to FIG. FIG. 4 is a block diagram showing the hardware configuration of computer system 400. As shown in FIG. Computer system 400 functions as gateway server 130 , cloud server 150 , push server 160 , or management server 200 .

The computer system 400 has, as main components, a CPU 1 that executes a program, a mouse 2 and a keyboard 3 that receive instructions input by the user of the computer system 400, data generated by the execution of the program by the CPU 1, or the mouse 2 Alternatively, a RAM 4 for volatilely storing data input via a keyboard 3, a hard disk 5 for nonvolatilely storing data, an optical disk drive 6, a communication interface (I/F) 7, and a monitor 8 are provided. include. Each component is connected to each other by a data bus. The optical disc drive device 6 is loaded with a CD-ROM 9 or other optical discs.

Processing in the computer system 400 is realized by each piece of hardware and software executed by the CPU 1 . Such software may be stored in the hard disk 5 in advance. Also, the software may be stored in a CD-ROM 9 or other recording medium and distributed as a computer program. Alternatively, the software may be provided as a downloadable application program by a so-called information provider connected to the Internet. Such software is read from the recording medium by the optical disk drive 6 or other reading device, or downloaded via the communication interface 7, and then temporarily stored in the hard disk 5. FIG. The software is read from the hard disk 5 by the CPU 1 and stored in the RAM 4 in the form of an executable program. CPU1 executes the program.

Each component that makes up the computer system 400 shown in FIG. 4 is conventional. Therefore, it can be said that one of the essential parts of the technical idea according to the present disclosure is software stored in RAM 4, hard disk 5, CD-ROM 9 or other recording media, or software that can be downloaded via a network. Recording media may include non-transitory computer-readable data recording media. Since the operation of each piece of hardware in computer system 400 is well known, detailed description will not be repeated.

The recording medium is not limited to CD-ROM, FD (Flexible Disk), hard disk, SSD (Solid State Drive), magnetic tape, cassette tape, optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile Disc)), IC (Integrated Circuit) card (including memory card), Optical card, Mask ROM, EPROM (Electronically Programmable Read-Only Memory), EEPROM (Electronically Erasable Programmable Read-Only Memory), Flash ROM A medium such as a semiconductor memory such as a fixed program may also be used.

The program here includes not only a program that can be directly executed by the CPU, but also a program in source program format, a compressed program, an encrypted program, and the like.

With reference to FIG. 5, the flow of deriving the optimum care program or rehabilitation plan from the resident's observation data will be described. FIG. 5 is a diagram showing the flow of deriving the optimum care program or rehabilitation plan from observation data of residents.

In one aspect, the database 510 holds data obtained by observing the condition of each resident. The database 510 is constructed on the hard disk 5 provided in the cloud server 150 or the management server 200 .

In step S520, the terminal 530 acquires the data of each resident from the database 510 based on the operation by the staff 540 or based on the satisfaction of the condition preset as the data acquisition timing. The preset conditions include, for example, arrival of a predetermined time every day, monitoring data showing an abnormal value (for example, monitoring data deviating from a predetermined threshold value), and the like. The terminal 530 is realized by the management server 200, the mobile terminal 220, and other information communication terminals.

In step S550, the staff refers to the data acquired from database 510 to determine the status of each resident.

In step S560, the recommendation device 570 uses the data acquired from the database 510 and past recommendations (suggestions) that have been applied to similar residents so far, and responds to the situation of each resident. Create recommendations for care plans or rehabilitation plans. The recommendation device 570 creates recommendations based on commands given by care staff, administrators, and other users, or based on the arrival of a time specified in advance as the timing for creating recommendations. The predetermined time is when the period set in advance as the period for creating or updating the recommendation has passed, when it is detected that the numerical value indicating the resident's condition has improved, and the resident's condition has deteriorated. when it is recognized that In one aspect, recommendation device 570 is implemented by the same computer device as terminal 530 . In another aspect, the recommendation device 570 may be realized by a computer device different from the terminal 530. FIG.

In step S580, the recommendation device 570 outputs the generated recommendations. In one aspect, the recommendation device 570 displays the recommendation in the form of text, graph, or the like. In another aspect, the recommendation device 570 can transmit the content of the recommendation to another information communication device such as the mobile terminal 220 . Staff then use the recommendations generated for each resident to develop a care plan or rehabilitation plan 590 .

An example of a configuration for realizing a change detection device 600 that detects changes in indices will be described with reference to FIG. FIG. 6 is a block diagram showing the functional configuration of change detection device 600 according to an embodiment. The change detection device 600 is realized by a well-known computer device having communication functions and data processing functions.

The change detection device 600 includes a signal input section 610 , a storage section 620 , a data processing section 630 and an output section 640 . The data processing unit 630 includes an index acquisition unit 631 , a change detection unit 632 , a trend detection unit 633 , a recommendation generation unit 634 and an alert generation unit 635 . Output unit 640 includes alert output unit 641 and recommendation display unit 642 . The index acquisition unit 631 includes an index calculation unit 650 .

Signal input unit 610 receives a signal input to change detection device 600 from the outside. The signal input unit 610 is implemented by a LAN (Local Area Network) interface card, a WiFi (Wireless Fidelity) module, or other input interface device.

Storage unit 620 holds data input to change detection device 600 and data generated by data processing unit 630 . Storage unit 620 is implemented by a RAM or other volatile data recording medium, or a hard disk, SSD or other nonvolatile data recording medium.

The data processing unit 630 uses the data provided to the change detection device 600 to generate recommendations for each resident. Recommendations include modified care plans or rehabilitation programs.

More specifically, in the data processing unit 630, the index acquisition unit 631 acquires an index indicating the ADL of the resident from monitoring data obtained by daily observation of each resident. Obtaining the metric from the monitoring data may include calculating the metric using the monitoring data.

The indicators include sleep-related indicators and behavior-related indicators. The sleep index includes, for example, sleeping time, bedtime, bed leaving time, and room stay time. Examples of behavioral indicators include walking speed, presence or absence of sway, degree of sway, frequency of sway, action area, frequency of visiting the action area, moving distance, repeated action, time of repeated action, stop time (length of time Sabetsu), etc. The index can be calculated on a daily basis, by day and night, or by a specific time period. Specific time periods may be, for example, activity periods between breakfast and lunch or between lunch and dinner, time periods specified by caregivers, nurses, care managers, and the like.

For example, the index calculation unit 650 calculates the index indicating the ADL of the resident using monitoring data obtained by observing each resident on a daily basis.

The change detection unit 632 detects a change in the resident's ADL using the acquired index and the reference index obtained from the data acquired so far for the index. A detected change is an increase or decrease in each index. In another aspect, the change detection unit 632 can also detect that the index has not changed (=the ADL state can be maintained). The type of change is short-term fluctuations on a daily basis, long-term fluctuations on a weekly or monthly basis, or such long-term trends. A long-term trend may be either an upward trend or a downward trend. Regarding the medium- to long-term fluctuations of each index, the change detection unit 632 uses a plurality of data such as data for several days, data for one week, data for one month, etc. for the index, and calculates variations, variances, and other statistical values. can be calculated.

The reference index is an index that indicates that the ADL state of the resident is being maintained. Reference indicators may be derived from observing the resident's behavior over time, or may be specified by the resident's caregiver, nurse or administrator.

In a certain aspect, if any of the above indices exceeds the corresponding reference index, or if any of the indices falls below the reference index, the change detection unit 632 detects a change in physical condition of the resident. A change in the physical condition or a change in the ADL condition of the resident is detected. For example, in the case of ADL where a smaller index value is desirable, such as bedtime, degree of swaying, frequency of swaying, etc., when the index exceeds the reference index, the change detection unit 632 detects that the resident's ADL has deteriorated. detect that

The trend detection unit 633 detects a long-term trend of the index as a change in the index. For example, the trend detection unit 633 detects that the bedtime of a certain resident tends to increase. In this case, the increasing trend may lead to a determination that the resident's ADL condition is deteriorating. In another aspect, the trend detection unit 633 may detect that the frequency or degree of swaying of a certain resident is trending downward. In this case, the decreasing trend may lead to a determination that the resident's ADL condition is improving.

The recommendation generation unit 634 generates a recommendation based on the change detected by the change detection unit 632 or the trend detected by the trend detection unit 633 and the recommendation data prepared in advance stored in the storage unit 620. Generate personalized recommendations. The recommendation includes, for example, a proposal to raise the level of the training program for improving the ADL of the resident or a proposal to lower the level of the training program.

The alert generation unit 635 is based on the change detected by the change detection unit 632 or the trend detected by the trend detection unit 633 and the alert data prepared in advance stored in the storage unit 620. Generate an alert to notify you of the condition. The alert includes a short-term change alert that notifies that the state of the resident has changed in the short term, and a medium- to long-term change alert that notifies that the state has changed over the medium term or long term. Also, reporting the condition of the resident can include both notifying that the condition of the resident has improved and that the condition of the resident has deteriorated.

Short-term change alerts warn that a resident may be unwell based on, for example, a sudden change in the resident's temperature. The alert generation unit 635 generates a short-term change alert when the value on the target day exceeds x times the standard deviation of the n-day moving average of each value constituting the trajectory representing the daily observation results. As an example, when the alert generation unit 635 detects that the bed-riding rate of a certain resident has increased, the record of the physical condition of the resident so far and the change in the bed-riding rate stored in the storage unit 620 are recorded. Based on this, it is determined that the resident may have become ill, and the determination result is generated as an alert. In another aspect, when a certain resident moves to the toilet in the living room more frequently than usual, the alert generation unit 635 determines that the resident may have an upset stomach. and generate the judgment result as an alert.

Medium-term change alerts are issued based on observations of medium-term or long-term changes in bodily function. The medium- to long-term change alert notifies, for example, that a certain resident is recovering as a result of training based on a rehabilitation plan. Alternatively, the medium- to long-term change alert notifies that aging of the resident progresses and physical ability may decline more than expected. The alert generation unit 635 issues a medium- to long-term change alert when a significant difference is detected in a nonparametric test, multiple comparison test, or the like.

The output unit 640 outputs recommendations generated by the recommendation generation unit 634 or alerts generated by the alert generation unit 635 . For example, the alert output unit 641 displays an alert as a message on the monitor 8 of the management server 200 or the display 226 of the mobile terminal 220 to notify that the condition of a certain resident has changed suddenly. In this case, the mobile terminal 220 may vibrate a vibrator (not shown) and notify the caregiver or other staff who are users of the mobile terminal 220 of the incoming alert through the vibration.

In another aspect, the alert output unit 641 outputs the alert as a voice from a speaker (not shown) of the mobile terminal 220 or a speaker (not shown) connected to the management server 200 . As a result, caregivers, nurses, and other staff can quickly go to the resident's room and check the resident's condition.

In another aspect, the output unit 640 outputs recommendations for a certain resident as the recommendation display unit 642 . A recommendation may include a recommended care program or rehabilitation plan. For example, the recommendation display unit 642 displays recommendations for the resident based on detection of an instruction to display the recommendations for the resident specified by the staff.

More specifically, when a certain staff member is in a room of a certain resident, the recommendation display application is activated on the mobile terminal 220 . When the staff designates a resident for which recommendations are requested, the mobile terminal 220 communicates with the management server 200 to acquire recommendations according to the ADL status of the resident and display them on the display 226 .

In addition, instead of the configuration in which the staff designates the resident, the mobile terminal 220 detects that the staff is in the resident's room, and the mobile terminal 220 sends a message to improve the resident's condition. You can display recommendations. For example, when the mobile terminal 220 receives the identification data emitted from the short-range wireless transmitter provided in each living room to transmit the identification data of each living room, the mobile terminal 220 is the user. Detect that the staff member is in the room to which the identification data is assigned. According to such a configuration, the staff can display recommendations for the resident on the portable terminal 220 without performing an operation to select the resident in front of the staff.

[data structure]
The data structure of the storage unit 620 included in the change detection device 600 will be described with reference to FIGS. 7 to 9. FIG. The change detection device 600 is implemented by the management server 200 or each mobile terminal 220 . 7 to 9 are diagrams conceptually showing one mode of data storage in storage unit 620. FIG.

As shown in FIG. 7, storage unit 620 includes tables 710 and 720 . Table 710 holds data obtained by observing each resident. As an example, table 710 includes resident ID, date and time 712, sleep time 713, bed time 714, bed leaving time 715, and indoor stay time 716.

The resident ID identifies the resident. The date and time 712 represents the date and time when each data was acquired. Sleep time 713 represents the sleep time of the resident. The bedtime 714 represents the time the resident was lying on the bed 113 . The out-of-bed time 715 represents the time during which the resident was not in bed 113 . Indoor stay time 716 represents the time the resident spent in the living room 119 . In one aspect, table 710 holds daily data for each resident. These data are acquired by the sensor box 119 .

Table 720 holds data on the daily behavior of each resident. For example, the table 720 includes date/time 721, walking speed 722, action area 723, movement amount 724, and rising time 725 as data related to daily actions.

The date/time/minute 721 represents the time when the data was acquired. Walking speed 722 represents the current walking speed of the resident. Activity area 723 represents where the resident was staying when the data was acquired. Places where a resident can stay as the action area 723 are the bed, living room 119, toilet 114, common space, rehabilitation room, dining room, bathing facility, and the like. Amount of movement 724 represents the distance traveled by the resident at that time. Rise time 725 represents the time required for the resident to rise from bed 113 . These data are acquired by the sensor box 119 .

Referring to FIG. 8, storage unit 620 further includes table 810 . The table 810 includes threshold values as criteria for determining whether or not the resident's condition has improved for each index. For example, table 810 includes index 811 , first threshold 812 and second threshold 813 .

Indicators 811 are used to derive alerts or recommendations for each resident. The indicators 811 are, for example, daytime sleep, daytime rate of stay in bed, daytime stay rate, daytime indoor activity, nighttime sleep, nighttime rest efficiency, nighttime in bed, number of times of getting out of bed at night, but are not limited to these, and other indicators may be used. may be

Daytime sleep represents the percentage of time that resident sleep was observed during the daytime when sleep was not required relative to a reference time (eg, 24 hours). The daytime stay-in-bed rate represents the ratio of the time the resident spent in bed 113 during the daytime with respect to the reference time. The daytime stay rate represents the ratio of the time spent in the living room 119 during the time period when the person can stay outside the room, such as in the common space or the rehabilitation room, with respect to the reference time. The daytime indoor activity is the time during which movement is observed indoors relative to the reference time, for example, the time during which the resident is out of bed in the living room.

Nocturnal sleep represents the percentage of time sleep was observed during the night relative to the reference time. The nighttime rest efficiency represents the ratio of the time that the resident was resting at night with respect to the reference time. Resting time represents the percentage of time in which no wandering was observed. As an example, the resting efficiency is calculated by dividing sleep time by bed time (=sleep time+wake time). Night time in bed represents the percentage of time spent in bed at night with respect to the reference time. The time spent in bed at night may or may not coincide with the time spent sleeping at night. A case where the time spent in bed at night does not match the time spent sleeping at night is a case where the resident is moving on the bed 113 . When the resident is moving on the bed 113, it is the time when the resident is awake on the bed 113, for example. The number of times the resident leaves the bed at night is the number of times the resident leaves the bed 113 at night.

A first threshold 812 and a second threshold 813 are thresholds for capturing changes in residents. Since each resident has individual differences, in a certain aspect, two types of thresholds are prepared in advance, and one of the thresholds can be selected according to the state of each resident. In another aspect, one threshold may be used to capture changes in residents, or three or more thresholds may be used. As the number of thresholds used increases, the state of the resident can be discriminated and grasped in detail.

Referring to FIG. 9, storage unit 620 further includes table 910 . Table 910 includes, as an example, data for evaluating the condition of each resident according to the daytime and nighttime conditions of each resident. For example, evaluation of the daytime condition of a resident is classified into four levels according to the relationship between the stay rate and the bed occupancy rate. For example, when the rate of stay in bed is higher than a preset reference value, the evaluation D1 that the time in bed is long is derived regardless of the length of time spent in the room during the daytime (the length of stay rate). Note that the manner of classification is not limited to the manner (matrix format) shown in FIG. 9, simpler classification may be performed, and more indices may be used for classification. For example, a decision tree-like aspect may be employed.

Similarly, the evaluation of the resident's nighttime state is classified into four stages according to the relationship between sleep time and time spent in bed. For example, when the sleep time at night is shorter than a preset reference value, the evaluation N1 that the sleep time at night is short is derived regardless of the time spent in bed.

In addition, the evaluation of the number of times of getting out of bed at night is classified into two levels depending on the magnitude relationship between the number of times of getting out of bed and a preset reference number of times. For example, when the number of times of getting out of bed at night is greater than a preset reference number of times, an evaluation N5 is derived that the number of times of getting out of bed at night is large.

The control structure of change detection device 600 will be described with reference to FIG. FIG. 10 is a flowchart showing part of the process executed by CPU 1 of computer system 400 functioning as change detection device 600 or control device 221 of portable terminal 220 . Below, the case where the CPU 1 of the computer system 400 executes the process will be described, but the case where the mobile terminal 220 executes the process is the same. In this case, the mobile terminal 220 downloads necessary data from the database before executing the process.

In step S<b>1010 , CPU 1 accesses database 510 and reads data of each index from database 510 .

In step S1020, CPU 1 calculates the rate of change (rate of change) and the degree of fluctuation (change) of each index from the read data.

In step S1030, CPU 1 determines whether or not a short-term or abrupt fluctuation (change) has been detected for each index. When CPU 1 determines that the variation has been detected (YES in step S1030), CPU 1 switches control to step S1050. Otherwise (NO in step S1030), CPU 1 switches control to step S1040.

In step S1040, CPU 1 determines whether medium- to long-term fluctuations have been detected for each index. When CPU 1 determines that the variation has been detected (YES in step S1040), CPU 1 switches control to step S1050. Otherwise (NO in step S1040), CPU 1 terminates the control.

At step S1050, CPU 1 outputs an alert based on the detection result.
[Screen example]
An example of the screen will be described with reference to FIGS. 11 to 17. FIG. 11 to 17 are diagrams illustrating screens displayed on the monitor 8 or the display 226, respectively.

As shown in FIG. 11, screen 1100 is displayed on monitor 8 or display 226 . The screen 1100 includes an area 1110 that displays a list of facility residents, and an area 1120 that displays the state of any selected resident. Area 1110 displays, as the list, the name (“user name”), the room number of the resident (“room number”), and one or more icons 1130 representing the characteristics of the resident. do. Each icon 1130 indicates whether the resident is in bed during the day, whether the resident is in bed at night, or sleeps at night. An icon indicating still another state of the resident may be displayed in a manner associated with the resident. This allows the administrator to easily grasp which resident is being alerted.

For example, when a value exceeding the normal range of fluctuation is detected, management server 200 or mobile terminal 220 displays an icon in area 1110 to notify that fact. Thereby, the user of the management server 200 or the portable terminal 220 can easily recognize that the state of the resident is different from the usual state.

Area 1120 displays information for the selected resident. For example, area 1120 may display graphs showing daytime and nighttime behavior, respectively, as selected by caregivers, staff including nurses, or other users. A graph 1140 represents changes in the percentage of daytime in-beds. A graph 1150 represents changes in the percentage of nighttime stays in bed. Graph 1160 represents changes in sleep at night.

Although the graphs 1140, 1150, and 1160 show transitions every three days, the transition interval is not limited to three days.

[Display example of short-term change]
FIG. 12 is a diagram showing an example of detecting and displaying short-term changes. Monitor 8 or display 226 displays screen 1200 . Screen 1200 displays a graph representing daily changes in a resident's condition. Status is indicated using, for example, bed rest rate during the day, although other indicators may be used. The graph shows short-term poor physical condition such as a resident catching a cold or upset stomach, and recovery from such poor physical condition as changes in the bedridden rate.

Solid lines represent measured values, and dotted lines represent moving averages. In actual measurements, points contained in regions 1210, 1220, 1230, 1240, and 1250 indicate short-term alerts. For example, the cloud server 150 or the CPU 1 of the management server 200 or the control device 221 of the mobile terminal 220 monitors the daily measured values, and the measured value of a certain day is the standard deviation of the n-day moving average of the measured values up to that day. A short-term alert is issued when it detects that it exceeds x times.

[Display example of medium- to long-term change]
FIG. 13 is an example of a screen 1300 that displays a box chart showing changes in daytime activity hours on a monthly basis. Boxplots display minimum 1310, first quartile 1320, second quartile (median) 1330, third quartile 1340, and maximum 1350 for each month.

For example, variation in daytime activity hours from month to month can be used when observing long-term changes in each resident's physical function. In this case, when physical ability is recovered through rehabilitation, the boxplot for daytime activity shifts upward, and conversely, when physical ability declines due to aging, the boxplot shifts downward. expected to

FIG. 14 is an example of a screen 1400 displaying a boxplot showing changes in daytime activity hours of a certain resident on a yearly basis. Boxplots display the minimum, 1st quartile, 2nd quartile (median), 3rd quartile and maximum for each year. For example, for the year to September 2019, a boxplot shows a minimum value of 1410, a first quartile of 1420, a second quartile (median) of 1430, a third quartile of 1440, and a maximum of 1450. In the case where is shown, the boxplot for the year up to September 2020 has shifted downward, clearly indicating that the daytime activity time of the resident has decreased, and physical ability has declined. It can be read that there is

As shown in FIGS. 12 to 14, cloud server 150, management server 200, or mobile terminal 220 switches the statistical method and display format during the analysis period to be determined. As a result, a user such as a nursing staff, a nursing staff, or an administrator can display a graph on the monitor 8 or the display 226 according to the purpose by specifying a period of time. can be determined accurately.

FIG. 15 is a diagram for explaining an example of noise removal when detecting a change in the trajectory of action. In a living room where a resident lives, for example, a sensor box 119 is arranged on the ceiling. A resident's behavior is determined using the data acquired by the sensor box 119 . In addition to the resident, nursing staff, nursing staff, or the resident's family may enter the living room. Since the sensor box 119 can detect movements of people other than the resident as behavior of the resident, there may be cases where the accurate behavior of the resident is not captured.

Therefore, the cloud server 150 or the CPU 1 of the management server 200 or the control device 221 of the mobile terminal 220 uses the median value of the observed data (actual values) for n days to smooth the actual values. Furthermore, the CPU 1 or the control device 221 removes data other than the resident's data in consideration of the bed-ridden rate during the daytime. For example, when the CPU 1 or the control device 221 observes the person leaving the bed for less than a predetermined time, the time is regarded as the time when the behavior based on the movement of a person other than the resident is observed, and the data within that time can be excluded. As a result, it is possible to detect an abnormality in a state in which staff other than residents and trajectories other than residents are reduced, so detection accuracy can be improved.

For example, as shown in state A, monitor 8 or display 226 is displaying screen 1510 . Screen 1510 includes actual values, average values, moving average values and short-term alerts. A moving average value is an average value of data for 7 days. It is assumed that a short-term alert is detected when the measured value is ±1.5σ or more of the average value.

The data used to display screen 1510 remains the data observed by sensor box 119 . Therefore, when someone other than the resident enters the living room, the movement of the other person can be observed as a locus of movement. Therefore, short-term alerts 1511, 1512, 1513, 1514, 1515, 1516, 1517, 1518, 1519 may be issued when a preset threshold is exceeded for neglecting the alert of the resident.

Therefore, as shown in state B, the median value of the data (measured values) for n days is used to smooth the measured values and remove noise caused by the actions of others other than the residents. As a result, the number of short-term alerts mixed with noise will be reduced and the noise-canceled short-term alerts 1521, 1522 will be generated.

The statistics reporting function will be described with reference to FIG. FIG. 16 is a diagram showing an example of a screen 1600 displayed by monitor 8 or display 226. As shown in FIG. Screen 1600 may display various reports for each selected resident. The report illustrated in FIG. 16 shows a sleep report. The sleep report includes statistics 1610 and graphs 1620, 1630, 1640.

Statistics 1610 show daytime out of bed time, indoor time out of bed, time in bed, and nighttime sleep time and time in bed for each month in a specified time period (e.g., 3 months, 6 months, 12 months, etc.). can display bed time and

Graph 1620 displays night time in bed, travel time, time out of bed, and time in bed for each month during the specified time period. A graph 1630 displays sleep time, indoor stay time, bed time, and travel time for each month in the specified period as how to spend the day. Graph 1640 displays night time in bed, travel time, time out of bed and time in bed for each month during the specified time period.

Also, by pressing software switch 1650 or 1660, the user can switch the display unit of the statistics between monthly unit and weekly unit. As a result, the user can check the condition of each resident in different periods, so that changes in the health condition of each resident can be easily detected.

FIG. 17 is a diagram showing an example of a screen 1700 displayed on monitor 8 or display 226. As shown in FIG. Screen 1700 displays an example of information about a resident's activities of daily living. As an example, screen 1700 displays tables 1710, 1740, 1750 and windows 1720, 1730 for a designated resident (for example, care recipient).

Table 1710 accepts input for selecting a daily living activity to be displayed on monitor 8 from among a plurality of daily living activities. In one aspect, when the user selects a check box displayed in table 1710 by mouse operation, touch operation, or voice input, the daily life activity associated with the check box is selected. In the example of FIG. 17, two activities of daily living have been selected, namely 'walking speed' and 'sway level', as two checkboxes have been identified.

Window 1720 displays information for the “walking speed” selected in table 1710 . More specifically, window 1720 displays graphs 1721 and 1722 . A graph 1721 represents changes in walking speed. In one aspect, graph 1721 displays the evolution of walking speed over a prespecified period of time. This period is set, for example, as a period of time before the date when the information shown in FIG. 17 was extracted. The fixed period of time to go back is, for example, one week, two weeks, one month, three months, etc., but the period is not limited to these, and the user can arbitrarily specify the period of display. A graph 1722 represents changes in the moving average of walking speed.

In window 1720, the two marks represent the date on which rehabilitation was performed and the content of rehabilitation, respectively. Since the graph displayed on the window 1720 and the rehabilitation are associated with each other by the mark, the user can recognize the relationship between the transition of the activities of daily living and the contents of the rehabilitation.

Window 1730 displays information for the “sway degree” selected in table 1710 . More specifically, window 1730 displays graphs 1731 and 1732 . A graph 1731 represents changes in the stagger degree. In one aspect, graph 1731 displays the evolution of the degree of sway over a prespecified period of time. This period is set, for example, as a certain period of time before the date when the information shown in FIG. 17 was extracted, similar to the graphs 1721 and 1722 of the window 1720 . This time period may be the same as the time period displayed in window 1720, or it may be different. A graph 1732 represents changes in the moving average of the degree of wobble.

Also in window 1720, two marks are displayed as the date of rehabilitation and the content of rehabilitation, respectively. This mark facilitates associating the graph displayed in window 1730 with rehabilitation.

Table 1740 displays the details of the rehabilitation that was performed. In one aspect, table 1740 displays details of the most recently performed rehabilitation. The period exemplified in the table 1740 is three days, but the displayed period is not limited and may be displayed in the past. In another aspect, the period displayed in table 1740 may move in conjunction with the change in period in window 1720 or window 1730 .

In yet another aspect, the content of rehabilitation performed during a period corresponding to the period specified in window 1720 or 1730 may be displayed. For example, if “September 8-September 15” is selected in window 1720 , table 1740 may display rehabilitation performed during the selected period. At this time, the periods may be displayed by scrolling according to the size of the area reserved for displaying the table 1740 .

The table 1750 contains information on the recommended rehabilitation menu for the care recipient. For example, in one aspect, table 1750 includes a menu, a recommended number of times to perform that menu, an activity time, and a message. The menu is determined based on the care recipient's current condition, the care recipient's wishes, and changes in past activities of daily living. In one aspect, the menu is determined by care staff, or in other aspects, is determined based on performance data from other care recipients who have seen similar conditions, wishes, and changes in past activities of daily living. obtain.

The graph representing changes in the daily living activities of the care recipient does not necessarily have to be an upward-sloping graph, as shown in FIG. 17 . In another aspect, the graph representing the change may be, for example, a downward-sloping graph, a graph showing that the variation is becoming smaller (=converging), or the like.

<Summary>
Some of the technical features disclosed above can be summarized as follows.

[Arrangement 1] According to one embodiment, a computer-implemented method for detecting a change in a resident's condition is provided. In this method, the CPU 1 or the control device 221 accesses time-series data obtained by observing the resident's living room. The CPU 1 or the control device 221 detects short-term or long-term changes in time-series data from the accessed data. CPU 1 or control device 221 determines whether short-term changes or long-term changes fall within a normal range predetermined as a normal range. CPU 1 or control device 221 outputs the result of the determination.

[Configuration 2] In a certain aspect, the CPU 1 or the control device 221 displays an alert calling the resident's attention on the monitor 8 or the display 226, or outputs the alert by voice.

[Configuration 3] In one aspect, the CPU 1 or the control device 221 displays on the monitor 8 or the display 226 the permissible range (for example, the upper limit value or the lower limit value) of the time-series data about the resident.

[Configuration 4] In a certain aspect, the CPU 1 or the control device 221 determines the state of the resident according to the period (short term (eg, weekly), medium term (eg, monthly), long term (eg, yearly)) Statistical processing is executed, and the results of statistical processing are displayed on monitor 8 or display 226 according to the period. Since the type of graph displayed can change depending on the period, the user can easily grasp changes in the state.

[Arrangement 5] In a certain aspect, the CPU 1 or the control device 221 removes from the time-series data data representing movements of others who are different from those of the resident. CPU 1 or control device 221 detects a short-term change or a long-term change from the time-series data after removing the data representing the movements of others.

[Structure 6] In one aspect, the CPU 1 or the control device 221 removes from the time-series data data in which the bed rest rate during the daytime is lower than a preset rate. As a result, even if the movement of a person other than the resident is acquired by the sensor, the data of the movement is excluded from the determination target, so that the deterioration of the change detection accuracy can be prevented.

[Configuration 7] In one aspect, the CPU 1 or the control device 221 outputs a message for the resident. The message includes, for example, a description of the resident's condition, a rehabilitation menu for improving the resident's condition, and the like. This makes it easier for the user to take action to improve the condition of the resident.

As described above, cloud server 150 or management server 200 or mobile terminal 220 according to the present embodiment uses time-series data acquired from sensor box 119 to detect changes in the state of the resident, If the change deviates from a predetermined permissible range for short-term or medium-term changes, an alert is issued to that effect. Alternatively, cloud server 150, management server 200, or mobile terminal 220 may issue an alert to that effect if the change deviates from a range predetermined as a permissible range of change over the long term. As a result, the user of the cloud server 150, the management server 200, or the mobile terminal 220 (that is, the manager or staff of a facility such as a nursing home or a hospital) can accurately grasp changes in the state of the resident being accommodated. Therefore, it becomes easier to take measures to improve the situation.

It should be considered that the embodiments disclosed this time are illustrative in all respects and not restrictive. The scope of the present invention is indicated by the scope of the claims rather than the above description, and is intended to include all modifications within the scope and meaning of equivalents of the scope of the claims.

1 CPU, 2 mouse, 3 keyboard, 4,103,223 RAM, 5 hard disk, 6 optical disk drive, 7,104,224 communication interface, 8 monitor, 9,102,222 ROM, 11,130 gateway server, 12, 135 exchange device, 15 wireless base station, 16 external network, 31 range, 100 system, 101, 221 control device, 105 camera, 106 Doppler sensor, 107 wireless communication device, 108, 228 storage device, 110, 120 living room, 111, 121 resident, 112 furniture, 113 bed, 114 toilet, 115 care call handset, 116 toilet sensor, 117, ID sensor, 118 door sensor, 119 sensor box, 140 access point, 141, 142, 143, 144 caregiver, 150 Cloud server, 160 push server, 161, 162, 163, 164, 220 mobile terminal, 180 facility, 200 management server, 226 display, 229 input device, 241 care call button, 290 vital sensor, 400 computer system, 510 database, 530 terminal , 540 staff, 570 recommendation device, 590 rehabilitation plan, 600 change detection device, 610 signal input unit, 620 storage unit, 630 data processing unit, 631 index acquisition unit, 632 change detection unit, 633 trend detection unit, 634 recommendation Generation unit 635 Alert generation unit 640 Output unit 641 Alert output unit 642 Recommendation display unit 710, 720, 810, 910, 1710, 1740, 1750 Table 712 Date and time 713 Sleep time 714 Bed time 715 Leaving bed time 716 Indoor stay time 721 Day and minute 722 Walking speed 723 Action area 724 Movement amount 725 Rising time 811 Index 812 First threshold 813 Second threshold 1100, 1200, 1300, 1400, 1510 , 1600, 1700 screen, 1110, 1120, 1210, 1220, 1230, 1240, 1250 area, 1130 icon, 1140, 1150, 1160, 1620, 1630, 1640, 1721, 1722, 1731, 1732 graph, 1310, 1410 minimum value , 1320, 1420 1st quartile, 1330, 1430 2nd quartile (median), 1340, 1440 3rd quartile, 1350, 1450 maximum, 1511, 1512, 1513, 1514, 1515, 1516, 1517, 1518, 1521, 1522 short-term alerts, 1610 statistics, 1650 software switches, 1720, 1730 windows.

Claims (9)

A computer-implemented method for detecting a change in a resident's condition, comprising:
accessing time-series data obtained by observing the resident's living room;
detecting short-term or long-term changes in the time-series data;
Determining whether the short-term change or long-term change falls within a predetermined normal range;
and outputting the result of the determination. 2. The method of claim 1, wherein the outputting step comprises outputting a cautionary alert to the resident. 2. The method of claim 1, wherein the outputting step includes displaying an acceptable range for the time series data about the resident. The step of outputting
a step of performing statistical processing according to the period for determining the state of the resident;
and displaying the result of the statistical processing according to the period of time. Further comprising a step of removing data representing movement of another person different from the resident from the time-series data,
5. Any one of claims 1 to 4, wherein the detecting step includes detecting the short-term change or the long-term change from the time-series data after removing the data representing the movement of the other person. described method. 6. The method of claim 5, wherein removing the data includes removing from the time-series data data in which a bedtime rate during the day is lower than a preset rate. A method according to any preceding claim, wherein said outputting step comprises outputting a message for said resident. A program that causes a computer to execute the method according to any one of claims 1 to 7. a memory storing the program according to claim 8;
A resident state change detection device, comprising: a processor that executes the program.

Images