JP7327397B2 - Computer-implemented programs, information processing systems, and computer-implemented methods - Google Patents

Description

The present disclosure relates to data processing, and more particularly to data processing based on walking trajectories.

Regarding grasping the activity level of a person who needs nursing care, for example, Japanese Patent Application Laid-Open No. 2006-012057 (Patent Document 1) describes, "Understanding the living situation of an elderly person who requires nursing care, etc., and estimating the degree of independent activity of the elderly person. Independent activity measurement system that measures" is disclosed. The system disclosed in Patent Document 1 "detects the position of a person A to be measured, who is an elderly person requiring nursing care, on a bed 94a by a bed sensor 12h, and determines the measurement target based on the position detected by the bed sensor 12h. The state of the person, the start time and the retention time of the state are detected and stored as detection results.In addition, it is possible to detect that the user is using a facility that is at a predetermined distance from the bed, such as a portable toilet 94b. ) is detected by the sensor 12i, and the usage start time and the usage time are stored as detection results, and the detection results by the bed sensor 12h and the portable toilet sensor 12i are displayed." (See [Summary].) ).

Japanese Patent Application Laid-Open No. 2015-215711 (Patent Document 2) discloses that "In order to monitor the relative changes in the behavior and ecology of the residents in a way that closely follows the lives of the residents, the progress of 'aging', which has significant individual differences, can be monitored. A watching system that can relatively evaluate according to the resident's ability and environment and present the specification of watching according to the evaluation” (see [Problem] in [Summary]).

JP 2006-012057 A JP 2015-215711 A

People who need to be watched or cared for often stay in facilities for a long period of time, and their behavior may change during that time. Therefore, there is a need for a technology that can easily grasp changes over time in behavioral situations.

In addition, care certification is performed based on, for example, interviews with the subject and caregivers, opinions of attending physicians, etc., and quantitative measurement is not possible, and it may take time to determine the degree of care required. Therefore, there is a need for a technique that enables quantitative measurement and does not take much time for determination.

The present disclosure has been made in view of the background as described above, and an object in one aspect is to provide a technique that can easily grasp changes in the behavioral situation of a person who needs nursing care. An object in another aspect is to provide techniques for deriving quantitative information that can be used for certification of care level.

According to one embodiment, a computer-executable program is provided for determining a level of care. The program instructs the computer to represent the resident's walking trajectory, obtain a plurality of trajectory data obtained on different days, and identify a plurality of daily behavioral states of the resident based on the plurality of trajectory data. and calculating the time for each of the identified behavioral states; multiplying the time for each behavioral state by a predetermined coefficient to calculate a product; and a step of calculating the amount of activity for the day.

According to one embodiment, the program further causes the computer to derive a change in the resident's activity status by comparing each calculated amount of activity for each of the plurality of days.

According to an embodiment, the plurality of behavioral states includes at least two or more of walking, sitting, lying, and moving in a wheelchair.

According to one embodiment, the plurality of behavioral states includes walking, sitting, and lying. The magnitude relationship among the first coefficient for walking time, the second coefficient for sitting time, and the third coefficient for lying time is first coefficient>second coefficient>third coefficient.

According to an embodiment, the obtaining step includes summarizing each trajectory data in the resident's living room.

According to another embodiment, an information processing device is provided. The information processing device includes a memory and a processor coupled to the memory. The processor obtains a plurality of trajectory data representing the resident's walking trajectory and obtained on different days, identifies a plurality of daily behavioral states by the resident based on the plurality of trajectory data, and identifies Calculate the time for each behavioral state, multiply the time for each behavioral state by a predetermined coefficient to calculate the product, and add each product to calculate the daily activity amount of the resident. is configured to

According to an embodiment, the processor is further configured to derive a change in the resident's activity status by comparing each calculated activity measure for each of the plurality of days.

In the information processing apparatus according to one embodiment, the plurality of behavioral states includes at least two or more of walking, sitting, lying down, and moving in a wheelchair.

In an information processing apparatus according to an embodiment, the plurality of behavioral states includes walking, sitting, and lying down. The magnitude relationship among the first coefficient for walking time, the second coefficient for sitting time, and the third coefficient for lying time is first coefficient>second coefficient>third coefficient.

In an information processing apparatus according to an embodiment, obtaining includes summing each trajectory data in a resident's living room.

According to another embodiment, a computer-implemented method is provided for determining a level of care. The method comprises the steps of obtaining a plurality of trajectory data representative of a resident's walking trajectory and obtained on different days; identifying a plurality of daily behavioral states of the resident based on the plurality of trajectory data; calculating the time for each of the identified behavioral states; multiplying the time for each behavioral state by a predefined coefficient to calculate a product; and calculating the amount of activity of

According to another embodiment, the method further comprises the step of deriving a change in the resident's activity status by comparing each calculated activity measure for each of the plurality of days.

In a method according to another embodiment, the plurality of behavioral states includes at least two or more of walking, sitting, lying, and moving in a wheelchair.

In a method according to another embodiment, the plurality of behavioral states includes walking, sitting and lying. The magnitude relationship among the first coefficient for walking time, the second coefficient for sitting time, and the third coefficient for lying time is first coefficient>second coefficient>third coefficient.

In a method according to another embodiment, the obtaining step includes summarizing each trajectory data in the resident's living room.

In a certain situation, it is possible to easily grasp changes in the behavior of a person who needs nursing care. In other aspects, quantitative information can be derived that can be used for certification of care levels.

The above and other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description of the invention taken in conjunction with the accompanying drawings.

It is a figure which shows an example of a structure of the watching system 100. FIG. It is a block diagram which shows the outline of a structure of the watching system 100. FIG. 3 is a block diagram showing the hardware configuration of computer system 300 functioning as cloud server 150. FIG. It is a figure which shows an example of an outline of an apparatus structure of the watching system 100 using the sensor box 119. FIG. 4 is a diagram showing one mode of data storage in hard disk 5 provided in cloud server 150. FIG. 4 is a diagram conceptually showing one mode of data storage in hard disk 5 of cloud server 150. FIG. 4 is a flow chart showing part of the processing executed by CPU 1 of cloud server 150 according to an embodiment. FIG. 10 is a diagram showing transitions between results of the resident's activity state displayed on the monitor 8 and the amount of indoor activity according to an embodiment;

Hereinafter, embodiments of the technical concept according to the present disclosure will be described with reference to the drawings. In the following description, the same parts are given the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

[Technical concept]
First, the technical idea according to the present disclosure will be described. In one aspect, the system measures the movement route of a person to be watched over (for example, a resident of a nursing care facility), performs image analysis using image data sent from a camera placed in the living room, It is possible to grasp the behavioral state of the person. The system then measures the subject's walking time, sitting time (time sitting on the edge of the bed or chair), and lying time (time lying on the bed). Since the amount of exercise varies depending on the subject's posture and activity content, a coefficient that reflects the exercise load is set in advance. The system calculates the product by multiplying each measured time by a coefficient set according to the behavioral state, and calculates the sum of the products as the amount of activity.

Further, the system calculates the amount of activity of the same resident on different days and compares the amounts of activity to detect changes in the amount of activity of the resident.

[Configuration of monitoring system]
FIG. 1 is a diagram showing an example of the configuration of the watching system 100. As shown in FIG. The watching target is, for example, a resident in each living room provided in the living room area 180 of the facility. In the monitoring system 100 of FIG. 1 , living rooms 110 and 120 are provided in a living room area 180 . A living room 110 is assigned to a resident 111 . A living room 120 is assigned to a resident 121 . In the example of FIG. 1, the number of living rooms included in the watching system 100 is two, but the number is not limited to this.

In monitoring system 100 , sensor boxes 119 installed in living rooms 110 and 120 , management server 200 installed in management center 130 , and access point 140 are connected via network 190 . Network 190 may include both intranets and the Internet.

In the watching system 100 , a mobile terminal 143 carried by a caregiver 141 and a mobile terminal 144 carried by a caregiver 142 can be connected to a network 190 via an access point 140 . Furthermore, sensor box 119 , management server 200 and access point 140 can communicate with cloud server 150 via network 190 .

Living rooms 110 and 120 each include a wardrobe 112, a bed 113, and a toilet 114 as equipment. A door sensor 118 is installed on the door of the living room 110 to detect opening and closing of the door. A toilet sensor 116 for detecting opening and closing of the toilet 114 is installed on the door of the toilet 114 . A bed 113 is provided with an odor sensor 117 that detects the odor of each of the residents 111 and 121 . Each resident 111,121 is equipped with a vital sensor 290 for detecting vital information of the resident 111,121. The detected vital information includes the resident's body temperature, respiration, heart rate, and the like. In living rooms 110 and 120, each resident 111 and 121 can operate a care call slave device 115, respectively.

The sensor box 119 incorporates sensors for detecting behavior of objects in the living rooms 110 and 120 . One example of a sensor is a Doppler sensor for detecting motion of an object. Another example is a camera. Sensor box 119 may include both a Doppler sensor and a camera as sensors.

The constituent elements of the watching system 100 will be described with reference to FIG. 2 . FIG. 2 is a block diagram showing an overview of the configuration of the watching system 100. As shown in FIG.

[Sensor box 119]
The sensor box 119 includes a control device 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a communication interface 104, a camera 105, a Doppler sensor 106, a wireless communication device 107, and a storage device. 108.

The control device 101 controls the sensor box 119 . The control device 101 is composed of, for example, at least one integrated circuit. Integrated circuits include, for example, at least one CPU (Central Processing Unit), MPU (Micro Processing Unit) and other processors, at least one ASIC (Application Specific Integrated Circuit), at least one FPGA (Field Programmable Gate Array), or these It is composed of a combination of

An antenna (not shown) and the like are connected to the communication interface 104 . The sensor box 119 exchanges data with an external communication device via the antenna. External communication devices include, for example, the management server 200, the mobile terminals 143, 144 and other terminals, the access point 140, the cloud server 150 and other communication terminals.

Camera 105, in one implementation, is a near-infrared camera. A near-infrared camera includes an IR (Infrared) projector that projects near-infrared light. By using a near-infrared camera, an image representing the inside of living room 110, 120 can be captured even at night. In other implementations, camera 105 is a surveillance camera that receives only visible light. In still other implementations, camera 105 may be a 3D sensor or a thermographic camera. The sensor box 119 and the camera 105 may be configured integrally or separately.

The Doppler sensor 106 is, for example, a microwave Doppler sensor that emits and receives radio waves to detect the behavior (movement) of objects in the living rooms 110 and 120 . Thereby, the biological information of the residents 111 and 121 in the living rooms 110 and 120 can be detected. In one example, the Doppler sensor 106 emits microwaves in the 24 GHz band toward the beds 113 of the living rooms 110 and 120 and receives reflected waves reflected by the residents 111 and 121 . The reflected waves are Doppler-shifted by the motions of the residents 111 and 121 . The Doppler sensor 106 can detect the respiratory state and heart rate of the residents 111 and 121 from the reflected waves.

Wireless communication device 107 receives signals from care call slave device 240 , door sensor 118 , toilet sensor 116 , odor sensor 117 , and vital sensor 290 and transmits the signals to control device 101 . For example, care call slave device 240 has care call button 241 . When the button is operated, care call slave device 240 transmits a signal indicating that the operation has been performed to wireless communication device 107 . Door sensor 118 , toilet sensor 116 , odor sensor 117 , and vital sensor 290 transmit their detection results to wireless communication device 107 .

The storage device 108 is, for example, a fixed storage device such as a flash memory or hard disk, or a recording medium such as an external storage device. The storage device 108 stores programs executed by the control device 101 and various data used for executing the programs. Various data may include the behavior information of residents 111 and 121 . Details of the action information will be described later.

At least one of the programs and data described above can be stored in a storage device other than the storage device 108 (for example, a storage area of the control device 101 (eg, cache memory), a ROM 102, a storage device that can be accessed by the control device 101). It may be stored in the RAM 103, external equipment (for example, the management server 200, mobile terminals 143, 144, etc.).

[Action information]
The above action information will be explained. Action information is, for example, information indicating that residents 111 and 121 have performed a predetermined action. In one example, the predetermined actions are "waking up" indicating that the residents 111 and 121 have woken up, "getting out of bed" indicating that the residents 111 and 121 have left the bedding, and the residents 111 and 121 have fallen off the bedding. It includes four actions of "falling down" indicating that the residents 111 and 121 have fallen down.

In one embodiment, the control device 101 generates each action information of the residents 111 and 121 associated with each living room 110 and 120 based on the images captured by the cameras 105 installed in each living room 110 and 120. do. For example, the control device 101 detects the heads of the residents 111 and 121 from the above images, and determines the " Detects "getting up", "getting out of bed", "falling" and "falling". A specific example of generating behavior information will be described in more detail below.

First, the storage device 108 stores the location area of each bed 113 in the rooms 110 and 120, the first threshold Th1, the second threshold Th2, and the third threshold Th3. The first threshold Th1 distinguishes the size of the resident's head between the lying posture and the sitting posture within the location area of the bed 113 . The second threshold Th2 identifies whether or not the resident is in a standing posture in the living rooms 110 and 120 excluding the area where the bed 113 is located, based on the size of the resident's head. The third threshold Th3 identifies whether or not the resident is lying down based on the head size of the resident in the living rooms 110 and 120 excluding the area where the bed 113 is located.

The control device 101 extracts a moving body region as a human region of the residents 111 and 121 from the target image by, for example, a background subtraction method or a frame subtraction method. The control device 101 further derives from the extracted moving object region, for example, a circular or elliptical Hough transform, pattern matching using a head model prepared in advance, and a neural network trained for head detection. The head regions of the residents 111 and 121 are extracted using the threshold values thus obtained. The control device 101 detects "getting up", "getting out of bed", "falling", and "falling" from the extracted position and size of the head.

The control device 101 determines that the position of the head extracted as described above is within the location area of the bed 113 and the size of the head extracted as described above is determined by using the first threshold value Th1. It may be determined that the action "wake up" has occurred when it detects a change from posture magnitude to sitting posture magnitude.

When the position of the head extracted as described above moves from within the location area of the bed 113 to outside the location area of the bed 113, the control device 101 controls the size of the head extracted as described above. By applying the second threshold Th2 to the head position, it may be determined that the action "getting out of bed" has occurred when it is detected that the size of the head has changed from a certain size to the size of the standing posture.

When the position of the head extracted as described above moves from within the location area of the bed 113 to outside the location area of the bed 113, the control device 101 controls the size of the head extracted as described above. By applying the third threshold value Th3 to , it may be determined that the action "fall" has occurred when it is detected that the head has changed from a certain size to the size of the lying posture.

The control device 101 determines that the position of the head extracted as described above is located in the living room 110 or 120 excluding the location area of the bed 113, and the size of the extracted head uses the third threshold value Th3. It may be determined that the action "fall" has occurred if it detects a change from a size to a lying size by .

As described above, in one specific example, the control device 101 of the sensor box 119 generates each action information of the residents 111 and 121 . In addition, in the monitoring system 100 according to another aspect, an element other than the control device 101 (for example, the cloud server 150) generates the behavior information of the residents 111 and 121 using the images in the living rooms 110 and 120. good too.

[Portable terminals 143, 144]
The mobile terminals 143 , 144 include a control device 221 , a ROM 222 , a RAM 223 , a communication interface 224 , a display 226 , a storage device 228 and an input device 229 . In one aspect, the mobile terminals 143 and 144 are implemented as, for example, smart phones, tablet terminals, wristwatch-type terminals, and other wearable devices.

The control device 221 controls the mobile terminals 143 and 144 . The control device 221 is composed of, for example, at least one integrated circuit. An integrated circuit is composed of, for example, at least one CPU, at least one ASIC, at least one FPGA, or a combination thereof.

An antenna (not shown) and the like are connected to the communication interface 224 . Mobile terminals 143 and 144 exchange data with external communication devices via the antenna and access point 140 . External communication equipment includes, for example, the sensor box 119, management server 200, and the like.

The display 226 is implemented by, for example, a liquid crystal display, an organic EL (Electro Luminescence) display, or the like. The input device 229 is implemented by a touch sensor provided on the display 226, for example. The touch sensor receives a touch operation on the mobile terminals 143 and 144 and outputs a signal corresponding to the touch operation to the control device 221 .

The storage device 228 is implemented by, for example, a flash memory, a hard disk or other fixed storage device, or a removable data recording medium.

[Cloud server configuration]
The configuration of the cloud server 150 as a type of computer will be described with reference to FIG. FIG. 3 is a block diagram showing the hardware configuration of computer system 300 that functions as cloud server 150. As shown in FIG.

The computer system 300 has, as main components, a CPU 1 that executes a program, a mouse 2 and a keyboard 3 that receive instructions input by the user of the computer system 300, data generated by the execution of the program by the CPU 1, or the mouse 2 Alternatively, a RAM 4 for volatilely storing data input via a keyboard 3, a hard disk 5 for nonvolatilely storing data, an optical disk drive 6, a communication interface (I/F) 7, and a monitor 8 are provided. include. Each component is connected to each other by a data bus. The optical disc drive device 6 is loaded with a CD-ROM 9 or other optical discs.

Processing in the computer system 300 is implemented by each piece of hardware and software executed by the CPU 1 . Such software may be stored in the hard disk 5 in advance. Also, the software may be stored in a CD-ROM 9 or other recording medium and distributed as a computer program. Alternatively, the software may be provided as a downloadable application program by a so-called information provider connected to the Internet. Such software is read from the recording medium by the optical disk drive 6 or other reading device, or downloaded via the communication interface 7, and then temporarily stored in the hard disk 5. FIG. The software is read from the hard disk 5 by the CPU 1 and stored in the RAM 4 in the form of an executable program. CPU1 executes the program.

Each component that makes up the computer system 300 shown in FIG. 3 is conventional. Therefore, it can be said that one of the essential parts of the technical idea according to the present disclosure is software stored in RAM 4, hard disk 5, CD-ROM 9 or other recording media, or software that can be downloaded via a network. Recording media may include non-transitory computer-readable data recording media. Since the operation of each piece of hardware in computer system 300 is well known, detailed description will not be repeated.

The recording medium is not limited to CD-ROM, FD (Flexible Disk), hard disk, magnetic tape, cassette tape, optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile Disc)). , IC (Integrated Circuit) card (including memory card), optical card, mask ROM, EPROM (Electronically Programmable Read-Only Memory), EEPROM (Electronically Erasable Programmable Read-Only Memory), flash ROM and other semiconductor memories It may be a medium that literally carries the program.

The program here includes not only a program that can be directly executed by the CPU, but also a program in source program format, a compressed program, an encrypted program, and the like.

[Device configuration of watching system 100]
Watching using the watching system 100 will be described with reference to FIG. FIG. 4 is a diagram showing an example of a schematic device configuration of the monitoring system 100 using the sensor box 119. As shown in FIG.

The watching system 100 is used to watch over residents 111 and 121 who are watching targets (monitoring targets) and other residents. As shown in FIG. 4, a sensor box 119 is attached to the ceiling of living room 110 . Sensor boxes 119 are similarly installed in other living rooms.

Range 410 represents the detection range by sensor box 119 . If the sensor box 119 has a Doppler sensor as described above, the Doppler sensor detects human behavior occurring within the range 410 . If sensor box 119 has a camera as a sensor, the camera takes an image within area 410 .

Sensor box 119 is installed, for example, in nursing homes, medical facilities, homes, and the like. In the example of FIG. 4, the sensor box 119 is attached to the ceiling and photographs the resident 111 and the bed 113 from the ceiling. The mounting location of the sensor box 119 is not limited to the ceiling, and may be mounted on the side wall of the living room 110 .

The watching system 100 detects a danger to the resident 111 based on a series of images (that is, video) obtained from the camera 105 . By way of example, detectable hazards include resident 111 falling, resident 111 being in a hazardous area (eg, bed rails, etc.), and the like.

When the watching system 100 detects that the resident 111 is in danger, the monitoring system 100 notifies the caregivers 141, 143, etc. of the fact. As an example of the notification method, the watching system 100 notifies the mobile terminals 143 and 144 of the caregivers 141 and 142 of the danger of the resident 111 . Upon receiving the notification, the mobile terminals 143 and 144 notify the caregivers 141 and 142 of the danger of the resident 111 by message, voice, vibration, or the like. As a result, the caregivers 141 and 142 can immediately grasp that the resident 111 is in danger, and can quickly rush to the resident 111 .

Note that FIG. 4 shows an example in which the monitoring system 100 includes one sensor box 119 , but the monitoring system 100 may include a plurality of sensor boxes 119 . Moreover, although FIG. 4 shows an example in which the watching system 100 is provided with a plurality of mobile terminals 143 and 144, the watching system 100 can also be realized by a single mobile terminal.

[data structure]
The data structure of the cloud server 150 will be described with reference to FIG. FIG. 5 is a diagram showing one mode of data storage in hard disk 5 included in cloud server 150. As shown in FIG.

The hard disk 5 holds a table 60. FIG. The table 60 sequentially stores data transmitted from each sensor provided in each living room. More specifically, table 60 includes room ID 61 , date and time 62 , X coordinate value 63 , and Y coordinate value 64 . The room ID 61 identifies the room of the resident. Date and time 62 identifies the date and time when the signal sent from the sensor was acquired. The X-coordinate value 63 represents the X-coordinate value of the point detected at the date and time, that is, the position of the resident. Y-coordinate value 64 represents the Y-coordinate value of the point detected at the date and time, that is, the position of the resident. In one aspect, the coordinate axes from which the X coordinate value and the Y coordinate value are based are defined, for example, with reference to the end point of the living room (for example, one corner of the room). In another aspect, the coordinate axes may be defined with reference to a certain point in the facility where each living room is provided.

Furthermore, the hard disk 5 holds image data sent from the sensor box 119 . Image data is acquired at predetermined time intervals. The CPU 1 can identify the state of the resident by performing image analysis using each image data. For example, the CPU 1 can extract the head from each image data, the time the resident is lying down, the time the resident is sitting on the bed 113, and the time the resident is walking.

In addition to the resident, there is a possibility that the person walking in the living room may be a caregiver or a family member. Therefore, the CPU 1 can exclude the walking trajectories of people other than the resident. For example, the CPU 1 may calculate the walking speed for each walking trajectory, and exclude the walking trajectory for which a speed equal to or higher than a certain speed, which is estimated to be the walking speed of a healthy person other than the resident, is calculated from the object of image analysis. In this case, as the walking speed of a healthy person, a walking speed measured in advance or a walking speed measured during image analysis can be used. For example, for each point detected at a plurality of times from the walking trajectory, the CPU 1 calculates the walking distance based on the x-coordinate value, the y-coordinate value, and the time from one point to the next point. Velocity can be calculated.

The data structure of the cloud server 150 will be further described with reference to FIG. FIG. 6 is a diagram conceptually showing one mode of data storage in the hard disk 5 of the cloud server 150. As shown in FIG. The hard disk 5 stores a table 600. FIG. Table 600 holds daily data observed for each resident.

Table 600 includes resident ID 610 , date 620 , walking time 630 , sitting time 640 , lying time 650 , and amount of indoor activity 660 . The resident ID 610 identifies the resident who is the object of observation. Date 620 represents the date when the movement was observed. The walking time 630 represents the time calculated as the time during which the resident is walking on that date. The sitting time 640 represents the time calculated as the sitting time of the resident on the date.

The lying time 650 represents the time calculated as the time during which the resident is lying on the date. The amount of indoor activity 660 represents the amount of indoor activity of the resident. Indoor activity amount 660 is calculated using walking time 630, sitting time 640, and lying time 650 in a certain aspect. For example, the CPU 1 multiplies each of the walking time 630 , the sitting time 640 and the lying time 650 by a preset coefficient, and calculates the sum of the products as the indoor activity amount 660 . Note that each coefficient can be set so as to satisfy the following magnitude relationship, for example. Coefficient of lying time<Coefficient of sitting time<Coefficient of walking time The conditions of the resident are not limited to the above three, and more conditions and coefficients corresponding to the conditions may be used. For example, a coefficient may be set for each of the states of moving in a wheelchair, standing, exercising, etc., and used to calculate the amount of indoor activity. Here, exercise includes, for example, heel raises, bending and stretching, and other exercises according to the athletic ability of the resident of the facility. In such a case, the magnitude relationship of each coefficient is as follows.
Coefficient of lying time < Coefficient of sitting time < Coefficient of standing time < Coefficient of time moving in a wheelchair < Coefficient of walking time < Coefficient of time exercising [Overview of CPU 1 operation]
(1) In a certain aspect, the CPU 1 obtains from the hard disk 5 a plurality of locus data representing the resident's walking locus and obtained on different days. The CPU 1 identifies a plurality of daily behavior states of the resident based on the plurality of trajectory data. The behavioral state includes, for example, a walking state, a sitting state, a lying state, and the like. In another aspect, the ambulatory state can include independent walking, walking with an assistive device, and movement in a wheelchair. CPU 1 calculates the time for each identified behavioral state. The CPU 1 multiplies the time of each behavioral state by a predetermined coefficient to calculate the product, and adds the products to calculate the daily activity amount of the resident.

(2) In a certain aspect, the CPU 1 derives a change in the resident's activity state by comparing each indoor activity amount calculated for each of a plurality of days. For example, the user of the management server 200 can specify one or more past dates as targets for indoor activity comparison. Management server 200 transmits the specified date to cloud server 150 . The CPU 1 reads from the hard disk 5 the walking locus data and the image data for the specified date. The CPU 1 performs image analysis using the image data and classifies the state of the person. In yet another aspect, when the CPU 1 detects an object moving with the image of the person, it determines that the resident is moving with a walking aid or wheelchair.

(3) In some aspects, the plurality of behavioral states includes walking, sitting, and lying. The magnitude relationship among the first coefficient for walking time, the second coefficient for sitting time, and the third coefficient for lying time is first coefficient>second coefficient>third coefficient. The first coefficient, the second coefficient, and the third coefficient may be defined, for example, in a positive correlation relationship between each behavioral state and energy consumption. Each coefficient can be changed according to the purpose of judgment as long as the magnitude relationship is maintained. For example, each coefficient can be set according to the age of each resident, the level of care required, the presence or absence of dementia, and the like.

[Control structure]
The control structure of the cloud server 150 will be described with reference to FIG. FIG. 7 is a flow chart representing part of the processing executed by CPU 1 of cloud server 150 according to an embodiment.

At step S710, the CPU 1 acquires image data from the hard disk 5. FIG.
At step S720, CPU 1 identifies a plurality of behavioral states of the resident for a day based on the plurality of image data. For example, the CPU 1 performs image analysis using image data to classify the state of the resident. In one aspect, the CPU 1 determines whether the resident is sitting, lying down, or walking based on the results of the image analysis. This determination is made based on the location and orientation of the person's head, presence/absence of movement, etc. obtained from the result of image analysis.

At step S730, CPU 1 calculates the time for each identified behavioral state. For example, the CPU 1 calculates the duration of each state, totals the calculation results, and calculates a walking time 630, a sitting time 640, and a lying time 650. FIG.

At step S740, CPU 1 calculates the product by multiplying the time of each action state by a coefficient predetermined for each action state. In addition, this coefficient may be the same coefficient or a different coefficient between tenants. Moreover, in a certain aspect, the coefficient may be defined according to the resident's age, sex, level of care need, and the like.

In step S750, the CPU 1 calculates the daily indoor activity amount of the resident by adding each product. Note that the unit for calculating the amount of indoor activity is not limited to one day. In another aspect, the CPU 1 may calculate, for example, the amount of indoor activity for one week, the amount of indoor activity for one month, or other units of activity according to the settings of the administrator or the like.

In step S760, CPU 1 compares each activity amount calculated for each of a plurality of days. For example, the CPU 1 can calculate the amount of indoor activity on the day specified as the day on which the amount of indoor activity is to be calculated, the day three months before, and the day six months before, and compare transitions.

At step S770, CPU 1 derives a change in the resident's activity state based on the result of the comparison. For example, the CPU 1 compares the current amount of indoor activity with the past amount of indoor activity for each resident, and detects whether or not there is a change in the amount of indoor activity. When the CPU 1 detects a resident whose amount of indoor activity has decreased, the CPU 1 can output to the monitor 8 or a form the results representing the amount of indoor activity of the resident along with the walking time, sitting time, and lying time.

[Display mode]
With reference to FIG. 8, an example of a display mode of changes in the state of activity of a resident will be described. FIG. 8 is a diagram showing transitions between the result of the resident's activity state displayed on the monitor 8 and the amount of indoor activity according to an embodiment.

When the user of the management server 200 instructs calculation of the amount of indoor activity, the management server 200 sends the instruction to the cloud server 150 . The CPU 1 of the cloud server 150 performs the processing shown in FIG. 7 and transmits the processing result to the management server 200 . The monitor 8 of the management server 200 displays the result of processing.

More specifically, the monitor 8 displays a bar graph 810 showing walking time, sitting time, and sitting time for a certain resident (Mr. A) for the current month (for example, yesterday), the day three months ago, and the day six months ago. and a bar graph 830 showing the lying time, respectively. Furthermore, the monitor 8 displays a graph 840 showing the amount of indoor activity calculated as described above using the walking time corresponding to the bar graph 810, the sitting time corresponding to the bar graph 820, and the lying time corresponding to the bar graph 830. In this way, the nursing care staff, care manager, and other users of the resident can objectively grasp changes in the behavioral state of the resident, so it is possible to objectively determine the level of nursing care required. , and the convincingness and transparency of the determination result can be enhanced.

In the example of FIG. 8, for example, the walking time coefficient=2.0, the sitting time coefficient=1.4, and the lying time coefficient=1.0, the indoor activity amount is derived by the following formula.

Amount of indoor activity = walking time x 2.0 + sitting time x 1.4 + lying time x 1.0
In another aspect, it may be classified into more times. In this case, the indoor activity coefficient can be set as follows.

Amount of indoor activity = Lying time x 0.5 + sitting time x 0.8 + standing time (not moving) x 1.0 + wheelchair running time x 1.3 + walking time x 1.5 + exercise time x 2 .0
Note that the past days to be compared are not limited to three months ago and six months ago. Comparisons may be made based on weekly performance, such as 1 week ago and 2 weeks ago. Alternatively, the comparison may be based on monthly performance, such as one month ago and two months ago.

Moreover, in another aspect, each indoor activity amount of a plurality of residents may be compared. The comparison makes it easier to detect changes in each resident's condition.

[Summary of Embodiment]
As described above, according to the present embodiment, the image data of the person being watched over is acquired sequentially. The system uses movement trajectory data and image data to classify the state of the person being watched over into walking, sitting, lying, moving in a wheelchair, standing, exercising, etc. do. The system aggregates the time in each classified state, multiplies the time in each state by a coefficient set for each state, calculates the product, and calculates the sum of the products as the amount of indoor activity. In this way, various behaviors of one resident can be represented by a single value, the amount of indoor activity. Therefore, even if the time in each state changes, the amount of activity of the resident as a whole can be grasped. can do. Since the amount of indoor activity is calculated using data sent from the sensor box 119 (for example, image data from the camera 105 and output data from the Doppler sensor 106), subjectivity by the judge is eliminated. As a result, since the amount of behavior of the resident is objectively indicated, for example, it is possible to objectively determine the degree of care required of the resident, and the convincingness of the determination result can be enhanced.

It should be considered that the embodiments disclosed this time are illustrative in all respects and not restrictive. The scope of the present invention is indicated by the scope of the claims rather than the above description, and is intended to include all changes within the meaning and scope equivalent to the scope of the claims.

This technology is applicable to information processing of data obtained in hospitals, nursing homes, nursing homes and other facilities.

51 washbasin, 52 desk, 60 table, 100 system, 101,221 control device, 105 camera, 106 Doppler sensor, 107 wireless communication device, 108,228 storage device, 110,120 room, 111,121 resident, 112 chest , 113 bed, 114 toilet, 115 care call handset, 116 toilet sensor, 117 sensor, 118 door sensor, 119 sensor box, 130 control center, 140 access point, 141, 142 caregiver, 143, 144 mobile terminal, 150 cloud server , 180 room area, 190 network, 200 management server.

Claims (15)

A program running on a computer, said program causing said computer to:
a step of acquiring data on the behavior of the resident with a camera or Doppler sensor provided in the living room;
a step of identifying a plurality of behavioral states indicating living conditions of the resident in the living room based on the data on the behavior of the resident, and calculating the time for each of the identified behavioral states;
a step of multiplying the time of each said behavioral state by a coefficient predetermined corresponding to each said behavioral state , calculating the product, and calculating the amount of behavior of the resident by adding each product; The program that causes the to run. 2. The program of claim 1, wherein the program further causes the computer to derive a change in the resident's activity status by comparing each of the activity measures calculated for a plurality of time periods. 3. The program according to claim 1, wherein said plurality of behavioral states include at least two of walking, sitting, lying down, and moving in a wheelchair. the plurality of behavioral states includes walking, sitting, and lying;
The first coefficient for the walking time, the second coefficient for the sitting time, and the third coefficient for the lying time are such that the first coefficient > the second coefficient > the third coefficient. 3. The program according to claim 1 or 2. 5. The program according to any one of claims 1 to 4, wherein said obtaining step includes obtaining movement trajectory data in said resident's living room. A camera or Doppler sensor installed in the living room,
a memory that stores information of the camera or Doppler sensor;
a processor coupled to the memory;
the processor obtains data about the behavior of the resident;
identifying a plurality of behavioral states indicating living conditions of the resident in the living room based on the data on the behavior of the resident, and calculating the time for each of the identified behavioral states;
Multiplying the time in each of the behavioral states by a predetermined coefficient corresponding to each of the behavioral states , calculating the product, and adding each product to calculate the daily activity amount of the resident. An information handling system configured to: 7. The information processing system of claim 6, wherein the processor is further configured to derive a change in the resident's activity status by comparing each of the activity measures calculated for multiple time periods. 8. The information processing system according to claim 6, wherein said plurality of behavioral states include at least two of walking, sitting, lying down, and moving in a wheelchair. the plurality of behavioral states includes walking, sitting, and lying;
The first coefficient for the walking time, the second coefficient for the sitting time, and the third coefficient for the lying time are such that the first coefficient > the second coefficient > the third coefficient. The information processing system according to claim 6 or 7. 10. The information processing system according to any one of claims 6 to 9, wherein said acquiring includes acquiring movement trajectory data in said resident's living room. A computer implemented method comprising:
a step of acquiring data on the behavior of the resident with a camera or Doppler sensor provided in the living room;
a step of identifying a plurality of behavioral states indicating living conditions of the resident in the living room based on the data on the behavior of the resident, and calculating the time for each of the identified behavioral states;
Multiplying the time in each of the behavioral states by a predetermined coefficient corresponding to each of the behavioral states , calculating the product, and adding each product to calculate the daily activity amount of the resident. and a method. 12. The method of claim 11, further comprising deriving a change in the resident's activity status by comparing each of the activity measures calculated for a plurality of time periods. 13. The method of claim 11 or 12, wherein the plurality of behavioral states includes at least two or more of walking, sitting, lying, and moving in a wheelchair. the plurality of behavioral states includes walking, sitting, and lying;
The first coefficient for the walking time, the second coefficient for the sitting time, and the third coefficient for the lying time are such that the first coefficient > the second coefficient > the third coefficient. , a method according to claim 11 or 12. The method according to any one of claims 11 to 14, wherein said obtaining step includes obtaining movement trajectory data in said resident's living room.

Images