WO2020003951A1 - Program executed by computer, information processing device, and method executed by computer - Google Patents

Abstract

A process executed by a CPU of a computer system includes: a step (S710) in which a plurality of items of movement trajectory data are acquired from a hard disk; a step (S720) in which each movement trajectory is classified into a plurality of groups on the basis of a pattern of the plurality of items of movement trajectory data; a step (S740) in which a determination is made that an occupant is exhibiting abnormal behavior, when the number of movement trajectories classified into any group is equal to or greater than a predetermined number (Yes in step S730); a step (S760) in which a change in the occupant is detected, when a new movement trajectory that has not been classified into any of the groups is detected (Yes in step S750); a step (S770) in which a data process for displaying a movement frequency of the occupant as a heatmap is executed; and a step (S780) in which a process result is output.

Description

Computer-executed program, information processing apparatus, and computer-executed method

The present disclosure relates to data processing, and more specifically, to data processing based on a movement trajectory.

技術 Techniques for watching residents are known. For example, Japanese Patent Application Laid-Open No. 2015-215711 (Patent Literature 1) states, "In order to watch the relative changes in behavior and ecology of residents in a manner close to the lives of residents, the progress of" aging "in which individual differences are remarkable. A monitoring system that can relatively evaluate the resident according to the abilities and environment of the resident and present a watching specification that matches the evaluation ”(see [Problem] in [Summary]).

Japanese Patent Application Laid-Open Publication No. 2017-151676 (Patent Document 2) discloses “a monitored person monitoring apparatus, a method, and a system capable of further improving work efficiency”. The monitored person monitoring device is configured such that an event notification communication signal for notifying the event, containing event information indicating the content of a predetermined event related to the monitored person to be monitored, is communicably connected. The management server SV as an example is a device that notifies the terminal device.The self-sustained determination processing unit 223 that determines whether the monitored person Ob can be independent based on an image acquired from the sensor device via the network. And a warning notification processing unit 224 that transmits a warning notification communication signal containing warning information indicating a warning to a predetermined terminal device when the monitored person Ob is determined to be unable to become independent by the independence determination processing unit 223. (See [Summary]).

Japanese Patent Application Laid-Open No. 2003-153868 (Patent Literature 3) discloses a method of automatically determining whether a human body, such as an elderly person, has a mental and / or behavioral disorder or does not have the disorder. An individual disease detection system that can be determined in a targeted manner ”(paragraph 0021). The system includes: a photographing unit 1 and a determination unit 7 that are installed in a room and detect a movement frequency that is a distance that a human body has moved in the room during a predetermined time interval; The human body has a mental and / or behavioral disorder based on the open / close sensor 9 and the determination means 7 for at least detecting the use frequency of the tool and the detected movement frequency and the detected use frequency. Judgment means for judging whether the state is a state or a state without a failure. "(Refer to [Summary]).

Japanese Patent Laying-Open No. 2017-215634 (Patent Literature 4) discloses that “a nurse can visit and see at what location in a hospital many times and how much time he / she stayed at each location. Technology "is disclosed. According to the technology, “a behavior analysis unit 12 that analyzes a place visited by a nurse, the number of visits, and a stay time based on the behavior history information of a nurse, and the analyzed visit place, the number of visits, and the stay time, And a display control unit 13 for displaying a heat map on a layout image representing an arrangement of each place in the hospital, and a first pattern change centered on the number of visits and a second pattern change centered on the stay time. Variably sets the display mode of the heat map according to the above, the number of visits to each place and the length of stay can be simultaneously grasped by the display mode of the heat map variably set according to two types of pattern changes, Can see at a glance which places in the hospital they have visited and how long they stayed. " ] Reference).

Japanese Patent Laying-Open No. 2017-215635 (Patent Document 5) states, “In a heat map display in which values representing the behavior of nurses in a hospital are totalized and visualized, it is possible to see at a glance whether the displayed contents are good or bad. "Disclose" technology. According to this technology, “when summarizing values representing the behavior of nurses in a hospital (number of visits to each location, staying time, etc.) and displaying a heat map with the totalized values visualized for each location, Is provided with a display control unit 13 for performing a heat map display with a different display system according to the location of the patient, and among the various places such as a hospital room, a nurse station, a consultation room, and a corridor existing in the hospital, the one having the larger total value is By displaying a heat map with a different display system for each of the good places and the places where the tally value is smaller, the places where the tally values are larger (or smaller) by the heat map can be used. It will help you understand at a glance whether it's good or bad. "(See [Summary].)

JP-A-2015-215711 JP 2017-151676 A JP 2003-153868 A JP 2017-215634 A JP 2017-215635 A

For people who need to be watched over or care, the degree of care is determined based on, for example, interviews with the target person and caregiver and the opinion of the attending physician. May take some time. Therefore, there is a need for a technique that enables quantitative measurement and does not require much time for determination.

The present disclosure has been made in view of the above background, and an object in one aspect is to provide a technique for deriving quantitative information that can be used for nursing care recognition.

According to one embodiment, a computer-executable program for determining the degree of care is provided. The program includes a step of obtaining a plurality of trajectory data representing a trajectory of a person in a living room, a step of classifying each trajectory into a plurality of groups based on a pattern of the plurality of trajectory data, And determining that the resident of the room is performing abnormal behavior based on the fact that the number of movement trajectories classified into any one of the groups is equal to or greater than a predetermined number.

According to one embodiment, the program causes the computer to change the resident based on detecting a new trajectory that is not classified into any of the plurality of groups based on the pattern from the plurality of trajectory data. Is further executed.

According to one embodiment, the determining step estimates a behavior purpose of the resident based on the position information registered as the location where each of the plurality of articles arranged in the living room is arranged and the movement trajectory data. Including steps.

According to an embodiment, the estimating step is such that the resident can act alone based on whether or not a movement trajectory to a predetermined item of the plurality of items is detected during a certain period. Or not.

According to an embodiment, the program causes the computer to generate a trajectory of the resident in each area based on each movement trajectory data and each area data defining each of the plurality of areas generated by dividing the room in advance. A step of displaying the movement frequency is further executed.

According to one embodiment, the step of displaying includes displaying a movement frequency based on the movement trajectory data detected in a certain time zone and the respective area data among the plurality of movement trajectory data.

According to another embodiment, an information processing apparatus including a memory and a processor coupled to the memory is provided. The processor acquires a plurality of trajectory data representing the trajectory of a person in the living room, classifies each trajectory into a plurality of groups based on a pattern of the plurality of trajectory data, and selects one of the plurality of groups. Is configured to determine that the resident of the living room is performing abnormal behavior based on the number of the moving trajectories classified into the predetermined number being equal to or greater than the predetermined number.

According to one embodiment, the processor detects a change in the occupant based on detecting a new trajectory that is not classified into any of the plurality of groups based on the pattern from the plurality of trajectory data. It is further configured to:

According to an embodiment, the processor estimates the behavior purpose of the resident based on the location information registered as the location of each of the plurality of articles arranged in the room and the respective movement trajectory data. It is configured.

According to one embodiment, the estimating is such that the resident can act alone based on whether a trajectory to a predetermined item of the plurality of items is detected during a certain period of time. Or not.

According to one embodiment, the processor determines the moving frequency of the resident in each area based on each movement trajectory data and each area data defining each of the plurality of areas generated by dividing the room in advance. Is further configured to be displayed.

According to one embodiment, the displaying includes displaying a movement frequency based on the movement trajectory data detected in a predetermined time zone among the plurality of movement trajectory data and each area data.

According to yet another embodiment, a computer-implemented method is provided. The method includes: acquiring a plurality of movement trajectory data representing a movement trajectory of a person in a living room; classifying each movement trajectory into a plurality of groups based on a pattern of the plurality of movement trajectory data; And determining that the occupant of the living room is performing abnormal behavior based on the number of movement trajectories classified into any one of the groups being equal to or greater than a predetermined number.

According to an embodiment, the method includes detecting a change in the resident based on detecting a new movement trajectory that is not classified into any of the plurality of groups based on the pattern from the plurality of movement trajectory data. The method further includes the step of detecting.

According to one embodiment, the determining step estimates a behavior purpose of the resident based on the position information registered as the location where each of the plurality of articles arranged in the living room is arranged and the movement trajectory data. Including steps.

According to an embodiment, the estimating step is such that the resident can act alone based on whether or not a movement trajectory to a predetermined item of the plurality of items is detected during a certain period. Or not.

According to an embodiment, the method includes the steps of moving a resident in each area based on each movement trajectory data and each area data defining each of a plurality of areas generated by dividing a room in advance. The method further includes displaying the frequency.

According to one embodiment, the step of displaying includes displaying a movement frequency based on the movement trajectory data detected in a certain time zone and the respective area data among the plurality of movement trajectory data.

に お い て In certain situations, it is possible to derive quantitative information that can be used to certify the degree of care.

The above and other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description of the present invention that is understood in connection with the accompanying drawings.

FIG. 1 is a diagram illustrating an example of a configuration of a watching system. FIG. 1 is a block diagram illustrating an outline of a configuration of a watching system. FIG. 3 is a block diagram illustrating a hardware configuration of a computer system 300 functioning as a cloud server 150. It is a figure showing an example of the outline of the device composition of watching system 100 using sensor box 119. FIG. 3 is a diagram illustrating one mode of data storage in a hard disk 5 included in the cloud server 150. It is a figure showing a change of a moving track of a resident in a certain situation. 9 is a flowchart illustrating a part of a process executed by CPU 1 of cloud server 150. FIG. 4 is a diagram illustrating a screen on which a movement trajectory in a living room 110 is displayed on a monitor 8 as a heat map. It is a figure showing the heat map 910 obtained based on the movement locus which can be observed when a resident's state changes. FIG. 6 is a diagram illustrating an example of a diagnosis result displayed by a monitor 8 of the management server 200.

Hereinafter, embodiments of the technical concept according to the present disclosure will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. Their names and functions are the same. Therefore, detailed description thereof will not be repeated.

[Technical Thought]
First, a technical idea according to the present disclosure will be described. In a certain situation, the system can accurately acquire the daily state of the target person by measuring the moving path of the watching target person (for example, a resident of a nursing care facility). The order of drop-in in the room is patterned from the locus of movement of the subject. If the pattern is repeated daily, the movement can be determined to be a habitual action. If the pattern is repeated several times in a short time (for example, one hour), it can be estimated that abnormal behavior such as "repeated behavior" in the symptoms of dementia is occurring.

{(Change of trajectory)} For example, the system observes the trajectory of the subject every day and applies it to the trajectory pattern to classify the trajectories into groups. When the system detects that the movement trajectory pattern has changed (the movement trajectory has deviated from the pattern or has stopped using the movement trajectory), the system outputs a determination result indicating that there has been a change in the body or mind of the watching target person. obtain. The facility manager can objectively determine the state of the watching target person by referring to this determination result. Reasons for the change of the movement trajectory pattern include, for example, behavioral changes such as anxiety at the watching target person's feet (losing power), a decrease in their physical strength, and anxiety when they do not walk. Conceivable. In this case, the system may output a message prompting a review of the degree of care required for the watching target.

{(Estimation of behavior)} The system can estimate what the watching target person is doing (movement for what purpose) by combining the movement trajectory of the watching target person and the arrangement information (position information) of the furniture. By associating the movement trajectory with the action purpose, the system can find an action whose purpose cannot be classified (the purpose cannot be estimated). If the number of unintended behaviors increases for a certain monitoring target person, the system informs the manager that the dementia of the monitoring target person is progressing or the need to review the degree of nursing care is required. Can notify. In this case, it is possible to objectively determine the degree of need for nursing by using data defined in advance for the relationship between the purposeless behavior and the degree of need for nursing.

(Change in habit) The system can continuously measure the estimated behavior of the watching target person and grasp that the behavior has changed. The fact that the behavior has changed (habits have changed) is presumed to have changed the physical condition and thinking of the watching target person, so that the system can re-determine the recognition of the degree of need for nursing care. Also in this case, since the determination is made based on objectively observed data, the objectivity of the determination result can be ensured.

{(What can be done by yourself)} The system can estimate what daily activities are performed by the watching target alone from the movement trajectory of the watching target. The daily activities may include, for example, whether the watching target person can go to the toilet alone, can take out clothes from the closet and change clothes, can wash his / her face and hands on the sink, and the like. The system or its users (facility managers and caregivers) can determine whether there is a difference between the content of the application and the reality by comparing it with the judgment of the degree of care required or the assessment of the care plan. The system measures daily activities that can be performed by one person continuously, grasps what the monitoring target has improved and what cannot be done, and outputs the inspection result, so it is possible to make objective judgments Become.

{(Heat Map)} The system can create a heat map of the moving area and display it on the monitor by coloring the area where the watching target person has moved in the room such as the living room and the frequency of entering the area. The meaning of the same heat map depends on whether the heat map is created with 30-minute action data (moving locus) of the watching target person or with overnight action data. It will be different. An analysis can be performed by linking the result of the heat map with the data of how long the heat map was created.

The system can analyze behavior during a time zone that should not be walked around by cutting out only a specific time (for example, a sleeping time zone such as nighttime) and creating a heat map. If the heat map created using only the nighttime data indicates movement between the bed and the toilet, the system may determine that the watching target has moved for the toilet. In this case, it can be determined that there is no particular problem for the watching target person. However, when the system displays a heat map that fills the entire room for another watching target, it can be estimated that the watching target is wandering in the room at night. Therefore, the user of the system may determine that the watching target is not sleeping at night or that the rhythm of the watching target is reversed day and night.

[Configuration of watching system]
FIG. 1 is a diagram illustrating an example of the configuration of the watching system 100. The watching target is, for example, a resident in each living room provided in the living room area 180 of the facility. In the watching system 100 in FIG. 1, living rooms 110 and 120 are provided in a living room area 180. The living room 110 is assigned to the resident 111. The living room 120 is assigned to the resident 121. In the example of FIG. 1, the number of living rooms included in the watching system 100 is two, but the number is not limited to this.

In the watching system 100, the sensor boxes 119 installed in the living rooms 110 and 120, the management server 200 installed in the management center 130, and the access point 140 are connected via the network 190. Network 190 may include both an intranet and the Internet.

In the watching system 100, the mobile terminal 143 carried by the caregiver 141 and the mobile terminal 144 carried by the caregiver 142 can be connected to the network 190 via the access point 140. Further, the sensor box 119, the management server 200, and the access point 140 can communicate with the cloud server 150 via the network 190.

Each of the living rooms 110 and 120 includes a closet 112, a bed 113, and a toilet 114 as facilities. The door of the living room 110 is provided with a door sensor 118 that detects opening and closing of the door. A toilet sensor 116 for detecting the opening and closing of the toilet 114 is installed on the door of the toilet 114. The bed 113 is provided with an odor sensor 117 for detecting the odor of each of the residents 111 and 121. Each resident 111, 121 is equipped with a vital sensor 290 for detecting vital information of the resident 111, 121. The detected vital information includes the resident's body temperature, respiration, heart rate, and the like. In the living rooms 110 and 120, the residents 111 and 121 can operate the care call slave 115, respectively.

The sensor box 119 has a built-in sensor for detecting the behavior of an object in the living rooms 110 and 120. One example of a sensor is a Doppler sensor for detecting the movement of an object. Another example is a camera. The sensor box 119 may include both a Doppler sensor and a camera as sensors.

With reference to FIG. 2, the components of the watching system 100 will be described. FIG. 2 is a block diagram showing an outline of the configuration of the watching system 100.

[Sensor box 119]
The sensor box 119 includes a control device 101, a read only memory (ROM) 102, a random access memory (RAM) 103, a communication interface 104, a camera 105, a Doppler sensor 106, a wireless communication device 107, and a storage device. 108.

(4) The control device 101 controls the sensor box 119. The control device 101 is composed of, for example, at least one integrated circuit. The integrated circuit is, for example, at least one CPU (Central Processing Unit), MPU (Micro Processing Unit) or other processor, at least one ASIC (Application Specific Integrated Circuit), at least one FPGA (Field Programmable Gate Array), or these. And the like.

ア ン テ ナ An antenna (not shown) and the like are connected to the communication interface 104. The sensor box 119 exchanges data with an external communication device via the antenna. External communication devices include, for example, management server 200, mobile terminals 143, 144 and other terminals, access point 140, cloud server 150, and other communication terminals.

The camera 105 is a near-infrared camera in one implementation. The near-infrared camera includes an IR (Infrared) projector that emits near-infrared light. By using the near-infrared camera, an image representing the inside of the living room 110 or 120 can be captured even at night. In another implementation, camera 105 is a surveillance camera that receives only visible light. In still other implementations, a 3D sensor or a thermographic camera may be used as camera 105. The sensor box 119 and the camera 105 may be configured integrally or may be configured separately.

The Doppler sensor 106 is, for example, a microwave Doppler sensor, and emits and receives radio waves to detect the behavior (movement) of objects in the living rooms 110 and 120. Thereby, the biological information of the resident 111, 121 of the living room 110, 120 can be detected. In one example, the Doppler sensor 106 emits microwaves in the 24 GHz band toward the beds 113 of the rooms 110 and 120, and receives reflected waves reflected by the residents 111 and 121. The reflected waves are Doppler shifted by the actions of the residents 111 and 121. The Doppler sensor 106 can detect the respiratory state and heart rate of the residents 111 and 121 from the reflected waves.

The wireless communication device 107 receives signals from the care call slave device 240, the door sensor 118, the toilet sensor 116, the odor sensor 117, and the vital sensor 290, and transmits the signals to the control device 101. For example, the care call slave unit 240 includes a care call button 241. When the button is operated, care call slave device 240 transmits a signal indicating that the operation has been performed to wireless communication device 107. The door sensor 118, the toilet sensor 116, the odor sensor 117, and the vital sensor 290 transmit respective detection results to the wireless communication device 107.

The storage device 108 is, for example, a fixed storage device such as a flash memory or a hard disk, or a recording medium such as an external storage device. The storage device 108 stores a program executed by the control device 101 and various data used for executing the program. The various data may include behavior information of the residents 111 and 121. The details of the action information will be described later.

At least one of the above-mentioned programs and data is a storage device other than the storage device 108 (for example, a storage area (for example, a cache memory) of the control device 101, a ROM 102, The RAM 103 and external devices (for example, the management server 200 and the portable terminals 143 and 144) may be stored.

[Behavior information]
The above behavior information will be described. The action information is, for example, information indicating that the residents 111 and 121 have performed a predetermined action. In one example, the predetermined action is “wake up” indicating that the resident 111, 121 has occurred, “getting out” indicating that the resident 111, 121 has left the bedding, and the resident 111, 121 has fallen from the bedding. This includes four actions of “fall” indicating that the resident has fallen, and “falling” indicating that the resident 111 or 121 has fallen.

In one embodiment, the control device 101 generates each piece of behavior information of the resident 111, 121 associated with each room 110, 120 based on an image captured by the camera 105 installed in each room 110, 120. I do. The control device 101 detects, for example, the heads of the residents 111 and 121 from the image, and based on the detected temporal changes in the sizes of the heads of the residents 111 and 121, “ "Wake up", "get out of bed", "fall" and "fall" are detected. Hereinafter, a specific example of the generation of the behavior information will be described in more detail.

First, the storage area of the beds 113 in the living rooms 110 and 120, the first threshold Th1, the second threshold Th2, and the third threshold Th3 are stored in the storage device. The first threshold Th1 identifies the size of the resident's head between the lying position and the sitting position in the area where the bed 113 is located. The second threshold value Th2 identifies whether or not the resident is in the standing posture, based on the size of the resident's head in the living rooms 110 and 120 excluding the area where the bed 113 is located. The third threshold value Th3 identifies whether or not the resident is in the recumbent posture in the living rooms 110 and 120 excluding the area where the bed 113 is located, based on the size of the resident's head.

The control device 101 extracts a moving object region from the target image as a region of the occupants 111 and 121 by, for example, the background difference method or the frame difference method. The control device 101 further derives from the extracted moving body region by, for example, a circular or elliptical Hough transform, by pattern matching using a prepared head model, or by a neural network learned for head detection. The head areas of the residents 111 and 121 are extracted using the thresholds thus set. The control device 101 detects “wake up”, “get out of bed”, “fall” and “fall” from the extracted position and size of the head.

The control device 101 determines that the position of the head extracted as described above is within the area where the bed 113 is located, and that the size of the head extracted as described above uses the first threshold Th1 to lie down. When it is detected that the size of the posture has changed to the size of the sitting posture, it may be determined that the action “wake up” has occurred.

When the position of the head extracted as described above moves from within the area where the bed 113 is located to outside the area where the bed 113 is located, the control device 101 controls the size of the head extracted as described above. When it is detected that the head has changed from a certain size to a standing posture size by applying the second threshold value Th2, it may be determined that the action “getting out of bed” has occurred.

When the position of the head extracted as described above moves from within the area where the bed 113 is located to outside the area where the bed 113 is located, the control device 101 controls the size of the head extracted as described above. When it is detected that the head has changed from a certain size to a recumbent posture by applying the third threshold value Th3, it may be determined that the action “fall” has occurred.

The control device 101 determines that the position of the head extracted as described above is located in the living rooms 110 and 120 excluding the area where the bed 113 is located, and the size of the extracted head uses the third threshold Th3. If it is detected that the size has changed from a certain size to the size of the recumbent posture, it may be determined that the action “fall” has occurred.

As described above, in one specific example, the control device 101 of the sensor box 119 generates the behavior information of the residents 111 and 121. In the watching system 100 according to another aspect, an element other than the control device 101 (for example, the cloud server 150) generates the behavior information of the residents 111 and 121 using the images in the living rooms 110 and 120. Is also good.

[Mobile terminals 143, 144]
The mobile terminals 143 and 144 include a control device 221, a ROM 222, a RAM 223, a communication interface 224, a display 226, a storage device 228, and an input device 229. In one aspect, the mobile terminals 143 and 144 are realized as, for example, a smartphone, a tablet terminal, a wristwatch-type terminal, or other wearable devices.

The control device 221 controls the mobile terminals 143 and 144. The control device 221 is configured by, for example, at least one integrated circuit. The integrated circuit includes, for example, at least one CPU, at least one ASIC, at least one FPGA, or a combination thereof.

ア ン テ ナ An antenna (not shown) and the like are connected to the communication interface 224. The mobile terminals 143 and 144 exchange data with an external communication device via the antenna and the access point 140. External communication devices include, for example, the sensor box 119, the management server 200, and the like.

The display 226 is realized by, for example, a liquid crystal display, an organic EL (Electro Luminescence) display, or the like. The input device 229 is realized by, for example, a touch sensor provided on the display 226. The touch sensor receives a touch operation on the mobile terminals 143 and 144, and outputs a signal corresponding to the touch operation to the control device 221.

The storage device 228 is realized by, for example, a flash memory, a hard disk or another fixed storage device, or a removable data recording medium.

[Cloud Server Configuration]
The configuration of the cloud server 150 as a type of computer will be described with reference to FIG. FIG. 3 is a block diagram illustrating a hardware configuration of computer system 300 functioning as cloud server 150.

The computer system 300 includes, as main components, a CPU 1 that executes a program, a mouse 2 and a keyboard 3 that receive an instruction input by a user of the computer system 300, and data generated by executing the program by the CPU 1 or a mouse 2. Alternatively, a RAM 4 for volatilely storing data input via the keyboard 3, a hard disk 5 for nonvolatilely storing data, an optical disk drive 6, a communication interface (I / F) 7, and a monitor 8 Including. Each component is mutually connected by a data bus. The optical disk drive 6 is loaded with a CD-ROM 9 and other optical disks.

The processing in the computer system 300 is realized by each hardware and software executed by the CPU 1. Such software may be stored in the hard disk 5 in advance. The software may be stored on the CD-ROM 9 or another recording medium and distributed as a computer program. Alternatively, the software may be provided as a downloadable application program by an information provider connected to the so-called Internet. Such software is temporarily stored in the hard disk 5 after being read from the recording medium by the optical disk drive 6 or another reading device, or downloaded via the communication interface 7. The software is read from the hard disk 5 by the CPU 1 and stored in the RAM 4 in the form of an executable program. CPU 1 executes the program.

各 Each component of the computer system 300 shown in FIG. 3 is a general component. Therefore, it can be said that one of the essential parts of the technical idea according to the present disclosure is software stored in the RAM 4, the hard disk 5, the CD-ROM 9, or other recording media, or software downloadable via a network. The storage medium may include a non-transitory, computer-readable data storage medium. Since the operation of each piece of hardware of computer system 300 is well known, detailed description will not be repeated.

The recording medium is not limited to a CD-ROM, FD (Flexible Disk), or hard disk, but may be a magnetic tape, cassette tape, optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile Disc)). , IC (Integrated Circuit) card (including memory card), optical card, mask ROM, EPROM (Electronically Programmable Read-Only Memory), EEPROM (Electronically Erasable Programmable Read-Only Memory), and fixed memory such as flash ROM It may be a medium that carries the program.

プ ロ グ ラ ム The program here includes not only a program directly executable by the CPU but also a program in a source program format, a compressed program, an encrypted program, and the like.

[Device Configuration of Watching System 100]
With reference to FIG. 4, watching using the watching system 100 will be described. FIG. 4 is a diagram illustrating an example of a schematic device configuration of the watching system 100 using the sensor box 119.

The watching system 100 is used for watching the residents 111 and 121 who are the monitoring target (monitoring target) and other residents. As shown in FIG. 4, a sensor box 119 is attached to the ceiling of the living room 110. Sensor boxes 119 are similarly attached to other rooms.

A range 410 represents a detection range of the sensor box 119. When the sensor box 119 has the above-mentioned Doppler sensor, the Doppler sensor detects a person's behavior that has occurred within the range 410. When the sensor box 119 has a camera as a sensor, the camera captures an image in the range 410.

The sensor box 119 is installed in, for example, a nursing care facility, a medical facility, or a home. In the example of FIG. 4, the sensor box 119 is attached to the ceiling, and the resident 111 and the bed 113 are imaged from the ceiling. The place where the sensor box 119 is mounted is not limited to the ceiling, and may be mounted on the side wall of the living room 110.

The watching system 100 detects a danger occurring to the resident 111 based on a series of images (that is, videos) obtained from the camera 105. As an example, the detectable danger includes a fall of the resident 111 and a state where the resident 111 is at a danger location (for example, a bed fence).

When the monitoring system 100 detects that the resident 111 is in danger, the monitoring system 100 notifies the caregivers 141, 143, etc. of that fact. As an example of the notification method, the watching system 100 notifies the danger of the resident 111 to the portable terminals 143 and 144 of the caregivers 141 and 142. Upon receiving the notification, the mobile terminals 143 and 144 notify the caregivers 141 and 142 of the danger of the resident 111 by a message, voice, vibration, or the like. Accordingly, the caregivers 141 and 142 can immediately recognize that the danger has occurred in the resident 111 and can rush to the resident 111 quickly.

Note that FIG. 4 illustrates an example in which the watching system 100 includes one sensor box 119, but the watching system 100 may include a plurality of sensor boxes 119. FIG. 4 shows an example in which the watching system 100 includes a plurality of mobile terminals 143 and 144. However, the watching system 100 can be realized by one mobile terminal.

[data structure]
The data structure of the cloud server 150 will be described with reference to FIG. FIG. 5 is a diagram illustrating one mode of data storage in the hard disk 5 included in the cloud server 150.

The hard disk 5 holds the table 60. The table 60 sequentially stores data transmitted from each sensor provided in each living room as movement trajectory data. More specifically, the table 60 includes a room ID 61, a date and time 62, an X coordinate value 63, and a Y coordinate value 64. The room ID 61 identifies the room of the resident. The date and time 62 identifies the date and time when the signal sent from the sensor was acquired. The X coordinate value 63 indicates the point detected at the date and time, that is, the X coordinate value of the position of the resident. The Y coordinate value 64 represents the point detected at the date and time, that is, the Y coordinate value of the position of the resident. In one aspect, the coordinate axes that are the basis of the X coordinate value and the Y coordinate value are defined, for example, with reference to the end point of the living room (for example, one corner of the room). In another aspect, the coordinate axis may be defined based on a certain point in the facility where each living room is provided.

[Operation Overview of CPU 1]
(1) In a certain situation, the CPU 1 represents a moving trajectory of a person in a living room and acquires a plurality of moving trajectory data acquired on different days. For example, based on an instruction from the management server 200, the CPU 1 of the cloud server 150 reads a plurality of pieces of movement path data stored in the hard disk 5 from the table 60 and temporarily writes each piece of movement path data in the RAM 4. The CPU 1 classifies each movement locus into a plurality of groups based on a plurality of patterns of the movement locus data. Each pattern is prepared in advance and stored in the hard disk 5. Each pattern is defined, for example, by a facility manager or other user. The pattern is, for example, a trajectory assumed when a simple reciprocation from the bed 113 to the toilet 114 is performed, a loci that reciprocates from the bed 113 to the toilet 114 but expands more than necessary (redundant), and a specific destination from the bed 113 to a specific destination. And a trajectory that returns to the bed 113 again without going to. The CPU 1 determines that the resident of the room is performing abnormal behavior based on the fact that the number of trajectories classified into any one of the plurality of groups is equal to or greater than a predetermined number. For example, when a certain number or more of the same patterns are detected within a predetermined period of time, the number of days, and the like, the CPU 1 determines that the resident is performing abnormal behavior.

(2) In a certain situation, the CPU 1 detects a change in a resident based on detecting a new moving locus that is not classified into any of the plurality of groups based on the pattern from the plurality of moving locus data. I do. For example, when the CPU 1 detects a locus that returns to the bed 113 without going to a specific destination from the bed 113 (when the resident starts wandering), there is a possibility that the resident has developed dementia. Something can be detected.

(3) In a certain situation, the CPU 1 determines the location information registered as the location of each of a plurality of articles (for example, the toilet 114, the wash basin 51, the closet 112, the desk 52, etc.) arranged in the living room, Based on the trajectory data, the behavior purpose of the resident is estimated. By associating the observation target point extracted from the signal sent from the sensor box 119 with the position information of each article in the living room, the starting point, the passing point, and the ending point of each trajectory become clear, so that the resident's destination Can be identified, and the behavioral purpose can also be estimated.

(4) In a certain situation, the CPU 1 determines whether or not the resident can act alone based on whether or not a movement trajectory to a predetermined article among the plurality of articles is detected during a predetermined period. Estimate. For example, for a resident whose trajectory from the bed 113 to the toilet 114 is reasonably short, the trajectory may then be displaced, or the trajectory may move along the wall of the room (to gain support). If the resident is detected, the CPU 1 can estimate that the resident is unlikely to act alone.

(5) In a certain situation, the CPU 1 determines the moving frequency of the resident in each area based on each movement trajectory data and each area data defining each of the plurality of areas generated by dividing the room in advance. Is displayed. For example, the CPU 1 divides each living room into 20 cm square meshes, and uses each mesh as region data. CPU 1 determines which mesh includes a point indicating a resident based on a signal sent from sensor box 119. In accordance with the determination result, the CPU 1 associates the area data corresponding to each mesh with the movement frequency based on each point, and displays the movement frequency in mesh units as in a heat map. Note that the size of the mesh is not limited to the size exemplified above.

{Circle around (6)} In a certain aspect, the CPU 1 displays the movement frequency based on the movement trajectory data detected in a certain time zone among the plurality of movement trajectory data and each area data. The certain time zone is, for example, a night time zone (eg, from 10:00 to 6:00 in the morning), a daytime zone (eg, from 6:00 to 10:00 in the morning), and the like. It is not limited to these. The CPU 1 calculates the occupant's movement frequency for each mesh as described above using the movement trajectory data detected in a certain time period, and displays the calculated frequency as a heat map. Thereby, for example, since the movement of the resident during the night time period when the activity is small is extracted, the manager or the other user is a purposeful movement for using the toilet 114 for the manager or other users. Or, it can be objectively determined whether it is merely wandering. In addition, since data for such a determination is obtained based on a signal sent from the sensor box 119 without depending on a caregiver or other staff, an increase in the burden on staff or the like can be suppressed.

[Changes in trajectory]
With reference to FIG. 6, the change of the trajectory of the resident will be described. FIG. 6 is a diagram illustrating a change in the trajectory of a resident in a certain situation.

{(Case A)} In a certain situation, two movement trajectories 610 and 611 are observed based on data transmitted from the sensor box 119. Each of the movement trajectories 610 and 611 indicates a reciprocation trajectory from the bed 113 to the toilet 114. Therefore, it is presumed that the resident went straight from bed 113 to toilet 114. For example, when the resident does not have difficulty moving, the path from the bed 113 to the toilet 114 can be observed as the movement trajectories 610 and 611 so as to take the shortest course.

(Case B) After that, if the resident comes to hinder normal movement, such as when his / her foot is injured, the resident may hinder movement even if he just goes from the bed 113 to the toilet 114. May not be able to walk straight. In such a case, the trajectory of the resident can be observed as the trajectories 620 and 621, for example.

Therefore, by observing the change in the trajectory of each resident, it is possible to objectively grasp the change in the state of the resident.

[Data processing]
The data processing in the cloud server 150 will be described with reference to FIG. FIG. 7 is a flowchart showing a part of a process executed by CPU 1 of cloud server 150.

In step S710, the CPU 1 acquires a plurality of movement trajectory data from the hard disk 5 based on the instruction of the data processing being given to the cloud server 150. For example, when a user (administrator, nursing staff, etc.) of the management server 200 accessing the cloud server 150 instructs data processing, the cloud server 150 transfers the accumulated moving trajectory data from the hard disk 5 to the RAM 4. read out.

In step S720, the CPU 1 classifies each locus into a plurality of groups based on a plurality of locus data patterns. The pattern of a plurality of movement trajectory data refers to a movement trajectory pattern composed of the movement trajectory data. The movement trajectory pattern includes, for example, a simple reciprocation from the bed 113 to a specific destination (for example, the toilet 114, the wash basin 51, the closet 112, and the desk 52), and a reciprocation from the bed 113 to the destination but a movement trajectory. Include those that are blurred, that are whirling around the bed 113 without going to a specific destination (wandering), and the like.

For example, the CPU 1 classifies the movement trajectory data starting from the bed 113 into a movement trajectory whose destination (return point) corresponds to any one of the toilet 114, the wash basin 51, the closet 112, and the desk 52. In this case, the positions of the toilet 114, the wash basin 51, the closet 112, and the desk 52 are defined as a rectangular area on a coordinate axis with a predetermined point as a reference point in the living room 110. For example, a rectangular area may be defined as an x- and y-coordinate value at its upper left and an x- and y-coordinate value at its lower right. These coordinate axes are the same as the coordinate axes that derive the x-coordinate value and y-coordinate value of the points constituting each locus shown in the table 60. Therefore, the CPU 1 determines whether or not the x-coordinate value and the y-coordinate value of each movement trajectory data included in the table 60 are included in each of the rectangular areas of the toilet 114, the wash basin 51, the closet 112, and the desk 52. Thereby, the moving trajectory can be classified into a trajectory toward the toilet 114, a trajectory toward the sink 51, a trajectory toward the closet 112, a trajectory toward the desk 52, and other trajectories.

In another aspect, the CPU 1 can classify the movement trajectory according to the distance from the bed 113 to each destination. For example, when the distance from the bed 113 to the toilet 114 is the longest of the distances from the bed 113 to the destination, a value obtained by adding a certain margin to the distance is set as the assumed distance from the bed 113 to the toilet 114. You. Therefore, if the CPU 1 detects a moving path that exceeds the assumed distance, the CPU 1 can determine that the moving path is not a moving path toward a specific destination. Similarly, when the distance from the bed 113 to the closet 112 is the shortest of the distances from the bed 113 to the destination, the CPU 1 detects a moving trajectory that is less than the distance, the moving trajectory is determined to be a specific trajectory. It can be determined that the trajectory is not the movement trajectory toward the destination.

In yet another aspect, the CPU 1 may detect a loop-like movement trajectory from the movement data. In this case, assuming that the resident is moving without heading for a specific destination, the occupant can classify the trajectory as a wandering trajectory.

In step S730, the CPU 1 determines whether or not the number of trajectories classified into any one of the groups is equal to or greater than a predetermined number. When CPU 1 determines that the number of the movement trajectories is equal to or greater than the predetermined number (YES in step S730), control is switched to step S740. Otherwise (NO in step S730), CPU 1 switches control to step S750.

に て At step S740, CPU 1 determines that the resident is performing an abnormal behavior such as a repetitive behavior. For example, it may be observed that the resident repeatedly goes to any one of the toilet 114, the wash basin 51, the closet 112, and the desk 52 in a predetermined short time. In such a case, it can be determined that the resident has any illness.

In step S750, the CPU 1 determines whether a new trajectory not classified into any group has been detected. If CPU 1 determines that the new movement trajectory has been detected (YES in step S750), CPU 1 switches the control to step S760. Otherwise (NO in step S750), CPU 1 switches control to step S770.

に て At step S760, CPU 1 detects a change in the resident. For example, when a path such as the movement trajectories 620 and 621 is observed in addition to the movement trajectories 610 and 611 from the bed 113 to the toilet 114, the CPU 1 hinders the resident from reaching the destination rationally. Can be determined to have occurred.

In step S770, CPU 1 executes data processing for displaying the occupant's movement frequency as a heat map. For example, the CPU 1 maps each observation point in the table 60 using its x-coordinate value and y-coordinate value, and counts the mapped point as the movement frequency. The CPU 1 generates data to be displayed as a heat map by adding a color according to the range of the movement frequency.

In step S710, the CPU 1 outputs the result of the processing to the monitor 8, the management server 200, or an information display device such as the mobile terminals 143 and 144. The information display device may display the trajectory of the resident in the room as a heat map based on the data indicating the result.

[Heat map]
One mode of a screen on the monitor 8 according to the present disclosure will be described with reference to FIGS. 8 and 9. FIG. 8 is a diagram showing a screen on which the movement trajectory in living room 110 is displayed on monitor 8 as a heat map. The monitor 8 displays an image of the living room 110. The image includes a heat map 810. Heat map 810 includes a plurality of cells. The x coordinate value and the y coordinate value of each point of the movement trajectory data included in the table 60 can be included in any of the cells. Each cell is color-coded according to the movement frequency. For example, the cells are color-coded according to the frequency (that is, the number of points) observed during a predetermined period of time (for example, one hour, bedtime from turning off to 6:00 in the morning, etc.).

According to the heat map 810 shown in FIG. 8, it is estimated that the resident is heading from the bed 113 to the toilet 114 along a reasonable route (that is, the shortest route). Therefore, it is determined that the health condition of the resident whose heat map 810 is observed has not changed.

FIG. 9 is a diagram illustrating a heat map 910 obtained based on a moving trajectory that can be observed when the state of the resident changes. The range with a higher frequency in the heat map 910 is wider than the range in the heat map 810. Therefore, the CPU 1 can estimate the possibility that the resident cannot take a reasonable (short) movement route when going from the bed 113 to the toilet 114, or the possibility that signs of wandering have appeared. .

[Diagnosis]
The display mode of the screen on the monitor 8 will be further described with reference to FIG. FIG. 10 is a diagram illustrating an example of a diagnosis result displayed on the monitor 8 of the management server 200. In a certain situation, the management server 200 can display a diagnosis result for each resident by using a calculation result by the CPU 1 of the cloud server 150. For example, for the resident (A), the diagnosis result that the night action is “only the toilet” is displayed. Therefore, the care staff can determine that the current resident (A) does not need any special patrol at night.

[Summary of Embodiment]
As described above, according to the present embodiment, the movement trajectory of the watching target is continuously observed, and a change in the situation of the watching target and the degree of care required are determined using the observation result. Can be. Therefore, an objective judgment can be made, and consent to the judgment result can be improved.

The embodiments disclosed this time are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

技術 This technology is applicable to information processing obtained in hospitals, nursing homes, nursing homes and other facilities.

51 wash basin, 52 desk, 60 table, 100 system, 101,221 control device, 105 camera, 106 Doppler sensor, 107 wireless communication device, 108,228 storage device, 110, 120 living room, 111, 121 resident, 112 closet , 113 beds, 114 toilets, 115 care call handsets, 116 toilet sensors, 117 sensors, 118 door sensors, 119 sensor boxes, 130 administration centers, 140 access points, 141, 142 caregivers, 143, 144 mobile terminals, 150 cloud servers , 180 living room area, 190 network, 200 management server.

Claims (18)

1. A program executed on a computer, wherein the program is stored in the computer,
   Acquiring a plurality of trajectory data representing a trajectory of a person in the living room;
   Classifying each of the trajectories into a plurality of groups based on a pattern of the plurality of trajectory data;
   Determining that the resident of the living room is performing abnormal behavior based on the number of movement trajectories classified into any one of the plurality of groups being equal to or greater than a predetermined number. Let the program.
2. The program causes the computer to detect a change in the resident based on detecting, from the plurality of movement trajectory data, a new movement trajectory that is not classified into any of the plurality of groups based on the pattern. The program according to claim 1, further causing the step of detecting to be executed.
3. The determining step includes a step of estimating the occupant's action purpose based on position information registered as an arrangement location of each of the plurality of articles arranged in the living room and each of the movement trajectory data. The program according to claim 1.
4. The estimating step determines whether or not the resident can act alone based on whether or not a movement trajectory to a predetermined article among the plurality of articles is detected during a predetermined period. 4. The program according to claim 3, including estimating.
5. The program causes the computer to move the resident in each of the areas based on each of the movement trajectory data and each of the area data defining each of the plurality of areas generated by dividing the living room in advance. 2. The program according to claim 1, further comprising the step of displaying a frequency.
6. 6. The display step includes displaying the movement frequency based on the movement trajectory data detected in a predetermined time zone among the plurality of movement trajectory data and each of the area data. The program described in.
7. Memory and
   A processor coupled to the memory,
   The processor is
   Obtain a plurality of movement trajectory data representing the movement trajectory of a person in the living room,
   Based on the pattern of the plurality of trajectory data, classify each of the trajectories into a plurality of groups,
   It is configured to determine that the resident of the living room is performing abnormal behavior based on the number of movement trajectories classified into any one of the plurality of groups being equal to or greater than a predetermined number. Information processing device.
8. The processor may detect a change in the occupant based on detecting a new movement trajectory that is not classified into any of the plurality of groups based on the pattern from the plurality of movement trajectory data. The information processing apparatus according to claim 7, further comprising:
9. The processor is configured to estimate the activity purpose of the resident based on the position information registered as the location of each of the plurality of articles arranged in the living room and each of the movement trajectory data. The information processing apparatus according to claim 7, wherein
10. The estimating determines whether or not the resident can act alone based on whether or not a movement trajectory to a predetermined article of the plurality of articles is detected during a predetermined period. The information processing apparatus according to claim 9, further comprising estimating.
11. The processor displays the moving frequency of the resident in each of the areas based on each of the movement trajectory data and each of the area data that defines each of the plurality of areas generated by dividing the living room in advance. The information processing apparatus according to claim 7, further configured to:
12. 12. The displaying includes displaying the movement frequency based on the movement trajectory data detected in a certain time zone among the plurality of movement trajectory data and each of the area data. An information processing apparatus according to claim 1.
13. A method executed on a computer, the method comprising:
    Acquiring a plurality of trajectory data representing a trajectory of a person in the living room;
    Classifying each of the trajectories into a plurality of groups based on a pattern of the plurality of trajectory data;
    Determining that the resident of the living room is performing abnormal behavior based on the fact that the number of movement trajectories classified into any one of the plurality of groups is equal to or greater than a predetermined number. ,Method.
14. Detecting a change in the resident based on detecting a new movement trajectory that is not classified into any of the groups based on the pattern from the plurality of trajectory data, The method according to claim 13.
15. The determining step includes a step of estimating the occupant's action purpose based on position information registered as an arrangement location of each of the plurality of articles arranged in the living room and each of the movement trajectory data. 14. The method of claim 13.
16. The estimating step determines whether or not the resident can act alone based on whether or not a movement trajectory to a predetermined article among the plurality of articles is detected during a predetermined period. 16. The method of claim 15, comprising estimating.
17. Displaying the moving frequency of the resident in each of the areas based on each of the moving trajectory data and each of the area data that defines each of the plurality of areas generated by dividing the living room in advance. 14. The method of claim 13, comprising.
18. 18. The method according to claim 17, wherein the displaying includes displaying the movement frequency based on the movement trajectory data detected in a predetermined time zone among the plurality of movement trajectory data and each of the area data. The method described in.

Images