JP2023108852A - Display device, display system, display method, and display program - Google Patents

Abstract

To provide a display device, a display system, a display method, and a display program which allow a caring staff to analyze a state of a cared person in detail.SOLUTION: A display device includes a time series data generation section 313, a map generation section 315, and a display section 33. The time series data generation section 313 generates time series data in which indexes of walking of a cared person 70 are arranged in time series order. The map generation section 315 generates a behavioral map of the cared person 70. The display section 33 displays the time series data and the behavioral map of the cared person 70, which corresponds to the time series data.SELECTED DRAWING: Figure 6

Description

The present invention relates to a display device, a display system, a display method, and a display program.

In Japan, life expectancy has increased remarkably due to improvements in living standards, sanitary environments, and medical standards that have accompanied the post-war period of rapid economic growth. For this reason, coupled with the declining birth rate, Japan is becoming an aging society with a high rate of aging. In such an aging society, it is expected that there will be an increase in the number of people requiring nursing care and nursing care (hereinafter referred to as "care recipients") due to illness, injury, aging, etc. be. In facilities such as hospitals and welfare facilities for the elderly (hereinafter simply referred to as "facility"), caregivers, nurses, physical therapists, occupational therapists, etc. (hereinafter referred to as "care staff") provide care to persons requiring care. Care and other measures are being taken.

In relation to this, Patent Literature 1 below discloses a rehabilitation management apparatus that analyzes exercise information from rehabilitation exercises performed by patients and manages overall histories of exercise functions of a plurality of patients.

JP-A-2005-352686

However, although the rehabilitation management device of Patent Document 1 has a function of displaying a graph of the patient's walking speed over time, there is a problem that medical staff cannot analyze the patient's condition in detail from the displayed graph over time.

The present invention has been made in view of the above-described problems. SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a display device, display system, display method, and display program that enable a care staff to analyze the walking condition of a care recipient in detail.

The above problems of the present invention are solved by the following means.

(1) a time-series data generation unit that generates time-series data in which indicators related to walking of the care recipient are arranged in time series; a map generation unit that generates a behavior map of the care recipient; the time-series data; and a display unit that displays the care recipient's behavior map corresponding to the time-series data.

(2) an acquisition unit that acquires a plurality of images obtained by photographing the care recipient and the dates and times of photography of the plurality of images; and an index related to walking of the care recipient based on the plurality of images and the photography dates and times wherein the time-series data generation unit generates time-series data in which the indices are arranged in time series for a predetermined time period including the shooting date and time, and the map generation unit visualizes the stay time of the care recipient for each area within the field of view of the plurality of images, generates a heat map of the stay time of the care recipient for each of the predetermined time periods as the action map, and The display device according to (1) above, wherein the display unit displays the time-series data and also displays a heat map corresponding to the predetermined time period of the time-series data.

(3) The display device according to (2) above, wherein the time-series data generator generates the time-series data for a daytime period and/or a nighttime period as the predetermined time period.

(4) The display device according to (2) above, wherein the display unit displays the time-series data and the heat map for each of a daytime period and a nighttime period as the predetermined time period.

(5) The display device according to any one of (1) to (4) above, wherein the plurality of images are moving images.

(6) The display device according to any one of (1) to (5) above, wherein the time series data is a graph in which the indicators are arranged in time series.

(7) further comprising a reception unit that receives selection of a day or time as a unit for generating the time-series data and selection of the index, wherein the display unit selects the time-series data related to the selected index; The display device according to any one of the above (2) to (4), which displays the heat map corresponding to the predetermined time period of the time-series data while displaying the time-series data according to the unit.

(8) The display according to any one of (1) to (7) above, wherein the indicator is any one of walking speed, sway, movement distance, number of stops, time to stop, number of repetitions, and activity time. Device.

(9) A display system comprising: an imaging unit that captures an image of the living room of the care recipient; and the display device according to any one of (1) to (8) above.

(10) step (a) of generating time-series data in which indices relating to walking of the care recipient are arranged in time series; step (b) of generating a behavior map of the care recipient; the time-series data; and a step (c) of displaying the behavior map of the care recipient corresponding to the time-series data.

(11) Before the step (a), a step (d) of obtaining a plurality of images of the person to be cared for and the shooting dates and times of the plurality of images; and a step (e) of calculating an index relating to walking of the care recipient based on said step (a), wherein in said step (a), said index is arranged in chronological order for a predetermined time period including said photographing date and time. generating series data, and in step (b), visualizing the stay time of the care recipient for each region in the field of view of the image, and heat map of the stay time of the care recipient for each of the predetermined time zones; is generated as the action map, and in the step (c), the time-series data is displayed, and a heat map corresponding to the predetermined time period of the time-series data is displayed. Method.

(12) A display program for causing a computer to execute the processing included in the display method described in (10) or (11) above.

According to the present invention, the time-series data in which the indicators related to walking of the care recipient are arranged in chronological order is displayed, and the behavior map corresponding to the predetermined time period of the time-series data is displayed. It is possible to analyze the walking state of the person in detail.

It is a figure which illustrates the whole watching system composition concerning one embodiment. FIG. 3 is a schematic diagram illustrating detection units installed around a bed in a living room of a person to be cared for; 2 is a block diagram illustrating a schematic configuration of a detection unit shown in FIG. 1; FIG. 2 is a block diagram illustrating a schematic configuration of a management server shown in FIG. 1; FIG. 2 is a block diagram showing a schematic configuration of an information manager terminal shown in FIG. 1; FIG. FIG. 6 is a functional block diagram illustrating some functions of a control unit shown in FIG. 5; It is a block diagram which shows schematic structure of the care staff terminal shown in FIG. 4 is a flow chart illustrating a processing procedure of a display method according to one embodiment; FIG. 4 is a schematic diagram illustrating a screen of software for visualizing and analyzing action data of a care recipient.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted. Also, the dimensional ratios in the drawings are exaggerated for convenience of explanation, and may differ from the actual ratios.

FIG. 1 is a diagram illustrating the overall configuration of a monitoring system according to an embodiment, and FIG. 2 is a schematic diagram illustrating detection units installed around a bed in a living room of a person to be cared for.

(Watching system 1)
As shown in FIG. 1 , the watching system 1 functions as a display system and includes multiple detection units 10 , a management server 20 , an information manager terminal 30 , and one or more care staff terminals 40 . These are connected by wire or wirelessly so as to be able to communicate with each other via a network 50 such as a LAN (Local Area Network), a telephone network, or a data communication network. Network 50 may include repeaters, such as repeaters, bridges, routers, or cross-connects that relay communication signals. In the example shown in FIG. 1, the detection unit 10, the management server 20, the information manager terminal 30, and the care staff terminal 40 are connected to each other via a wireless LAN including the access point 51 (for example, a LAN conforming to the IEEE802.11 standard). They are communicably connected by a network 50 .

The watching system 1 is arranged at an appropriate place according to the person to be cared 70 . The care recipient 70 is, for example, a patient who needs nursing care due to illness or injury, a cared person who needs care due to a decline in physical ability due to old age, a solitary person living alone, or the like. In particular, from the viewpoint of enabling early detection and early treatment, the person to be cared for 70 can be a person who needs detection when a predetermined inconvenient event such as an abnormal condition occurs to the person. Therefore, the watching system 1 is preferably installed in buildings such as hospitals, welfare facilities for the elderly, and dwelling units according to the type of the person to be cared for 70 . In the example shown in FIG. 1, the monitoring system 1 is placed in a facility that includes a plurality of rooms (residential rooms) in which a plurality of care recipients 70 reside and a plurality of rooms including a care station.

The detection unit 10 is arranged in each living room, which is the observation area of the care recipient 70 . In the example shown in FIG. 1, four detection units 10 are arranged in the living rooms of persons A, B, C, and D who are care recipients 70, respectively. As shown in FIG. 2, the observation area of the detector 10 includes a bed 60 . A care staff member 80 who provides nursing or nursing care to the care recipient 70 carries a care staff terminal 40, which is a portable terminal. However, the position, the number, and the like of each component included in the watching system 1 are not limited to the example shown in FIG. 1 . For example, management server 20 may not be located at the care station, but may be an external server unit connected to network 50 .

In addition, the care broadly includes care provided by the care staff 80 to the care subject 70, for example, living assistance such as eating, water intake, excretion, bathing, cleaning, and vital checks (body temperature, blood pressure, pulse, etc.). measurement). The care staff 80 is a person who provides various services (care) to the care recipient 70 in accordance with the work. Services may include medical services and nursing services.

(Detector 10)
FIG. 3 is a block diagram showing a schematic configuration of the detector. As shown in the figure, the detection unit 10 includes a control unit 11, a communication unit 12, a camera 13, a care call unit 14, and an audio input/output unit 15, which are interconnected by a bus.

The control unit 11 includes a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), and other memories, and performs control and arithmetic processing of each unit of the detection unit 10 according to a program. Note that the control unit 11 may further include a HDD (Hard Disk Drive) or an SSD (Solid State Drive) as memory.

The communication unit 12 is an interface circuit (such as a LAN card) for communicating with other devices such as the management server 20, the information manager terminal 30, or the care staff terminal 40 via the network 50. FIG.

The camera 13 is arranged, for example, on the ceiling or the upper part of the wall of the living room, photographs the direction of the bed 60 of the care recipient 70 directly below as an observation area, and outputs a photographed image (image data). When the care recipient 70 is in the living room, the camera 13 photographs the care recipient 70 from above. The captured images include still images and moving images. Hereinafter, it is assumed that the captured image is a moving image of 1 fps or an image of each frame of the moving image. Although the camera 13 is a near-infrared camera, a visible light camera may be used in place of this, or these may be used in combination. The captured image is transmitted to the management server 20 together with information regarding the date and time when the image was captured. Camera 13 functions as an imaging unit.

The care call unit 14 includes a push-button switch, and detects a care call (nurse call) when the care recipient 70 pushes the switch. A voice microphone may be used to detect a care call instead of the push-button switch. When the switch of the care call unit 14 is pressed, that is, when a care call is detected, the control unit 11 transmits a notification of the care call to the management server 20 or the like via the communication unit 12 and the network 50 .

The voice input/output unit 15 is, for example, a speaker and a microphone, and enables voice communication by transmitting and receiving voice signals to and from the information manager terminal 30 via the communication unit 12 . Note that the voice input/output unit 15 may be connected to the detection unit 10 via the communication unit 12 as an external device of the detection unit 10 .

(Management server 20)
FIG. 4 is a block diagram showing a schematic configuration of the management server. The management server 20 includes a control section 21 , a communication section 22 and a database 23 . This management server 20 may be provided in the same building as the living room for the care recipient 70, or may be provided in a remote location and connectable via a network. For example, the management server 20 may be a cloud server virtually constructed by a plurality of servers arranged on a network such as the Internet. Each component is communicatively connected to each other by a bus.

Since the control unit 21 and the communication unit 22 have functions similar to those of the components of the detection unit 10, detailed description thereof will be omitted.

The database 23 stores captured images captured by the camera 13 of the detection unit 10, an event list, information about the care recipient 70, information about the care staff 80, care records, and the like.

The event list includes information on various events including a predetermined event (for example, a fall event) that may lead to an accident of the care recipient 70 . The information about the care recipient 70 includes, for example, the name of the care recipient 70, date of birth, ID number, room number, usual meal amount, water intake other than meals, past diseases, level of care required, current Information such as outpatient status is included. The information about the care staff 80 includes information such as the name of the care staff 80, the staff ID, and the date of birth, for example.

A care record is a document that records information about care for the care recipient 70 at the facility. For example, based on a nursing care plan prepared for each care subject 70, the care staff 80 provides the care subject 70 with various living assistance such as eating, water intake, excretion, bathing, cleaning, etc. Perform a vital check (measurement of body temperature, blood pressure, pulse, etc.). Care such as assistance and vital checks performed by the care staff 80 for the care subject 70, and the behavior and state of the care subject 70 are recorded in the care record. The care record is updated by inputting care information for the care recipient 70 by the care staff 80 . Care records are input through the input unit 44 of the care staff terminal 40, for example.

The management server 20 determines which care recipient 70 the event detected by the detection unit 10, such as getting up, getting out of bed, falling, care call, etc., relates to, either alone or in cooperation with the detection unit 10. to (identify) This determination determines the care recipient 70 (ie, the resident of the room) associated with the room number in which the detection unit 10 that detected the event is installed. Then, the determined event type and the care recipient 70 are associated with each other and added to the event list of the database 23 . In this embodiment, when the care subject 70 carries an IC tag, the care subject 70 is determined by reading the IC tag with an RFID reader provided in each room. good too. When a plurality of care recipients 70 exist in one room, such as a shared room, the care recipient 70 may be determined by arranging the detector 10 for each bed 60 .

(Information manager terminal 30)
FIG. 5 is a block diagram showing a schematic configuration of the information manager terminal, and FIG. 6 is a functional block diagram illustrating some functions of the control unit shown in FIG. The information manager terminal 30 is a so-called PC (Personal Computer) and functions as a display device. The information manager terminal 30 includes a control section 31, a communication section 32, a display section 33, an input section 34, and an audio input/output section 35, which are interconnected by a bus. The control unit 31 has a configuration similar to that of the control unit 11 of the detection unit 10, and includes a CPU, a RAM, a ROM, and the like.

As shown in FIG. 6 , the control unit 31 functions as an acquisition unit 311 , an index calculation unit 312 , a time-series data generation unit 313 , a reception unit 314 and a heat map generation unit 315 . These functions are realized by the CPU executing the display program.

The acquisition unit 311 cooperates with the communication unit 32 to acquire, from the database 23 of the management server 20 , a plurality of captured images captured by the camera 13 and information regarding the shooting date and time of the captured images.

The index calculation unit 312 calculates an index related to walking of the care recipient 70 based on the photographed image and the photographing date and time. The index calculation unit 312 calculates, for example, the movement trajectory of the care recipient 70 in the living room from the captured image, and calculates an index (hereinafter referred to as (also simply referred to as an “index”).

The movement trajectory of the care recipient 70 can be calculated by, for example, the following method. Detecting a human silhouette (hereinafter referred to as "human silhouette") from the captured image by background subtraction method or time subtraction method, and photographing a rectangular area containing the human silhouette (hereinafter referred to as "human rectangle"). Detect each frame of the image. The person rectangle may be detected using a neural network that has been trained using teacher data of a combination of the captured image and the correct label of the person rectangle.

Subsequently, tracking processing is performed to track the movement of the human rectangle detected from each frame of the captured image. For the tracking process, for example, a known method can be adopted in which person rectangles that overlap in a plurality of frames adjacent in time series are determined to be person rectangles of the same person. By performing tracking processing, even if a plurality of person rectangles are detected from frames adjacent in time series of the captured image, the person rectangles of the same person can be associated with the plurality of person rectangles in each frame. .

For example, the movement trajectory of the person of care 70 can be calculated as the movement of the center of the associated person rectangle over a number of frames. Alternatively, the moving trajectory of the person to be cared 70 may be calculated as the movement of the center of gravity of the human silhouette in a plurality of frames. The movement locus may be calculated by a program processed by the CPU of the control unit 31, or by a built-in processing circuit.

The indicators related to walking of the care subject 70 are, for example, walking speed (m/min), sway (mm), moving distance (m), number of stops (times), time to stop (min), number of repetitions (times), and It can be activity time (min), but is not limited to these.

The walking speed and the active time can serve as indices representing the motor function of the care recipient 70, for example. In addition, staggering can be an index representing the physical condition of the care recipient 70, for example. In addition, the number of times of stopping, the time of stopping, and the number of times of repetition can serve as indices representing the degree of dementia symptoms of the care recipient 70, for example.

The walking speed as an index can be calculated as the moving distance per unit time along the moving trajectory of the care recipient 70 at each time point.

The fluctuation as an index can be calculated as, for example, the difference between position information at each point in the movement trajectory of the care recipient 70 and a value obtained by smoothing (for example, moving average) the position information.

The movement distance as an index can be calculated as, for example, the movement distance along the movement trajectory of the care recipient 70 in a predetermined time period. The predetermined time period can be, for example, a daytime period (7:00 to 19:00) or a nighttime period (19:00 to 7:00 the next day). The time setting for the daytime period and the nighttime period can be appropriately changed by the information manager 90 for each facility. Also, in the facility, it can be set for each floor of the building, for each unit, or individually.

The number of stops as an index is the number of times the care recipient 70 stops during a predetermined time period. For example, if the walking speed remains below a predetermined speed for a predetermined period of time, it can be determined that the vehicle has stopped.

The stopping time as an index is the time during which the person to be cared 70 stopped in a predetermined time period. For example, it is the time during which the walking speed is less than a predetermined speed.

The number of repetitions as an index is the number of times the care subject 70 returns to the original area immediately after moving from one area in the living room to another adjacent area.

The activity time as an index is the total time spent by the care recipient 70 after waking up (moving). That is, the active time is the total time during which the person to be cared 70 is out of bed. Recognition of getting up, leaving the bed, etc. is performed based on the posture of the care recipient 70 (for example, standing, sitting, lying down, etc.) from the detected human silhouette, and from the position relative to an object in the living room such as the bed 60. It is done by recognizing another such as.

The time-series data generation unit 313 generates time-series data in which indices are arranged in time series for a predetermined time period including the shooting date and time. As will be described later, the user (care staff 80, information manager 90, etc.) inputs a day or time as a unit for generating time-series data, and the reception unit 314 receives selection of the input day or time. . The user also inputs an index for generating time-series data, and the reception unit 314 receives selection of the input index.

The heat map generating unit 315 visualizes the stay time of the care recipient 70 for each area within the field of view of the captured image, and generates a heat map of the stay time of the care recipient 70 for each predetermined time period. A heat map (behavior map) can be used for the purpose of understanding how the care recipient 70 is behaving in the living room. A map of the movement trajectory of the person to be cared 70 can also be used for the above purpose without being limited to the heat map. The heat map generator 315 functions as a map generator. Details of the functions of the time-series data generation unit 313, reception unit 314, and heat map generation unit 315 will be described later.

The communication unit 32 is an interface for various local connections such as a network interface for wired communication based on standards such as Ethernet (registered trademark) and an interface for wireless communication based on standards such as Bluetooth (registered trademark) and IEEE 802.11. It communicates with each terminal connected to 50 .

The display unit 33 is, for example, a liquid crystal display, and displays various information. The display unit 33 displays, for example, a graph that visualizes the behavior data of the care recipient 70, a heat map of the stay time of the care recipient 70, and the like.

The input unit 34 includes a keyboard, numeric keypad, mouse, etc., and inputs various information. Also, the voice input/output unit 35 is, for example, a headset equipped with a speaker and a microphone.

In this embodiment, the information manager terminal 30 is used as a terminal for the information manager 90, displays various information in the facility to the information manager 90, and receives instructions. Further, voice communication with the care staff 80 or the like is enabled by transmitting and receiving voice signals to and from the detection unit 10 or the care staff terminal 40 via the communication unit 32 . Here, the information manager 90 is a manager who supervises a plurality of care staffs 80 . The information manager 90 may comprehensively manage the care staff 80 working not only at one facility but also at a plurality of facilities.

In addition, the care staff 80 or the technical staff, etc., through the information manager terminal 30, when the detection unit 10 is attached to each room (living room), associate the room number and the detection unit 10, the bed 60, etc. Positional information of objects installed in the living room, that is, calibration and designation of outline information in an upward view by the camera 13 on the ceiling are performed. In addition, identification information such as the name and ID number of the care recipient 70 who is hospitalized or resides in the room is associated with each room number.

(Care staff terminal 40)
FIG. 7 is a block diagram showing a schematic configuration of a care staff terminal. The care staff terminal 40 includes a control section 41, a wireless communication section 42, a display section 43, an input section 44, and an audio input/output section 45, which are interconnected by a bus. The control unit 41 has a configuration similar to that of the control unit 11 of the detection unit 10, and includes a CPU, a RAM, a ROM, and the like. The wireless communication unit 42 enables wireless communication using standards such as Wi-Fi and Bluetooth (registered trademark), and wirelessly communicates with each device via the access point 51 or directly. The display unit 43 and the input unit 44 are touch panels, and a touch sensor as the input unit 44 is superimposed on the display surface of the display unit 43 made of liquid crystal or the like. The display unit 43 and the input unit 44 display various operation screens listing a plurality of events included in the event list to the care staff 80, and accept various operations through the operation screens. In addition, when the care staff 80 cares for the care recipient 70 , the care staff 80 inputs information related to care for the care recipient 70 into the care record through the input unit 44 .

The voice input/output unit 45 is, for example, a speaker and a microphone, and enables voice communication by the care staff 80 with other care staff terminals 40 via the wireless communication unit 42 . The care staff terminal 40 is a device that functions as a user interface of the monitoring system 1, and can be configured by portable communication terminal devices such as tablet computers, smart phones, or mobile phones.

The care staff 80 performs login authentication processing through the assigned care staff terminal 40 at the start of work. The care staff 80 inputs the staff ID and password through the touch panel (display unit 43, input unit 44) of the care staff terminal 40, and transmits them to the management server 20. FIG. The management server 20 compares the authentication information stored in the database 23 and transmits the authentication result corresponding to the authority of the care staff 80 to the care staff terminal 40, thereby completing the login authentication.

Note that the detection unit 10, the management server 20, the information manager terminal 30, and the care staff terminal 40 may include components other than the above components, or may not include some of the above components. good too.

(Display method)
Next, with reference to FIGS. 8 and 9, an overview of the processing procedure of the display method of this embodiment will be described. FIG. 8 is a flowchart illustrating the processing procedure of the display method of this embodiment. The processing of the flowchart of FIG. 1 is realized by the CPU of the control unit 31 executing the display program.

FIG. 9 is a schematic diagram illustrating a screen of software for visualizing/analyzing the behavior data of the care recipient 70. As shown in FIG. The control unit 31 causes the display unit 33 to display a screen 200 of software for visualizing/analyzing the behavior data of the care recipient 70 . The control unit 31 causes the screen 200 to display the care recipient selection area 220 and the behavior display area 230 when the user selects "behavior" 211 from the menu area 210 .

The care recipient selection area 220 is an area for selecting the care recipient 70 whose behavior is to be displayed, and has a floor selection section 221 and a user search section 222 . The floor selection unit 221 allows the user to select the person to be cared 70 for each floor of the facility building in a pull-down format. A list of the names of the care recipients 70 on the selected floor is displayed in the user name display area 231 . In the example shown in the figure, Mr. A in room 101 to Mr. M in room 113 are displayed. By selecting one of the user names displayed in the user name display area 231, the user selects the care recipient 70 whose behavior is to be displayed. The selected user name is displayed in user display area 232 . In the example shown in the figure, "Mr. A, Room 101" is displayed.

The user search unit 222 searches for the care recipient 70 by the user name (the name of the care recipient 70) entered by the user, and selects the care recipient 70 corresponding to the user name as an object whose actions are to be displayed. is selected as the care recipient 70 of the The user can use either the floor selection unit 221 or the user search unit 222 to select the care recipient 70 whose behavior is to be displayed.

As shown in FIG. 8, first, a photographed image and the photographing date and time of the photographed image are obtained (step S101). The acquisition unit 311 acquires, from the database 23 of the management server 20, a plurality of photographed images (moving images) of the care recipient 70 whose behavior is to be displayed, and information regarding the photographing dates and times of these photographed images. If the person to be cared 70 is in the living room when the photographed image is taken, there is a high possibility that the person to be cared 70 is shown in the photographed image.

Next, an index relating to the walking of the care recipient 70 is calculated (step S102). The index calculation unit 312 calculates an index related to walking of the care recipient 70 based on the photographed image and the photographing date and time. For example, the index calculation unit 312 calculates the movement trajectory of the care recipient 70 in the living room from the photographed image, and calculates an index related to the walking of the care recipient 70 based on the movement trajectory and the date and time of the photographed image. .

The index setting area 233 is an area for selecting an index for generating time series data. The index is selected from, for example, walking speed, stagger, distance traveled, number of stops, time to stop, number of repetitions, and activity time, but the options are not limited to these. In the example shown in the figure, "walking speed" is selected. Accepting unit 314 accepts the selected index.

Next, time series data is generated by arranging the indices in time series (step S103). The time-series data generation unit 313 generates time-series data in which indices related to walking of the care recipient 70 are arranged in time series for a predetermined time period (for example, daytime time period and nighttime time period) including the shooting date and time of the captured image. Generate. Time-series data is automatically generated up to the day before it is displayed.

The period setting area 234 is an area for the user to set a target period for displaying time-series data. In the example shown in the figure, "September 06, 2020-October 03, 2020" is set.

The day setting button 235 is a button for selecting "day" as the unit for generating time series data, and the time setting button 236 is a button for selecting "hour" as the unit for generating time series data. When the user presses the date setting button 235, the accepting unit 314 accepts “day” as the unit for generating time-series data, and sets the horizontal axis of the graph to date. On the other hand, when the user presses the time setting button 236, the accepting unit 314 accepts "time" as the unit for generating time-series data, and sets the horizontal axis of the graph to time. If the graph's horizontal axis is set to time, the user must specify the date. The time on the horizontal axis of the graph is, for example, the daytime period (eg, 7:00 to 19:00) or the nighttime period (19:00 to 7:00 the next day).

For example, when "day" is selected as the unit for generating the time-series data, the walking speed in the predetermined time period is the median value of the moving speeds of the care recipient 70 calculated at each point in the predetermined time period. sell. When the person to be cared 70 stops frequently while walking, the average value of the moving distance in the predetermined time period becomes very small. Therefore, it is preferable to set the walking speed in the predetermined time period as the median value of the moving distance. . On the other hand, when the person to be cared 70 seldom stops while walking, the walking speed in the predetermined time period may be used as the average value of the movement distance after appropriate processing.

Further, when "time" is selected as the unit for generating the time-series data, the walking speed at each time is the median value at each time of the movement speed of the care recipient 70 calculated at each point in the predetermined time period. Possible.

Next, a heat map is generated (step S104). The heat map generating unit 315 visualizes the stay time of the care recipient 70 for each area within the field of view of the captured image, and generates a heat map of the stay time of the care recipient 70 for each predetermined time period.

Next, time-series data and a heat map are displayed (step S105). The behavior display area 230 is an area for displaying the state of behavior (behavior data) of the care recipient 70, and includes an area for displaying time-series data and a heat display area corresponding to a predetermined time period of the time-series data. and an area for displaying a map. In the example shown in FIG. 9, the screen 200 includes time-series data and heat maps for each of the daytime hours and nighttime hours. Time-series data can be, for example, a graph in which indices are arranged in time series (hereinafter also simply referred to as “graph”). In the same drawing, a graph 237 in the daytime period and a graph 238 in the nighttime period are exemplified, in which the vertical axis is the walking speed (m/min) and the horizontal axis is the date. Although not shown, a graph with the horizontal axis set to time can be similarly displayed.

The daytime zone heat map 239 corresponds to the daytime zone graph 237 and is a heat map of the daytime zone of the date specified by the user in the daytime zone graph 237 . Also, the night time period heat map 240 corresponds to the night time period graph 238 and is a heat map of the night time period of the date specified by the user in the night time period graph 238 . For example, in the daytime graph 237, the date (October 3, 2020) where the horizontal axis and the cursor C1 overlap is specified, and in the nighttime graph 238, the date where the horizontal axis and the cursor C2 overlap. (October 3, 2020) is designated. Note that even when "hour" is selected as the unit for generating time series data, it is the same as when "day" is selected as the unit for generating time series data. A heat map for each zone is displayed.

Since graphs 237, 238 and heat maps 239, 240 are displayed at the same time in this manner, the user can easily view graph 237 and heat map 239 for the daytime period and graph 238 and heat map 240 for the nighttime period. can be compared to

In the heat map 239 of the daytime zone and the heat map 240 of the nighttime zone, the color, density, pattern, etc. of each area are displayed differently according to the length of stay of the care recipient 70 . For example, the longer the stay time of the care recipient 70 is, the darker the color (for example, dark red) is displayed, and the shorter the stay time is, the lighter the color (for example, light red) is. In this specification, for the convenience of the display of the drawings, the pattern of each area is displayed differently depending on the length of stay time (the correspondence relationship between each pattern and the length of stay time is shown in FIG. 9). (shown on the right side of the page). In addition, in the example shown in the figure, the case where the stay time is in three stages of long, medium, and short is illustrated, but it may be four stages or more. In addition, it is possible to display a heat map of a time period specified by the user as well as a heat map of a predetermined time period.

As described above, in the process of the flowchart shown in FIG. 8, the photographed image of the care subject 70 and the photographing date and time of the photographed image are acquired, and based on the photographed image and the photographing date and time, the care subject 70 is displayed. Calculate the index related to walking. Subsequently, for a predetermined time period including the shooting date and time, time-series data is generated in which indicators related to walking are arranged in chronological order, and the duration of stay of the care subject 70 within the field of view of the captured image is visualized for each region, and a predetermined time period is generated. A heat map of the stay time of the care recipient 70 is generated for each time zone. Then, along with displaying the time-series data, a heat map corresponding to the predetermined time period of the time-series data is displayed on the display unit 33 .

Specifically, the control unit 31 controls to simultaneously display a graph of the index selected by the user for a predetermined time period and a heat map for a predetermined time period on the same day as the date of the graph side by side on the screen 200. . Thereby, the user can easily confirm the behavior of the person to be cared 70 during the predetermined time period on the same day as the date of the graph by using the heat map.

For example, by simultaneously displaying a graph of indicators (walking speed, swaying, etc.) related to walking of the care recipient 70 in the living room and a heat map of the corresponding predetermined time period, the care staff 80 can easily understand the indicators and the care recipient. The relationship with the behavior of the person 70 can be easily grasped. For example, the care staff 80 can easily determine in which area the person to be cared 70 is often in the living room, in which area in the living room, at what walking speed, and with what degree of sway. I can grasp it.

The display device, display system, display method, and display program of this embodiment described above have the following effects.

Along with displaying time-series data in which indicators related to walking of the care recipient 70 are arranged in chronological order, a behavior map (heat map) corresponding to a predetermined time period of the time-series data is displayed, so that the care staff 80 can easily understand the care recipient. 70 states can be analyzed in detail.

The present invention is not limited to the above-described embodiments, but can be modified in various ways within the scope of the claims.

For example, in the above-described embodiment, the information manager terminal 30 functions as a display device, but the present invention is not limited to such a case, and the care staff terminal 40 is configured to function as a display device. may be

Further, in the above-described embodiment, the case where the control unit 31 of the information manager terminal 30 calculates the movement locus of the care recipient 70 and recognizes the behavior has been described. However, the present invention is not limited to such a case, and the control unit 11 of the detection unit 10 may be configured to calculate the movement trajectory of the care recipient 70 and recognize the behavior. In this case, the control unit 11 of the detection unit 10 and the information manager terminal 30 function as a display device.

In addition, the control unit 31 or the control unit 11 machine-learns the walking speed of a person moving in the living room, the walking style of the person such as the stride, and determines whether the person in the living room is the care staff 80 or the care recipient 70 can be configured to determine whether

Also, the display program may be provided by a computer-readable recording medium such as a USB memory, a flexible disk, or a CD-ROM, or may be provided online via a network such as the Internet. In this case, the program recorded on the computer-readable recording medium is usually transferred and stored in a memory, storage, or the like. Also, this display program may be provided, for example, as independent application software, or may be incorporated in the software of each device as one function of the server.

Also, part or all of the processing executed by the program in the embodiment can be executed by replacing it with hardware such as a circuit.

1 monitoring system,
10 detection unit,
11 control unit,
12 communication unit,
13 camera,
14 Care Call Department,
15 voice input unit,
20 management server,
21 control unit,
22 communications department,
23 databases,
30 information manager terminal,
40 care staff terminals,
41 control unit,
42 wireless communication unit,
43 display unit,
44 input section,
45 audio input/output unit,
50 network,
51 access points,
60 beds,
70 subject of care,
80 care staff.

Claims (12)

a time-series data generation unit that generates time-series data in which indices related to walking of the care recipient are arranged in time-series;
a map generation unit that generates a behavior map of the care recipient;
and a display unit that displays the time-series data and a behavior map of the care recipient corresponding to the time-series data. an acquisition unit that acquires a plurality of images of the person to be cared for and the shooting dates and times of the plurality of images;
an index calculation unit that calculates an index related to walking of the care recipient based on the plurality of images and the shooting date and time;
The time-series data generation unit generates time-series data in which the indicators are arranged in time series for a predetermined time period including the shooting date and time,
The map generation unit visualizes the stay time of the care recipient for each region within the field of view of the plurality of images, and generates a heat map of the stay time of the care recipient for each predetermined time period as the behavior map. generate and
2. The display device according to claim 1, wherein said display unit displays said time-series data and also displays a heat map corresponding to said predetermined time period of said time-series data. 3. The display device according to claim 2, wherein said time-series data generator generates said time-series data for a daytime period and/or a nighttime period as said predetermined time period. 3. The display device according to claim 2, wherein said display unit displays said time-series data and said heat map for each of a daytime period and a nighttime period as said predetermined time period. 5. The display device according to claim 1, wherein said plurality of images are moving images. The display device according to any one of claims 1 to 5, wherein said time series data is a graph in which said indices are arranged in time series. further comprising a reception unit that receives selection of a day or time as a unit for generating the time-series data and selection of the index;
5. The display unit displays the time-series data of the selected indicator in a selected unit, and displays a heat map corresponding to the predetermined time period of the time-series data. 1. The display device according to claim 1. 8. The display device according to any one of claims 1 to 7, wherein the index is any one of walking speed, stagger, movement distance, number of times of stopping, time of stopping, number of repetitions, and activity time. an imaging unit that captures the interior of the living room of the care recipient;
A display device according to any one of claims 1 to 8;
a display system. a step (a) of generating time-series data in which indices relating to walking of the care recipient are arranged in time series;
step (b) of generating a behavioral map of the subject of care;
and a step (c) of displaying the time-series data and a behavior map of the care recipient corresponding to the time-series data. a step (d) of acquiring, prior to step (a), a plurality of images of the person to be cared for and the date and time of photography of the plurality of images;
a step (e) of calculating an index related to walking of the care recipient based on the plurality of images and the photographing date and time;
In step (a), generating time-series data in which the indicators are arranged in time series for a predetermined time period including the shooting date and time;
In the step (b), the stay time of the care recipient is visualized for each region within the field of view of the image, and a heat map of the stay time of the care recipient is generated as the action map for each of the predetermined time periods. death,
11. The display method according to claim 10, wherein in said step (c), said time-series data is displayed, and a heat map corresponding to said predetermined time period of said time-series data is displayed. A display program for causing a computer to execute the processing included in the display method according to claim 10 or 11.