JP2023177885A - Information processing device, information processing system, and information processing method - Google Patents

Abstract

To provide an information processing device capable of identifying a dangerous area for a target person, such as an elderly person, in a given area such as a room.SOLUTION: An information processing device is provided, comprising an acquisition unit 210 configured to acquire a behavior indicator of a target person in a given area, an identification unit 220 configured to identify a dangerous area where the target person is at risk of falling in the given area on the basis of the behavior indicator, and an output unit 230 for outputting dangerous area information showing the dangerous area.SELECTED DRAWING: Figure 8

Description

The present disclosure relates to an information processing device, an information processing system, and an information processing method.

Conventionally, it has been required to prevent the occurrence of accidents in various facilities. For example, Japanese Patent Application Laid-open No. 2011-186988 (Patent Document 1) states, ``In a hospital, it is necessary to understand situations where accidents (fall accidents, etc.) are likely to occur among individual patients with different abilities, and to avoid such situations. The company discloses a system whose purpose is to devise specific countermeasures for each patient.

Furthermore, in recent years, Japan's population has been aging, and the number of people requiring care is increasing. Therefore, there is a need to prevent accidents even in nursing care facilities, residences, etc.

JP2011-186988A

Falls are known to pose a risk of significantly reducing the quality of life (QOL) of elderly people. In elderly care facilities, preventing falls in living rooms, which are out of the reach of facility staff, is a major issue.

Locations in the living room that are considered to have a high risk of falling include the toilet, around the bed, and in front of wardrobes. However, such locations are only locations that are generally considered. Places with a high risk of falling also depend on the condition (lifestyle, behavioral patterns) of each elderly person. Therefore, in reality, facility staff have a hard time grasping which parts of individual living rooms are dangerous and what measures should be taken to prevent falls.

The present disclosure has been made in view of the above-mentioned problems, and its purpose is to provide information that can identify dangerous areas where subjects such as elderly people may fall in a predetermined area such as a living room. An object of the present invention is to provide a processing device, an information processing system, and an information processing method.

According to an aspect of the present disclosure, the information processing device includes an acquisition unit that acquires a behavior index of a subject within a predetermined area, and a dangerous area where the subject may fall within the predetermined area based on the behavior index. The device includes a specifying unit that specifies the dangerous area, and an output unit that outputs dangerous area information indicating the dangerous area.

Preferably, the behavioral indicator is associated with the time at which the behavioral indicator was obtained. The identification unit identifies a dangerous area for each predetermined time period. When a time period is specified, the output unit outputs dangerous area information in the specified time period.

Preferably, the risk area includes one or more element areas. The identification unit calculates the degree of risk in each element area based on the behavioral index. The risk area information includes risk level information indicating the level of risk of each element area.

Preferably, the acquisition unit acquires the behavioral index based on movement information indicating the movement mode of the subject in the predetermined area. The behavioral indicators include at least one of walking speed, staggering, stopping, and repeated back and forth.

Preferably, the movement information is trajectory information indicating a movement trajectory of the subject.
Preferably, the predetermined area is the subject's room. A dangerous area is an area where a subject may fall.

Preferably, the information processing device is a server device. The output unit transmits the dangerous area information to a terminal device communicably connected to the information processing device.

According to another aspect of the present disclosure, an information processing system includes a sensing device that senses a predetermined area, a server device that acquires sensing results from the sensing device, and a terminal device that can communicate with the server device. The server device obtains a behavioral index of the subject within a predetermined area based on the sensing results. The server device identifies a dangerous area where the subject may fall within the predetermined area based on the behavioral index. The server device outputs dangerous area information indicating the dangerous area to the terminal device.

According to still another aspect of the present disclosure, an information processing method includes the steps of: acquiring a behavior index of a subject within a predetermined area; and a dangerous area where the subject may fall within the predetermined area based on the behavior index. and outputting dangerous area information indicating the dangerous area.

According to the present disclosure, it is possible to specify a dangerous area where a subject such as an elderly person may fall in a predetermined area such as a living room.

1 is a diagram showing an example of the configuration of a nursing care support system according to the present embodiment. It is a diagram for explaining the hardware configuration of each device that constitutes the nursing care support system. FIG. 3 is a diagram showing an example of information obtained by imaging a living room using a camera. FIG. 3 is a diagram in which the imaging region is divided into a plurality of element regions. FIG. 2 is a diagram showing a user's dangerous area during the daytime. FIG. 3 is a diagram showing a dangerous area of a user during a predetermined time period at night. FIG. 7 is a diagram in which a trajectory of a specified time period is superimposed on the image shown in FIG. 6; FIG. 2 is a functional block diagram for explaining the functional configuration of a server device and a terminal device that constitute the nursing care support system. FIG. 2 is a diagram showing a screen displayed on a display of a terminal device in a certain situation. FIG. 2 is a sequence diagram showing a processing flow executed in the nursing care support system. FIG. 2 is a flowchart for explaining the flow of behavioral index acquisition processing executed by the server device. FIG. 3 is a flow diagram for explaining the flow of processing executed in the server device based on instructions from the terminal device.

The nursing care support system 1 as an information processing system will be described below with reference to the drawings. In the embodiments described below, the same parts and equivalent parts are given the same reference numbers, and overlapping explanations may not be repeated. In the following description, when referring to a plurality of the same configurations, they may be expressed as configurations 123A and 123B. Moreover, when they are collectively referred to, they are expressed as configuration 123.

<A. System configuration>
FIG. 1 is a diagram showing an example of the configuration of a nursing care support system 1. As shown in FIG. The nursing care support system 1 is assumed to be used mainly in residences where elderly people live, special nursing homes, nursing care facilities, etc. (hereinafter collectively referred to as "facilities"). In the following explanation, the cared person who receives care within the facility will be referred to as a "user." Furthermore, the caregivers who perform nursing care work within the facility are referred to as "staff." The user becomes the target of data acquisition.

As shown in FIG. 1, a facility 1000 where the nursing care support system 1 is used has a plurality of living rooms 900A and 900B. Room 900A is assigned to user 800A. Living room 900B is assigned to user 800B. In the example of FIG. 1, the number of rooms is two, but the number is not limited to this.

The care support system 1 includes sensor boxes 100A and 100B, a server device 200, an access point AP, and terminal devices 300A and 300B. In the nursing care support system 1, sensor boxes 100A and 100B, a server device 200, and an access point AP are connected via a network NW.

Terminal device 300A is used by staff 810A. Terminal device 300B is used by staff 810B. The terminal devices 300A and 300B are typically portable devices such as smartphones and tablets. Terminal devices 300A and 300B are connected to network NW via access point AP.

Sensor box 100A is installed in living room 900A. Sensor box 100B is installed in living room 900B. Specifically, the sensor boxes 100A and 100B are installed on the ceiling of the living room 900A and the living room 900B, respectively. Sensor boxes 100A and 100B can communicate with server device 200 via network NW.

In this example, the living rooms 900A and 900B include a bed 910, a television 920 (specifically, a television stand and a television), a chest of drawers 930, and a toilet room 940 having a sink (not shown) as facilities. The sink may be provided outside the toilet room 940. Note that the present invention is not limited to this, and furniture, equipment, etc. (not shown) may be installed in the living rooms 900A and 900B.

The sensor box 100A has a built-in sensor for detecting the behavior of objects in the living room 900A. In this example, a camera that captures an image of a subject is used as the sensor. Note that the present invention is not limited to this, and it is also possible to use a Doppler sensor that detects the motion of an object instead of the camera.

Further, the sensor box 100A may include a camera and a Doppler sensor. Note that the sensor box 100B has the same configuration as the sensor box 100A, so a description thereof will not be given here.

<B. Hardware configuration>
FIG. 2 is a diagram for explaining the hardware configuration of each device configuring the nursing care support system 1. As shown in FIG. Specifically, FIG. 2 is a diagram showing the hardware configuration of the sensor box 100, the server device 200, and the terminal device 300.

(b1. Sensor box 100)
The sensor box 100 includes a control device 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a communication interface 104, a camera 105, and a storage device 109.

Control device 101 controls the overall operation of sensor box 100. Control device 101 is configured, for example, by at least one integrated circuit. The integrated circuit includes, for example, at least one CPU (Central Processing Unit), at least one ASIC (Application Specific Integrated Circuit), at least one FPGA (Field Programmable Gate Array), or a combination thereof.

An antenna (not shown) or the like is connected to the communication interface 104. The sensor box 100 exchanges data with an external communication device via the antenna. External communication devices include, for example, the server device 200, the terminal device 300, the access point AP, and other communication terminals. Note that the sensor box 100 may be connected to the network NW wirelessly or by wire.

Camera 105, in one implementation, is a near-infrared camera. The near-infrared camera includes an IR (Infrared) projector that projects near-infrared light. By using a near-infrared camera, an image showing the inside of living room 900 can be captured even at night. In other implementations, camera 105 is a surveillance camera that receives only visible light. In still other implementations, the camera 105 may be a 3D sensor or a thermography camera.

Note that when the sensor box 100 is configured to include a Doppler sensor (not shown), a microwave Doppler sensor, for example, can be used as the Doppler sensor. The Doppler sensor detects the behavior (movement) of objects in the living room 900 by emitting and receiving radio waves.

The storage device 109 is, for example, a storage medium such as a hard disk or an external storage device. The storage device 109 stores programs executed by the control device 101 and various data used to execute the programs.

Note that, in a broad sense, the sensor box 100 is an example of a "sensing device." In a narrow sense, the camera 105 or the Doppler sensor is an example of a "sensing device."

(b2. Server device 200)
Server device 200 includes a control device 201, ROM 202, RAM 203, communication interface 204, display interface 205, operation interface 207, and storage device 209.

The control device 201 controls the overall operation of the server device 200. Control device 201 is configured, for example, by at least one integrated circuit. The integrated circuit is configured, for example, by at least one CPU, at least one ASIC, at least one FPGA, or a combination thereof.

An antenna (not shown) or the like is connected to the communication interface 204. The server device 200 exchanges data with an external communication device via the antenna. External communication equipment includes sensor box 100, for example.

The display interface 205 is connected to the display 206 and sends an image signal for displaying an image to the display 206 according to a command from the control device 201 or the like.

The operation interface 207 is, for example, a USB (Universal Serial Bus) terminal, and is connected to the input device 208. Operation interface 207 receives signals indicating user operations from input device 208 . Input device 208 is, for example, a mouse, keyboard, touch panel, or other device capable of accepting user input operations.

The storage device 209 is, for example, a storage medium such as a hard disk or an external storage device. In one implementation, storage device 209 stores a program executed by control device 201, but the program may be stored in another storage device accessible by control device 201.

(b3. Terminal device 300)
Terminal device 300 includes a control device 301 , a RAM 303 , a communication interface 304 , a display 305 , an input device 306 , and a nonvolatile memory 309 .

Control device 301 controls the overall operation of terminal device 300. Control device 301 is configured, for example, by at least one integrated circuit. The integrated circuit is configured, for example, by at least one CPU, at least one ASIC, at least one FPGA, or a combination thereof.

An antenna (not shown) or the like is connected to the communication interface 304. The terminal device 300 exchanges data with an external communication device via the antenna and the access point AP (FIG. 1). External communication equipment includes, for example, the server device 200.

The display 305 is realized by, for example, an organic EL (Electro Luminescence) display. The input device 306 is realized, for example, by a touch sensor (specifically, a touch panel) overlaid on the display 305. The touch sensor accepts various operations on the terminal device 300 through touch operations, and outputs the contents of the operations to the control device 301 .

Nonvolatile memory 309 is, for example, a storage medium such as eMMC (Embedded MultiMediaCard). As an example, the non-volatile memory 309 stores a program executed by the control device 301, but the program may be stored in a storage device other than the non-volatile memory 309 as long as it is a storage device that the control device 301 can access. You can leave it there.

Note that the nursing care support system 1 may include a camera (single unit) that can transmit moving images to the server device 200 instead of the sensor boxes 100A and 100B.

<C. Information obtained by imaging the living room 900>
FIG. 3 is a diagram showing an example of information obtained by imaging the living room 900 using the camera 105. As shown in FIG. 3, the image R is an image obtained by superimposing an image based on movement information indicating the movement mode of the user 800 on an image photographed by the camera 105. FIG. 3 shows trajectory information representing a movement trajectory 400 (also referred to as a "behavior line") of the user 800 as an example of "movement information." Specifically, image R is an image obtained by superimposing trajectory 400 on an image of living room 900 when user 800 is absent.

The trajectory information is information in which two-dimensional coordinates and time information are associated. The trajectory information is generated by the server device 200. Image data for displaying image R is generated by server device 200. Image R is displayed on terminal devices 300A, 300B, etc.

Hereinafter, the area of the living room 900 that is imaged by the camera 105, as shown in FIG. 3, will also be referred to as the "imaging area W." Also, action areas and dangerous areas will be explained. The action area and the dangerous area are areas within the living room 900 (specifically, within the imaging area W). The "action area" is an area where the user 800 exists (including passing through). The "dangerous area" is an area where the user 800 may fall. Note that the dangerous area in any time period is included in the action area in that time period.

<D. Action area>
FIG. 4 is a diagram in which the imaging region W is divided into a plurality of element regions E. As shown in FIG. 4, in this example, the element area E is a rectangular area. The server device 200 determines whether each element area E is an action area. Specifically, the server device 200 determines whether each elemental region E is an action region based on trajectory information obtained by imaging with the camera 105.

FIG. 5 is a diagram showing the activity area M of the user 800 during the daytime. Note that the daytime time period includes, for example, the time period from 7 a.m. to 7 p.m. As shown in FIG. 5, the server device 200 specifies the action area M of the user 800 from the imaging area W based on the trajectory information in the above time period.

In this example, the element area E included in the action area M is divided into four levels using four types of hatching. In the example of FIG. 5, the longer the user 800 is present, the denser the hatching becomes. Of the action areas M, the densely hatched areas m1, m2, m3, and m4 are areas where the user 800 was present for a long time (more than a certain time) during the above time period.

In this example, for convenience of illustration, the length of time the user 800 is in each element area E is expressed using the density of hatching. The length of time that the user 800 is in each element area E is expressed using the shading of the tiles. Further, the length of time the user 800 is in each element area E may be expressed using a contour map. Note that the display mode of each element area E is not particularly limited as long as the staff 810 can recognize the length of time spent in each element area E.

Next, a method for specifying the action area M will be explained. The action area M is a set of multiple element areas E in which the user 800 exists (including passes through) during a certain time period.

The server device 200 identifies an element region E that overlaps with the locus 400 in a certain time period as an action region in that time period. In the server device 200, time information at which the user 800 was present is associated with each element area E (specifically, identification information of the element area E, etc.). For example, in an element area E that the user 800 has passed through multiple times, time information for multiple times is stored in association with the element area E. Therefore, the server device 200 can specify the action area M for each time zone (for example, the time zone from 7 a.m. to 7 p.m., the time zone from 12 a.m. to 5 a.m., etc.).

<E. Dangerous area>
Similar to the action area M, the server device 200 determines whether each element area E is a dangerous area. Specifically, similar to the action area M, the server device 200 determines whether each of the plurality of elemental areas E is a dangerous area in a certain time period based on the trajectory 400 in a certain time period. In this way, the dangerous area is an area determined by the server device 200 to be a "dangerous area for the user 800."

FIG. 6 is a diagram showing the dangerous area of the user 800 during a predetermined time period at night. In addition, in the example of FIG. 6, the dangerous area of the time zone from 0:00 a.m. to 5:00 a.m. is shown as an example. As shown in FIG. 6, the server device 200 identifies a dangerous area D where the user 800 may fall from the imaging area W based on the trajectory information in the above time period. In this example, the server device 200 specifies two areas Da and Db as the dangerous area D. Note that the area Da includes 100 or more element areas E. Area Db includes two element areas E.

In FIG. 6 as well, similarly to FIG. 5, the element region E included in the action region M is divided into four sections using four types of hatching. As described above, the area with the densest hatching corresponds to the dangerous area D.

In this example, the degree of danger of the elemental region E included in the dangerous region D is divided into four danger levels using four types of hatching. In the example of FIG. 6, the higher the risk for the user 800 (area where there is a higher risk of being a storefront), the denser the hatching becomes. Of the dangerous area D, the areas d1, d2, and d3 with the densest hatching are the areas with the highest level of danger for the user 800.

In this example, for convenience of illustration, the level of danger of each element area E (i.e., location) for the user 800 is expressed using the density of hatching, but typically, it is expressed by a predetermined color. The danger level of each element area E is expressed using the shading of solid painting (tile painting). Further, the risk level of each element region E may be expressed using a contour map. Note that the display mode of each element area E is not particularly limited as long as the staff 810 can recognize the danger level of each element area E.

In addition, in FIG. 6, a portion of the bed 910 is displayed in a manner indicating a dangerous area. However, since it is night time, it is assumed that the user 800 is sleeping on the bed 910. Therefore, the staff 810 can determine that the area d1 is not a true danger area. Therefore, at night, it can be seen that the region d2 on the side and the region d3 on the front side of the toilet room 940 are the dangerous regions with the highest danger level.

Next, a method for identifying the dangerous area D will be explained. As described above, the dangerous area D is an area where the user 800 may fall.

The server device 200 identifies a dangerous area D within the living room 900 (specifically, within the imaging area W) based on the behavioral index within the living room 900 (specifically, within the imaging area W). The server device 200 further specifies the degree of risk in each element area E based on the behavioral index.

Specifically, the server device 200 acquires (calculates) a behavior index based on movement information (trajectory information in this example) indicating the movement mode of the user 800 in the living room 900. The behavioral indicators include at least one of walking speed, unsteadiness, stopping, repetition of going back and forth, and destination.

Of walking speed, unsteadiness, stopping, repeated going back and forth, and destination, unsteadiness and stopping are considered to be factors that cause falls. Therefore, in this example, the server device 200 calculates the degree of risk for each element region E, focusing on wandering and stopping. This is because falls tend to occur when changing the direction of travel (once stopping) or after stopping for errands, and stopping can also be a factor in falling. Note that the movement destination indicates the action area M, and is a place (for example, the toilet room 940) specified by the element area E over which the trajectory 400 is superimposed.

When wobbling occurs, the server device 200 assigns (links) points to the element area E corresponding to the location where the wobbling occurs, along with time information when the wobbling occurs. In this example, the server device 200 gives higher points as the wobbling becomes larger. In this way, the server device 200 quantifies the degree of wobbling. If one wobbling spans two or more elemental areas E, the server device 200 may assign points to the two or more elemental areas E, or may assign points to points corresponding to the starting position of the wobbling. Points may be given to only one element area E. Note that, since a person rarely walks in a straight line, swaying is always detected.

The magnitude of the wobbling is calculated based on the distance (more specifically, the area) between a trajectory connecting the coordinates of the detected element region E by a line segment and a straight line or curve obtained by smoothing the trajectory. Can be detected. Since the technique for detecting the magnitude of wobbling is well known, a detailed explanation will not be given here.

Similarly, when the user 800 stops, the server device 200 assigns points to the element area E corresponding to the place where the user 800 stopped, together with the time information when the user 800 stopped.

In the server device 200, points are added to each element area E as time passes. This point becomes the risk level. Points linked to element area E have time information. Therefore, the server device 200 can calculate the total number of points for each time period (for example, the time period from 7 a.m. to 7 p.m., the time period from 12 a.m. to 5 a.m., etc.).

The server device 200 classifies the risk of each element area E into a plurality of risk levels (four risk levels in the example of FIG. 6) using the calculated points (that is, the risk) and the plurality of threshold values.

Note that, in the above, the server device 200 calculates the degree of risk for each element region E by paying attention to wandering and stopping, but the degree of risk may be calculated based on only either one of them. That is, the server device 200 may identify the dangerous area D based on the wandering or stopping.

The present invention is not limited to the above, and the server device 200 may calculate the degree of risk by taking other indicators such as walking speed into account. For example, regarding walking speed, points may be given to element regions E where the speed has changed significantly.

(Superimposition of trajectories)
FIG. 7 is a diagram in which the trajectory of a designated time period is superimposed on the image shown in FIG. 6. More specifically, in FIG. It is a diagram in which trajectories for several minutes between 1:00 and 2:00 a.m. are superimposed.

Although details will be described later, the time period of the trajectory is specified by the staff member 810. The time zone of the trajectory can be specified separately from the time zone of the dangerous area D. By referring to the image shown in FIG. 7, the staff 810 of the facility 1000 can know the dangerous area D as well as the movement mode of the user 800 during the time period desired by the staff 810. A more detailed configuration for performing the above-described processing will be described below.

<F. Functional configuration>
FIG. 8 is a functional block diagram for explaining the functional configuration of the server device 200 and the terminal device 300 that constitute the nursing care support system 1. As shown in FIG. 8, the server device 200 includes an acquisition section 210, an identification section 220, and an output section 230. The acquisition unit 210 includes a trajectory generation unit 211 and a trajectory analysis unit 212. The identification unit 220 includes a behavioral index storage unit 221 and a risk calculation unit 222.

The terminal device 300 includes the control device 301, the display 305, and the input device 306, as described above. The control device 301 includes an input reception section 311, an acquisition section 312, and a display control section 313. Below, first, the server device 200 will be explained, and then the terminal device 300 will be explained.

(f1. server device 200)
(1) Processing of the acquisition unit 210 The acquisition unit 210 acquires image data from the sensor box 100. Note that the image data is obtained by imaging with the camera 105. The acquisition unit 210 acquires behavioral indicators of the user 800 in the living room 900. Specifically, the acquisition unit 210 acquires the behavior index based on movement information indicating the movement mode of the user 800 in the living room 900 (in this example, trajectory information representing the movement trajectory of the user 800). The acquisition unit 210 sends the acquired behavior index to the identification unit 220.

Specifically, the trajectory generation section 211 in the acquisition section 210 generates trajectory information based on the image data acquired from the sensor box 100. The trajectory information includes a plurality of coordinate values and time information associated with each coordinate value. Since the method of generating trajectory information is publicly known, a specific method of generating it will not be described here.

The trajectory analysis unit 212 identifies behavioral indicators by analyzing trajectory information. As described above, the behavioral indicators include walking speed, unsteadiness, stopping, repetition of going back and forth, and destination. Each behavioral index is associated with the element region E from which the behavioral index was acquired. Each behavioral indicator is also associated with the time at which the behavioral indicator was acquired.

The acquisition unit 210 associates the behavior index with the identification information of the element region E and the time information and sends it to the identification unit 220. Furthermore, the acquisition unit 210 sends trajectory information to the identification unit 220 in order to be able to identify the action area M and the dangerous area D for each time period.

(2) Processing of the identification unit 220 The identification unit 220 identifies the dangerous area D in the living room 900 based on the behavioral index. The identifying unit 220 sends dangerous area information indicating the identified dangerous area D to the output unit 230. Note that the dangerous area D is an area where the user 800 may fall, as described above.

Specifically, the identifying unit 220 identifies the dangerous area D in the living room 900 based on an instruction from the terminal device 300 (an instruction to display the dangerous area D, which will be described later). More specifically, the identification unit 220 identifies the dangerous area D for each predetermined time period. The time zone is specified by the terminal device 300.

More specifically, the behavior index storage section 221 in the identification section 220 sequentially accumulates the behavior indexes sent from the acquisition section 210. The risk calculation unit 222 calculates the risk of each element area E in the specified time period based on the accumulated behavioral indicators. The specifying unit 220 determines whether each element area E in the specified time period is a dangerous area based on the degree of risk (total number of points) in the specified time period. Thereby, the specifying unit 220 specifies the dangerous area D in the specified time period.

The specifying unit 220 sends the dangerous area information indicating the identified dangerous area D to the output unit 230 in a state in which risk level information indicating the degree of risk of each element area E is included. The dangerous area information is information specifying a dangerous area (area where the degree of risk is equal to or higher than a reference value) within the living room 900 (specifically, within the imaging area W). The dangerous area information includes at least information that specifies an elemental area E whose degree of danger is equal to or higher than a reference value among the plurality of elemental areas E (for example, identification information or position information (coordinate information) of the elemental area E), and and the risk information of the element area E. The identification unit 220 sends dangerous area information indicating the dangerous area D to the output unit 230 based on an instruction from the terminal device 300 (an instruction to display the dangerous area D, which will be described later).

Further, the specifying unit 220 specifies the action area M in the specified time period based on the trajectory information obtained from the obtaining unit 210. Specifically, the specifying unit 220 specifies the activity area M in the living room 900 based on an instruction from the terminal device 300 (an instruction to display the activity area M, which will be described later). Based on the identification of the action area M, the identification unit 220 sends action area information indicating the action area M to the output unit 230. More specifically, based on the instruction from the terminal device 300, the specifying unit 220 sends the trajectory information of the specified time period to the output unit 230 along with the activity area information.

Note that, as described based on FIG. 7, the time period specified by the staff 810 for the dangerous area D or the action area M does not have to match the time period specified by the staff 810 for the trajectory information. For example, if the specified time period for danger area D or action area M is from 12:00 a.m. to 5:00 a.m., the specified time period for trajectory information is from 12:00 a.m. to 5:00 a.m. It may be a part of the time slot (for example, a part of the time slot from 1 a.m. to 2 a.m.).

(3) Processing of the output unit 230 The output unit 230 outputs dangerous area information indicating the dangerous area D in a certain situation. In this example, the output unit 230 transmits the dangerous area information to the terminal device 300 that is the source of the request signal. Specifically, based on the server device 200 receiving an instruction to display the dangerous area D from the terminal device 300, the output unit 230 outputs (transmits) dangerous area information to the terminal device 300. Specifically, when a time slot is designated by the terminal device 300, the output unit 230 outputs dangerous area information for the designated time slot.

The output unit 230 outputs action area information indicating the action area M in another situation. In this example, the output unit 230 transmits the action area information to the terminal device 300 that is the source of the request signal. Specifically, based on the server device 200 receiving an instruction to display the action area M from the terminal device 300, the output unit 230 outputs (transmits) action area information to the terminal device 300. Specifically, when a time slot is designated by the terminal device 300, the output unit 230 outputs activity area information in the designated time slot.

Based on instructions from the terminal device 300, the output unit 230 transmits the trajectory information of the specified time period to the terminal device 300 together with the dangerous area information or the action area information. Based on the information output from the output unit 230, a display based on the information is performed on the terminal device 300.

The server device 200 may transmit the trajectory information to the terminal device 300 in a state where it is included in the dangerous area information or the action area information. The dangerous area information and the action area information include at least image data obtained by imaging the interior of the living room 900. In this example, image data is typically used as the data format for displaying the trajectory information, the dangerous area D, and the action area M on the terminal device 300. However, it is not limited to this.

Specifically, in this example, HTTP (Hypertext Transfer Protocol) can be used as a communication protocol between the server device 200 and the terminal device 300. In this case, the terminal device 300 uses a Web browser to access a server device as a Web server. Note that the communication protocol is not limited to this.

(f2. terminal device 300)
(1) In a certain situation, the acquisition unit 312 receives, from the server device 200, dangerous area information indicating the dangerous area D within the living room 900 (more specifically, within the imaging area W) and the movement mode of the user 800 within the living room 900. (in this example, trajectory information representing the trajectory of movement of the user 800).

In this case, the display control unit 313 causes the display 305 to display an image showing the dangerous area D and an image showing the movement mode (hereinafter also referred to as "trajectory image") based on the dangerous area information and movement information. . Specifically, the display control unit 313 causes the display 305 to display the image showing the dangerous area D and the trajectory image in such a manner that the trajectory image is superimposed on the image showing the dangerous area D. Note that the image showing the dangerous area D corresponds to the images of the dangerous areas Da and Db shown in FIG. 7, for example. The trajectory image corresponds to the trajectory 400 shown in FIG. 7, for example.

(2) In yet another aspect, the input receiving unit 311 receives a designation of the time period T1 regarding the movement mode based on a staff operation on the input device 306. The input receiving unit 311 receives the designation of the time period T2 for the dangerous area D based on the staff's operation on the input device 306.

In this case, the acquisition unit 312 acquires information (trajectory information in this example) indicating the movement mode in the designated time period T1 from the server device 200 as the movement information. The acquisition unit 312 also acquires dangerous area information in the specified time period T2. As shown in FIG. 7, the display control unit 313 causes the display 305 to display a trajectory image based on the trajectory information in the time period T1 superimposed on an image showing the dangerous area D in the time period T2. Note that the time period T1 is preferably included in the time period T2. The acquisition unit 312 typically acquires the trajectory information and the dangerous area information at the same timing (as a response signal to one request signal).

(3) In yet another aspect, the input receiving unit 311 receives a designation of the time period T1 regarding the movement mode based on a staff operation on the input device 306. The input receiving unit 311 receives the designation of the time period T3 for the action area M based on the staff's operation on the input device 306.

In this case, the acquisition unit 312 acquires information (trajectory information in this example) indicating the movement mode in the designated time period T1 from the server device 200 as the movement information. The acquisition unit 312 also acquires activity area information in the specified time period T3. The display control unit 313 causes the display 305 to display a trajectory image based on the trajectory information in the time period T1 superimposed on an image showing the action area M in the time period T3.

(4) As shown in FIGS. 6 and 7, the display control unit 313 displays the dangerous area D on the display 305 in a manner according to the risk level based on the risk level information. For example, as described above, the display control unit 313 displays areas with a high degree of risk in the dangerous area D in a darker color than areas with a lower degree of risk.

<G. User interface>
Next, an example of a user interface displayed on the terminal device 300 will be described. FIG. 9 is a diagram showing a screen 305G displayed on the display 305 of the terminal device 300 in a certain situation. Note that each selection described below is made by the staff 810.

As shown in FIG. 9, the screen 305G shows a state in which an image 351 indicating a fall hazard map item is selected from a plurality of selection items displayed in a vertical line on the left side. In this state, on the screen 305G, the first floor is selected in the pull-down menu 352 for selecting a floor. Therefore, the names of the three users on the first floor and the room numbers of each user are displayed in a list on the screen 305G. Furthermore, on the screen 305G, an object image 353 indicating the user of room 101 is selected. Therefore, personal information 360 regarding the user of room 101 is displayed on the screen 305G. Note that users can also be narrowed down by entering their last name, first name, or given name in the search field 354.

The personal information 360 includes a column 361 in which the user image is described, a column 362 in which fall countermeasures are described, a column 363 in which the causes of falls are described by category, an input column 364 for displaying areas and trajectories, and a server device. image 365 obtained from 200.

Note that the contents displayed in columns 361, 362, and 363 are contents registered in advance by staff 810. In this example, the image 365 is displayed at a predetermined position below the input field 364 (hereinafter also referred to as "image display position").

Next, the input field 364 regarding display of areas and trajectories will be explained. The input field 356 is configured so that the staff member 810 can select either the dangerous area D or the action area M as a display target (specifically, an image) to be displayed at the above-mentioned image display position. Furthermore, the input field 356 is configured to allow selection of the date and selection of either daytime or nighttime.

In this example, daytime is, for example, a time period from 7 a.m. to 7 p.m. Nighttime is the time period from 7:00 pm to 7:00 am. Note that the daytime and nighttime hours are not limited to these. Alternatively, the day may be divided into three or more sections (eg, early morning, morning, daytime, evening, night, late night, etc.), and each section may be selected. Furthermore, a configuration may be adopted in which a short time zone can be input by setting a specific time instead of daytime and nighttime.

When the region selection, date, and daytime or nighttime are input (selected), the terminal device 300 transmits a request signal (see FIG. 8) to the server device 200. Thereby, the terminal device 300 receives a response signal (FIG. 8) corresponding to the request signal from the server device 200. Note that in FIG. 9, "dangerous area" is selected as the area (specifically, the area type).

In the terminal device 300, the trajectory can also be displayed on the image 365. In the input field 364, the staff member 810 selects (designates) a time period using a pull-down menu for the item "time period selection (trajectory)". In this example, the time around 14:00 (14:00:00 to 14:59:59) is selected. Next, the staff member 810 selects (designates) one time from the plurality of times at which trajectories are stored in the relevant time period from the pull-down menu for the item "trajectory selection." In this example, 14:23 is selected. The selection can also be made using two triangular buttons (a button for moving forward and a button for going back one step) placed on both sides of the time display.

In this example, trajectories are stored in the server device 200 in association with multiple times, such as "14:07", "14:23", "14:30", "14:39", etc. , information about the time can be displayed on the terminal device 300. Note that these times do not indicate a one-minute trajectory, but indicate the time when the user 800 started moving. For example, "14:23" is the time when movement is determined to have started (for example, 14:23:25) to the time when movement is determined to have stopped (for example, 14:27:10). It represents the trajectory up to.

Note that the method of specifying the time period of the trajectory is not limited to this. The terminal device 300 may have a configuration that accepts input of a time zone, for example, may have a configuration that accepts input of two times (for example, 14:20 and 14:30).

As described above, the terminal device 300 receives a response signal (image data) from the server device 200 in which the trajectory information for the specified time period (the trajectory information of the user in room 101) is superimposed on the image showing the dangerous area D. signal) can be obtained. In this case, the terminal device 300 displays the acquired image data at the image display position. That is, the terminal device 300 updates the image at the image display position. As a result, the terminal device 300 can display, for example, the image 365 shown in FIG. 9 at the image display position.

Note that the user interface of the terminal device 300 shown in FIG. 9 is an example, and the user interface is not limited thereto.

<H. Processing flow>
FIG. 10 is a sequence diagram showing the processing flow executed in the nursing care support system 1. As shown in FIG. 10, image data continues to be transmitted from the sensor box 100 to the server device 200. In sequence SQ1, the terminal device 300 receives an input specifying a display target (user, date, time zone, area type, necessity of a trajectory, etc.), as shown in screen 305G in FIG. Note that in the following description, it is assumed that a dangerous area is selected as the area type on the screen 305G.

In sequence SQ2, terminal device 300 transmits a request signal to server device 200 based on an input specifying a display target. In sequence SQ3, the server device 200 identifies the dangerous area D in the specified time period based on the request signal. In sequence SQ4, server device 200 transmits a response signal (a signal including image data) to terminal device 300.

In sequence SQ5, terminal device 300 displays the received image data. Specifically, the terminal device 300 displays information according to the area type. In this example, since the dangerous area is selected as the area type as described above, the terminal device 300 displays information indicating the dangerous area D. Note that in the screen 305G, when the action area is selected as the area type, the terminal device 300 displays information indicating the action area M. Further, when displaying the trajectory is instructed by the staff 810, the terminal device 300 displays the trajectory image in a manner superimposed on the dangerous area D or the action area M.

FIG. 11 is a flow diagram for explaining the flow of the behavioral index acquisition process executed by the server device 200. As shown in FIG. 11, in step S11, the server device 200 starts acquiring image data obtained by imaging with the camera 105. In step S12, the server device 200 generates trajectory information indicating the trajectory of the movement of the user 800 based on the image data. In step S13, the server device 200 analyzes the behavioral index of the user 800 by analyzing the trajectory information.

FIG. 12 is a flow diagram for explaining the flow of processing executed in the server device 200 based on instructions from the terminal device 300. In the following, it is assumed that a dangerous area is selected as the area type on the screen 305G. That is, a case will be described in which the instruction from the terminal device 300 is to display the dangerous area D.

As shown in FIG. 12, in step S21, the server device 200 receives a request signal from the terminal device 300. In step S22, the server device 200 calculates the risk level of each element area E from the behavior index in the specified time period based on the request signal. In step S23, the server device 200 identifies the dangerous area D in the specified time period.

In step S24, the server device 200 generates a response signal. Specifically, the server device 200 generates dangerous area information including image data indicating the dangerous area D. When the instruction from the terminal device 300 includes displaying a trajectory, the server device 200 generates trajectory information and dangerous area information. In step S25, the server device 200 transmits the trajectory information and the dangerous area information to the terminal device 300 as a response signal to the request signal.

Note that when the instruction from the terminal device 300 is to display the action area M, the server device 200 generates action area information including image data indicating the action area M. When the instruction from the terminal device 300 includes displaying a trajectory, the server device 200 generates trajectory information and action area information. Further, the server device 200 transmits the trajectory information and the action area information to the terminal device 300 as a response signal to the request signal.

<I. Summary>
(i1. Server device 200)
The processing of the server device 200 regarding the dangerous area D can be summarized as follows.

(1) The server device 200 includes an acquisition unit 210 that acquires behavioral indicators of the user 800 (target person) of the facility 1000 in the living room 900 (within a predetermined area), and a The device 800 includes a specifying unit 220 that specifies a dangerous area D where there is a risk of falling, and an output unit 230 that outputs dangerous area information indicating the dangerous area D. According to such a configuration, the server device 200 can specify a dangerous area D where the user 800 may fall. Furthermore, by displaying the dangerous area information on the terminal device 300, the staff 810 can grasp the dangerous area D in the living room 900 where the user 800 may fall.

(2) The behavioral index is associated with the time at which the behavioral index was acquired. The identification unit 220 identifies the dangerous area D for each predetermined time period. When a time slot is designated by the staff, the output unit 230 outputs dangerous area information for the designated time slot. According to such a configuration, the staff 810 can use the terminal device 300 to grasp the dangerous area D where the user 800 may fall for each time period.

(3) The dangerous area D includes one or more element areas E. The identification unit 220 calculates the degree of risk in each element area E based on the behavioral index. The risk area information includes risk level information indicating the level of risk of each of the element areas. According to such a configuration, the staff 810 can grasp the degree of risk (the magnitude of the degree of risk) in the dangerous area D using the terminal device 300.

(4) The acquisition unit 210 acquires a behavior index based on movement information indicating the movement mode of the user 800 in the room 900. The behavioral indicators include at least one of walking speed, staggering, stopping, and repeated back and forth. According to such a configuration, the server device 200 can accurately specify the dangerous area D where the user 800 may fall. Therefore, the staff 810 can fully grasp the dangerous area D where the user 800 may fall.

(i2. terminal device 300)
The processing of the terminal device 300 regarding the dangerous area D can be summarized as follows.

(1) The terminal device 300 acquires dangerous area information indicating a dangerous area D where the user 800 may fall within the living room 900 and movement information indicating the movement mode of the user 800 within the living room 900. unit 312, and a display control unit 313 that causes the display 305 to display an image showing the dangerous area D and an image showing the movement mode based on the dangerous area information and movement information. According to such a configuration, the staff 810 can grasp the dangerous area D for the user 800 in the living room 900 and the user's movement mode using the terminal device 300.

(2) The display control unit 313 causes the display 305 to display the image showing the movement mode and the image showing the dangerous area D in such a manner that the image showing the movement mode is superimposed on the image showing the dangerous area D. According to such a configuration, the staff member 810 can check the movement mode of the user 800 while superimposing the dangerous area D where the user 800 may fall.

(3) The terminal device 300 further includes an input reception unit 311 that accepts specification of a time zone. The acquisition unit 312 acquires, as movement information, information indicating a movement mode in a specified time period. According to such a configuration, the staff 810 can grasp the movement mode of the user during the time period specified by the staff 810.

(4) The acquisition unit 312 acquires information indicating the dangerous area D in the specified time period as the dangerous area information. According to such a configuration, the staff 810 can grasp the dangerous area D where the user 800 may fall during the time period specified by the staff 810.

(5) The dangerous area D includes one or more element areas. The risk area information includes risk level information indicating the level of risk of each element area E. The display control unit 313 displays the dangerous area D on the display 305 in a manner according to the risk level based on the risk level information. According to such a configuration, the staff 810 can grasp the degree of risk (the magnitude of the degree of risk) in the dangerous area D.

<J. Modified example>
(1) The server device may start saving the moving image data obtained by imaging with the camera 105 based on the user 800 performing a specific first action. Furthermore, in this case, it is preferable that the identifying unit 220 is configured to end the video data storage process based on the user performing a specific second action. According to such a configuration, the amount of data accumulated in the server device 200 can be reduced.

The first action is typically the action of the user 800 getting up or the action of leaving the bed after the user 800 wakes up (an action representing leaving the bedding such as the bed 910). The second motion is the motion of the user 800 landing on bed. According to such a configuration, the server device 200 no longer specifies the area of the bed 910 as the dangerous area D.

Note that the server device 200 may determine the wake-up motion, bed leaving motion, and bed landing motion from the trajectory information, or the sensor box 100 may determine the motion using the following method. The control device 101 of the sensor box 100 extracts a moving object region as a region of the person of the user 800 from the target image by, for example, a background subtraction method or a frame difference method. The control device 101 may further detect "waking up," "leaving bed," and "landing on bed" from the extracted moving body region by pattern matching using a human body model prepared in advance. It is not particularly limited as long as it can detect when the user 800's upper body rises (wake-up) and when a part of the user's 800 body leaves the bed area (leave the bed).

(2) The server device 200 can selectively save (backup save) the moving image data obtained by imaging with the camera 105. For example, when the server device 200 detects an event such as a fall of the user 800, it is possible to save the moving image data before and after the event without overwriting the same image data obtained later.

(3) The imaging target is not limited to the living room. The target of data acquisition is not limited to the elderly. The above system configuration is not limited to facilities for the elderly, but can also be applied to inpatient facilities, facilities requiring monitoring, and the like.

(4) Movement information is not limited to trajectory information. The movement information does not need to be connected by line segments, and may be composed of a plurality of points (discrete points).

(5) Instead of the server device 200, an information processing device (not shown) such as a PC (Personal Computer) installed in the facility 1000 may be caused to execute the various processes described above that are executed by the server device 200. .

<Additional notes>
(1) Obtaining behavioral indicators of the subject within a predetermined area;
identifying a dangerous area where the subject may fall within the predetermined area based on the behavioral index;
outputting dangerous area information indicating the dangerous area; and a program for causing a processor of the information processing device (server device 200) to execute the step.

(2) A non-temporary recording medium that stores the above program.
The embodiments disclosed this time should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is indicated by the claims rather than the above description, and it is intended that all changes within the meaning and range equivalent to the claims are included.

1 nursing care support system, 100A, 100B sensor box, 105 camera, 200 server device, 305 display, 210, 312 acquisition unit, 211 trajectory generation unit, 212 trajectory analysis unit, 220 identification unit, 221 behavior index accumulation unit, 222 risk level Calculation unit, 230 Output unit, 300A, 300B Terminal device, 305G Screen, 311 Input reception unit, 313 Display control unit, 351, 365, R Image, 352 Pull-down menu, 353 Object image, 356, 364 Input field, 400, 430 Trajectory, 800A, 800B User, 810A, 810B Staff, 900A, 900B Room, 910 Bed, 920 TV, 930 Chest of drawers, 940 Toilet room, 1000 Facility, AP Access point, D, D1, D2 Danger area, d1, d2, d3, m1, m2, m3, m4 area, E element area, M action area, NW network, W imaging area.

Claims (9)

an acquisition unit that acquires a behavior index of a subject within a predetermined area;
an identification unit that identifies a dangerous area where the subject may fall within the predetermined area based on the behavioral index;
An information processing device comprising: an output unit that outputs dangerous area information indicating the dangerous area. The behavioral index is associated with a time at which the behavioral index is obtained,
The identifying unit identifies the dangerous area for each predetermined time period,
The information processing apparatus according to claim 1, wherein when a time slot is designated, the output unit outputs the dangerous area information in the designated time slot. The dangerous area includes one or more element areas,
The identification unit calculates the degree of risk in each element area based on the behavioral index,
3. The information processing apparatus according to claim 1, wherein the risk area information includes risk information indicating a risk level of each element area. The acquisition unit acquires the behavioral index based on movement information indicating a movement mode of the subject in the predetermined area,
The information processing device according to claim 1 or 2, wherein the behavioral index includes at least one of walking speed, staggering, stopping, and repetition of going back and forth. The information processing apparatus according to claim 4, wherein the movement information is trajectory information indicating a movement trajectory of the subject. The predetermined area is a room of the subject,
The information processing apparatus according to claim 1 or 2, wherein the dangerous area is an area where the subject may fall. The information processing device is a server device,
The information processing device according to claim 1, wherein the output unit transmits the dangerous area information to a terminal device communicably connected to the information processing device. a sensing device that senses a predetermined area;
a server device that acquires sensing results from the sensing device;
comprising a terminal device capable of communicating with the server device,
The server device includes:
Obtaining behavioral indicators of the subject within the predetermined area based on the sensing results;
identifying a dangerous area where the subject may fall within the predetermined area based on the behavioral index;
An information processing system that outputs dangerous area information indicating the dangerous area to the terminal device. obtaining behavioral indicators of the subject within a predetermined area;
identifying a dangerous area where the subject may fall within the predetermined area based on the behavioral index;
An information processing method comprising: outputting dangerous area information indicating the dangerous area.

Images