JP2023070277A - Risk prediction system for care facility, program, and risk prediction method for care facility - Google Patents

Abstract

To provide a risk prediction system for a care facility, program, and risk prediction method for a care facility capable of remotely watching a cared person by notifying of, by prediction, a state where the cared person is likely to fall.SOLUTION: A risk prediction system 1 includes: reception means 201 for receiving a photographed video from a user terminal 3; posture estimation means 203 for estimating a posture of a cared person from the photographed video received by the reception means 201; action detection means 205 for detecting an action state of the cared person from object information estimated by the posture estimation means 203; action prediction means 206 for predicting a next action of the cared person from the action state detected by the action detection means 205; and display instruction means 601 for allowing a care support terminal 7 to display a state image based on the action state detected by the action detection means 205 and prediction information predicted by the action prediction means 206.SELECTED DRAWING: Figure 1

Description

The present invention relates to a risk prediction system for nursing care facilities, a program, and a risk prediction method for nursing care facilities.

Nursing care facilities and the like have conventionally provided various types of watching systems for watching over care recipients. For example, Patent Literature 1 discloses a technique for detecting biometric information by providing a range image camera on the ceiling of a room in a medical facility, a nursing facility, or a home, and capturing an image of the entire bed. In addition, Patent Document 2 discloses a technique for monitoring the state of a pedestrian reflected in the captured moving image by capturing a pedestrian with a camera and analyzing the captured moving image obtained thereby, and watching over the pedestrian. disclosed. In addition, in Patent Document 3, an infrared laser sensor that covers the entire area around the bed within its angle of view automatically scans the monitored object, including the bed of the care recipient, with an infrared laser. There is disclosed a technique for identifying a location on a bed on which a care-receiver is sleeping by acquiring an image. In addition, in Patent Document 4, an image of a care recipient is acquired by a digital camera such as a web camera (Web camera), and by detecting the physical condition of the care recipient based on the physiological data of the care recipient, is in an abnormal state. Further, in Patent Document 5, a temperature region indicating a human body is extracted from a first temperature distribution and a second temperature distribution detected by two two-dimensional temperature distribution sensors installed in a room of a care recipient, A technique for determining the standing, sitting, or lying position of the human body from these two extracted human body temperature regions is disclosed.

Japanese Patent Application Laid-Open No. 2020-110422 International publication number WO2016/152182A1 JP 2019-49849 A Japanese Patent Application Laid-Open No. 2020-074110 JP 2014-106636 A

In conventional monitoring systems, it was possible to remotely report when there was a problem such as a fall of a care recipient in a nursing facility, etc. did not previously exist.

The present invention has been made in consideration of these points, and is a risk prediction system, program, and risk prediction method for nursing facilities that can predict the danger of a care recipient and notify the caregiver. intended to provide

The danger prediction system for nursing homes of the present invention is
Receiving means for receiving a shot video from a user terminal;
posture estimating means for estimating the posture of the cared person from the captured moving image received by the receiving means;
action detection means for detecting the action status of the cared person from the object information estimated by the posture estimation means;
action prediction means for predicting the next action of the cared person from the action situation detected by the action detection means;
display instruction means for displaying a situation image on a care support terminal based on the action status detected by the action detection means and the prediction information predicted by the action prediction means;
characterized by comprising

The program of the present invention is
the computer,
Receiving means for receiving a shot video from a user terminal;
posture estimating means for estimating the posture of the cared person from the captured moving image received by the receiving means;
action detection means for detecting the action status of the cared person from the object information estimated by the posture estimation means;
action prediction means for predicting the next action of the cared person from the action situation detected by the action detection means;
display instruction means for displaying a situation image on a care support terminal based on the action status detected by the action detection means and the prediction information predicted by the action prediction means;
It is characterized by functioning as

The information processing method for nursing care facilities of the present invention includes:
a step of receiving a captured video from a user terminal;
a step of estimating the posture of the care-receiver from the received captured video;
a step of detecting the action status of the care recipient from the estimated object information;
a step of predicting the next action of the cared person from the detected action situation;
a step of displaying a situation image on a care support terminal based on the detected action situation and the predicted prediction information;
characterized by comprising

According to the risk prediction system for nursing facilities, the program, and the risk prediction method for nursing facilities of the present invention, it is possible to predict the danger of the care recipient and notify the caregiver of the danger.

1 is a block diagram showing the configuration of a danger prediction system according to an embodiment of the present invention; FIG. 1. It is explanatory drawing which shows the flow of information when watching over a care-receiver by the danger prediction system shown in FIG. 1. It is a chart figure which shows transmission/reception of information in each server, a terminal, etc. when watching over a care-receiver by the danger prediction system shown in FIG. 1. It is a flowchart which shows the flow of a series of operation|movement when watching over a care-receiver by the danger prediction system shown in FIG. FIG. 4 is an explanatory diagram showing a human skeleton model including a photographed moving image photographed by a user terminal and the positions of a plurality of parts that characterize the movement of the care-receiver in the photographed moving image; 6 is an explanatory diagram showing a human skeleton model extracted from FIG. 5; FIG. It is a figure which shows the situation image displayed on a care support terminal. It is a figure which shows the other situation image displayed on a care support terminal. FIG. 4 is an explanatory diagram showing a method of generating a learned model to be used for behavior detection means and behavior prediction means of a learning server;

<Overview of Danger Prediction System 1>
BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the danger prediction system 1 according to the present embodiment, and FIG. 2 is an explanation showing the flow of information when the danger prediction system 1 shown in FIG. 1 watches over the cared person. It is a diagram. 3 and 4 are charts and flow charts when the danger prediction system 1 shown in FIG. 1 is used to watch over the cared person. FIG. 5 is an explanatory diagram showing a moving image captured by the user terminal 3 and a human skeleton model including the positions of a plurality of parts that characterize the movement of the care receiver in this moving image, and FIG. 6 is an explanatory diagram showing a human skeleton model extracted from FIG. 5; FIG. 7 and 8 are diagrams showing various screens displayed on the care support terminal 7. FIG. Also, FIG. 9 is an explanatory diagram showing a method of generating a learned model to be used in the behavior detection means and behavior prediction means of the learning server.

A danger prediction system 1 according to this embodiment includes a learning server 2, a video stream service 4, an application server 6, and the like. These learning server 2, video stream service 4, application server 6, etc. are connected to communicate with each other via a communication network 8 such as an Internet line. A storage device 9 is communicably connected to the learning server 2 . A database 5 is communicably connected to the application server 6 . The learning server 2 is a server that can be used through an Internet line, and is provided in a state in which a virtual dedicated server is constructed within a physical server that can be accessed through the Internet. The video stream service 4 and the application server 6 are also databases, servers, etc. that can be used through the Internet line. A user terminal 3 installed in the cared person's room is communicatively connected to the learning server 2, the video stream service 4, and the like via a communication network 8. FIG. The user terminal 3 is a terminal owned by a care recipient or a terminal lent to the care recipient from a care facility, and the terminal has a camera. As a result, compared with the case where a camera or the like is installed on the ceiling of the care recipient's room, the care recipient does not feel that he is constantly being monitored, and the care recipient can be relieved. The user terminal 3 is, specifically, a smart phone, a PC tablet, a smart speaker, a smart device, or the like. A care support terminal 7 installed in an office of a care facility or carried by a caregiver is communicatively connected to the application server 6 via a communication network 8 . Nursing care support terminal 7 is, specifically, a smart phone, a PC tablet, a computer, or the like.

<Structure of Learning Server 2>
As shown in FIG. 1, by executing a program, the learning server 2 includes a reception unit 201, a photographing instruction unit 202, a posture estimation unit 203, a care equipment detection unit 204, an action detection unit 205, an action prediction unit 206, It is a computer that functions as posture information transmission means 207 and action information transmission means 208 . The program is stored in the learning server 2 or stored in a server other than the learning server 2, and is read from the learning server 2 to the other server.

The accepting means 201 accepts captured moving images over time from the video stream service 4, which will be described later. The photographing instruction means 202 receives the photographing start signal from the user terminal 3 and transmits the photographing instruction signal to the user terminal 3 .

The posture estimating means 203 estimates the posture of the cared person from the captured video received by the receiving means 201 . Specifically, posture estimating means 203 converts a human skeleton model (skeleton model) including the positions of a plurality of parts that characterize the movement of the care receiver in the captured moving image received by receiving means 201 into object information of the care receiver. (attitude information). The details of the method of estimating the care-receiver's posture will be described later. Nursing-care equipment detection means 204 detects information on nursing-care equipment from the captured video received by receiving means 201 . Examples of care equipment include beds, wheelchairs, care lifts, handrails, and the like.

Action detection means 205 detects the action status of the cared person from the object information estimated by posture estimation means 203 . Specifically, the action detection means 205 detects the action status of the care receiver based on a human skeleton model including the positions of a plurality of parts that characterize the actions of the care receiver. The details of the detection method of the action status of the care-receiver will be described later.

The action prediction means 206 predicts the next action of the cared person from the action situation detected by the action detection means 205 . The details of the method for predicting the care-receiver's next action will be described later.

Posture information transmitting means 207 transmits the object information of the cared person estimated by posture estimating means 203 to storage device 9 . The behavior information transmitting means 208 also transmits to the application server 6 the behavior status of the cared person detected by the behavior detecting means 205 and prediction information of the care recipient's next behavior obtained by the behavior predicting means 206 .

Details of the functions of the reception means 201, the photographing instruction means 202, the posture estimation means 203, the care equipment detection means 204, the action detection means 205, the action prediction means 206, the posture information transmission means 207, and the action information transmission means 208 will be described later. do.

Such a learning server 2 is communicably connected to a video stream service 4, an application server 6, and the like via a communication network 8 such as the Internet and a LAN.

The user terminal 3 is, as described above, a terminal possessed by the care recipient or a terminal lent to the care recipient from the care facility. The user terminal 3 is placed at a predetermined position in the cared person's room. For example, a charger for the user terminal 3 is fixed at a predetermined position in the cared person's room, and the user terminal 3 is placed on the charger. In addition, the user terminal 3 is equipped with an imaging unit such as a camera, and moving images captured by the imaging unit are transmitted from the user terminal 3 to the video stream service 4 over time. In addition, an infrared camera, an RGBD (depth camera), and a vital sensor are additionally installed in the care recipient's room. It may be possible to

The video stream service 4 is installed in a server or the like that can be used through the Internet line, and processes the captured video transmitted from the user terminal 3 and transmits the processed video to the learning server 2 . The video stream service 4 is composed of WebRTC and the like, and transmits and receives large amounts of digital data such as audio and video in real time on a WEB browser. WebRTC is an open standard with open source code.

The storage device 9 stores the object information transmitted from the learning server 2 by the attitude information transmission means 207 . Specifically, the storage device 9 stores, as object information, a human skeleton model of the care receiver and moving images of information of a plurality of parts of care equipment other than the care receiver. In addition, the database 5 stores the care receiver's action status and predictive information transmitted from the application server 6 .

The application server 6 is a computer that functions as display instruction means 601 by executing a program.

The display instruction means 601 causes the nursing care support terminal 7 to display a situation image based on the action status detected by the action detection means 205 and the prediction information predicted by the action prediction means 206 . More specifically, the application server 6 or a server different from the application server 6 stores explanatory images for each action situation (specifically, illustrations of various action situations). includes an explanatory image corresponding to the action situation detected by the action detection means 205 in the situation image. As a result, an explanatory image corresponding to the action situation detected by the action detection means 205 is displayed on the care support terminal 7 . The application server 6 or a server other than the application server 6 stores whether or not there is a dangerous situation for each piece of prediction information. is in a dangerous situation, the situation image includes a message to notify of this. As a result, when the prediction information predicted by the action prediction means 206 indicates a dangerous situation, a message is displayed on the nursing care support terminal 7 to notify this fact. The details of the function of such display instruction means 601 will be described later.

As described above, the care support terminal 7 is installed in an office of a care facility or carried by a caregiver, and displays a situation image transmitted from the application server 6 . Details of the situation image will be described later.

<Information processing method by danger prediction system 1>
Next, an information processing method by the danger prediction system 1 will be described with reference to FIGS. 2 to 9. FIG.

In carrying out the information processing method by the danger prediction system 1 of the present embodiment, the user terminal 3 is installed in the cared person's room. As described above, the user terminal 3 is a terminal owned by a care recipient or a terminal lent to the care recipient from a care facility. In such a user terminal 3, a danger prediction application can be downloaded from an online store or a management company of the danger prediction system 1. FIG. Then, after the danger prediction application is downloaded to the user terminal 3, when the watching application is activated or the shooting start command is input by the watching application, the captured video acquired by the user terminal 3 is sent to the video stream service 4. will be sent over time to More specifically, the entire room of the care receiver is captured by the camera of the user terminal 3, and the captured video includes the care receiver, caregiver, and care equipment. Also, at this time, when the watching application is activated or a shooting start command is input by the watching application, a shooting start signal is transmitted from the user terminal 3 to the learning server 2 (“YES” in step St1). The user terminal 3 may start photographing after the photographing instruction signal is returned from the learning server 2 to the user terminal 3 (step St2). This allows the learning server 2 to recognize that the user terminal 3 has started shooting.

Alternatively, instead of downloading the monitoring application to the user terminal 3, a photographing instruction may be sent to the camera of the user terminal 3 on the browser. In this case, by accessing the website for risk prediction in the browser on the user terminal 3 and entering the member ID or e-mail address and password, the member site can be entered. Then, when a shooting start command is input at the member's site, the shot moving image acquired by the user terminal 3 is transmitted to the video stream service 4 with time. Also, at this time, when a shooting start command is input at the member site, a shooting start signal is transmitted from the user terminal 3 to the learning server 2 (“YES” in step St1), and the learning server 2 instructs shooting. After the signal is returned to the user terminal 3, the user terminal 3 may start photographing (step St2). This allows the learning server 2 to recognize that the user terminal 3 has started shooting.

When the captured video is transmitted from the user terminal 3 to the video stream service 4 (step St3), the video stream service 4 transmits and receives large-capacity digital data such as audio and video in real time on the WEB browser, and the captured video is converted into video. It is transmitted from the stream service 4 to the learning server 2 (step St4).

When a captured moving image is transmitted from the video stream service 4 to the learning server 2, the receiving means 201 receives the captured moving image. Then, posture estimating means 203 estimates the posture of the cared person from the captured moving image received by receiving means 201 (step St5). Specifically, as shown in FIGS. 5 and 6, the posture estimating means 203 generates a skeletal model including positions of a plurality of parts that characterize the movement of the care-receiver in the moving image received by the receiving means 201. The created human skeleton model is used as the object information of the cared person. The posture information transmitting means 207 also transmits the human skeleton model created by the posture estimating means 203 as object information to the storage device 9, and stores it in the storage device 9 (step St6). Nursing-care equipment detection means 204 detects information on nursing-care equipment (indicated by two-dot chain lines in FIGS. 5 and 6 ) from the captured moving image received by receiving means 201 . Information on the detected care equipment is also transmitted from the learning server 2 to the storage device 9 and stored in the storage device 9 together with the cared person's skeleton model.

Then, the action detection means 205 detects the action status of the cared person from the object information estimated by the posture estimation means 203 with reference to a first learned model described later (step St7). More specifically, the action detection means 205 detects at least one of the object information estimated by the posture estimation means 203, and the care-receiver's depth recognition (depth detection) and space recognition in the photographed moving image received by the reception means 201. Or refer to both and detect the behavioral situation of the care recipient. For example, if the nursing care equipment is a bed, the behavioral status of the care recipient includes the state of lying on the bed, the state of trying to get up from the state of lying on the bed, the state of getting up and sitting on the bed, and the state of getting up from the bed. There is a state of trying to sleep, a state of standing up from the bed, and a state of starting to walk while holding a handrail. The behavior detection unit 205 also refers to information on the position of the care receiver with respect to care equipment other than the care receiver detected by the care equipment detection unit 204, and detects the behavior of the care receiver. For example, referring to the position and posture of the care receiver with respect to the bed detected by the care equipment detection section 204, the action detection section 205 detects the action status of the care receiver. It should be noted that the action status of the cared person detected by the action detection means 205 is not limited to the bed, but may be the action status of the care receiver with respect to other types of care equipment, or with care equipment. may be unrelated behavioral situations of the care receiver.

The first trained model is obtained by deep learning based on the object information of the cared person (specifically, the human skeleton model) acquired by the posture estimation means 203 . Specifically, as shown in FIG. 9, the learning server 2 is provided with a first model generation means 210, and the first model generation means 210 generates a first trained model. The first model generation means 210 is composed of a neural network (NN) such as a convolutional neural network (CNN). Such a neural network has a function of outputting a first model consisting of multiple weights learned from the care receiver's object information and behavioral situations as teaching data, and a function of outputting the care receiver's object information and peripheral object information. a middle layer that performs calculations using the input object information of the care receiver and multiple weights learned from the object information; and an output layer that outputs the care receiver's action status, It has a function to use the output of the output layer as learning data for re-learning. By using the first trained model generated by the first model generation means 210 in this manner, the behavior detection means 205 can accurately estimate the behavior of the care-receiver from the object information estimated by the posture estimation means 203. can be detected.

Further, the action prediction means 206 predicts the next action of the cared person by referring to the second learned model, which will be described later, based on the action status of the cared person detected by the action detection means 205 (step St8). Specifically, for example, when the action detecting means 205 detects that the care receiver is in a state of getting up from lying on the bed, the action predicting means 206 detects that the care receiver will stand up as the next action of the care receiver. foresee what will happen. After that, the action information transmitting means 208 transmits to the application server 6 the action status of the cared person detected by the action detecting means 205 and prediction information of the next action of the cared person detected by the action predicting means 206 ( Step St9).

The second learned model is obtained by deep learning based on the behavioral situation of the care receiver detected by the behavior detection means 205 . Specifically, as shown in FIG. 9, the learning server 2 is provided with a second model generating means 212, and the second model generating means 212 generates a second trained model. The second model generation means 212 is composed of a neural network (NN) such as a convolutional neural network (CNN). Such a neural network has a function of creating a second trained model consisting of multiple weights learned by learning the behavioral situation of the care recipient and the next behavioral emotion of the care recipient as teaching data, a middle layer that performs calculations using multiple weights learned from the input behavioral situation of the care recipient, and an output layer that outputs the next behavioral situation of the care recipient, It has a function to use the output of the output layer as learning data for re-learning. By using the second learned model generated by the second model generation means 212 in this manner, the behavior prediction means 206 detects the care recipient's It is possible to accurately predict the next action.

When the action status of the cared person and the prediction information of the next action are transmitted from the learning server 2 to the application server 6, the display instruction means 601 of the application server 6 detects the action status and action detected by the action detection means 205. Based on the prediction information predicted by the prediction means 206, a situation image is displayed on the care support terminal 7 (step St10). FIG. 7 is a situation image displayed on the care support terminal 7. FIG. The care support terminal 7 is a smartphone, a PC tablet, a computer, or the like installed in an office of a care facility or carried by a caregiver.

Specifically, as described above, the application server 6 or a server other than the application server 6 stores explanatory images for each action situation. Then, the display instructing means 601 causes the care support terminal 7 to display the explanatory image corresponding to the action situation detected by the action detecting means 205 by including the explanatory image corresponding to the action situation in the situation image. For example, in FIG. 7 , an illustration of a cared person getting up and sitting on the bed is displayed on the care support terminal 7 as the explanatory image 71 .

In addition, the display instruction means 601 causes the care support terminal 7 to display a message related to prediction information of the next action of the care-receiver detected by the action prediction means 206 in the situation image. For example, in FIG. 7, a message 72 is displayed on the care support terminal 7 indicating that the care receiver is about to stand up as the next action.

The application server 6 or a server other than the application server 6 stores whether or not each piece of prediction information is in a dangerous situation. Then, if the prediction information predicted by the action prediction unit 206 indicates a dangerous situation, the display instruction unit 601 causes the situation image to include an additional message such as "fall risk". As a result, when the prediction information predicted by the action prediction means 206 indicates a dangerous situation, an additional message is displayed on the care support terminal 7 . For example, in FIG. 7, when it is predicted that the care-receiver is about to stand up as the next action of the care-receiver, an additional message 73 informing the care-receiver that there is a risk of falling is sent to the care receiver. It is displayed on the support terminal 7 .

In this way, since the explanation image 71, the message 72, and the warning message 73 are displayed on the care support terminal 7, the caregiver senses that the caregiver is at risk of falling and immediately moves to the caregiver's room. can go to

Also, the action status of the cared person and the predictive information of the next action sent from the learning server 2 to the application server 6 are sent to the database 5 and stored in the database 5 . Further, the object information (specifically, the skeletal model of the cared person) estimated by the posture estimation means 203 is transmitted from the learning server 2 to the care support terminal 7 via the video stream service 4, and the care support terminal 7 It may be displayed on the support terminal 7 . FIG. 8 is a diagram showing another situation image displayed on the care support terminal 7. As shown in FIG. As shown in FIG. 8, the care support terminal 7 displays an image 71 of the cared person standing up and sitting on the bed, and a skeletal model 74 of the cared person. Is displayed. This allows the caregiver to know the more detailed posture of the care-receiver.

According to the danger prediction system 1, the program, and the information processing method of the present embodiment configured as described above, when the accepting means 201 accepts a captured moving image from the user terminal 3, the posture estimating means 203 causes the accepting means 201 to The posture of the cared person is estimated from the received moving image, and the action detection means 205 detects the action status of the cared person from the object information estimated by the posture estimation means 203 . Further, the action prediction means 206 predicts the next action of the cared person from the action situation detected by the action detection means 205, and the display instruction means 601 detects the action situation detected by the action detection means 205 and the action prediction. Based on the prediction information predicted by the means 206, the care support terminal 7 is made to display the situation image. As a result, the caregiver looking at the care support terminal 7 can recognize the caregiver's current behavior and the caregiver's next behavior even at a distance from the caregiver. Therefore, it is possible to predict and notify the care-receiver that the care-receiver is likely to fall or the like.

In addition, instead of displaying the moving image captured by the user terminal 3 as it is on the care support terminal 7, a situation image corresponding to the action situation detected by the action detection means 205 and the prediction information predicted by the action prediction means 206 is displayed. By displaying it on the care support terminal 7, the privacy of the cared person can be protected.

Note that the danger prediction system 1 and the information processing method according to the present embodiment are not limited to the aspects described above, and various modifications can be made.

For example, in the above description, an example in which the learning server 2 and the application server 6 are provided separately has been described, but a server in which the learning server 2 and the application server 6 are integrated may be used. In this case, in one server, a computer executes a program to obtain information such as reception means 201, photographing instruction means 202, posture estimation means 203, care equipment detection means 204, behavior detection means 205, behavior prediction means 206, and posture information transmission. It functions as means 207 , action information transmission means 208 and display instruction means 601 . In addition, the reception means 201, the photographing instruction means 202, the attitude estimation means 203, the behavior detection means 205, the behavior prediction means 206, the attitude information transmission means 207, the behavior information transmission means 208, and the display instruction means 601 described above can be combined with a plurality of computers or servers. In this case, which means functions on which computer or server can be set arbitrarily.

Also, the learning server 2, the application server 6, and the like may be locally installed servers or computers instead of being executed in the cloud. Also, the storage device 9 and the database 5 may be recording devices installed locally instead of on the cloud.

Alternatively, the video stream service 4 may not be provided in the danger prediction system 1 , and moving images captured by the user terminal 3 may be directly transmitted from the user terminal 3 to the learning server 2 . In addition, the ``receiving means for receiving a captured moving image from the user terminal'' in the present specification and claims refers to a mode in which the receiving means 201 directly receives a captured moving image from the user terminal 3, and a mode in which the receiving means 201 receives a video stream from the user terminal 3. It is designed to include both aspects of receiving a captured moving image via the service 4.

Further, the action prediction means 206 may predict the next action of the cared person based on the action status detected by the action detection means 205 and the cared person's care record. The care-receiver's care record refers to the care-receiver's age, sex, height, weight, chronic disease, degree of dementia, etc. received from the care record database. Such physical characteristics of the care recipient are stored in advance in the care record database in association with the identification information of the care recipient. The identification information of the user terminal 3 is also linked to the identification information of the care recipient, and the identification information of the user terminal 3 is transmitted when the captured moving image is transmitted from the user terminal 3 to the learning server 2. Thus, the learning server 2 can acquire the identification information of the cared person corresponding to the identification information of the user terminal 3 . In this case, the action prediction means 206 can predict the next action of the care receiver according to the care record of the care receiver. Object information acquired by posture estimation means 203, behavioral situations detected by behavior detection means 205, prediction information predicted by behavior prediction means 206, etc. are transmitted from database 5 to the nursing care record database. may be stored in

In addition, when the prediction information predicted by the action prediction unit 206 indicates a dangerous situation, the display instruction means 601 includes an additional message in the situation image, and when judging whether or not the situation is dangerous, Information on the physical characteristics of the caregiver may be reflected. In other words, even if the prediction information predicted by the behavior prediction means 206 is the same, the judgment as to whether or not there is a dangerous situation may differ when the physical characteristics of the care-receiver are different. For example, if the care recipient is relatively healthy, even if it is predicted that the care recipient is about to get up from the state of sitting on the bed, the situation image displayed on the care support terminal 7 does not include a warning message. If the care receiver has a chronic illness or is elderly, the situation image displayed on the care support terminal 7 is displayed when it is predicted that the care recipient is about to get up from the state of sitting on the bed. A warning message about the possibility of a fall will be included.

Further, in the above description, it has been described that the entire room of the care recipient is photographed by the camera of the user terminal 3, and that the photographed video includes the care recipient, the caregiver, and the care equipment. The posture estimation unit 203 may estimate the posture of not only the caregiver but also the caregiver to obtain object information, and the behavior detection unit 204 may detect the caregiver's action status. Information relating to the caregiver's action status may be transmitted from the application server 6 to the care support terminal 7 and displayed on the care support terminal 7 . This makes it possible to detect abusive behavior by the caregiver to the care-receiver. In addition, not only the care receiver's room but also the whole area of a large number of people, such as a shared space, is photographed by the camera of the user terminal 3, so that all actions of multiple caregivers and care receivers in this large number of people area can be captured. Simultaneous analysis may be possible.

1 Danger prediction system 2 Learning server 3 User terminal 4 Video stream service 5 Database 6 Application server 7 Nursing care support terminal 8 Communication network 9 Storage device 71 Explanation image 72 Message 73 Warning message 74 Skeleton model 201 Receiving means 202 Photographing instruction means 203 posture estimation means 204 care equipment detection means 205 action detection means 206 action prediction means 207 posture information transmission means 208 action information transmission means 210 first model generation means 212 second model generation means 601 display instruction means

Claims (17)

Receiving means for receiving a shot video from a user terminal;
posture estimating means for estimating the posture of the cared person from the captured moving image received by the receiving means;
action detection means for detecting the action status of the cared person from the object information estimated by the posture estimation means;
action prediction means for predicting the next action of the cared person from the action situation detected by the action detection means;
display instruction means for displaying a situation image on a care support terminal based on the action status detected by the action detection means and the prediction information predicted by the action prediction means;
A danger prediction system for nursing homes. An explanatory image for each action situation is stored in advance,
2. The danger prediction system for nursing homes according to claim 1, wherein said display instructing means causes said situation image to include a pre-stored explanatory image corresponding to the action situation detected by said action detecting means. Whether or not there is a dangerous situation is stored in advance for each prediction information,
3. The display instruction means according to claim 1, wherein said display instruction means includes a message in said situation image when the prediction information predicted by said action prediction means indicates a dangerous situation based on information stored in advance. Danger prediction system for nursing homes. The care receiver's care record is linked to this care receiver's identification information,
The identification information of the user terminal is transmitted when the captured video is transmitted from the user terminal, thereby acquiring the identification information of the care recipient,
When the display instruction means includes a message in the situation image when the prediction information predicted by the action prediction means is a dangerous situation, the care recipient's care in judging whether the situation is dangerous or not. The danger prediction system for nursing homes according to claim 3, wherein the recorded information is reflected. 5. The method of claim 1, wherein said posture estimating means uses a skeletal model of the cared person including positions of a plurality of parts that characterize the movement of the cared person in the photographed moving image received by said receiving means as object information of the cared person. The danger prediction system for nursing homes according to any one of the preceding items. 6. The danger prediction system for nursing homes according to claim 5, wherein said display instruction means includes said human skeleton model in said situation image. 7. The danger prediction system for nursing homes according to claim 5 or 6, wherein said behavior detection means detects the behavior of the care recipient based on said human skeleton model. Further comprising a nursing care equipment detection means for acquiring information of nursing care equipment other than the care recipient in the photographed moving image received by the reception means,
8. The danger prediction system for nursing care facilities according to claim 7, wherein said behavior detection means also refers to information on the position of the care recipient with respect to care equipment other than the care recipient to detect the behavior status of the care recipient. 9. The action detection means according to claim 7, wherein the action detection means detects the action status of the care-receiver by referring to at least one of depth recognition and spatial recognition of the care-receiver in the photographed moving image received by the reception means. Danger prediction system for nursing homes. Further comprising a first model generating means for generating a first trained model by deep learning based on the object information of the care receiver acquired by the posture estimating means,
10. The behavior detector according to any one of claims 1 to 9, wherein said behavior detector refers to said first learned model from the object information estimated by said posture estimator and detects the behavior of the care-receiver. hazard prediction system for nursing homes. Further comprising a second model generation means for generating a second trained model by deep learning based on the behavioral status of the care recipient detected by the behavior detection means,
11. The action prediction means according to any one of claims 1 to 10, wherein the action prediction means predicts the next action of the care-receiver by referring to the second learned model from the action status of the care-receiver detected by the action detection means. or the danger prediction system for nursing homes according to any one of the above items. The danger prediction system for nursing care facilities according to any one of claims 1 to 11, wherein the user terminal is a terminal owned by the care receiver or a terminal lent to the care receiver from the nursing care facility. further comprising imaging instruction means for transmitting an imaging instruction signal to the user terminal when an imaging start signal is received from the user terminal;
13. The system according to any one of claims 1 to 12, wherein when a photographing instruction signal is transmitted to said user terminal by said photographing instruction means, a photographed moving image acquired by said user terminal is transmitted to said danger prediction system over time. The danger prediction system for nursing homes described in . A danger prediction application can be downloaded to the user terminal, and when the danger prediction application is activated or a shooting start command is input by the danger prediction application, the photographed video acquired by the user terminal is used as the danger prediction. 13. A care facility danger prediction system according to any one of claims 1 to 12, transmitted to the system over time. The care receiver's care record is linked to this care receiver's identification information,
The identification information of the user terminal is transmitted when the captured video is transmitted from the user terminal, thereby acquiring the identification information of the care recipient,
The behavior prediction means also refers to the care receiver's care record linked to the acquired identification information of the care receiver, and based on the behavior status detected by the behavior detection means, predicts the care receiver's next behavior. 15. The danger prediction system for nursing homes according to any one of claims 1 to 14, which predicts the behavior of the care facility. the computer,
Receiving means for receiving a shot video from a user terminal;
posture estimating means for estimating the posture of the cared person from the captured moving image received by the receiving means;
action detection means for detecting the action status of the cared person from the object information estimated by the posture estimation means;
action prediction means for predicting the next action of the cared person from the action situation detected by the action detection means;
display instruction means for displaying a situation image on a care support terminal based on the action status detected by the action detection means and the prediction information predicted by the action prediction means;
A program that functions as A computer-implemented method of hazard prediction for nursing homes, comprising:
a step of receiving a captured video from a user terminal;
a step of estimating the posture of the care-receiver from the received captured video;
a step of detecting the action status of the care recipient from the estimated object information;
a step of predicting the next action of the cared person from the detected action situation;
a step of displaying a situation image on a care support terminal based on the detected action situation and the predicted prediction information;
A danger prediction method for nursing homes.

Images