JP2023041168A - Monitor system, monitor method, and monitor program - Google Patents

Abstract

To provide a monitor system, monitor method, and monitor program which can prevent or suppress a false alarm or missing alarm of a movement of a care-receiver who is required give alarm such as his/her slip-down, overturn, and the like.SOLUTION: A monitor system 1 includes an imaging part 103, a bed sensor 95, and a determination part. The imaging part images an image region including at least a bed. The bed sensor 95 detects a load on the bed, The determination part determines an object that has been fallen-out from the bed on the basis of, the information obtained by the imaging part and the information obtained by the bed sensor 95.SELECTED DRAWING: Figure 1

Description

The present invention relates to a monitoring system, a monitoring method, and a monitoring program.

In nursing care facilities such as nursing care facilities for the elderly and hospitals, care recipients and care recipients (hereinafter referred to as "cared care recipients" or "users") who live in or are hospitalized may: You may fall while walking or fall out of bed and get injured. Therefore, in order to enable nurses, caregivers, etc. (hereinafter referred to as “caregiver”) to rush to the caregiver when the caregiver becomes such a condition, Development of a monitoring system for constant detection is underway.

Patent Literature 1 listed below discloses a technique for detecting "skidding" of a person to be monitored from a bed based on an image. With this technology, when it is detected that 30% or more and less than 100% of the volume of a moving object exceeds the bed frame while the upper or lower body is protruding from the bedside in a supine position, the "slipping It is judged as "Falling".

WO2019/030880

In the technique of Patent Document 1, as a result of diligent studies by the inventors, even if an object other than the person to be monitored (the person being cared for), such as a futon, protrudes from the bed, it is erroneously detected that the person to be monitored has slipped down. It was found that there were problems that the monitored person slipped off the bed very slowly, and that the object was not recognized as a moving object, resulting in a false alarm.

The present invention has been made to solve such problems. That is, it is an object of the present invention to provide a monitoring system, a monitoring method, and a monitoring program capable of preventing or suppressing erroneous and unreported conditions of a care recipient, such as slipping, falling, and falling.

The above problems of the present invention are solved by the following means.

(1) an imaging unit that captures an imaging area including at least a bed; a bed sensor that detects a load on the bed; and a determination unit that determines an object protruding from the object.

(2) Based on the information obtained from the imaging unit and the information obtained from the bed sensor, the determination unit determines whether the object protruding from the bed includes only a futon or at least a part of the body. The monitoring system according to (1) above, which determines.

(3) The monitoring system according to (2) above, further comprising a control unit that performs control not to report the determination result when it is determined that the only object protruding from the bed is the futon.

(4) The monitoring system according to (2) above, further comprising a control unit that performs control to report a determination result when it is determined that the object protruding from the bed includes at least a part of the body. .

(5) The monitoring system according to any one of (1) to (4) above, wherein the information obtained from the imaging unit is information obtained by analyzing an image captured by the imaging unit.

(6) The monitoring system according to any one of (1) to (5) above, wherein the information obtained from the bed sensor is information obtained by analyzing the load detected by the bed sensor.

(7) The monitoring system according to any one of (1) to (4) and (6) above, wherein the information obtained from the imaging section is image information captured by the imaging section.

(8) The monitoring system according to any one of (1) to (5) and (7) above, wherein the information obtained from the bed sensor is load information detected by the bed sensor.

(9) The above-described (7), further comprising an acquisition unit that acquires information from the bed sensor, changes image recognition sensitivity based on the information acquired from the bed sensor, and then acquires the image information. monitoring system.

(10) The monitoring system according to (7) above, further including an acquisition unit that acquires the image information from the imaging unit installed on the bed.

(11) The monitoring system according to any one of (1) to (10) above, further comprising an output section that outputs a determination result of the determination section.

(12) an image capturing step in which an image capturing unit captures an image of an image capturing area including at least a bed; a detection step in which a bed sensor detects a load on the bed; information obtained from the image capturing unit; and a determination step of determining an object protruding from the bed based on the information obtained.

(13) In the determining step, based on the information obtained from the imaging unit and the information obtained from the bed sensor, it is determined whether the object protruding from the bed includes only the futon or at least a part of the body. The monitoring method according to (12) above.

(14) The monitoring method according to (13) above, further comprising a control step of not notifying the determination result when it is determined in the determination step that the only object protruding from the bed is the futon.

(15) The above (13), further comprising a control step of notifying the determination result when it is determined in the determination step that the object protruding from the bed includes at least a part of the body. monitoring method described in .

(16) The monitoring method according to any one of (12) to (15) above, wherein the information obtained from the imaging unit is information obtained by analyzing an image captured by the imaging unit.

(17) The monitoring method according to any one of (12) to (16) above, wherein the information obtained from the bed sensor is information obtained by analyzing the load detected by the bed sensor.

(18) The monitoring method according to any one of (12) to (15) and (17) above, wherein the information obtained from the imaging unit is image information captured by the imaging unit.

(19) The monitoring method according to any one of (12) to (16) and (18) above, wherein the information obtained from the bed sensor is load information detected by the bed sensor.

(20) The monitoring method according to (18) above, wherein information is obtained from the bed sensor, image recognition sensitivity is changed based on the information obtained from the bed sensor, and then the image information is obtained.

(21) The monitoring method according to any one of (12) to (20) above, further including an output step of outputting the determination result of the determination step.

(22) A monitoring program for causing a computer to execute the monitoring method according to any one of (12) to (21) above.

According to the present invention, based on the information obtained from the imaging unit and the information obtained from the bed sensor, an object protruding from the bed is determined. It is possible to prevent or suppress erroneous and unreported conditions of the cared-for person who needs to be notified.

It is a figure showing the whole surveillance system composition concerning a 1st embodiment. FIG. 2 is a schematic diagram illustrating a bed of a cared person on which the detection device shown in FIG. 1 is installed; FIG. 3 is a schematic diagram of the bed of the person being cared for shown in FIG. 2 viewed from the side; 2 is a block diagram showing the hardware configuration of the detection device shown in FIG. 1; FIG. 5 is a block diagram illustrating main functions of an arithmetic control unit shown in FIG. 4; FIG. 2 is a block diagram showing the hardware configuration of the server shown in FIG. 1; FIG. 6 is a flow chart illustrating a processing procedure of a detection device monitoring method according to the first embodiment; FIG. 4 is a diagram illustrating determination of protrusion from a bed based on image information and bed sensor information; FIG. 4 is a schematic diagram illustrating an image when the cared person is lying down; FIG. 10 is a schematic diagram illustrating an image when the cared person is out of bed; FIG. 11 is a schematic diagram illustrating an image when a futon object protrudes from a predetermined area; FIG. 10 is a schematic diagram illustrating an image when a part (leg) of the cared person object is protruding from a predetermined area; FIG. 10 is a schematic diagram illustrating an image when the cared person is out of bed; It is a schematic diagram which illustrates an image when a cared person being cared for slips down. FIG. 10 is a schematic diagram illustrating an image when the cared person is lying down as an initial state; FIG. 4 is a diagram illustrating determination of protrusion from a bed based on image information and bed sensor information; FIG. 4 is a schematic diagram illustrating an image when the cared-for caregiver falls or falls; FIG. 11 is a schematic diagram illustrating an image when a futon object protrudes from a predetermined area; 11 is a flow chart illustrating a processing procedure of a monitoring method by a detection device according to the third embodiment; FIG. 10 is a schematic diagram illustrating an image when the cared person is lying down as an initial state; FIG. 10 is a diagram exemplifying state determination of a cared person based on image information and bed sensor information; FIG. 10 is a schematic diagram illustrating an image when the cared person is out of bed; FIG. 4 is a schematic diagram illustrating an image when the cared-for caregiver falls or falls; As a comparative example, it is a schematic diagram illustrating an image when the cared person is out of bed. As a comparative example, it is a schematic diagram which illustrates an image when a cared person to be cared for falls or falls.

Hereinafter, a monitoring system, a monitoring method, and a monitoring program according to embodiments of the present invention will be described with reference to the drawings. In the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted. Also, the dimensional ratios in the drawings are exaggerated for convenience of explanation, and may differ from the actual ratios.

(First embodiment)
FIG. 1 is a diagram showing the overall configuration of a monitoring system 1 according to the first embodiment. In this embodiment, the monitoring system 1 is used in a nursing care facility or the like. The monitoring system 1 includes a detection device 10 , a server 20 , multiple mobile terminals 30 and a wireless AP (access point) 31 . The detection device 10, the server 20, and the mobile terminal 30 are connected to communicate with each other via a LAN (Local Area Network) and a wireless AP31. The portable terminal 30 is carried by a nursing caregiver 70 who nurses or cares for a nursing care recipient 80 .

(Bed 90)
FIG. 2 is a schematic diagram illustrating a bed 90 of a cared person 80 on which the detection device 10 shown in FIG. 1 is installed. 3 is a schematic diagram of the bed 90 shown in FIG. 2 as viewed from the side. In FIG. 3, the direction from left to right in the paper (the direction from the head side to the foot side of the bed 90) is the X-axis direction, and the direction from the front to the back of the paper (the width direction of the bed 90) is the Y-axis. The Z-axis direction is defined as the direction and the direction from the bottom to the top of the paper surface (the height direction of the bed 90).

As shown in FIGS. 2 and 3, bed 90 includes frame portion 91 , headboard 92 , footboard 93 , mattress 94 and bed sensor 95 . A futon 96 is put on the mattress 94 . Futon 96 may include bedding such as comforters, blankets, towel blankets, and the like. Also, the bed 90 can be, for example, an electric bed.

The bed sensor 95 is a load sensor that detects the pressing force due to the weight of the cared person 80 on the bed 90 . Thereby, the bed sensor 95 detects whether or not the cared person 80 is on the bed 90 . The bed sensor 95 can be composed of a sheet-like bag containing air and a piezoelectric sensor that detects variations in the air pressure. The bed sensor 95 can be arranged, for example, in a portion of the mattress 94 with which the back, waist, buttocks, etc. of the cared person 80 come into contact. A detection result (bed sensor information) of the bed sensor 95 is transmitted to the arithmetic control unit 101 . The bed sensor information can be information about the load measured by the load sensor (load sensor information). In the present embodiment, the bed sensor information indicates, for example, the presence or absence of a load on the care recipient 80 as "weighted" and "unweighted", respectively.

(Detection device 10)
FIG. 4 is a block diagram showing the hardware configuration of the detection device 10 shown in FIG. 1. As shown in FIG. The detection device 10 has an arithmetic control unit 101, a communication unit 102, and a camera 103, which are interconnected by a bus. The detection device 10 is installed, for example, on the upper end of the headboard 92 of the bed 90 .

The arithmetic control unit 101 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like. Also, the arithmetic control unit 101 can include an HDD or an SSD (Solid State Drive). The arithmetic control unit 101 performs control and arithmetic processing of each unit of the detection device 10 according to the monitoring program. Main functions of the arithmetic control unit 101 will be described later.

The communication unit 102 is an interface circuit (for example, a LAN card, a wireless communication circuit, etc.) for communicating with, for example, the server 20 via a LAN.

The camera 103 captures an image within a monitoring area including at least the bed 90 and the peripheral area of the bed 90, and outputs a captured image (image information). The camera 103 constitutes an imaging unit. This image information includes still images and moving images. A moving image may include a plurality of still images captured intermittently in time series. The camera 103 is a near-infrared camera, but a visible light camera may be used instead, or both of them may be used.

FIG. 5 is a block diagram illustrating major functions of the arithmetic control unit 101 shown in FIG. Calculation control unit 101 functions as acquisition unit 111 , determination unit 112 , and output unit 113 .

Acquisition unit 111 acquires image information and bed sensor information. Judgment unit 112 judges protrusion from bed 90 based on image information and bed sensor information. The details of the determination method of the protrusion from the bed 90 will be described later. The output unit 113 outputs the determination result of the determination unit 112 . Output from the output unit 113 is transmitted to the mobile terminal 30 by the communication unit 102 .

(Server 20)
FIG. 6 is a block diagram showing the hardware configuration of the server 20. As shown in FIG. The server 20 has an arithmetic control unit 201, a communication unit 202, and a storage unit 203, which are interconnected by a bus.

The arithmetic control unit 201 includes a CPU, RAM, ROM, and the like. The arithmetic control unit 201 controls each unit of the server 20 and performs arithmetic processing according to a program. In addition, the arithmetic control unit 201 can be configured to cooperate with the determination unit 112 to determine whether the person protrudes from the bed 90 .

The communication unit 202 is configured by an interface circuit for communicating with, for example, the detection device 10 and the mobile terminal 30 via a LAN. The communication unit 202 receives the protrusion determination result from the detection device 10 . Further, the communication unit 202 receives image information and bed sensor information from the detection device 10 when the arithmetic control unit 201 performs the protrusion determination. The protrusion determination result is stored in the storage unit 203 .

Storage unit 203 is configured by an HDD or SSD, and stores various programs and various data.

(monitoring method)
FIG. 7 is a flowchart illustrating a processing procedure of a monitoring method by the detection device according to the first embodiment; The processing of the flowchart in FIG. 1 is realized by the CPU of the arithmetic control unit 101 executing the monitoring program. FIG. 8 is a diagram illustrating determination of protrusion from the bed 90 based on image information and bed sensor information. FIG. 9 is a schematic diagram illustrating an image when the cared person 80 is lying down. In order to simplify the explanation, illustration of the object on the image corresponding to the footboard 93 is omitted in FIG. 9 (the same applies to FIGS. 10 to 14 below).

As shown in FIG. 7, first, the camera 103 starts photographing the monitored area (step S101). The arithmetic control unit 101 controls the camera 103 to start (or continue) photographing within the monitoring area. In this embodiment, as an example, it is assumed that the cared person 80 lying on the bed 90 gets up from the bed 90, gets down on the floor, and tries to walk. That is, the initial state of the cared person 80 is bed rest, and then transitions to getting out of bed. The camera 103 photographs the cared person 80 and the bed 90 within the monitoring area.

Next, image information and bed sensor information are obtained (step S102). Acquisition unit 111 acquires an image captured by camera 103 and transmits it to determination unit 112 as image information. Acquisition unit 111 also transmits bed sensor information to determination unit 112 .

Next, it is determined whether or not the object protrudes from the bed 90 (step S103). A determination unit 112 detects a moving object from the image information acquired by the acquisition unit 111 . A moving object can be detected, for example, by extracting a range of pixels with relatively large differences by a time difference method of extracting differences between images (frames) captured at different times. A moving object may be detected by a background subtraction method that extracts a difference between a captured image and a background image. In this embodiment, it is assumed that the cared-for person 80 is detected as a moving object in the process of changing the state of the cared-for person 80 from lying down to getting out of bed. Further, when the cared person 80 is covered with a futon 96, the futon 96 may move. Therefore, in this embodiment, it is considered that the cared person 80 and/or the futon 96 may be detected as a moving object. In this embodiment, the bed 90 may be an electric bed, but while the care recipient 80 is being watched over, reclining and other operations are stopped, and the mattress 94 does not move.

However, it is difficult to distinguish between the state where the futon 96 protrudes and the state where the cared-for person 80 slips down only by the detection result of the moving object obtained from the image information. On the other hand, it is difficult to distinguish between a state in which a part of the body of the care recipient 80 protrudes from the bed 90 and a state in which the care recipient 80 is lying on the bed 90 only with the bed sensor information. Therefore, in the present embodiment, the determination unit 112 determines protrusion from the bed 90 based on both image information and bed sensor information. Here, protruding from the bed 90 means that part of the body of the care recipient 80 and/or the futon 96 protrudes from the bed 90 . Then, the determination unit 112 determines an object protruding from the bed 90 based on image information (information obtained from the camera 103) and bed sensor information (information obtained from the bed sensor 95). Below, the object of the care recipient 80 on the image is the care recipient object 800, the object of the mattress 94 is the mattress object 940 (shaded area in FIG. 9), the object of the bed sensor 95 is the bed sensor object 950, the futon 96 objects are called futon objects 960 .

(1) Judgment pattern 1
As shown in FIG. 9, the determination unit 112 determines that there is no change in the image around a predetermined area (imaging area indicated by a dashed line) A including the mattress object 940 (that is, there is no object moving from the inside to the outside of the predetermined area). ) and the bed sensor information from the bed sensor 95 is "weighted" (the bed sensor object 950 is shown in light gray), it is determined that the bed 90 is "not protruding". More specifically, the determination unit 112 detects (determines) the presence or absence of a load when the load exceeds a predetermined threshold value set corresponding to the weight of each cared person 80 . A predetermined threshold for load can be set in advance for each care recipient 80 and stored in the RAM of the arithmetic control unit 101 . The predetermined area A is set to a size slightly larger than the mattress object 940 . That is, along the outer edge of the mattress object 940 , the outer edge of the predetermined area A can be set at a position separated by m from the outer edge of the mattress object 940 . This is because it is assumed that the bedding object 960 protrudes from the mattress object 940 to some extent even when the bedding 96 is appropriately placed on the cared person 80 lying in bed.

Note that FIG. 9 illustrates a part of the predetermined area A for the sake of simplification of explanation, but the outer edge of the mattress object 940 outside the monitoring area is also separated from the outer edge of the mattress object 940 by m. An outer edge of the predetermined area A can be set at the position. The setting of the outer edge of the predetermined area A is the same in FIGS. 10 to 14 below.

Furthermore, since there is no protrusion from the bed 90, the determination unit 112 determines that the cared person 80 is lying down (see FIG. 8).

(2) Judgment pattern 2
FIG. 10 is a schematic diagram illustrating an image when the cared person 80 is out of bed. The determination unit 112 determines that the image does not change around the predetermined area A (no moving object is detected by the time subtraction method or the background subtraction method), and the bed sensor information from the bed sensor 95 is "unweighted." , it is determined that there is no protrusion from the bed 90 . For example, when the cared person 80 moves slowly, the image around the predetermined area A may not change. Even if the moving object cannot be detected from the image information, the determination unit 112 determines that the cared person 80 is out of bed because the bed sensor information is "no weight".

(3) Judgment pattern 3
FIG. 11 is a schematic diagram illustrating an image when the futon object 960 protrudes from the predetermined area A, and FIG. 4 is a schematic diagram illustrating an image; FIG. If there is a change in the image around the predetermined region A and the bed sensor information from the bed sensor 95 is "weighted", the determination unit 112 determines that "the object protrudes". If there is nothing protruding from the predetermined area A other than the care recipient object 800 and the futon object 960, it is estimated that either or both of the care recipient object 800 and the futon object 960 have protruded. . Therefore, the determination unit 112 determines that the cared person 80 is lying down or that a part of the body protrudes from the state that the care recipient 80 is not at least slipping down or getting out of bed.

(4) Judgment pattern 4
FIG. 13 is a schematic diagram illustrating an image when the cared-for person 80 is out of bed, and FIG. 14 is a schematic diagram illustrating an image when the cared-for person 80 slips down. When there is a change in the image around the predetermined area A and the bed sensor information from the bed sensor 95 is "no weight", the determination unit 112 determines that "the care recipient protrudes from the bed 90". do. Further, the determination unit 112 determines that the care recipient object 800 has moved out of the predetermined area A and does not exist on the mattress object 940, so that the care recipient 80 has left the bed or slipped down.

Next, the determination result is output (step S104). The output unit 113 outputs the determination result of protrusion from the bed 90 and/or the determination result of the state of the cared person 80 . The determination result is transmitted to the mobile terminal 30 via the server 20 . The output unit 113 does not output the determination result when, for example, the nursing caregiver 80 is lying down or a part of the body of the nursing caregiver 80 protrudes. When it is necessary to rush to the nursing caregiver 80, the determination result can be output. In addition, the output unit 113 outputs the determination result in the case of slipping, tumbling, falling, etc., and also sends an instruction to the mobile terminal 30 to issue an alarm. It is also possible to send an image for confirming the state of the mobile terminal 30 via the server 20. - 特許庁In addition, the output unit 113 may be configured to output the determination result in the case of getting out of bed, and to inform the nursing caregiver 70 according to the level of care required of the nursing caregiver 80 and the like. In addition, for example, when a part of the body of the care recipient 80 or the futon 96 protrudes for a long time, the output unit 113 outputs the determination result from the viewpoint of the safety and physical condition of the care recipient 80. and the caregiver 70 may be notified.

In this way, in the processing of the monitoring method of the flowchart shown in FIG. 7, image information and bed sensor information are acquired, and protrusion from the bed 90 is determined based on both the image information and bed sensor information.

Since the detecting device 10 of this embodiment described above determines an object protruding from the bed 90 based on the information obtained from the camera 103 and the information obtained from the bed sensor 95, the cared person 80 erroneous and unreported information about the condition of the cared person 80 who needs to be notified, such as when the patient slips off the bed 90 or falls down, can be prevented or suppressed.

(Second embodiment)
In the second embodiment, a case in which an object in an image has moved downward outside the predetermined area A will be described, based on bed sensor information to determine whether the object protrudes from the bed 90 . In the following description, detailed descriptions of the same configurations as those of the first embodiment will be omitted in order to avoid duplication of description.

FIG. 15 is a schematic diagram illustrating an image when the care recipient 80 is in bed as an initial state, and FIG. 16 illustrates determination of protrusion from the bed 90 based on image information and bed sensor information. It is a diagram. 17 is a schematic diagram illustrating an image when the cared person 80 falls or falls, and FIG. 18 illustrates an image when a part of the futon object 960 protrudes from the predetermined area A. It is a schematic diagram. 15, 17, and 18, the illustration of the object corresponding to the footboard 93 is omitted for the sake of simplicity of explanation.

In this embodiment, a threshold value TH is set for distinguishing whether the object has moved upward or downward outside the predetermined area A. FIG. The threshold TH is set, for example, to the position of the upper end of the mattress object 940 on the image. If the position of the object in the vertical direction on the image exceeds the threshold value TH, the determination unit 112 determines that the object has moved outside and above the predetermined area A in the image. On the other hand, when the position of the object is less than the threshold TH, the determination unit 112 determines that the object has moved downward.

As shown in FIG. 16, when there is a change in the image below the predetermined area A and the bed sensor information from the bed sensor 95 is "no weight", the determination unit 112 determines that the "weight" from the bed 90 is "not weighted". There is protruding caregiver".

Furthermore, as shown in FIG. 17, the determination unit 112 determines that the care recipient object 800 on the image has moved outside the predetermined region A and does not exist on the mattress object 940, so that the care recipient 80 falls. , or fall.

On the other hand, if there is a change in the image below the outside of the predetermined area A in the image and the bed sensor information from the bed sensor 95 is "weighted", the determination unit 112 determines that "there is an object protruding from the bed 90". ” is determined. If there is nothing protruding from the predetermined area A other than the care recipient object 800 and the futon object 960, it is estimated that either or both of the care recipient object 800 and the futon object 960 have protruded. . FIG. 18 exemplifies a case where the futon object 960 protrudes from the predetermined area A. As shown in FIG.

As described above, in the present embodiment, when the object in the image moves downward outside the predetermined area A, it is determined whether the object protrudes from the bed 90 based on the bed sensor information. Therefore, the determination unit 112 can determine whether some object that has moved downward outside the predetermined area A is due to the protrusion of the cared person 80 or the protrusion of the futon 96 . Furthermore, the determination unit 112 can determine whether the cared person 80 is in a state of falling or falling, or is bedridden (not falling or falling).

(Third embodiment)
In the third embodiment, when the bed sensor information is "unweighted", based on whether the object in the image has moved upward or downward outside the predetermined area A, the cared person 80 determine the state of In the following description, detailed descriptions of the same configurations as those of the first embodiment will be omitted in order to avoid duplication of description.

FIG. 19 is a flow chart illustrating the processing procedure of the monitoring method by the detecting device 10 according to the third embodiment. The processing of the flowchart in FIG. 1 is realized by the CPU of the arithmetic control unit 101 executing the monitoring program. FIG. 20 is a schematic diagram illustrating an image when the care recipient 80 is in bed as an initial state, and FIG. 21 is a state of the care recipient 80 based on image information and bed sensor information. It is a figure which illustrates determination. FIG. 22 is a schematic diagram illustrating an image when the care recipient 80 is out of bed, and FIG. 23 is a schematic diagram illustrating an image when the care recipient 80 falls or falls. . Further, FIG. 24 is a schematic diagram illustrating an image when the care recipient 80 is out of bed as a comparative example, and FIG. 25 is an image when the care recipient 80 falls or falls as a comparative example. It is a schematic diagram to illustrate. 20 and 22 to 25, the illustration of the object corresponding to the footboard 93 is omitted in order to simplify the explanation.

As shown in FIG. 19, first, the camera 103 starts photographing the monitored area (step S201). The arithmetic control unit 101 controls the camera 103 to start (or continue) photographing within the monitoring area. In this embodiment, as in the second embodiment, as an initial state, it is assumed that the cared person 80 is lying down, and an object that has moved outside the predetermined area A is outside the predetermined area A. A threshold TH is set for distinguishing between upward movement and downward movement (see FIG. 20).

Next, bed sensor information is acquired (step S202). Acquisition unit 111 transmits the detection result of bed sensor 95 to determination unit 112 as bed sensor information.

Next, it is determined whether or not the bed sensor information is "no weight" (step S203). As shown in FIG. 21, when the bed sensor information is "unweighted" (step S203: YES), the image recognition sensitivity is changed (step S204). The arithmetic control unit 101 controls the camera 103 so as to increase the recognition sensitivity of the image around the bed 90 . For example, the arithmetic control unit 101 controls to increase the image recognition sensitivity by increasing the frame rate of photographing by the camera 103 . By increasing the frame rate, the image quality is improved, and the detection accuracy of the position of the object in the image is improved. On the other hand, if the bed sensor information is not "unweighted", ie, "weighted" (step S203: NO), the image recognition sensitivity is not changed, and the process proceeds to the next step S205. In the case of "weighted", the cared person 80 has not moved from the bed 90, so there is no need to increase the image recognition sensitivity.

Next, image information is acquired (step S205). Acquisition unit 111 acquires an image captured by camera 103 and transmits it to determination unit 112 as image information.

Next, the state of the cared person 80 is determined (step S206). As shown in FIG. 22 , when there is some object that has moved upward outside the predetermined area A, the determining unit 112 determines that the cared person 80 has left the bed (falling or not falling). Further, as shown in FIG. 23, when there is some object that has moved downward outside the predetermined area A and there is no object that has moved upward, the determination unit 112 determines that the cared person 80 has fallen or has fallen. I judge.

On the other hand, as shown in FIG. 24, when the bed sensor 95 is not arranged on the mattress 94, the bed sensor information cannot be obtained, so the image recognition sensitivity is not changed. Therefore, there is a possibility that the determination unit 112 cannot detect even if the cared person 80 leaves the bed. Similarly, as shown in FIG. 25, if the bed sensor 95 is not placed on the mattress 94, there is a possibility that the bed sensor 95 may not be detected even if the cared person 80 falls or falls.

Next, the determination result is output (step S207). The output unit 113 outputs the determination result of the condition of the care recipient 80 . The determination result is transmitted to the mobile terminal 30 via the server 20 .

In this way, in the processing of the flowchart shown in FIG. 19, the bed sensor information is acquired, and if the bed sensor information is "unweighted", the image recognition sensitivity is changed, then the image information is acquired, and the image information and The state of the care recipient 80 is determined based on the bed sensor information.

According to the detecting device 10 of the present embodiment described above, when the cared person 80 slips off the bed or falls, etc., the state of the cared person 80 that needs to be notified is false or missing. can be prevented or suppressed.

The configuration of the monitoring system 1 described above is the main configuration for describing the features of the above-described embodiment, and is not limited to the above-described configuration, and can be variously modified within the scope of the claims.

For example, in the first to third embodiments described above, the case where the detection device 10 is installed on the headboard 92 has been described, but the detection device 10 may be installed on a location other than the headboard 92, such as the footboard 93. may be Further, the detection device 10 is not limited to the upper end portions of the headboard 92 and the footboard 93, and can be installed, for example, in the plane facing the center of the bed 90. FIG.

Further, in the above-described first to third embodiments, the determination unit 112 detects an object from the bed 90 based on whether there is a change in the vicinity of the predetermined area A of the image information and whether there is weight in the bed sensor information. The case of judging the protrusion of is described above. However, the present invention is not limited to such a case. and whether or not the weight of the bed sensor information is weighted.

An object protruding from the bed 90 may be only the futon 96 , the futon 96 and a part of the body of the cared person 80 , or only a part of the body of the cared person 80 . When image analysis is performed using a machine learning technique, the determination unit 112 can determine a protruding object, for example, as follows. In advance, conditions such as the position and state of the futon 96 on the bed 90 are changed, and the predetermined region A and its surroundings are photographed when only the futon 96 protrudes from the bed 90, and a plurality of photographed images and correct labels are displayed. It is input to the deep neural network as training data and made to learn. In this case, the correct labels are: 100% probability (likelihood) that only the futon 96 protrudes from the bed 90; The probability that only a part of the body of the caregiver 80 protrudes is assumed to be 0%. During monitoring of the cared person 80, the determination unit 112 inputs the image (information) obtained from the camera 103 to the deep neural network, and obtains the probability that only the futon 96 protrudes from the bed 90 as an analysis result. . Based on the analysis result and the bed sensor information (information obtained from the bed sensor 95), the determination unit 112 determines whether the object protruding from the bed 90 is only the futon 96 or includes at least a part of the body. judge. For example, when the probability that only the futon 96 protrudes from the bed 90 is equal to or greater than a predetermined value and the bed sensor information is "weighted", the determination unit 112 determines that the futon 96 is the only object protruding from the bed 90. I judge. On the other hand, when the probability that only the futon 96 protrudes from the bed 90 is less than the predetermined value and the bed sensor information is "weighted", the determination unit 112 determines that the object protruding from the bed 90 is at least It is determined that a part of the body of the person 80 is included.

When the determination unit 112 determines that the only object protruding from the bed 90 is the futon 96, control is performed so as not to output (notify) the determination result. As a result, it is possible to prevent erroneous reporting of the condition of the cared person 80 that does not require reporting. As a result, the burden on the caregiver 70 can be reduced. On the other hand, when the determination unit 112 determines that at least part of the body is included in the object protruding from the bed 90, the output unit 113 performs control to output the determination result.

Moreover, although the case where the determination part 112 performs image analysis was demonstrated, this invention is not limited to such a case. For example, the camera 103 analyzes the image of the predetermined area A and its surroundings to determine whether or not the bed 90 protrudes. It can also be configured to determine protruding objects. In addition, the image (image information) captured by the camera 103 is transmitted to the server 20 as it is without analysis, and the image is analyzed in the server 20 to determine whether the bed 90 protrudes. It is also possible to determine an object protruding from the bed 90 based on.

Further, the bed sensor 95 detects the load (load information) on the bed 90, analyzes the detected load information, and determines whether or not the bed 90 protrudes. It can also be configured to determine an object protruding from the bed 90 based on the analysis result. Further, the load information detected by the bed sensor 95 is transmitted to the server 20 as it is without being analyzed, and the server 20 analyzes the load information on the bed 90 to determine whether the bed 90 protrudes from the bed 90. Image information, And, based on the analysis result of the load information, it is also possible to determine an object protruding from the bed 90 .

Further, the above-described flowchart may include steps other than the steps shown in the flowchart, or may not include some steps. Also, the order of steps is not limited to the embodiment described above. Furthermore, each step may be combined with other steps and executed as one step, may be executed while being included in other steps, or may be divided into a plurality of steps and executed.

Also, the means and methods for performing various processes in the above-described embodiments can be realized either by dedicated hardware circuits or programmed computers. The program may be provided by a computer-readable recording medium such as a USB memory or a DVD (Digital Versatile Disc)-ROM, or may be provided online via a network such as the Internet. In this case, the program recorded on the computer-readable recording medium is usually transferred to and stored in a storage unit such as a hard disk. Further, the program may be provided as independent application software, or may be incorporated as a function into the software of the device such as the detection unit.

1 surveillance system,
10 detection device,
101 arithmetic control unit,
102 communication unit,
103 camera,
111 acquisition unit;
112 determination unit,
113 output unit,
20 servers,
201 arithmetic control unit,
202 communication unit;
203 storage unit,
30 mobile terminals,
31 access points,
90 beds,
91 frame part,
92 headboards,
93 footboards,
94 mattresses,
95 sensor unit,
96 futons.

Claims (22)

an imaging unit that captures an imaging area including at least the bed;
a bed sensor that detects a load on the bed;
a determination unit that determines an object protruding from the bed based on information obtained from the imaging unit and information obtained from the bed sensor. The determination unit determines whether the object protruding from the bed includes only the futon or at least a part of the body, based on the information obtained from the imaging unit and the information obtained from the bed sensor. A surveillance system according to claim 1 . 3. The monitoring system according to claim 2, further comprising a control unit that performs control not to report the determination result when it is determined that the object protruding from the bed is only the futon. 3. The monitoring system according to claim 2, further comprising a control unit that, when it is determined that at least a part of the body is included in the object protruding from the bed, performs control to report a determination result. 5. The monitoring system according to any one of claims 1 to 4, wherein the information obtained from said imaging unit is information obtained by analyzing an image taken by said imaging unit. The monitoring system according to any one of claims 1 to 5, wherein the information obtained from the bed sensor is information obtained by analyzing the load detected by the bed sensor. 7. The monitoring system according to any one of claims 1 to 4 and 6, wherein the information obtained from said imaging unit is image information captured by said imaging unit. The monitoring system according to any one of claims 1 to 5 and 7, wherein the information obtained from the bed sensor is load information detected by the bed sensor. 8. The monitoring system according to claim 7, further comprising an acquisition unit that acquires information from the bed sensor, changes image recognition sensitivity based on the information acquired from the bed sensor, and then acquires the image information. 8. The monitoring system according to claim 7, further comprising an acquisition section that acquires the image information from the imaging section installed on the bed. 11. The monitoring system according to any one of claims 1 to 10, further comprising an output section for outputting the judgment result of said judgment section. an imaging step in which the imaging unit images an imaging region including at least the bed;
a detection step in which a bed sensor detects a load on the bed;
a determination step of determining an object protruding from the bed based on information obtained from the imaging unit and information obtained from the bed sensor. In the determination step, it is determined whether the object protruding from the bed includes only the futon or at least a part of the body, based on the information obtained from the imaging unit and the information obtained from the bed sensor. Item 13. The monitoring method according to Item 12. 14. The monitoring method according to claim 13, further comprising a control step of not notifying the determination result when it is determined in the determination step that the only object protruding from the bed is the futon. 14. The monitoring according to claim 13, further comprising a control step of notifying the determination result when it is determined in the determination step that the object protruding from the bed includes at least a part of the body. Method. 16. The monitoring method according to any one of claims 12 to 15, wherein the information obtained from said imaging unit is information obtained by analyzing an image taken by said imaging unit. The monitoring method according to any one of claims 12 to 16, wherein the information obtained from the bed sensor is information obtained by analyzing the load detected by the bed sensor. 18. The monitoring method according to any one of claims 12 to 15 and 17, wherein the information obtained from said imaging unit is image information captured by said imaging unit. The monitoring method according to any one of claims 12 to 16 and 18, wherein the information obtained from said bed sensor is load information detected by said bed sensor. 19. The monitoring method according to claim 18, wherein information is obtained from the bed sensor, image recognition sensitivity is changed based on the information obtained from the bed sensor, and then the image information is obtained. 21. The monitoring method according to any one of claims 12 to 20, further comprising an output step of outputting the determination result of said determination step. A monitoring program for causing a computer to execute the monitoring method according to any one of claims 12 to 21.

Images