JP7147837B2 - Anomaly detection system, anomaly detection method and anomaly detection program - Google Patents

Description

The present invention relates to an anomaly detection system, an anomaly detection method, and an anomaly detection program for detecting an anomaly in a detection target area.

2. Description of the Related Art Conventionally, various systems have been proposed for monitoring the actions of care recipients in nursing care facilities and the like. For example, in the system disclosed in Patent Literature 1, an image captured by a camera is analyzed to determine whether a person has fallen, and when it is determined that the person has fallen, time measurement is started. Then, when the measured time reaches the reference time, a notification is sent to a pre-registered notification destination. A notification is issued only when the measured time is long (only when it is assumed that the elderly person has fallen or crouched down), and when the measured time is short (such as when he gets up immediately). Therefore, only in the former case, a supervisor (for example, a staff member) needs to arrange for a confirmation person or an ambulance, thereby reducing the staff's monitoring burden.

By the way, when a large number of care recipients reside in a nursing care facility, notifications may be concentrated on a small number of staff members from each room. In this case, if there are many erroneous notifications (hereinafter also referred to as erroneous reports), the frequency of staff rushing to the site will increase, which is likely to lead to an increase in work load. Therefore, it is desirable to prevent false alarms from occurring frequently.

In addition, in the living room, the care recipient may fall violently, or the pain in the internal organs may cause the person to slowly squat down on the spot. When judging the care recipient's fall based on the image taken by the camera installed on the ceiling, the image from the ceiling accurately captures the change in the care recipient's behavior in the vertical direction to the ceiling. is difficult. Therefore, it is difficult to determine in a short period of time whether the cared person is falling or squatting on the basis of the image. Also, the care recipient may temporarily squat down to pick up something in the first place. Therefore, if notification is given in a short time after the fall is determined, false alarm may be caused. In this regard, the system of Patent Document 1 described above is considered to be very effective in preventing frequent occurrence of false alarms because notification is made after a certain period of time has elapsed after the determination of the turnover.

Japanese Patent Application Laid-Open No. 2010-40017 (Claim 4, paragraphs [0025], [0064] to [0076], see FIGS. 13 to 16, etc.)

However, in the system of Patent Document 1, there is no concept of utilizing the camera image when the fall is first determined (when there is a suspicion of the fall) for post-analysis. There is no concept of recording information on whether there is a suspicion of a fall, and the above information is not recorded at all. For this reason, even if the camera image used for fall detection is recorded as it is, it is difficult to quickly find the image at the time when the fall is suspected (image at the moment of the fall) from the recorded image. is difficult to reuse. In particular, when the number of recorded images (for example, the number of frames per second) is large, it becomes more difficult to search for the required image from the recorded images, and the reusability of the recorded images is further reduced.

Therefore, while suppressing the frequent occurrence of false alarms, the video when there is a suspicion of the occurrence of an abnormality (for example, a fall) is recorded without omission, and the video can be easily searched later (reusability of recorded video). It is desirable to build a system that can improve However, such a system has not yet been proposed.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and its purpose is to avoid the loss of video at the time when there is a suspicion of an abnormal occurrence while suppressing the frequent occurrence of false alarms, and to reproduce the recorded video. It is an object of the present invention to provide an anomaly detection system, an anomaly detection method, and an anomaly detection program capable of improving usability.

An anomaly detection system according to one aspect of the present invention includes an imaging unit that captures an image by capturing a detection target area, an information acquisition unit that acquires area information indicating a state in the detection target area, and the area an abnormality suspicion determining unit for determining whether or not there is a suspicion of occurrence of an abnormality in the detection target area based on internal information; and an output unit for notifying the occurrence of the abnormality to the outside when the occurrence of the abnormality is confirmed by the abnormality confirmation determination unit, based on the internal information. and, when the abnormality suspicion determination unit determines that the occurrence of the abnormality is suspected, the image captured by the imaging unit including the determination timing of the suspicion in terms of time is used for the determination. a temporary holding unit for temporarily holding timing information in association with timing information; and the image and the timing held in the temporary holding unit when the occurrence of the abnormality is confirmed by the abnormality confirmation determination unit. and a video recording unit that acquires information and records the acquired video and the timing information in association with the determination result by the abnormality determination determination unit.

An anomaly detection method according to another aspect of the present invention acquires area information indicating a state in the detection target area by an information acquisition unit including an imaging unit that acquires an image by capturing an image of the detection target area. an abnormality suspicion determination step of determining whether or not there is a suspicion of occurrence of an abnormality in the detection target area based on the area information; a temporary holding step of temporarily holding, in a temporary holding unit, the image captured by the imaging unit including timing for judging whether there is presence in terms of time, in association with timing information indicating the timing for judging the image; After determining that there is a suspicion of occurrence of an abnormality, an abnormality confirmation determination step of determining whether or not the abnormality has occurred based on the in-area information; a recording step of acquiring the image and the timing information held in a unit, associating the obtained image and the timing information with the confirmed result of occurrence of the abnormality and recording the result in the image recording unit; and an output step of notifying the occurrence of the abnormality to the outside when it is determined that the abnormality has occurred.

An anomaly detection program according to still another aspect of the present invention is a program that causes a computer to execute the above anomaly detection method.

While suppressing frequent occurrence of false alarms, it is possible to avoid loss of video at the time when there is a suspicion of occurrence of an abnormality, and improve the reusability of recorded video.

1 is a block diagram showing a schematic configuration of a care support system, which is an example of an anomaly detection system according to an embodiment of the present invention; FIG. It is a block diagram which shows the concrete structure of the room terminal which the said care support system has. FIG. 4 is an explanatory diagram schematically showing a state in which a subject standing up in a living room is viewed from the front; FIG. 3B is an explanatory diagram schematically showing a state in which the subject in FIG. 3A is viewed from the ceiling side of the living room; FIG. 10 is an explanatory diagram schematically showing a front view of a subject who has fallen in a living room; FIG. 4B is an explanatory diagram schematically showing a state in which the subject in FIG. 4A is viewed from the ceiling side of the living room; It is a block diagram which shows the concrete structure of the management server which the said care support system has. It is a block diagram which shows the concrete structure of the notification terminal which the said care support system has. It is a flowchart which shows an example of the flow of operation|movement in the said care support system. FIG. 4 is an explanatory diagram showing an example of information held in a temporary holding unit of the room terminal; It is a flowchart which shows the other example of the flow of operation|movement in the said care support system.

An embodiment of the present invention will be described below with reference to the drawings. In the following description, the term “image” refers to a visible image in a narrow sense, but here, in a broad sense, it includes not only the visible image but also the data (pixel values) of each pixel that constitutes the image. as a concept. In addition, the present invention is not limited to the following contents.

[About Care Support System]
FIG. 1 is a block diagram showing a schematic configuration of a care support system 1, which is an example of an abnormality detection system of this embodiment. The care support system 1 is a system for supporting the daily lives of care recipients residing in nursing homes or general housing and patients (care recipients) hospitalized in hospitals, and is also called a monitoring system. . The cared person and the cared person are subjects whose living activities are supported by the care support system 1 . Here, as an example, a case where the care support system 1 is constructed in a nursing facility will be described.

The care support system 1 is configured by connecting a room terminal 10, a management server 20, and a notification terminal 30 via a communication line 40 so that they can communicate with each other. The communication line 40 includes a wired LAN (Local Area Network), a wireless LAN, an Internet line, a dedicated communication line for mobile phones, and the like. In this embodiment, the room terminal 10 and the management server 20, and the management server 20 and the notification terminal 30 can communicate with each other.

The room terminal 10 is installed on the ceiling of the living room R of the subject T in the nursing facility. When there are multiple living rooms R in the care facility, the room terminal 10 is installed for each living room R. The living room R is an area targeted for abnormality detection by the care support system 1 (detection target area). The management server 20 collectively manages information output from each room terminal 10 .

The notification terminal 30 is configured by a mobile terminal such as a smart phone, a tablet terminal, a notebook personal computer, or the like. The staff S who monitors behavioral anomalies of the subject T can carry the notification terminal 30, and based on the information output from the notification terminal 30 after being notified from the management server 20 to the notification terminal 30, each living room It is possible to grasp the condition of the subject T of R.

Details of the care support system 1 will be described below.

(room terminal)
FIG. 2 is a block diagram showing a specific configuration of the room terminal 10. As shown in FIG. The room terminal 10 includes an information acquisition section 11 , an action recognition section 12 , a temporary holding section 13 , a notification management section 14 and a control section 15 . The control unit 15 is composed of a central processing unit (CPU) that controls the operation of each unit of the room terminal 10, and operates according to an operation program stored in a storage unit (not shown).

The information acquisition unit 11 is a block for acquiring in-area information indicating the state in the living room R (in particular, the state of the subject T), and has an imaging unit 11a and a sensor unit 11b. The imaging unit 11a is composed of a camera that captures the interior of the living room R from the ceiling toward the floor surface and obtains an image (for example, an infrared image) from a directly overhead viewpoint whose viewing direction is directly below. The imaging unit 11a basically acquires a moving image as the image by continuous shooting, but may be configured to intermittently acquire still images by intermittent shooting. The image acquired by the imaging unit 11a constitutes the area information.

The sensor unit 11b is a sensor that detects the state inside the living room R, and includes, for example, a pyroelectric sensor that detects thermal energy (infrared rays), a vibration sensor that detects vibration, a sound sensor that detects impact sound, and a A pressure sensor for detecting seating on a chair (including a wheelchair), a distance measuring sensor for measuring the distance to the subject T, and the like are appropriately selected and configured.

For example, if a plurality of pyroelectric sensors are installed at different height positions on the wall of living room R, infrared rays are detected by pyroelectric sensors at lower positions and infrared rays are not detected by pyroelectric sensors at higher positions. In addition, it can be detected that the target person T is falling down or crouching down. Further, by installing a vibration sensor on the floor of the living room R and detecting vibration with the vibration sensor, it is possible to detect the falling of the subject T. Further, the fall of the subject T can be detected by detecting the impact sound at the time of the fall with the sound sensor. Furthermore, by installing a pressure sensor on the seat surface of the chair in the living room R, it can be detected that the subject T is sitting on the chair and has not fallen down when the pressure is detected. In addition, by installing a distance measuring sensor on the ceiling of the room R, the distance to the subject T can be measured, and whether the subject T is standing or lying on the floor can be determined to some extent (with high accuracy). not) can be detected. The information detected and acquired by various sensors constituting the sensor unit 11b constitutes the above-mentioned area information.

As the distance measurement sensor, a high-precision distance measurement sensor that can accurately detect whether the target person T lying on the floor is in a state of falling or squatting may be used. Such highly accurate ranging sensors are expensive. Therefore, when using a distance measuring sensor, the distance measuring sensor should be selected in consideration of the balance between the required accuracy and cost. In the present embodiment, as will be described later, based on the image acquired by the imaging unit 11a, the suspicion of a fall and the definite determination are performed. can be detected with high accuracy.

The behavior recognition unit 12 recognizes the presence or absence of an abnormality in the behavior of the target person T in the living room R, the type of behavior, and the like, based on the in-area information acquired by the information acquisition unit 11 . Such an action recognition unit 12 is configured by a GPU (Graphics Processing Unit), which is a computing device specialized for real-time image processing, for example, but may be configured by the same or separate CPU as the control unit 15. . Recognition of the behavior of the target person T by the behavior recognition unit 12 is performed, for example, as follows.

3A schematically shows a state in which the subject T standing up in the living room R is viewed from the front, and FIG. 3B schematically shows a state in which the subject T in FIG. 3A is viewed from the ceiling side of the living room R. ing. 4A schematically shows a front view of the subject T who has fallen in the room R, and FIG. 4B schematically shows a state of the subject T in FIG. 4A viewed from the ceiling side of the living room R. shown in The action recognition unit 12 extracts the head region of the subject T from the image acquired by the imaging unit 11a, obtains its diameter value Lm (mm), and combines the obtained diameter value Lm with a preset threshold value Thb (mm ), and the presence or absence of a fall of the subject T is determined based on the magnitude relationship. It should be noted that extraction of the head region from the image can be performed using, for example, a machine-learned neural network, or can be performed by pattern matching with a preset shape.

If the diameter value Lm of the extracted head region is equal to or greater than the threshold value Thb, the action recognition unit 12 determines that the subject T has not fallen in the living room R (see FIGS. 3A and 3B), and the diameter value Lm is the threshold value. If it is less than Thb, it is determined that the subject T has fallen in the living room R (see FIGS. 4A and 4B). Such a determination is possible because, if the subject T is standing, the distance between the room terminal 10 installed on the ceiling and the head of the subject T becomes short, and the image acquired by the imaging unit 11a becomes While the size of the head of the subject T increases, if the subject T is in a falling state, the distance between the room terminal 10 installed on the ceiling and the head of the subject T increases, and the imaging unit 11a acquires This is based on the fact that the head size of the subject T becomes smaller on the image.

However, even if the behavior recognition unit 12 determines that the target person T is falling down, it is temporarily determined to be in the “suspicious abnormality” state, and from this determination time, the diameter value of the head that is equal to or greater than the preset threshold value Thb When the state in which Lm is not detected continues for a predetermined period of time, a determination is made to determine that "abnormality has occurred". On the other hand, when the diameter value Lm of the head equal to or greater than the preset threshold value Thb is detected from the time when the "abnormality suspected" state is determined, the behavior recognition unit 12 cancels the "abnormality suspected" state and "abnormality occurrence None” is determined.

Note that the action recognition unit 12 may determine whether there is a suspicion of occurrence of an abnormality (fall) and/or whether or not an abnormality has occurred, based on other fall detection or state detection. For example, when the movement speed of the target person T equal to or greater than a predetermined threshold value is not detected in the image acquired by the imaging unit 11a, the action recognition unit 12 determines that there is a suspicion of occurrence of an abnormality (fall), and/or A definite determination that there is an abnormality may be made. Further, when it is detected that the target person T is on the bed in the image acquired by the imaging unit 11a, the action recognition unit 12 cancels the suspicion of occurrence of an abnormality and/or confirms that there is no occurrence of an abnormality. A definitive judgment may be made.

Furthermore, the action recognition unit 12 determines whether there is a suspicion of occurrence of an abnormality and/or determines whether an abnormality has occurred, based on information detected by the sensor unit 11b in addition to the image acquired by the imaging unit 11a. you can go For example, when the movement of the target person T is not detected by the pyroelectric sensor (for example, when infrared rays are detected by the low-position pyroelectric sensor for a predetermined time or longer), the action recognition unit 12 determines that there is a suspicion of an abnormality, And/or it may be determined that there is an abnormality. Furthermore, when the distance from the range sensor installed on the ceiling changes to be low, when vibration is detected by a floor mat-like vibration sensor, or impact sound is detected by a sound sensor, it is determined that there is a suspicion of an abnormal occurrence. And/or a definite determination that there is an abnormality may be made.

From the above, the action recognition unit 12 determines whether an abnormality has occurred in the detection target area based on the area information (information acquired by the imaging unit 11a or the sensor unit 11b) acquired by the information acquisition unit 11. It has a function as an abnormality suspicion determination unit 12a that determines the presence or absence of suspicion, and after it is determined that there is a suspicion of occurrence of an abnormality, an abnormality that determines whether or not an abnormality has occurred based on the in-area information. It can be said that it has a function as the confirmation determination section 12b.

In addition, the action recognition unit 12 acquires timing information indicating the timing of each determination at the same time as making determinations as to whether or not there is a suspicion of occurrence of an abnormality and whether or not an abnormality has occurred. As the timing information, for example, time information indicating the point in time when the action recognition unit 12 made each determination can be considered. Therefore, the action recognition unit 12 also functions as a timekeeping unit that measures time, thereby obtaining the time information. The time information includes not only the time of the day but also the calendar day (year, month, day). . Note that the timing information may be a simple tag indicating the point in time when the action recognition unit 12 has determined that there is suspicion, in addition to the time information described above.

When the behavior recognition unit 12 (abnormality suspicion determination unit 12a) determines that there is a suspicion of occurrence of an abnormality, the temporary holding unit 13 acquires by capturing the image with the imaging unit 11a including the timing of determination of suspicion in terms of time. The image obtained is temporarily stored in association with the timing information indicating the determination timing. Such a temporary holding unit 13 is composed of a volatile memory such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory). Further, the temporary holding unit 13 is configured by a ring buffer that can hold (store) a new video overwritten on an old video when the remaining data capacity becomes smaller than a predetermined value.

The notification management unit 14 is an interface for communicating with the outside (management server 20, notification terminal 30) and includes an input/output port for communication. For example, information acquired by the room terminal 10 (for example, video held by the temporary holding unit 13 or real-time video acquired by the imaging unit 11a) is output to the outside via the notification management unit 14 and the communication line 40. be. Note that the notification management unit 14 may include an antenna, a transmission/reception circuit, a modulation circuit, a demodulation circuit, and the like so as to correspond to wireless communication with the outside.

(management server)
FIG. 5 is a block diagram showing a specific configuration of the management server 20. As shown in FIG. The management server 20 temporarily aggregates the notification results from the room terminals 10 in each living room R and notifies the notification terminal 30 of them. Such a management server 20 includes a video recording section 21 , an application management section 22 and a control section 23 . The control unit 23 is composed of a CPU that controls the operation of each unit of the management server 20, and operates according to an operation program stored in a storage unit (not shown).

The image recording unit 21 acquires the image and the timing information held in the temporary holding unit 13 of the room terminal 10 when the action recognition unit 12 (abnormality confirmation determination unit 12b) confirms that an abnormality has occurred. The obtained image and timing information are recorded in association with the result of confirmation determination by the abnormality confirmation determination unit 12b (for example, the confirmation result indicating that an abnormality has occurred). Such a video recording unit 21 is composed of, for example, a hard disk, but may be composed of other non-volatile memory such as flash memory.

The application management unit 22 is an interface for communicating with the outside (room terminal 10, notification terminal 30), and includes input/output ports for communication. For example, information acquired by the management server 20 (for example, information recorded by the video recording unit 21) is output to the outside (for example, the notification terminal 30) via the application management unit 22 and the communication line 40. Note that the application management unit 22 may be configured including an antenna, a transmission/reception circuit, a modulation circuit, a demodulation circuit, etc. so as to correspond to wireless communication with the outside.

In addition to the above, the management server 20 may be configured to further include a display unit such as a liquid crystal display and an instruction input unit such as a mouse and keyboard. In this case, the management server 20 can be easily configured using a terminal device such as a personal computer.

In addition, in the management server 20, applications such as ON/OFF setting of the notification function, initial setting for action recognition in the room terminal 10, setting of selection of the notification terminal 30 to be notified, judgment and notification history management, etc. are normally performed. Various settings and records are managed for the .

(notification terminal)
FIG. 6 is a block diagram showing a specific configuration of the notification terminal 30. As shown in FIG. The notification terminal 30 is configured by a mobile terminal such as a smartphone, for example, as described above, and includes an output section 31 , a communication section 32 and a control section 33 . The control unit 33 is composed of a CPU that controls the operation of each unit of the notification terminal 30, and operates according to an operation program stored in a storage unit (not shown).

The output unit 31 outputs the results notified from the management server 20 via the communication line 40 . In particular, when the abnormality determination determination section 12b of the room terminal 10 determines that an abnormality has occurred, the output section 31 receives a notification from the management server 20 and notifies the abnormality to the outside. Such an output unit 31 includes a display unit such as a liquid crystal display, an audio output unit (speaker), and the like. The display section may be a touch panel display that also serves as an input section. Note that the method of notification such as screen display and notification sound in the notification terminal 30 is not particularly limited, and any method can be used.

The communication unit 32 is an interface for communicating with the outside (room terminal 10, management server 20), and includes an antenna, a transmission/reception circuit, a modulation circuit, a demodulation circuit, and the like. Therefore, the notification terminal 30 acquires, for example, information output from the management server 20 (for example, information recorded by the video recording unit 21) via the application management unit 22, the communication line 40, and the communication unit 32, and outputs the information. It can be output by the unit 31 .

[Notification method]
Next, a method of notifying occurrence of abnormality (including a method of detecting abnormality) in the care support system 1 having the configuration described above will be described. Here, it is assumed that the detection target area is the living room R where the target person T conducts daily activities, and the abnormality in the detection target area to be recognized is the behavioral abnormality of the target person T in the living room R. A case in which the subject T falls on the floor of the living room R will be described as an example of the abnormal behavior.

FIG. 7 is a flow chart showing an example of the flow of operations in the care support system 1 of this embodiment. First, the imaging unit 11a of the room terminal 10 captures the inside of the living room R, and acquires a moving image (area information indicating the state of the living room R) as a moving image (S1; information acquisition step). The image is temporarily held in the temporary holding section 13 under the control of the control section 15 (S2). In S1, in-area information indicating the state of the living room R may be acquired by the sensor unit 11b instead of the imaging unit 11a. In this case, the intra-area information acquired by the sensor unit 11b is temporarily stored in the temporary storage unit 13 in S2. Note that when the post facto analysis is performed, if the image temporally before the judgment time of suspicion of occurrence of abnormality is unnecessary (when only the image after the suspicion judgment time is necessary), S2 is omitted. It is also possible to

Next, the abnormality suspicion determination unit 12a of the action recognition unit 12 recognizes the occurrence of an abnormality (whether or not the subject T has fallen) in the living room R based on the image acquired in S1, and determines whether the abnormality has occurred. Suspicion is determined (S3; abnormality suspicion determination step). For example, as described above, the abnormality suspicion determination unit 12a determines the size relationship between the diameter value Lm of the head region of the subject T extracted from the image acquired by the imaging unit 11a and the preset threshold value Thb. Therefore, it is possible to recognize whether or not the target person T has fallen, thereby determining whether or not the subject T has fallen (see FIGS. 3B, 4B, etc.). If it is determined in S3 that the subject T has fallen (suspicious occurrence of an abnormality), the process proceeds to the next S4. It is determined whether or not the person T has fallen.

When it is determined that there is a fall in S3, under the control of the control unit 15, the temporary holding unit 13 stores the image captured by the imaging unit 11a that temporally includes the timing for determining that there is a fall (abnormal occurrence is suspected). The captured image is temporarily stored in association with timing information indicating determination timing (S4; temporary storage step). For example, FIG. 8 shows an example of information held in the temporary holding unit 13. As shown in FIG. Here, for the image (ID=0001) acquired by the imaging unit 11a, the time held in the temporary holding unit 13 in S2, that is, before the suspicion of falling in S3 is determined to be "8:15:29". , the time when the suspicion of overturning is determined in S3 is "8:17:15", and the confirmed determination time of overturning in S5, which will be described later, is "8:20:07". is shown as That is, from 8:15:29, the video (ID = 0001) shot including the suspicion determination time (8:17:15) of the presence of a fall is the above suspicion determination time (8:17:15). is associated with and held in the temporary holding unit 13 . If the time at which the image is held in the temporary holding unit 13 in S2 and the time at which the fall is suspected at S3 is short, the former time can be substantially substituted for the latter time. .

Next, the abnormality confirmation determination unit 12b of the action recognition unit 12 recognizes the occurrence of an abnormality (whether or not the subject T has fallen) in the living room R based on the image acquired in S1, and determines whether the abnormality has occurred. Determination to confirm the presence or absence is performed (S5; abnormality confirmation determination step). Specifically, the abnormality confirmation determination unit 12b determines whether or not the suspicion of a fall based on the image has continued for a predetermined period of time. is performed, and if it does not continue for a predetermined period of time, it is decided that there is no overturn. Note that the predetermined time may be set to a sufficiently long time (fixed determination time) in advance, or even if it is not a sufficiently long time, it is determined that the actual notification should be made based on the suspicion that there is a fall. may be set to a time until the reliability of is sufficiently increased.

If it is determined in S5 that there is no fall, the process returns to S3, and it is determined whether there is a suspicion of occurrence of an abnormality (whether or not the subject T has fallen) based on the image acquired in S1. On the other hand, if it is determined that there is a fall in S5, the confirmed result (there is a fall) and the fact that there is a fall are displayed at the held image (ID=0001) and the suspicion determination time (8:17:15). The fixed judgment time (8:20:07) is further associated and held in the temporary holding unit 13 (S6).

The information held in the temporary holding unit 13 in S6 is transmitted to the management server 20 via the notification management unit 14 and the communication line 40 (S7), and recorded in the video recording unit 21 of the management server 20 (S8 ; recording step). Then, under the control of the control unit 23 of the management server 20, the application management unit 22 notifies the notification terminal 30 that an abnormality has occurred, that is, that the subject T has fallen in the living room R (S9). In the notification terminal 30, the content of notification in S9 is output (for example, displayed) by the output unit 31 (S10; output step). As a result, the staff member S, who owns the notification terminal 30, sees the output contents, grasps the fall of the target person T, and can take necessary measures such as rushing to the living room R immediately.

〔effect〕
As described above, in the present embodiment, when the abnormality suspicion determination unit 12a determines that an abnormality (for example, a fall) is suspected (S3), and then the abnormality confirmation determination unit 12b determines that an abnormality has occurred. Then (S5), the output unit 31 of the notification terminal 30 notifies the outside of the occurrence of the abnormality (S10). Therefore, even if there is a suspicion of the occurrence of an anomaly, if it is later confirmed that no anomaly has occurred (for example, if the anomaly has been resolved in a short period of time), external notification should not be made. can be As a result, it is possible to suppress unnecessary notifications to the outside, that is, frequent occurrence of false alarms. As a result, it is possible to reduce the frequency of rushing to the detection target area (living room R) of the supervisor (for example, the staff S) of the notification destination, thereby reducing the staff S' monitoring burden.

In addition, the image acquired by the imaging unit 11a temporally including the determination timing of suspicion is held in the temporary holding unit 13, and the image is acquired from the temporary holding unit 13 in the image recording unit 21. is recorded (S8). As a result, the image at the point of doubt can be left in the image recording unit 21 without omission, and the loss of the image can be avoided.

Further, when the abnormality suspicion determination unit 12a determines that there is a suspicion of occurrence of an abnormality (S3), the image acquired by the imaging unit 11a temporally including the determination timing of suspicion has timing information indicating the determination timing. (for example, time information) and temporarily stored in the temporary storage unit 13 (S4). Then, when the occurrence of an abnormality is confirmed by the abnormality confirmation determination section 12b (S5), the image and the timing information held in the temporary holding section 13 are associated with the confirmation result by the abnormality confirmation determination section 12b, and the image is recorded. It is recorded in the section 21 (S8). As a result, when the occurrence of an abnormality is confirmed, using the timing information as a clue, the image recorded in the image recording unit 21 at the time when the occurrence of an abnormality is suspected is recorded, even if the number of recorded images (for example, 1 second) is recorded. Even if the number of frames per frame) is large, it is possible to quickly find out later. As a result, it is possible to immediately analyze the situation at the time when there is a suspicion of the occurrence of an abnormality based on the video, thereby improving the reusability of the recorded video.

In particular, when the management server 20 simultaneously monitors a plurality of locations, that is, when the management server 20 collectively manages the information output from each room terminal 10 in a plurality of living rooms R, the information output from one room terminal 10 The number of recorded images in the image recording unit 21 is significantly increased compared to the case where only the information contained in the image is managed. Therefore, the configuration of this embodiment, which can improve the reusability of recorded images as described above, is particularly effective when the management server 20 simultaneously monitors a plurality of locations.

Further, the abnormality suspicion determination unit 12a recognizes the occurrence of abnormality based on the image acquired by the imaging unit 11a, and determines whether or not there is suspicion of the occurrence of abnormality (S1, S3). As a result, the image can be used not only for monitoring the detection target area, but also for determining suspicion of the occurrence of an abnormality, and the image can be effectively used.

Further, the abnormality determination determination section 12b recognizes the occurrence of abnormality based on the image acquired by the imaging section 11a, and determines whether or not the abnormality has occurred (S1, S5). As a result, the image can be used not only for monitoring the detection target area, but also for determining whether an abnormality has occurred, and the image can be effectively used.

In addition, since the timing information described above is the time information of the timing at which it is determined that there is a suspicion of the occurrence of an abnormality, the timing of the determination that there is a suspicion of the occurrence of an abnormality is clearly specified. As a result, when the occurrence of an abnormality is confirmed, it is possible to quickly search for the image recorded in the image recording unit 21 at the time when the occurrence of the abnormality is suspected, using the time information as a clue. .

Further, the detection target area is the living room R of the target person T whose living activity is to be supported. Then, the abnormality determination determination unit 12b determines whether there is an abnormality related to the behavior of the subject T in the living room R based on the above-mentioned intra-area information. As a result, it is possible to realize a care support system 1 that determines whether or not there is a suspicion of occurrence of an abnormality and confirms it regarding the behavior of the subject T in the living room R, and notifies the occurrence of the abnormality to the outside when it is confirmed. That is, in the care support system 1 constructed for the purpose of supporting the living activities of the subject T, the above-described effects of the present embodiment can be obtained.

In addition, the care support system 1 of the present embodiment includes a room terminal 10 installed in a living room R, a management server 20 that manages information acquired by the room terminal 10, and the above information. A notification terminal 30 for notifying the staff S is further included. The room terminal 10 includes an information acquisition section 11 , an abnormality suspicion determination section 12 a , an abnormality confirmation determination section 12 b , and a temporary storage section 13 . Also, the management server 20 includes a video recording unit 21 , and the notification terminal 30 includes an output unit 31 . In this case, in the care support system 1 including the room terminal 10, the management server 20, and the notification terminal 30, the effect of the present embodiment described above can be obtained.

Moreover, in the care support system 1, the abnormality related to the behavior of the subject T includes the subject T's fall. In this case, it is possible to obtain the above-described effects of the present embodiment in a configuration in which it is determined whether or not there is a suspicion of a fall within the living room R of the subject T, and when it is confirmed that the subject T has fallen, the outside is notified. .

[Other notification methods]
FIG. 9 is a flow chart showing another example of the flow of operations in the care support system 1 of this embodiment. The flowchart of FIG. 9 is the same as that of FIG. 7 except that S9 and S10 shown in FIG. 7 are replaced with S9' and S10'. The parts different from FIG. 7 will be described below.

In S5, after it is determined that there is a fall, in S6, the image (ID=0001) acquired including the time point of suspicion determination is associated with the suspicion determination time, the confirmed result, and the definite determination time that there is a fall. and held in the temporary holding unit 13 . Here, the temporary storage unit 13 includes an image temporally later than the timing of the suspicion determination in S3, and this includes the timing of the definite determination that there is a fall in S5. The image acquired at (confirmation determination time) is also included. Therefore, the information held in the temporary holding unit 13 is transmitted to the management server 20 via the notification management unit 14 and the communication line 40 in S7, and recorded in the video recording unit 21 of the management server 20 in S8. Then, the video recorded in the video recording unit 21 includes the video acquired by the imaging unit 11a at the timing when the occurrence of an abnormality is determined by the abnormality determination determination unit 12b (hereinafter also referred to as the video at the time of determination). It can be said that

In this case, the management server 20, under the control of the control unit 23, notifies the notification terminal 30 of the occurrence of an abnormality (that the subject T has fallen) from the application management unit 22, and also transmits the timing information for confirmation determination. Based on this, the confirmation video recorded in the video recording unit 21 is transmitted (S9'). At this time, the management server 20 acquires from the room terminal 10 the real-time image being captured by the imaging unit 11a, and transmits it to the notification terminal 30 together with the image at the time of confirmation or in place of the image at the time of confirmation. good. Then, the output unit 31 of the notification terminal 30 notifies the outside (for example, the staff S) of the occurrence of the abnormality by display or the like, and outputs at least one of the confirmation video and the real-time video (S10').

In such a notification method, the staff S, who is the owner of the notification terminal 30, sees the notification of the occurrence of an abnormality, grasps that an abnormality has occurred in the subject T, and outputs the confirmation time video and the real-time video. by looking at at least one of , and confirming the situation in the detection target area (living room R) at the time of confirmation or at present (for example, whether or not the abnormal state is still continuing), and can rush to the living room R.

In addition, the video captured including the timing of the determination of the suspicion of the occurrence of an abnormality in S3 (video at the time of suspicion) includes the video at the moment of the fall, and as described above, the situation before and after the actual fall. This image is necessary for later searching and confirming. On the other hand, the image acquired at the timing of determination of confirmation of the occurrence of an abnormality in S5 (image at the time of confirmation) is an image necessary for notifying the staff S of the situation at the time when the fall was confirmed in real time and having the staff S rush to the scene. In this respect, the purpose of use is different from that of the video in case of suspicion.

〔supplement〕
The temporary holding unit 13 of the room terminal 10 described above may be provided in the management server 20 , and the video recording unit 21 of the management server 20 may be provided in the room terminal 10 . Also, the room terminal 10 may include each component of the management server 20 .

An example of detecting a fall of the target person T as an abnormality in the behavior of the target person T has been described above. The anomaly detection system and anomaly detection method of this embodiment can also be applied when notifying the outside.

In the above description, the room R of the target person T is considered as the detection target area, and an abnormality in the behavior or posture of the target person T in the room R is detected and notified to the outside. is not limited to living room R. For example, in a plant such as a factory, an area where gas pipes are laid may be set as the detection target area. The anomaly detection system and anomaly detection method of the present embodiment can also be applied to detecting a gas leak in the above area and notifying it to the outside. In other words, the anomaly detection system and anomaly detection method of the present embodiment can be applied to detection targets other than anomalies in human behavior and posture.

Further, for example, the information acquisition unit 11 acquires a video (infrared image) acquired by the imaging unit 11a as a thermography (temperature distribution), thereby detecting an abnormality as to whether or not a fire has occurred in the living room R. can also Furthermore, the sound sensor as the sensor unit 11b can detect an impact sound (for example, a breaking sound of a window glass) in the living room R. As described above, the information acquisition unit 11 includes the imaging unit 11a and the sensor unit 11b, so that an abnormality other than the behavior of the target person T (abnormality other than a person) in the living room R is detected and notified to the outside. Also in this case, it is possible to apply the notification method described in this embodiment.

The abnormality suspicion determination unit 12a basically determines the suspicion of abnormality occurrence by detecting "change", and the abnormality confirmation determination unit 12b confirms the occurrence of abnormality by detecting "state". Although it is possible to make a judgment, for example, an impact sound is detected to determine the suspicion of an abnormality, and a weight distribution is detected by a pressure sensor to confirm the occurrence of an abnormality. It is also possible to use a different sensor to perform the confirmation determination of .

The room terminal 10, the management server 20, and the notification terminal 30 (especially the output unit 31) described in this embodiment may be configured integrally (standalone type), and may be separately arranged in the same living room R. may Therefore, the notification terminal 30 may be a terminal device installed in the living room R. Also, the management server 20 may be a client server or a cloud server.

Moreover, the effects described in the present embodiment are not restricted by the form and configuration of the system (standalone, client server, cloud server, etc.). Also, the output destination of the notification result may be the management server 20 (including an external server) instead of the portable individual terminal owned by the staff S (the notification terminal 30). Furthermore, the notification terminal 30 may be notified directly from the room terminal 10 without going through the management server 20 . Also, it is possible to provide the management server 20 with the function of the action recognition unit 12 of the room terminal 10 . Further, there may be a plurality of room terminals 10, management servers 20, and notification terminals 30, respectively. Further, the notification terminal 30 may be a terminal installed in an external building or company (for example, a security company, an insurance company, etc.) instead of being owned by the staff S.

Further, the care support system 1 described in this embodiment can be configured by a computer (PC) in which an abnormality detection program (application software) is installed (for example, stand-alone type). In this case, a computer (for example, the control unit 15 as a CPU) reads and executes the program, thereby operating each unit of the care support system 1 to execute each process (each process) described above. Such a program is acquired by external download via a network, for example, and stored in a storage unit (not shown). Further, the program is recorded in a computer-readable recording medium such as a CD-ROM (Compact Disk-Read Only Memory), and the computer reads the program from the recording medium and stores it in a storage unit. good too.

〔others〕
The anomaly detection system, the anomaly detection method, and the anomaly detection program of this embodiment described above can also be expressed as follows.

1. an information acquisition unit that acquires in-area information indicating a state within the detection target area, including an imaging unit that acquires an image by capturing a detection target area;
an abnormality suspicion determination unit that determines whether there is a suspicion of occurrence of an abnormality in the detection target area based on the area information;
an abnormality confirmation determination unit that confirms whether or not the abnormality has occurred based on the in-area information after the abnormality suspicion determination unit determines that the abnormality is suspected to occur;
an output unit for notifying the occurrence of the abnormality to the outside when the occurrence of the abnormality is confirmed by the abnormality confirmation determination unit;
When the abnormality suspicion determination unit determines that the occurrence of the abnormality is suspected, the image captured by the imaging unit temporally including the determination timing of the suspicion is captured at the determination timing. a temporary holding unit that temporarily holds the timing information in association with the indicated timing information;
acquiring the image and the timing information held in the temporary holding unit when the occurrence of the abnormality is confirmed by the abnormality confirmation determination unit, and adding the abnormality confirmation to the acquired image and the timing information; An anomaly detection system, comprising: a video recording unit that records a confirmation result by the determination unit in association with the image recording unit.

2. The abnormality suspicion determination unit is characterized by recognizing the occurrence of the abnormality based on the image acquired by the imaging unit as the area information, and determining whether or not the occurrence of the abnormality is suspected. 2. The anomaly detection system according to 1 above.

3. The above-mentioned 1, wherein the abnormality confirmation determination unit recognizes the occurrence of the abnormality based on the image acquired by the imaging unit as the in-area information, and determines whether or not the abnormality has occurred. 3. The anomaly detection system according to 2.

4. The video recorded in the video recording unit further includes a confirmation time video acquired by the imaging unit at the timing when the occurrence of the abnormality is confirmed by the abnormality confirmation determination unit,
4. Any one of 1 to 3 above, wherein the output unit notifies the occurrence of the abnormality to the outside and outputs at least one of the image at the time of confirmation and the real-time image being captured by the imaging unit. An anomaly detection system as described.

5. 5. The anomaly detection system according to any one of 1 to 4, wherein the timing information is time information of a timing at which the occurrence of the anomaly is suspected.

6. The detection target area is a living room of a target person who is a target of support for daily activities,
The abnormality suspicion determination unit determines whether or not there is a suspicion of occurrence of an abnormality related to the behavior of the subject in the living room based on the in-area information,
6. The abnormality according to any one of 1 to 5, wherein the abnormality confirmation determination unit determines whether or not an abnormality related to the behavior of the subject in the living room has occurred based on the area information. detection system.

7. further comprising: a room terminal installed in the living room and connected so as to be able to communicate with each other; a management server for managing information acquired by the room terminal; and a notification terminal for notifying the staff of the information;
the room terminal includes the information acquisition unit, the suspected abnormality determination unit, the abnormality confirmation determination unit, and the temporary storage unit;
The management server includes the video recording unit,
7. The anomaly detection system according to 6 above, wherein the notification terminal includes the output unit.

8. 8. The anomaly detection system according to 6 or 7 above, wherein the anomaly related to the behavior of the subject includes falling of the subject.

9. an information acquisition step of acquiring in-area information indicating a state within the detection target area by an information acquisition unit including an imaging unit that acquires an image by capturing an image of the detection target area;
an abnormality suspicion determination step of determining whether or not there is a suspicion of occurrence of an abnormality in the detection target area based on the area information;
When it is determined that there is a suspicion of occurrence of the abnormality, the image captured by the imaging unit temporally including the determination timing of the suspicion is associated with the timing information indicating the determination timing. a temporary holding step of temporarily holding in the temporary holding portion;
an abnormality confirmation determination step of determining whether or not the abnormality has occurred based on the area information after it is determined that the abnormality is suspected;
When it is determined that the abnormality has occurred, the image and the timing information held in the temporary storage unit are acquired, and the confirmation result of the occurrence of the abnormality is associated with the acquired image and the timing information. a recording step of recording in a video recording unit with
and an output step of notifying the occurrence of the abnormality to the outside when it is confirmed that the abnormality has occurred.

10. In the abnormality suspicion determination step, the occurrence of the abnormality is recognized based on the image acquired by the imaging unit as the area information, and whether or not there is a suspicion of the occurrence of the abnormality is determined. 9. The abnormality detection method according to 9 above.

11. 9. The above-mentioned 9, wherein in the abnormality confirmation determination step, the occurrence of the abnormality is recognized based on the image acquired by the imaging unit as the area information, and whether or not the abnormality has occurred is determined. 11. The anomaly detection method according to 10.

12. The video recorded in the video recording unit further includes a confirmation time video acquired by the imaging unit at the timing when the occurrence of the abnormality is confirmed in the abnormality confirmation determination step,
12. Any one of 9 to 11, wherein in the output step, the occurrence of the abnormality is notified to the outside, and at least one of the image at the time of confirmation and the real-time image being captured by the imaging unit is output. The described anomaly detection method.

13. 13. The anomaly detection method according to any one of 9 to 12, wherein the timing information is time information of a timing at which it is determined that the occurrence of the anomaly is suspected.

14. The detection target area is a living room of a target person who is a target of support for daily activities,
In the abnormality suspicion determination step, based on the area information, it is determined whether or not there is a suspicion of occurrence of an abnormality related to the behavior of the subject in the living room;
14. The abnormality according to any one of 9 to 13 above, wherein in the abnormality confirmation determination step, it is determined whether or not there is an abnormality related to the behavior of the subject in the living room based on the area information. Detection method.

15. 15. The abnormality detection method according to 14 above, wherein the abnormality related to the behavior of the subject includes falling of the subject.

16. 16. An anomaly detection program that causes a computer to execute the anomaly detection method according to any one of 9 to 15 above.

17. 17. A computer-readable recording medium recording the anomaly detection program according to 16 above.

Although the embodiments of the present invention have been described above, the scope of the present invention is not limited thereto, and can be implemented by being expanded or modified without departing from the gist of the invention.

INDUSTRIAL APPLICABILITY The present invention is applicable to an anomaly detection system, an anomaly detection method, and an anomaly detection program for detecting an anomaly in a detection target area.

1 Care support system (abnormality detection system)
10 room terminal 11 information acquisition unit 11a imaging unit 12a abnormality suspicion determination unit 12b abnormality confirmation determination unit 13 temporary storage unit 20 management server 21 video recording unit 30 notification terminal 31 output unit R living room (detection target area)
S staff T target person

Claims (16)

an information acquisition unit that acquires in-area information indicating a state within the detection target area, including an imaging unit that acquires an image by capturing a detection target area;
an abnormality suspicion determination unit that determines whether there is a suspicion of occurrence of an abnormality in the detection target area based on the area information;
an abnormality confirmation determination unit that confirms whether or not the abnormality has occurred based on the in-area information after the abnormality suspicion determination unit determines that the abnormality is suspected to occur;
an output unit for notifying the occurrence of the abnormality to the outside when the occurrence of the abnormality is confirmed by the abnormality confirmation determination unit;
When the abnormality suspicion determination unit determines that the occurrence of the abnormality is suspected, the image captured by the imaging unit temporally including the determination timing of the suspicion is captured at the determination timing. a temporary holding unit that temporarily holds the timing information in association with the indicated timing information;
acquiring the image and the timing information held in the temporary holding unit when the occurrence of the abnormality is confirmed by the abnormality confirmation determination unit, and adding the abnormality confirmation to the acquired image and the timing information; An anomaly detection system, comprising: a video recording unit that associates and records a determination result by the determination unit. The abnormality suspicion determination unit recognizes occurrence of the abnormality based on an image acquired by the imaging unit as the area information, and determines whether or not the occurrence of the abnormality is suspected. An anomaly detection system as described. 3. The apparatus according to claim 1, wherein the abnormality determination determination unit recognizes occurrence of the abnormality based on the image acquired by the imaging unit as the area information, and confirms whether or not the abnormality has occurred. An anomaly detection system as described. The video recorded in the video recording unit further includes a confirmation time video acquired by the imaging unit at the timing when the occurrence of the abnormality is confirmed by the abnormality confirmation determination unit,
4. The abnormality according to any one of claims 1 to 3, wherein the output unit notifies the occurrence of the abnormality to the outside and outputs at least one of the image at confirmation and the real-time image being captured by the imaging unit. detection system. 5. The anomaly detection system according to claim 1, wherein said timing information is time information of a timing at which said occurrence of said anomaly is suspected. The detection target area is a living room of a target person who is a target of support for daily activities,
The abnormality suspicion determination unit determines whether or not there is a suspicion of occurrence of an abnormality related to the behavior of the subject in the living room based on the in-area information,
The abnormality detection system according to any one of claims 1 to 5, wherein the abnormality confirmation determination unit determines whether or not an abnormality related to the behavior of the subject in the living room occurs based on the in-area information. further comprising: a room terminal installed in the living room and connected so as to be able to communicate with each other; a management server for managing information acquired by the room terminal; and a notification terminal for notifying the staff of the information;
the room terminal includes the information acquisition unit, the suspected abnormality determination unit, the abnormality confirmation determination unit, and the temporary storage unit;
The management server includes the video recording unit,
7. The anomaly detection system according to claim 6, wherein said notification terminal includes said output unit. The anomaly detection system according to claim 6 or 7, wherein the anomaly related to the behavior of the subject includes falling of the subject. an information acquisition step of acquiring in-area information indicating a state within the detection target area by an information acquisition unit including an imaging unit that acquires an image by capturing an image of the detection target area;
an abnormality suspicion determination step of determining whether or not there is a suspicion of occurrence of an abnormality in the detection target area based on the area information;
When it is determined that there is a suspicion of occurrence of the abnormality, the image captured by the imaging unit temporally including the determination timing of the suspicion is associated with the timing information indicating the determination timing. a temporary holding step of temporarily holding in the temporary holding portion;
an abnormality confirmation determination step of determining whether or not the abnormality has occurred based on the area information after it is determined that the abnormality is suspected;
When it is determined that the abnormality has occurred, the image and the timing information held in the temporary storage unit are acquired, and the confirmation result of the occurrence of the abnormality is associated with the acquired image and the timing information. a recording step of recording in a video recording unit with
and an output step of notifying the occurrence of the abnormality to the outside when it is confirmed that the abnormality has occurred. 10. The method according to claim 9, wherein, in said abnormality suspicion determination step, occurrence of said abnormality is recognized based on an image acquired by said imaging unit as said area information, and whether or not said abnormality is suspected to occur is determined. The described anomaly detection method. 11. The apparatus according to claim 9, wherein, in said abnormality confirmation determination step, occurrence of said abnormality is recognized based on an image acquired by said imaging unit as said area information, and whether or not said abnormality has occurred is determined. The described anomaly detection method. The video recorded in the video recording unit further includes a confirmation time video acquired by the imaging unit at the timing when the occurrence of the abnormality is confirmed in the abnormality confirmation determination step,
12. The abnormality according to any one of claims 9 to 11, wherein in the outputting step, the occurrence of the abnormality is notified to the outside, and at least one of the confirmation image and the real-time image being captured by the imaging unit is output. Detection method. 13. The abnormality detection method according to any one of claims 9 to 12, wherein said timing information is time information of a timing at which said occurrence of said abnormality is suspected. The detection target area is a living room of a target person who is a target of support for daily activities,
In the abnormality suspicion determination step, based on the area information, it is determined whether or not there is a suspicion of occurrence of an abnormality related to the behavior of the subject in the living room;
14. The abnormality detection method according to any one of claims 9 to 13, wherein, in said abnormality confirmation determination step, whether or not an abnormality related to the behavior of said subject in said living room has occurred is determined based on said intra-area information. 15. The abnormality detection method according to claim 14, wherein the abnormality related to the behavior of the subject includes falling of the subject. An anomaly detection program that causes a computer to execute the anomaly detection method according to any one of claims 9 to 15.

Images