JP2016067641A - Fall detection processing apparatus and fall detection system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enhance reliability in fall detection of a person.SOLUTION: A fall detection processing apparatus comprises: an acquisition unit for acquiring a distance image of a space of a fall detection target and an IR image from a TOF sensor; a determination unit for determining a size of the target, a motion of the target for a predetermined period of time, and a pattern, color, or gradation of the target based on the acquired distance image and IR image; and an estimation unit for estimating whether the motion of the target is falling of a person according to the size of the target, the motion of the target for the predetermined period of time, and the pattern, color, or gradation of the target, which have been determined.SELECTED DRAWING: Figure 7

Description

  The present invention relates to a fall detection processing apparatus and a fall detection system.

In an aging society, there is a high demand for an environment that allows elderly people in nursing homes and elderly people who need home care to be healthy and live safely. For example, in the elderly, minor injuries and illnesses can have a significant impact on later life. For this reason, when an elderly person falls, it is necessary to notify the family, a caregiver, and other related parties immediately. Then, the method of detecting automatically and reporting the fall of an elderly person is proposed (for example, refer patent documents 1-3).
JP 2011-113406 A JP 2009-268854 A JP 2002-232870 A

  However, in the detection method described in the above patent document, it is difficult to distinguish a person from other moving objects due to the characteristics and arrangement of sensors used to determine whether a person has fallen. It may be difficult to judge. If the result of a person's fall is not correct, it will be misreported, and the trust of the system will eventually be suspected and not used.

  In view of the above problem, in one aspect, the present invention aims to improve the reliability of human fall detection.

  In order to solve the above-mentioned problem, according to one aspect, based on the acquired distance image and the IR image, an acquisition unit that acquires a distance image and an IR image of a space to be detected from a TOF sensor, A determination unit configured to determine the size of the target object, movement of the target object for a predetermined period of time, a pattern, a color, or a shade of the target object; a size of the determined target object; and a predetermined time of the target object A fall detection processing device is provided that includes a motion and an estimation unit that estimates whether the motion of the object is a person's fall according to the pattern, color, or shade of the object.

  According to one aspect, it is possible to improve the reliability of human fall detection and reduce false alarms.

The figure which shows an example of the fall detection system which concerns on one Embodiment. The figure which shows an example of a function structure of the fall detection processing apparatus which concerns on one Embodiment. The figure which shows an example of the notification destination registration table which concerns on one Embodiment. The figure which shows an example of the log information table which concerns on one Embodiment. The figure which shows an example of the characteristic / health information table which concerns on one Embodiment. The figure which shows an example of the exception information table which concerns on one Embodiment. The figure for demonstrating the conditions of the fall detection which concerns on one Embodiment. The flowchart which shows an example of the fall detection process which concerns on one Embodiment. The figure for demonstrating the fall detection which concerns on one Embodiment. The flowchart which shows an example of the monitoring process which concerns on one Embodiment. The figure which shows an example of the fall detection system of the fall detection processing apparatus which concerns on one Embodiment.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. In addition, in this specification and drawing, about the substantially same structure, the duplicate description is abbreviate | omitted by attaching | subjecting the same code | symbol.

[Overall configuration of fall detection system]
First, a fall detection system 1 according to an embodiment of the present invention will be described with reference to FIG. The fall detection system 1 according to the present embodiment is a space in which elderly persons who are to detect falls in a care facility A where the elderly live or an elderly home (home care home B) that requires home care, etc. Applies to In the fall detection system 1 according to the present embodiment, a TOF (Time Of Flight) sensor 9 is used in the care facility A or the home care home B as a sensor for photographing an object to be detected for fall. The TOF sensor 9 is installed on, for example, a ceiling portion of a space (for example, an elderly person's room) to detect a fall, and takes a distance image, an IR (Infrared) image, and a color image of the space. The TOF sensor 9 has a projector function for projecting infrared rays and a camera function. A color image or a black and white image can be taken using the camera function of the TOF sensor 9.

  The TOF sensor 9 projects infrared rays. The TOF sensor 9 detects the distance of the object from the time difference between the output infrared light and the infrared light that reflects back from the object. In this way, the TOF sensor 9 can take a distance image by measuring the distance from the time required for the infrared light to travel back and forth to the target. Further, the TOF sensor 9 can project an IR image by projecting infrared rays.

  The TOF sensor 9 is installed in a target space for detecting a fall, such as a bathroom in which a person is likely to fall or a common room where people are likely to gather. The TOF sensor 9 is installed on a ceiling part or a side wall part in a space where a fall is detected. The TOF sensor 9 may be placed in any space in which a fall is detected. However, if the TOF sensor 9 is placed at the center of the ceiling in the space, the range of objects that can be detected by the TOF sensor 9 is increased (see FIG. 7). When the TOF sensor 9 is placed on the side wall in the space, the range of objects that can be detected by the TOF sensor 9 is narrowed and the number of obstacles increases, making it difficult to detect the object. A plurality of TOF sensors 9 may be installed.

  The TOF sensor 9 is connected to the fall detection processing device 10 via a USB (Universal Serial Bus). The fall detection processing device 10 processes the distance image and the IR image acquired from the TOF sensor 9, and determines whether or not the object is a person based on these images, and determines that the object is a person. In this case, it is estimated whether the movement is a person's fall. The fall detection processing device 10 notifies a predetermined device when it is estimated that the movement of the object is a fall of a person. The predetermined device may be a device in the space where the TOF sensor 9 is installed (including the fall detection processing device 10), or a device that is registered in advance as a notification destination and can communicate with the network 30. Good. FIG. 1 shows a smartphone 40, a fixed telephone device 41, a PC (desktop, notebook, tablet, etc.) 42, and a mobile phone device 50 as examples of report destination devices that can communicate via the network 30. However, the notification destination device is not limited to these devices, and may be a household electrical appliance capable of communication, a navigation device mounted on an automated person, or other electronic devices. In the case of the fixed telephone device 41, only voice data may be notified at the time of falling notification. The owner of the notification destination device can be, for example, an elderly family member, a relative, or an elderly caregiver who lives in a space that is subject to fall detection.

  One or a plurality of the fall detection processing devices 10 may be provided in the care facility A or the home care home B. The fall detection processing device 10 is connected to the monitoring server 20 via the network 30. For example, the fall detection processing device 10 may be connected to the monitoring server 20 using WiFi, 3G (3rd. Generation), or Bluetooth. The monitoring server 20 may be a cloud server.

  When the fall detection processing device 10 determines that a person has fallen, the fall detection processing device 10 notifies the notification destination devices 40 and 50 and the monitoring server 20. The monitoring server 20 stores image data and sound data of a target space such as a care facility A, and of the stored data, at least any of image data and sound data including the state of the fall time of the person who estimated the fall Is transmitted to a device registered in advance as a notification destination. Thereby, for example, the owner of the smartphone 40 that is the notification destination can determine the situation of the person who has fallen, and can quickly deal with the person who has fallen. The monitoring server 20 may be arranged at an emergency center or the like. Thereby, medical practices such as arranging an ambulance for a fallen person and taking care of an injury can be quickly performed.

  Among image data and audio data, there is data that is not appropriate to be transferred from the viewpoint of protection of personal information or copyright. In particular, since a visible image is easier to identify a person than a distance image, it may not be suitable as data to be transferred to a notification destination. Therefore, when transferring the image data to the notification destination, it is preferable not to transmit the visible image, but to transmit the IR image or the IR image after performing processing so that it is not easy to identify a person. A distance image may be transmitted together with the IR image. Further, a visible image may be transmitted to a specific notification destination such as a family registered in advance.

  Information that a person has fallen can be reported to the smartphone 40, the mobile phone device 50, and the like owned by a close relative of the elderly by a known communication means. As publicly known communication means, a public telephone line network, an Internet communication network, or the like can be used as appropriate.

  In the conventional fall detection, it is not always possible to correctly detect a person's fall. For example, there is an error in the person's fall detection, such as detecting a fall of a stationary object as a person's fall. Conventional camera fall detection detects a moving body such as an elderly person based on image data acquired by the camera, and determines that there is a possibility of falling detection when the moving body stays in the same place for a certain period of time. Was.

  However, in this fall detection, it was difficult for the system to distinguish whether a “person” fell or an object fell. For this reason, in the conventional technology, there have been many cases of notifying a caregiver of misinformation regarding the fall of an elderly person. In contrast, the fall detection processing device 10 according to the present embodiment can improve the accuracy of the fall detection in the care facility A or the home care home B based on the distance image and the IR image acquired from the TOF sensor 9. . Thereby, it is possible to reduce the notification of the misinformation regarding the fall of the elderly person to the assistant, and to provide the elderly person and his assistant with the effective fall detection system 1. As a result, it is possible to realize an environment in which an elderly person in a care facility or an elderly person who needs home care can live in a healthy and safe manner. Hereinafter, functions and operations of the fall detection processing apparatus 10 capable of accurately detecting fall detection will be described in order.

[Functional configuration of the fall detection processing device]
An example of the functional configuration of the fall detection processing apparatus 10 according to an embodiment of the present invention will be described with reference to FIG. The fall detection processing apparatus 10 according to the present embodiment includes a detection unit 11, an acquisition unit 12, a storage unit 13, a determination unit 14, an estimation unit 15, a notification unit 16, a communication unit 17, a recording unit 18, and a voice recognition unit 19. .

  The detection unit 11 is connected to the TOF sensor 9, detects the object using the TOF sensor 9, and obtains a distance image and an IR image of the object. The detection unit 11 is separately connected to a sensor for obtaining a distance image and a sensor for taking an IR image instead of the TOF sensor 9, detects the object using each sensor, and detects the distance image of the object. And an IR image may be obtained.

  The acquisition unit 12 acquires a distance image and an IR image in the space to be detected from falling from the detection unit 11.

  The storage unit 13 stores predetermined information in the notification destination registration table 21, log information table 22, feature / health information table 23, and exception information table 24. As illustrated in FIG. 3, the notification destination registration table 21 stores notification destination device information 212 in association with an ID 211 indicating a person who is to be detected to fall. The notification destination device information 212 is, for example, a device possessed by the family, relatives, and caregivers of the person identified by the ID 211. For example, when the fall of the person whose ID 211 is “M1” is estimated, the fall notification is transmitted to the devices a and b. When the fall of the person whose ID 211 is “M2” is estimated, the fall notification is transmitted to the device c. When the fall of the person whose ID 211 is “M3” is estimated, the fall notification is transmitted to the device d and the device e. The information stored in the notification destination registration table 21 is transferred to the monitoring server 20 and stored in the monitoring server 20. The fall notification is transmitted to the monitoring server 20 in addition to the notification destination device registered in the notification destination registration table 21 in this way.

  The log information table 22 stores log information such as distance images and IR images acquired by the acquisition unit 12. The recording unit 18 acquires sound from a speaker (see the speaker 8a in FIG. 7) attached to the target space. The voice recognition unit 19 performs voice recognition processing on the voice acquired from the speaker 8a. The data after the voice recognition process is hereinafter also referred to as “voice data”.

  The storage unit 13 may store the audio data in the log information table 22. In the log information table 22, as shown in an example in FIG. 4, the acquisition time information 221 stores the date and time when the image data and audio data were acquired, and the acquisition location information 222 stores the location where the image data and audio data were acquired. Is done. In the log information table 22, image data and audio data are stored in the image / audio information 223. As image data stored in the image / audio information 223, at least one of a distance image and an IR image is stored. In addition to this, visible image data may be stored in the image / audio information 223. For example, image data and audio data taken in a room where the acquisition time information 221 is “2014.9.10.10:50” and the acquisition location information 222 is “(care facility) A1” are image / audio. 1 is stored in the image / audio information 223 of the log information table 22. Thereafter, images and sounds are taken at the same place “A1” at intervals of 10 seconds, and stored in the image / audio information 223 as image / audio data 2 to 5. In the log information table 22, in the room of the other acquisition location information “(care facility) A 2” indicated by the acquisition location information 222, the acquisition time information 221 “2014.9.1.10.14: 00” is recorded. Audio data 6 is stored in the image / audio information 223. As described above, the log information table 22 stores at least one of the image data and the sound data acquired at the time and location indicated by the acquisition time information 221 and the acquisition location information 222. Information stored in the log information table 22 is transferred to the monitoring server 20 and stored in a storage area of the monitoring server 20 or a storage area on the cloud. The information stored in the log information table 22 of the fall detection processing device 10 may be deleted after a predetermined time has elapsed since being stored.

  The feature / health information table 23 stores information related to the feature and health status of the person in association with the ID 231 indicating the person who is to detect the fall. The feature / health information table 23 stores an ID 231, height / weight 232, clothes (pattern, color, shading) 233, walking speed 234, and health condition information 235 as shown in FIG. 5 as an example. For example, in the feature / health information table 23, “150 cm / 40 kg” for the height / weight 232 and “red” for the clothes (pattern, color, shading) 233 as information on the feature and health status of the person whose ID 211 is “M1”. ”Polka dot pattern”, “2 km / h” is stored in the walking speed 234, and “initial dementia” is stored in the health state information 235. In the feature / health information table 23, in addition to the items shown in FIG. 5, sex, operation pattern, and other features (whether glasses are worn, etc.) can be stored. In the feature / health information table 23, not only the pattern, color, and shade of the clothes of the object, but also the pattern (including clothes) of the object in the state of wearing shoes, a hat, glasses, and the like. Color, shading information may be stored.

  The exception information table 24 stores condition information that is excluded from an object for detecting a fall even if it is a moving object that exists in the space for which the fall is detected. As shown in FIG. 6, the exception information table 24 stores information on an exception area 242 and an exception time 243 in association with an ID 241 that indicates a facility or a house whose fall is to be detected. For example, when the ID 241 is “care facility A”, an object that exists in “the place where the bed numbers 1 to 20 are placed” set in the exception area 242 is excluded from the objects for detecting the fall. The exception time 243 to which this exclusion condition is applied is stored as “all time”. That is, when the person in the place where the numbers 1 to 20 of the beds in the care facility A are placed goes to the sleeping state, the exception information table 24 falls over its operation at all times. Not detected.

  In addition, when the ID 241 is “care facility A”, an object existing in the “tatami room” set in the exception area 242 is excluded from objects to be detected to fall. The exception time 243 to which this exclusion condition is applied is “9:00 PM to 8:00 AM”. In other words, when the person in the tatami room in the care facility A changes from the sitting state to the sleeping state, the exception information table 24 performs the operation from 9 o'clock in the evening to 8 o'clock in the morning. Does not detect a fall.

  Based on the acquired distance image and IR image, the determination unit 14 determines the size of the object existing in the space where the fall is detected, the movement of the object for a predetermined time, and the pattern, color, or shade of the object. Determine. In the following, as an example, the clothing (pattern, color, shading) of the object is compared with the image, but not only the clothing, pattern, color, shading of the object but also wearing items such as shoes, hats, and glasses. The pattern, color, and shade of the object (including clothes) in the state can be used as comparison targets.

  The determination unit 14 acquires in advance image data indicating the state of the room when there is no movement based on the distance image. And the determination part 14 analyzes the image data which shows the state of the room when there is no movement, and the image data which shows that the state of the interior of the room has changed by a so-called background difference method, Distinguish from animals (including humans). Thereby, the determination unit 14 can determine the size of the object and the movement of the object for a predetermined time. Further, the determination unit 14 determines the pattern, shade, color, and shade difference between the top and bottom of the clothing and the color difference based on the IR image.

The estimation unit 15 determines that the movement of the object is a person's fall according to the size of the determined object, the movement of the object for a predetermined time, and the clothes (pattern, color, or shade) of the object. Estimate. At that time, the estimation unit 15 estimates that the person has fallen when all the conditions (1) to (3) are satisfied based on the distance image and the IR image.
(1) The size of the target object is greater than or equal to a predetermined value. For example, when the size of the target object determined as an animal is equal to or greater than the predetermined value (for example, 1 m or greater), the target object may be a human being. Presume that there is. For example, since the size of the object OJ1 shown in FIG. 7 is 1 m or more, it is estimated that the object OJ1 may be human. On the other hand, since the size of the object OJ3 is less than 1 m, it is determined that the object OJ3 is not a person. The threshold for estimating that the size of the object determined to be an animal is the size of a person is not limited to “1 m”. For example, a case of a man, a case of a woman, and a case of a child are set separately. Also good.

When the TOF sensor 9 is arranged on the ceiling, as shown by a dotted line in FIG. 7, the distance image acquired by the TOF sensor 9 and the upper side of the object to be captured in the IR image become closer to the center of the room depending on the viewing angle. It is cut and invisible. For example, the upper body of the object OJ2 at the end of the room is not copied to the distance image and the IR image acquired from the TOF sensor 9. In this case, since the size of the object OJ2 on the image data is less than 1 m, it is determined that the object OJ2 is not a person. From the above, in the present embodiment, the threshold value for determining that the size of the object is a person is set by changing as shown in FIG. 7 according to the viewing angle. According to this, the threshold value for the object located on the end side of the room is smaller than the threshold value for the object located on the center side of the room. Thereby, since the magnitude | size of the target object OJ2 becomes the same value as a threshold value, the estimation part 15 can estimate that the target object OJ2 may be a person.
However, since the upper body of a person is cut off from the image data and cannot be seen at the end of the room, it is difficult to distinguish it from animals. Whether the object OJ2 is actually a person or a moving body other than a person is difficult to estimate only under the condition (1). Even in the center of the room, it is easy to make an error if it is estimated whether the object is a person or a moving body other than the condition (1), and it is difficult to improve the accuracy of detecting a person's fall. Therefore, in the present embodiment, the estimation unit 15 estimates that a person has fallen when not only satisfying the condition (1) but also satisfying the following condition (2).
(2) An object that moves at or above a predetermined height satisfying (1) has shifted to a low state and has not moved for a predetermined time or has moved less for a predetermined time. When an object that moves above a certain height that satisfies () goes to a low state and does not move for a specified time or moves less for a specified time, the object is a person and the person falls It is estimated that For example, the estimation unit 15 is based on the height of a person at the time of an everyday movement in which a target object having a height of 1 m or more that has not been stationary for a predetermined time or longer is walking, standing, or sitting from a moving state. In a low state (for example, 1 m or less) for a certain period of time (for example, 10 to 15 seconds or more), or when a certain period of time (for example, 10 to 15 seconds) or more has passed after moving to the low state, It is estimated that the object has changed from a moving state to a non-moving state or a state with little movement.

  However, if the movement again becomes larger than the predetermined moving place after changing to a state that does not move or moves less, the estimation unit 15 estimates that the person has returned to the normal state, and the person falls Withdraw the estimate.

  The essential requirements for determining that a person has fallen have been described above. Further, the following (3) to (6) are not indispensable conditions for determining that a person has fallen, but for detecting whether a person has fallen in order to further improve the accuracy of fall detection, can do.

(3) The clothing (pattern, color, or shade) of the object matches or resembles pre-registered clothing (pattern, color, or shade). The estimation unit 15 satisfies the conditions of (1) and (2) Even if it is estimated that a person has fallen if the clothes (patterns, colors, or shades) of the object that satisfy the condition are at least partially matched or similar to pre-registered clothes (patterns, colors, or shades) Good. The estimator 15 uses the information on the height / weight 232, clothes (pattern, color or shade) 233, walking speed 234, and health condition information 235 stored in the feature / health information table 23 to determine the clothes ( It may be determined whether the pattern (color, color, or shade) matches or resembles pre-registered clothes (pattern, color, or shade). For example, as an example of the condition of (3), the clothes (pattern, color, or shade) of the object extracted from the IR image are the same as the clothes (pattern, color, or shade) that are worn in the care facility. It is mentioned that it is coincident or similar in the upper and lower parts.

(4) The object consists of three parts (head, torso, legs) The estimation unit 15 assumes that a person has fallen when the object consists of three parts (head, torso, legs). It may be estimated. This makes it possible to form a temporary proof that the fallen moving object is a person.

(5) No other animal features (for example, no tail) The estimation unit 15 may estimate that a person has fallen if there is no other animal feature (for example, no tail). Thereby, a temporary proof that the fallen moving body is an animal other than a human can be formed.

(6) The amount of activity is small or absent The estimation unit 15 is based on the walking speed 234 stored in the feature / health information table 23 and other information storing daily activities, and the walking speed of the object is slow. If it is determined that the amount of activity is small or not, it may be estimated that the fall is a sign. Further, it may be possible to shorten the time for determining that the vehicle falls from this sign.

  As described above, the estimation unit 15 can improve the accuracy of human fall detection by grasping the health state, height, weight, and the like and estimating the human fall based on the IR image. For example, when the estimation unit 15 determines that the walking speed of the object has slowed based on the walking speed 234 stored in the feature / health information table 23, the estimation unit 15 is in a state in which it is likely to fall over due to poor health. It may be estimated that If the estimation unit 15 detects that the person who has fallen has a bad health condition based on the walking speed 234 or the health condition information 235 stored in the feature / health information table 23, the timing of the person's fall notification is advanced. You may request | require to the report part 16 as follows.

  The estimation unit 15 excludes the objects detected in the exception area and the exception time stored in the exception information table 24 from the objects for detecting the fall. Thereby, for example, it is possible to prevent an elderly person from presuming that a person falls on a bed or a tatami room with a futon lying on the bed or a relaxing action is a person falling.

  When it is estimated that the movement of the object is a person's fall, the reporting unit 16 indicates the state of the fall time of the person who estimated the fall out of the log information stored in the log information table 22 by the storage unit 13. Report at least one of image data including data and audio data to a predetermined device. For example, the reporting unit 16 may transmit at least one of image data and audio data including the fall time of the person who estimated the fall and the states before and after the fall time. For example, when the fall of a person is estimated at the time when the acquisition time information 221 is “2014.9.10.11:10” among the log information stored in the log information table 22 of FIG. Transmits data including image / sound data 3 when it is estimated that a fall has occurred. Thereby, the notified person can determine whether or not the person has fallen. In addition, the notified person can promptly instruct the monitoring server 20 and other appropriate medical institutions what to do when it is determined that the person has fallen.

  In a case where the reporting unit 16 transmits at least one of image data and audio data including the fall time of the person who estimated the fall and the states before and after the fall time, the acquisition time information 221 is “2014.9.10.11: The image / sound data 3 at the time “10” and the image / sound data before and after that may be transmitted. As a result, the person who has been notified falls down based not only on the image / sound data 3 at the time of the fall, but also on the previous image / sound data 1, 2 and the subsequent image / sound data 4, 5 Whether or not can be determined more easily. As a result, the notified person can know not only the state at the time of the fall but also the state before the fall and the state after the fall, and the situation at the time of the fall can be easily grasped, so that an accurate response is promptly performed. It becomes possible. Based on the feature / health information table 23, the reporting unit 16 may notify health information 235, clothing information 233, and the like stored in association with the ID 231 of the fallen person in addition to the above data.

  Note that the predetermined device that is the notification destination of the fall detection notified by the reporting unit 16 may be at least one of a room at the site where the fall occurred and a notification destination registered in advance. The reporting unit 16 may make it possible to make a call with the device that has been reported at the same time as the report of the fall, and check whether there is a response that the call is safe. In addition, the fall detection processing device 10 may determine the notification destination for reporting the fall of a person based on the notification destination registration table 21. Moreover, the monitoring server 20 that has received the notification from the reporting unit 16 may determine the notification destination for reporting the fall of a person, and notify the necessary information from the monitoring server 20.

  The communication unit 17 transmits necessary information and reports to the monitoring server 20, the notification destination smartphone 40 and the mobile phone device 50, and receives predetermined information from the monitoring server 20.

[Fall detection processing]
Next, control and processing in the fall detection processing device 10 of the present embodiment configured as described above will be described. FIG. 8 shows a flowchart of the fall detection process executed by the fall detection processing apparatus 10 according to an embodiment of the present invention.

  When this process is started, the acquisition unit 12 acquires a distance image and an IR image using the TOF sensor 9 (step S10). The acquisition unit 12 may acquire a color image or a black and white image. Next, the determination unit 14 determines the size of the target object, the movement of the target object, and the pattern, color, and shade of the target object based on the distance image and the IR image. When the target is a moving object having a height equal to or higher than the threshold based on the size of the target extracted from the distance image, the estimating unit 15 estimates the target as a person, and a moving object having a height lower than the threshold (for example, a pet). (Step S12). The estimation unit 15 detects a change in the height direction of the object estimated to be a person (step S12).

  Next, the estimation unit 15 estimates the state of the person who has fallen from the distance between the TOF sensor 9 in the fall state and the object based on the distance image at the time when the fall occurred and the distance image acquired thereafter, The size and operation of the object are confirmed (step S14). As a result, the estimation unit 15 excludes an object estimated as a stationary object or an object estimated as a moving body other than a person such as a pet from a person's fall detection target (step S14). For example, in accordance with the condition (1), the estimation unit 15 estimates that the object is a person when the object is equal to or greater than a predetermined threshold (such as 1 m), and if the object is less than the threshold, the object is Presumed to be a non-human moving body. Moreover, the estimation part 15 detects the change of the height direction of a target object according to the conditions of said (2), and estimates whether a target object is a person's fall by this.

  Next, the estimation unit 15 confirms the IR image, and confirms the appearance features of the object such as the clothes of the object, the shade difference of the clothes, the shade difference between the top and bottom of the clothes, the color difference of the clothes, and the difference of the pattern. (Step S16). For example, the estimation unit 15 confirms the feature on the appearance of the object extracted from the IR image based on the information stored in the feature / health information table 23 according to the condition (3). It is estimated whether the person is a person and whether the person has fallen. The conditions (1) and (2) are indispensable requirements for determining human fall detection, but the condition (3) must not be an indispensable requirement for determining human fall detection. Also good. That is, step S16 can be omitted in the fall detection process according to the present embodiment.

  Next, the estimation unit 15 determines whether or not the movement for a certain period of time after estimating the fall of a person is less than a predetermined threshold (step S18). If the estimation unit 15 determines that the person's fall has been equal to or greater than a predetermined threshold after estimating the person's fall, the estimator 15 determines that no person has fallen and ends this process. On the other hand, when the estimation unit 15 determines that the movement for a certain period of time after estimating a human fall is less than a predetermined threshold, the determination unit 14 detects a moving object having a height equal to or higher than a threshold (for example, 1 m). Is determined (step S20).

  If the determination unit 14 determines that a moving object having a height equal to or higher than a threshold (for example, 1 m) has been detected, the determination unit 14 determines that the person who has fallen has returned, and ends the process. on the other hand. If the determination unit 14 determines that a moving object having a height equal to or higher than a threshold (for example, 1 m) has not been detected, the notification unit 16 reports a person's fall (step S22), and ends the process.

  For example, when reporting the fall of a person, the reporting unit 16 may automatically ask the speaker 8a shown in FIG. 7 "Are you all right?" In response to this, when a person shows a response such as “OK”, “NO PROBLEM”, “HELP”, etc., the microphone 8b catches this response or no response. At the time of reporting, the reporting unit 16 transmits a voice file storing the above question and an answer thereto to the notification destination smartphone 40 or the like. By putting information that can grasp the situation at the site in this way, the person concerned can grasp the degree of urgency and can efficiently respond. Further, it may be automatically determined that there is no problem by utilizing a voice recognition function or the like for the response. As a result, the accuracy of human fall detection can be improved, and false alarms can be reduced to alleviate concerns of related parties.

  As described above, according to the fall detection system 1 of the present embodiment, it is determined based on the distance image and the IR image whether or not the fall detection processing device 10 satisfies all the conditions for determining a person's fall. . Thereby, the fall detection system 1 which raised the fall reliability of the person and reduced the misinformation can be provided.

  In particular, according to the fall detection system 1 of the present embodiment, the distance image and the IR image are obtained using the TOF sensor 9. As a result, since there is no positional shift of the object captured by the distance image and the IR image, it is possible to accurately estimate whether the object is a person and whether the person has fallen based on these images. Can do. For example, as shown in FIG. 9, based on the distance image and the IR image obtained using the TOF sensor 9, the part of the person who fell and the fall state and the appearance characteristics of the person (clothing pattern, color, shading, etc. ) Is extracted, it is possible to improve the accuracy of estimation that a person has fallen based on the extracted image data. Thereby, the misinformation about a person's fall can be reduced.

[Monitoring process]
Next, control and processing in the monitoring server 20 of this embodiment will be described. FIG. 10 shows a flowchart of the monitoring process executed by the monitoring server 20 according to the embodiment of the present invention.

  When this process is started, the monitoring server 20 determines whether a report from the fall detection processing device 10 has been received (step S30). If the monitoring server 20 determines that the report from the fall detection processing device 10 has not been received, the process ends.

  On the other hand, when the monitoring server 20 determines that the report from the fall detection processing device 10 has been received, the image data and the voice data at the time when the person's fall is estimated and the image data and the voice data before and after that are notified to the notification destination. Transmit (step S32).

  Next, the monitoring server 20 determines whether an instruction for a person's falling or a predetermined time has elapsed (step S34). If the monitoring server 20 receives an instruction for a person's fall or determines that a predetermined time has elapsed, the monitoring server 20 ends the process. Otherwise, the process returns to step S32, and the determination process in step S34 indicates “ The processes of steps S32 and S34 are repeated until “Yes” is determined.

  As described above, according to the fall detection system 1 of the present embodiment, the monitoring server 20 detects the image data and audio data at the time when the fall occurred in response to the report from the fall detection processing device 10 and the data before and after that. Transmit image data and audio data. As a result, the notified person obtains not only the image data and audio data when the fall occurs, but also information on the person's fall based on the previous image data and audio data, and the subsequent image data and audio data. . As a result, the notified person can know not only the state of the person at the time of the fall but also the state of the person before the fall and the state of the person after the fall, and the situation at the time of the fall of the person becomes easy to understand. It is possible to perform an accurate response promptly.

  However, in step S32, the time when the person's fall was estimated and the image data and sound data before and after the time were transmitted to the notification destination, but the present invention is not limited to this, and the image data and sound at the time when the person's fall was estimated. Data including at least one of the data may be transmitted to the notification destination.

(Method to complement fall detection)
As described above, the fall detection method performed by the fall detection processing device 10 according to the present embodiment has been described. However, the following method that complements the fall detection method and the fall detection method can be combined. Thereby, the precision of fall detection can be raised more.

  For example, the fall detection processing device 10 may calculate the heart rate of a person estimated to have fallen from the image data acquired by the acquisition unit 12. Thereby, the situation of the person who falls from the heart rate can be known in more detail. For example, the information about the heart rate of the person who falls can be notified to the notification destination.

  For example, when the acceleration sensor is arranged on the bed, the acquisition unit 12 acquires the triaxial acceleration detected by the acceleration sensor. In this case, the fall detection processing device 10 can obtain information on the heart rate and physical motion of the fallen person based on the triaxial acceleration. Thereby, it is possible to estimate in detail whether or not the person has fallen from the heart rate and the movement of the body, and to know in more detail the situation of the person who has been estimated to have fallen.

  Furthermore, for example, when a person who falls down possesses a vibration sensor, the acquisition unit 12 acquires a vibration value detected by the vibration sensor. In this case, the fall detection processing device 10 can determine whether vibration has occurred when a person falls based on the vibration value.

  As described above, the fall detection processing device 10 according to the present embodiment can further improve the accuracy of human fall by complementing the fall detection method of the present embodiment with the above method.

(Example of fall detection system)
Finally, an example of the fall detection system of the fall detection processing apparatus 10 according to the present embodiment will be described with reference to FIG. FIG. 11 is a diagram illustrating a fall detection system example of the fall detection processing device 10 according to the present embodiment.

  The fall detection processing device 10 includes an input device 101, a display device 102, an external I / F 103, a RAM (Random Access Memory) 104, a ROM (Read Only Memory) 105, a CPU (Central Processing Unit) 106, a communication I / F 107, and An HDD (Hard Disk Drive) 108 is provided. Each part is connected to each other by a bus B.

  The input device 101 includes a keyboard and a mouse and is used to input each operation signal to the fall detection processing device 10. The display device 102 includes a display and displays various processing results.

  The communication I / F 107 is an interface that connects the fall detection processing device 10 to a network. Thereby, the fall detection processing device 10 can perform data communication with the monitoring server 20 and the notification destination device via the communication I / F 107.

  The HDD 108 is a non-volatile storage device that stores programs and data. The stored programs and data include basic software and application software that control the entire apparatus. For example, the HDD 108 stores various DB information, programs, and the like.

  The external I / F 103 is an interface with an external device. The external device includes a recording medium 103a. Thereby, the fall detection processing device 10 can read and / or write the recording medium 103 a via the external I / F 103. Examples of the recording medium 103a include a CD (Compact Disk), a DVD (Digital Versatile Disk), an SD memory card, a USB memory (Universal Serial Bus memory), and the like.

  The ROM 105 is a nonvolatile semiconductor memory (storage device) that can retain internal data even when the power is turned off. The ROM 105 stores programs and data such as network settings. The RAM 104 is a volatile semiconductor memory (storage device) that temporarily stores programs and data. The CPU 106 is an arithmetic unit that realizes control of the entire apparatus and mounting functions by reading programs and data from the storage device (for example, “HDD 108”, “ROM 105”, etc.) onto the RAM 104 and executing processing.

  With the above-described fall detection system, the fall detection processing device 10 according to the present embodiment can perform a fall detection process. For example, the CPU 106 executes a fall detection process using data and programs stored in the ROM 105 and the HDD 108. As a result, in the present embodiment, it is possible to provide the owner of a device registered in advance that a person has fallen. The information in the notification destination registration table 21, log information table 22, feature / health information table 23, and exception information table 24 is stored on the cloud connected to the fall detection processing device 10 via the RAM 104, the HDD 108, or the network 30. It can be stored in a server or the like.

  As mentioned above, although the fall detection processing device and the fall detection system have been described in the above embodiment, the fall detection processing device and the fall detection system according to the present invention are not limited to the above embodiment, and various within the scope of the present invention. Modifications and improvements are possible. The matters described in the above embodiments can be combined within a consistent range.

  For example, the configuration of the fall detection system according to the above-described embodiment is an example, and does not limit the scope of the present invention, and it goes without saying that there are various system configuration examples depending on applications and purposes. For example, the system configuration in which the fall detection processing device 10 and the monitoring server 20 are connected to each other via the network 30 is an aspect of the fall detection system according to the present embodiment, and is not limited thereto. For example, the number of monitoring servers 20 included in the fall detection system according to the present embodiment may be one or two or more. When a plurality of monitoring servers 20 are installed, the notification processing to the notification destination can be distributed by the plurality of monitoring servers 20. Note that these processing functions may be selectively aggregated in one monitoring server 20 among a plurality of units according to the purpose and purpose.

  Moreover, the function of each part of the fall detection processing apparatus 10 according to the present invention may be configured by hardware, may be configured by software, or may be configured by combining hardware and software.

  Further, for example, the fall detection processing device 10 according to the above-described embodiment estimates a person's fall based on the distance image and the IR image. However, the fall detection processing device 10 according to the present invention is not limited to this, and can estimate a person's fall using a color image or a monochrome image that can be acquired in addition to the distance image and the IR image.

1: fall detection system 9: TOF sensor 10: fall detection processing device 11: detection unit 12: acquisition unit 13: storage unit 14: determination unit 15: estimation unit 16: notification unit 17: communication unit 21: notification destination registration table 22 : Log information table 23: Feature / health information table 24: Exception information table A: Care facility B: Home care home

Claims (8)

An acquisition unit that acquires from the TOF sensor a distance image and an IR image of a target space to detect a fall;
A determination unit that determines the size of the object, the movement of the object for a predetermined time, and the pattern, color, or shade of the object based on the acquired distance image and the IR image;
An estimation unit that estimates whether the movement of the object is a fall of a person according to the determined size of the object, the movement of the object for a predetermined time, and the pattern, color, or shade of the object; ,
A fall detection processing apparatus. A storage unit for storing at least one of image data and audio data of the target space;
When it is estimated that the movement of the object is a person's fall, at least one of image data and audio data including the state of the fall time of the person who estimated the fall out of the data stored in the storage unit Has a reporting unit that reports to the specified device by sending
The fall detection processing device according to claim 1. The reporting unit
Transmitting at least one of image data and audio data including states before and after the fall time of the person who estimated the fall,
The fall detection processing device according to claim 2. A storage unit that stores information indicating at least one of the characteristics and health status of one or more persons;
The estimation unit includes
Based on the acquired distance image and the IR image, and at least one of the characteristics and health status of the person stored in the storage unit, the movement of the object is a person's fall, or the predetermined device Estimate whether it is necessary to report a person's fall,
The fall detection processing device according to any one of claims 1 to 3. The determination unit is based on the acquired distance image and IR image of the distance image and the IR image, and the distance image and IR image where the object is not captured. Determining the size of the object, the movement of the object for a predetermined time, and the pattern, color or shade of the object;
The fall detection processing device according to any one of claims 1 to 4. A storage unit that stores at least one of the exception area and the exception time in the target space;
The estimation unit includes
The object detected in the exception area and the exception time is removed from the object to detect a fall,
The fall detection processing device according to claim 1. A fall detection system in which a fall detection processing device that detects a fall of a person and a monitoring server that monitors the fall of a person can communicate,
The fall detection processing device is
An acquisition unit that acquires from the TOF sensor a distance image and an IR image of a target space to detect a fall;
A determination unit that determines the size of the object, the movement of the object for a predetermined time, and the pattern, color, or shade of the object based on the acquired distance image and the IR image;
An estimation unit that estimates whether the movement of the object is a fall of a person according to the determined size of the object, the movement of the object for a predetermined time, and the pattern, color, or shade of the object; ,
When it is estimated that the movement of the object is a human fall, a reporting unit that reports to a predetermined device including the monitoring server;
Have
The monitoring server is
Storing at least one of image data and audio data of the target space;
Of the stored data, at least one of image data and sound data including the state of the fall time of the person who estimated the fall is transmitted to a device registered in advance as a notification destination,
Fall detection system. The monitoring server is
Transmitting at least one of image data and audio data including states before and after the fall time of the person estimated to fall.
The fall detection system according to claim 7.

Images