Background
In the senior minority society, the elderly lack the care of the evening, and the elderly or young people are busy working to force the elderly to be at home more and more, and when the elderly falls down accidentally in home, serious injury is most likely to happen. In addition to taking care of the old people by the resident clothing personnel, in order to find the falling situation of the old people in time, the old people can wear a wearable device (such as a smart watch), the action track of the user is detected by utilizing the built-in accelerometer and gyroscope, whether the user has falling signs or not is judged, and when the falling situation of the user is detected, a warning message is sent to a default emergency contact person.
However, the wearable device needs to be worn on the body to detect the falling, and if the user forgets to wear the wearable device, the user cannot perform the function of detecting the falling. In addition, a home monitoring system is provided, and a mode of capturing images of a user and analyzing the images by devices such as a video camera and a camera installed in home is adopted to judge whether the user falls down. However, this method may capture the face of the person, which may be unacceptable to persons who are concerned about privacy. Still another method is to use a depth camera to capture a gray-scale image of depth by using a Time of flight (Time of flight) ranging method, and judge each behavior through an image data design algorithm, for example, represent a fall when the depth changes, if the fall surface is active, it represents that a person falls on the ground. However, if the area of the object, the height change, etc. are used as the measurement criteria, the falling will be triggered if the behaviors are similar, for example, when the person walks quickly, the moving area will meet the threshold value of the falling area, and simultaneously meet the huge change of the pixel depth, so that the falling warning is triggered erroneously.
In view of the foregoing, the present invention addresses the above-mentioned drawbacks and future needs of the prior art by providing a fall detection system and a fall detection method thereof, and the following detailed structures and embodiments thereof are provided:
disclosure of Invention
The main objective of the present invention is to provide a fall detection system and a detection method thereof, which uses an average depth distribution difference value calculated by the depth value and the depth average value of each pixel in the depth image data to determine whether the depth value distribution of the fall behavior is concentrated or dispersed, so as to more accurately distinguish whether the object to be detected is truly fallen.
In order to achieve the above object, the present invention provides a fall detection system adapted to detect whether an object to be detected falls, the fall detection system comprising: a depth sensing device for sensing the object to be measured and generating a plurality of depth image data of the object to be measured; the falling judgment device is connected with the depth sensing device and receives the depth image data, the depth image data of each time point are respectively provided with a depth value, a depth change value is generated according to the depth values of at least two time points in succession, if the depth change value is larger than or equal to a falling value, the falling judgment device judges that a falling sign occurs to an object to be detected, then the falling judgment device judges that a falling sign occurs to the object to be detected by utilizing the depth image data of the falling sign, if the falling area data is larger than or equal to a default falling value, the falling judgment device analyzes a depth value of each pixel in the depth image data of the falling sign and a depth average value of the depth image data, an average depth distribution difference value is generated by utilizing the depth value and the depth average value, the average depth distribution difference value is compared with a preset threshold, and when the average depth distribution difference value is smaller than the preset threshold, the falling judgment device judges that the object to be detected is in a falling state.
According to the embodiment of the invention, the drop area data is a depth image data of the occurrence of drop signs, an estimation range is set around the depth variation value, and a maximum value of the contour area of the object to be measured in the estimation range is calculated.
According to the embodiment of the invention, the depth average value is the sum of the depth values of each pixel in the depth image data divided by the number of pixels.
According to the embodiment of the invention, the average depth distribution difference value is obtained by subtracting the depth value of each pixel in the depth image data from the depth average value, taking an absolute value, and dividing the absolute value by the number of pixels after summing the absolute values.
The invention further provides a fall detection method, which comprises the following steps: sensing an object to be measured by using a depth sensing device and generating a plurality of depth image data of the object to be measured; receiving depth image data by using a falling judging device, wherein the depth image data of each time point respectively has a depth value, and generating a depth change value according to the depth values of at least two time points in succession; the falling judgment device judges whether the object to be detected has a falling sign according to the depth change value; the falling judgment device judges whether the object to be detected has a falling sign according to falling area data of the object to be detected; the falling judgment device analyzes a depth value of each pixel in the depth image data with the falling sign and a depth average value of the depth image data, and generates an average depth distribution difference value by using the depth value and the depth average value; and the falling judgment device compares the average depth distribution difference value with a preset threshold value, and judges that the object to be detected is in a falling state when the average depth distribution difference value is smaller than the preset threshold value.
According to an embodiment of the present invention, the step of determining, by the fall determining device, that the subject has a sign of falling further includes: judging whether the depth change value is larger than or equal to a falling value or not; and when the depth change value is greater than or equal to the falling value, judging that the object to be tested has a falling sign.
According to an embodiment of the present invention, the step of determining that the object to be detected has an indication of falling further includes: the falling judgment device judges falling area data of the object to be detected by using the depth image data of the falling signs; judging whether the falling area data is greater than or equal to a default falling value; and when the falling area data is larger than or equal to the falling value, judging that the object to be tested has a falling sign.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by a person skilled in the art without making any advanced effort are within the scope of the present invention.
It will be understood that the terms "comprises" and "comprising," when used in this specification and in the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in the present specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
The invention provides a fall detection system and a detection method thereof, please refer to fig. 1, which is a block diagram of the fall detection system of the invention. The fall detection system 10 includes a depth sensing device 12 and a fall determination device 14. The depth sensing device 12 is a depth camera, and is disposed in an active space of an object 18, such as a bedroom, a living room, a restaurant, etc., for sensing and capturing the object and generating a plurality of depth image data. The depth image data is a gray-scale image generated by using a time-of-flight ranging algorithm, so that no obvious facial expression exists, and the depth image data is suitable for people who pay attention to privacy. The depth image data sensed by the depth sensing device 12 is also displayed on a display screen 16, such as a computer, a smart phone, etc., so that the caregivers can monitor the status of the object 18 at any time.
The fall judging device 14 is connected to the depth sensing device 12 by a transmission line, receives the depth image data sensed by the depth sensing device 12, and judges whether the state of the object 18 to be detected is a falling sign, a falling sign or a falling state according to the received depth image data. The depth image data at each time point has a depth value, and the fall judging device 14 converts the depth into the height of the person, and then generates a depth variation value according to the depth values of at least two consecutive time points. For example, the depth value at time t1 is the same as the depth value at time t2, indicating that the object 18 to be measured has no height change. The fall determination device 14 also determines whether the subject 18 is "falling" or "falling", the sign of falling 20 is shown in fig. 2, but not falling, and the sign of falling 22 is shown in fig. 3. Fig. 2 and 3 are both depth image data.
Please refer to fig. 4, which is a flowchart illustrating a fall detection method according to the present invention. In step S10, the depth sensing device 12 is used to sense the object 18 to be measured and generate a plurality of depth image data of the object 18 to be measured. Next, in step S12, the fall determining device 14 receives the depth image data sensed by the depth sensing device 12, and generates a depth variation value according to the depth values of at least two consecutive time points. In step S14, the fall determining device 14 determines that the object 18 is subject to be detected to have a sign of falling according to the depth change value. Next, in step S16, the fall determining device 14 further uses a fall area data of the object 18 to determine that the object 18 has a fall sign. After determining that the object 18 to be detected has the inverted sign, it is further determined whether there is a possibility of misjudgment, so in step S18, the falling determination device 14 analyzes a depth value of each pixel in the depth image data with the inverted sign and a depth average value of the depth image data, and generates an average depth distribution difference value by using the depth value and the depth average value. The depth average value is the sum of the depth values of each pixel in the depth image data divided by the number of pixels. The average depth distribution difference value is obtained by subtracting the depth value of each pixel in the depth image data from the depth average value, taking an absolute value, and dividing the absolute value by the number of pixels after summing the absolute values. Next, in step S20, the fall determining device 14 compares the average depth distribution difference value with a predetermined threshold, and if the average depth distribution difference value is smaller than the predetermined threshold, the fall determining device 14 determines that the object 18 is in a falling state and needs to send a fall warning.
In step S14, the fall judgment device 14 defaults to a fall value representing the sign of fall. When the depth change values of tn and tn+1 are equal to or greater than the fall value, the fall judgment means 14 judges that the subject 18 is subject to be measured for signs of falling.
In step S16, the fall determination device 14 further defaults to a fall value representing the sign of fall. The falling judgment device 14 judges that the object 18 is falling when the falling area data is equal to or larger than the falling value by judging the falling area data of the object 18 by using the depth image data (time point tn+1) of the falling signs. The drop area data is obtained by setting an estimated range around the depth change value from the depth image data of the drop sign, and calculating a maximum value of the contour area of the object 18 to be measured within the estimated range. In the prior art, the determination of the fall of the object 18 to be measured has been completed, however, if a person walks quickly, the moving area will meet the threshold value of the falling area, that is, the falling value, and simultaneously meet the huge change of the pixel depth, that is, the falling value, so that the falling warning is triggered erroneously.
Therefore, the fall determining device 14 of the present invention further needs to analyze the depth distribution of the falling behavior through steps S18 to S22 by using the evaluation function, which is the depth average, the standard deviation fluctuation, and the average depth distribution difference, as described in detail later. Through comparison of experimental data, it can be observed that the distribution of depth values of the falling behavior is more concentrated, but not more discrete, as in the embodiments of fig. 5, 6, 7 and 8, wherein fig. 5 is the depth image data of the falling state, fig. 6 is the depth distribution map of the depth image data of fig. 5, fig. 7 is the depth image data of the non-falling state, and fig. 8 is the depth distribution map of the depth image data of fig. 7. The depth value distribution set is evident from the depth distribution map of fig. 5, whereas the depth value distribution of fig. 7 is more discrete. In addition, fig. 9 is a standard deviation fluctuation chart of the depth value of each pixel in the depth image data of fig. 5, and fig. 10 is a standard deviation fluctuation chart of the depth value of each pixel in the depth image data of fig. 6, it is also obvious from the chart that the fluctuation in falling is low, but the fluctuation in non-falling state is high. Thus, a depth value dep (i, j) of each pixel (i, j) in the depth image data and a depth average delta of the depth image data, where the inverse sign occurs, generate an average depth distribution difference μ, as shown in the following formulas (1), (2):
and then comparing the average depth distribution difference value with a preset threshold value. The preset threshold value is an empirical value obtained after experiments, the threshold value range is 15-25, and the optimal threshold value is 20. When the average depth distribution difference value is smaller than the preset threshold, the fall judgment means 14 judges that the object 18 to be measured is falling. In the embodiment of fig. 5 and 6, the average depth distribution difference μ is 9, while in the embodiment of fig. 7 and 8, the average depth distribution difference μ is 39. Obviously, the average depth distribution difference value calculated by the depth value and the depth average value of the depth image data can rapidly distinguish whether the object 18 to be measured falls truly.
It should be noted that the body type of the object 18 to be measured does not affect the threshold range, for example, it is assumed that the depth image datagram of the object a to be measured (thinner) contains 4 pixels, the depth distribution is 100, 120, the depth average delta is 110, the standard deviation is 10, the average depth distribution difference value μ=10, and the depth image data of the object B (fatter) to be measured contains 6 pixels, the depth distribution is 90, 110, the depth average δ is 100, the standard deviation is 10, and the average depth distribution difference μ=10.
After the fall detection system and the detection method according to the present invention are used, it is assumed that only the step S16 in fig. 4 is performed to detect 50 falls among 43 actual falls. After confirming the average depth distribution difference value, the average depth distribution difference value can be corrected to 43 times of falling and 7 times of non-falling through the calculation step of the evaluation function. Therefore, the invention can truly improve the accuracy of the falling detection system and avoid false judgment to falsely trigger falling alarm.
In summary, the present invention provides a fall detection system and a detection method thereof, wherein after capturing depth image data of an object to be detected by using a depth sensing device, a fall determining device determines whether the object to be detected is a sign of falling by using whether a depth value of each pixel in the depth image data varies greatly. And judging whether the object to be detected is an inverted sign by utilizing whether the falling area data of the object to be detected is large enough. Since the falling sign is judged through the falling area, in order to avoid that the falling judgment device misjudges that the object to be detected is the falling sign due to certain actions (such as too fast movement and larger area) of the object to be detected, the invention further provides a judging function, and an average depth distribution difference value is calculated by using the depth value of each pixel in the depth image data of the falling sign and the depth average value of the depth image data, so as to judge whether the depth value distribution of the falling action is concentrated or dispersed, further accurately distinguish whether the object to be detected truly falls, and effectively improve the accuracy of falling detection.
The foregoing description is only of the preferred embodiments of the invention and is not intended to limit the scope of the invention. It is therefore intended that all such equivalent variations or modifications as fall within the spirit and scope of the invention as defined in the appended claims be embraced thereby.