TW201329915A - Somatosensory fall-detection method - Google Patents

Abstract

A somatosensory fall-detection method is disclosed and comprising steps as following: (a) providing a electronic device and a detection device, wherein the detection device is connected with the electronic device; (b) a color image capturing module and a depth image capturing module of the detection device capture a color image and a depth image, respectively; (c) the electronic device receives the color image and the depth image; (d) the electronic device computes and analyzes the color image and the depth image; and (e) the electronic device identifies somatosensory actions according to the varies of the color image and the depth image. Under this circumstance, the present invention achieves the advantages of detecting at any time without directly contacting, using in darkness, and without disturbing daily lives, and further achieves comprehensive protection.

Description

Somatosensory detection fall method

The present invention relates to a detection method, and more particularly to a somatosensory detection fall method using a software operation and analyzing a color image and a depth image for performing a somatosensory detection and recognition action.

In recent years, due to the age structure of the population and the changes in the social and family structure, the average number of elderly people needed to support the young and middle-aged population has increased day by day, which has also caused the young and middle-aged population to have longer working hours to maintain the daily economic resources. It is more and more common to make the elderly alone at home.

In view of this, relying on the progress of today's technology, many related products and applications of telemedicine have been continuously developed and proposed, and its importance can be seen. For example, the somatosensory detection system is widely used in the field of telemedicine. It is mainly used to reduce the danger and injury caused by falls in elderly or hospital patients. It can be immediately detected when the subject falls or falls. Send a warning sound or notify a specific target, so that the loved ones, children, friends or medical staff of the detected person can immediately find out, thus preventing the expansion of the danger, not only reducing the risk of the elderly or the patient alone in the home or ward, but also letting them Relatives and children can work or work in peace of mind.

However, most of the traditional somatosensory detection systems use a gyroscope-based detection device and are worn on the subject to detect the object. This requires not only higher manufacturing costs, but also more damage. The inconvenience of the detective's daily life, and if the subject feels uncomfortable or is not easy to wear because of the difficulty of wearing and removing, the function and effect of the somatosensory detection cannot be exerted. What's more, due to the discomfort of wearing the detection device, the subject is often not wearing during sleep, so that the subject is at a very high risk in the darkness of the night, completely unable to guard against the middle of the night. The danger and injury of falling or falling down, and even many accidents and misfortunes, are bothersome.

Therefore, how to develop a somatosensory detection and fallback method that is sufficient to improve the above-mentioned conventional technology and can effectively reduce the manufacturing cost and achieve no need to contact the human body and is easy to install and does not affect the living environment is an urgent problem to be solved. .

The main purpose of the case is to provide a somatosensory detection and fallback method. The manufacturing cost of the conventional somatosensory detection system and the somatosensory detection and fallback method is high, and the inconvenience of daily life is caused by wearing and disposing and wearing. And the inability to prevent the risk of the person being detected falling or falling in the middle of the night.

Another object of the present invention is to provide a somatosensory detection and fallback method, which can acquire color images and depth images by using a color image capturing module and a depth image capturing module, and perform somatosensory recognition operations through electronic device calculation and analysis. It can be detected at any time without direct contact with the subject without affecting daily life, and can be used in the dark to achieve comprehensive protection.

Another object of the present invention is to provide a somatosensory detection and fallback method, which can achieve the effect of eliminating the need for a gyroscope and reducing the manufacturing cost through the setting of the color image capturing module and the depth image capturing module.

In order to achieve the above object, a preferred embodiment of the present invention provides a somatosensory detection and fallback method, comprising at least the steps of: (a) providing an electronic device and a detecting device, wherein the detecting device is associated with the electronic device (b) a color image capturing module and a depth image capturing module respectively capture a color image and a depth image; (c) the electronic device receives the color image and the depth image (d) the electronic device calculates and analyzes the color image and the depth image; and (e) the electronic device performs an integrated recognition operation based on the color image and the change in the depth image.

In order to achieve the above object, another preferred embodiment of the present invention provides a somatosensory detection and fallback method, comprising at least the steps of: (a) providing an electronic device and a detecting device, the detecting device and the electronic device (b) a color image capturing module and a depth image capturing module respectively capture a color image and a depth image; (c) the electronic device receives the color image and the depth (h) starting a first program; (g) determining, according to the color image and the depth image, whether an angle between a first vector and a second vector is greater than 70 degrees; (h) switching to a second program; (i) determining whether the angle of the angle between the first vector and the second vector changes with time is greater than 120 degrees per second; (j) determining whether the variation of the included angle with time is between 100 degrees per second and 120 per second. (k) recorded as a fall; (1) recorded as a high fall risk; (m) recorded as a natural sitting or lying down; (n) controlling the color image capture module of the detecting device to continue Take a continuous image; (o) convert the continuous image into a movie and upload it to a web (p) issuing an email or a message for notification; and (q) determining whether the angle is less than 10 degrees; wherein, when the determination of step (g) is no, step (g) is performed again, and when (g) when the determination result is YES, step (h) is performed; when the judgment result of step (i) is no, step (j) is performed, and when the judgment result of step (i) is YES, the step is performed ( k); when the judgment result of the step (j) is YES, the step (1) is performed, and when the judgment result of the step (j) is NO, the step (m) is performed; and when the judgment result of the step (q) is Otherwise, step (q) is re-executed, and when the result of step (q) is YES, step (f) is re-executed.

Some exemplary embodiments embodying the features and advantages of the present invention are described in detail in the following description. It should be understood that the present invention is capable of various modifications in the various aspects of the present invention, and the description and drawings are intended to be illustrative and not limiting.

Please refer to FIG. 1 and FIG. 2 , which are block diagrams of the somatosensory detection system components applicable to the somatosensory detection and fall detection method of the preferred embodiment of the present invention, and the somatosensory detection and fall method of the preferred embodiment of the present invention. Schematic diagram of the somatosensory detection system. As shown in FIG. 1 and FIG. 2, the somatosensory detection system 1 applicable to the somatosensory detection and fall detection method includes at least an electronic device 2 and a detecting device 3, wherein the electronic device 2 is configured to control the somatosensory detection. The operation of the system 1 can be, for example, but not limited to, a personal computer, a notebook computer, a workstation, a server, or a game console. The detection device 3 is connected to the electronic device 2, and can be, for example, a three-dimensional stereo camera or a network camera. The digital camera or the skeleton positioning camera or the like is not limited thereto, and the detecting device 3 includes at least a control module 31, a color image capturing module 32, and a depth image capturing module 33. The control module 31 is configured to control the operation of the detection device 2, and the color image capture module 32 is coupled to the control module 31 for capturing color images, and the depth image capture module 33 is also coupled to the control module. Group 31 is connected for capturing color images. After the color image capturing system 32 and the depth image capturing module 33 respectively capture the color image and the depth image, the electronic device 2 receives the color image and the depth image, and performs calculation and analysis on the color image and the depth image.进行 Perform an integrated recognition action based on changes in the color image and the depth image, such as identifying whether the subject has an action or behavior such as falling, sitting or lying down, by the color image capturing module 32 and the depth image capturing mode The group 33 acquires the color image and the depth image, and performs the somatosensory recognition operation through the calculation and analysis of the electronic device 2, so that the detection can be performed at any time without directly contacting the detected person without affecting daily life, and can be dark. Used in order to achieve comprehensive protection and other effects. In addition, the settings of the color image capturing module 32 and the depth image capturing module 33 can achieve the effect of eliminating the need to provide a gyroscope and reducing the manufacturing cost.

Please refer to FIG. 3 , which is a block diagram of the components of the somatosensory detection system applicable to the somatosensory detection and fall detection method according to another embodiment of the present invention. As shown in FIG. 3, the electronic device 2 of the somatosensory detection system 1 of the present invention may further include a control unit 21, a storage unit 22, an image processing unit 23, and a transmission interface 24. The control unit 21 is configured to control the operation of the electronic device 2, such as but not limited to a central processing unit, a microprocessor, etc., and the storage unit 22 is connected to the control unit 21 for storing color images, depth images, and The parameters of the somatosensory detection system or the data generated by the electronic device 2 after calculation and analysis are not limited thereto. The image processing unit 23 is connected to the control unit 21 for performing image calculation and analysis. For example, but not limited to, calculating a three-dimensional coordinate of an image or a slice layer processing of an image, the transmission interface 24 is connected to the control unit 21 for use. To receive color images and/or depth images by connecting to the detection device 3 or the network 4, or to transmit color images, depth images, computed and analyzed data, processed image data or other data to the network 4, but not limited to this. In some embodiments, the image processing unit 23 can be, for example, but not limited to, a graphics processor, a graphics processing unit, or a graphics chip, and can also be integrated with the control unit 21 or built into the control unit 21, and the transmission interface 24 can be Universal Serial Bus (USB) transmission interface, Thunderbolt transmission interface, Firewire transmission interface, Serial Advanced Technology Attachment (SATA) transmission interface, and Peripheral Component Interconnect Express , PCI-E) transmission interface, CAT-5e, CAT-6, CAT-6e or CAT-7 network Ethernet (Ethernet) transmission interface or a combination of more than one transmission interface, and the network can be the Internet Internet (Internet), Local Area Network (LAN), or Wireless Local Area Network (WLAN) are not limited to this.

According to the concept of the present invention, the detecting device 3 can further include an audio capturing module 34 and a transmission module 35, and the depth image capturing module 33 can further include an infrared emitter 331 and an infrared camera 332. The depth image capturing module 33 transmits infrared rays through the infrared emitter 331, and captures the reflected infrared rays by the infrared camera 332 to capture the depth image. The audio capture module 34 is connected to the control module 31 of the detecting device 3 for capturing sound information, and can simultaneously capture the continuous image with the color image capturing module 32, and then through the electronic device 2 The audio and continuous images are converted into video data and stored in the storage unit 22 or uploaded to the network 4 through the transmission interface 24, but not limited thereto. The transmission module 35 is connected to the control module 31 and the transmission interface 24 of the electronic device 2 for transmitting color images, depth images, audio and continuous images, and the like to the electronic device 2, and the transmission module 35 can include a universal The serial bus transmission interface, the Thunder transmission interface, the FireWire transmission interface, the sequence high-tech configuration transmission interface, the fast peripheral interconnection standard transmission interface, the network transmission interface, or a combination thereof of at least one of the foregoing.

In other embodiments, the detecting device 3 of the present invention may further include a driving module 36, such as but not limited to a motor module or a multi-directional rotating shaft, and the driving module 36 is connected to the control module 31. The body of the detecting device 3 is rotated to achieve the function of capturing the target of the color image and the depth image by the color image capturing module 32 and the depth image capturing module 33. In other words, the detection device of the present case has the function of locking the focus, and can lock the designated subject, such as an elderly person or a hospital patient, but without limitation, the image capturing and the somatosensory recognition are performed without interruption. The action can be detected at any time without direct contact with the subject without affecting daily life, and can be used in the dark to achieve comprehensive protection and other effects.

Please refer to FIG. 4 and FIG. 2, wherein FIG. 4 is a flow chart of the somatosensory detection and fall method of the preferred embodiment of the present invention. As shown in FIG. 4, the detection method of the present invention includes at least the steps. First, as shown in step S100, the electronic device 2 and the detecting device 3 are provided. The detecting device 3 is connected to the electronic device 2, and the electronic device 2 is connected. The operation of the control device 3 and the electronic device 2 itself is performed. Secondly, as shown in step S200, the color image capturing module 32 and the depth image capturing module 33 of the detecting device 3 respectively capture a color image. And a depth image; then, as shown in step S300, the electronic device 2 receives the color image and the depth image; then, as shown in step S400, the electronic device 2 performs calculation and analysis on the color image and the depth image; and finally, as in step S500 As shown, the electronic device 2 performs a somatosensory recognition action based on changes in the color image and the depth image, for example, to identify whether the subject has an action or behavior of falling, sitting, or lying down.

After the somatosensory recognition operation shown in steps S400 and S500 is completed, according to the concept of the present invention, the body feeling detecting device 1 can further have a function of notification and warning. Please refer to Figure 5 and Figure 6 together with Figures 2 and 4, where Figure 5 and Figure 6 are the flow chart of the somatosensory recognition action and subsequent actions of the somatosensory detection and fallback method respectively. A schematic diagram of a first vector, a second vector, and an included angle thereof in the preferred embodiment of the present invention. As shown in FIG. 2, FIG. 4, FIG. 5, and FIG. 6, the somatosensory detection fall method can be started in step S400, and the first program is started as shown in step S601, and the first program is stored in The storage unit 22 of the electronic device 2 is mainly used to identify whether the detected person falls, sits down or lies when standing in the daily life, and after the electronic device 2 completes the calculation and analysis of the color image and the depth image, as in step S602. As shown, it is determined whether the angle θ between the first vector V1 and the second vector V2 is greater than 70 degrees, wherein the first vector V1 and the second vector V2 respectively select the subject of the detected image in the color image and the depth image through the electronic device 2 Several skeleton joint nodes are connected with the initial vector and the changed vector, and the direction is from the ground closer to the ground, such as the line connecting the lumbar-vertebrate-hindbra, and the direction is from the waist to the hindbrain The vector, or the line connecting the center of the waist to the center of the shoulder, and the direction is from the center of the waist to the center of the shoulder, but not limited to this. If the result of the determination in the step S602 is no, the step S602 is re-executed. In other words, if the detected person does not have the action or behavior of falling, sitting or lying down, the first vector V1 and the second vector V2 are continuously detected. Whether the angle is greater than 70 degrees to achieve the effect of continuous detection, determination and identification; if the determination result in step S602 is YES, step S603 is executed to switch to the second program, wherein the second program is based on the color image and the depth image. The change is performed in an integrated recognition operation, and is stored in the storage unit 22 of the electronic device 2 for clearly determining and recognizing that the motion of the detected person is naturally sitting down, lying down or falling. As shown in step S604, it is determined whether the amount of change in the angle θ of the first vector V1 and the second vector V2 with time is greater than 120 degrees per second. If the result of the determination in step S604 is YES, it represents that the subject is at an extremely angular velocity ( That is, the amount of change of the angle with time falls or falls. Therefore, as shown in step S605, the electronic device 2 records the detected person as a fall; if the result of the determination in step S604 is negative, proceeds to step S606 to perform detailed analysis. Determining whether the amount of change in the angle θ of the first vector V1 and the second vector V2 with time is between 100 degrees and 120 degrees per second. If the result of the determination in step S606 is YES, it is a high-risk action due to falling or falling. Or behavior, so as shown in step S607, the electronic device 2 records the subject as a high fall risk; if the result of the determination in step S606 is no, since the angular velocity belongs to the range of sitting or lying down naturally, as in step S608. It is shown that the electronic device 2 records the subject as being naturally seated or lying down.

To distinguish whether the subject is determined to be a fall, a high fall risk, or to sit down or lie down naturally, to ensure the safety of the subject, if the electronic device 2 records the risk of a fall or a high fall, the steps are as follows. As shown in S609, the electronic device 2 controls the color image capturing module 32 of the detecting device 3 to continuously capture the continuous image, and after receiving the continuous image, the electronic device 2 converts the continuous image into a video and uploads it to the movie. The network 4, as shown in step S611, sends an email or message to notify the relatives, family members, or caregivers of the subject, and in some embodiments, steps S610 and S611 can be performed simultaneously. And the email or message sent by the electronic device 2 in step S611 includes the link URL of the video uploaded to the network 4 for viewing by the relatives, family members or medical care personnel of the subject to confirm the acceptance. The physical condition, movement or behavior of the detector is not limited to this.

If the detected person belongs to the action or behavior of sitting or lying down naturally, or after completing the email or message notification in step S611, the electronic device 2 proceeds to step S612, that is, determines the first vector V1. Whether the angle θ with the second vector V2 is less than 10 degrees, if the result of the determination in step S612 is NO, it means that the subject has not resumed standing or getting up, so step S612 is performed again to continuously detect the first vector V1 and the first Whether the angle between the two vectors V2 is less than 10 degrees, so as to achieve the effect of continuous detection, judgment and recognition; if the judgment result in step S612 is yes, it means that the subject has resumed standing or getting up, so switch to the first program again. To continue detection.

In summary, the present invention provides a somatosensory detection and fallback method. The color image capture module and the depth image capture module are arranged, and the manufacturing cost can be reduced without setting the gyroscope, and the color image is reduced by the color image. Taking the module and the depth image capturing module to obtain the color image and the depth image, and performing the somatosensory recognition operation through the operation and analysis of the electronic device, and judging by the first program and the second program, the detection can be performed at any time, and There is no need to directly contact the subject without affecting daily life, and it can be used in the dark, while reducing false positives to achieve comprehensive protection and other effects.

In addition, the conventional somatosensory detection system is prone to the situation in which the scooping of the skeleton joint nodes is caused by random numbers when the detector is seated or lying down, and often a misjudgment occurs. Compared with the conventional somatosensory detection system, the somatosensory detection system and the detecting device and method thereof of the present invention identify and judge the subject's fall, sit or lie down motion through the first program, and When the subject is in a risk-free situation such as sitting or lying down, switch to the second program to filter the random number determination to avoid misjudgment, thereby achieving high recognition accuracy.

The present invention has been described in detail by the above-described embodiments, and may be modified by those skilled in the art, without departing from the scope of the appended claims.

1. . . Somatosensory detection system

2. . . Electronic device

twenty one. . . control unit

twenty two. . . Storage unit

twenty three. . . Image processing unit

twenty four. . . Transmission interface

3. . . Detection device

31. . . Control module

32. . . Color image capture module

33. . . Depth image capture module

331. . . Infrared emitter

332. . . Infrared camera

34. . . Audio capture module

35. . . Transmission module

36. . . Drive module

4. . . network

S100~S500. . . step

S601~S612. . . step

V1. . . First vector

V2. . . Second vector

θ. . . Angle

Figure 1 is a block diagram of the components of the somatosensory detection system applicable to the somatosensory detection and fallback method of the preferred embodiment of the present invention.

FIG. 2 is a schematic diagram of a somatosensory detection system applicable to the somatosensory detection and falldown method of the preferred embodiment of the present invention.

Figure 3 is a block diagram of the components of the somatosensory detection system applicable to the somatosensory detection and fallback method of another embodiment of the present invention.

Figure 4 is a flow chart of the somatosensory detection fall method of the preferred embodiment of the present invention.

Figure 5 is a detailed flow chart of the somatosensory recognition action and subsequent actions of the somatosensory detection and falldown method.

Figure 6 is a schematic diagram of the first vector, the second vector, and the included angle of the preferred embodiment of the present invention.

1. . . Somatosensory detection system

2. . . Electronic device

3. . . Detection device

Claims (10)

A somatosensory detection fall method includes at least the steps:
(a) providing an electronic device and a detecting device, the detecting device being connected to the electronic device;
(b) a color image capturing module and a depth image capturing module of the detecting device respectively capture a color image and a depth image;
(c) the electronic device receives the color image and the depth image;
(d) the electronic device calculates and analyzes the color image and the depth image; and (e) the electronic device performs an integrated recognition operation based on the color image and the change in the depth image. The method for detecting a fall of a somatosensory detection according to the first aspect of the patent application, wherein the step (d) further comprises the steps of:
(f) initiating a first program;
(g) determining whether an angle between a first vector and a second vector is greater than 70 degrees; and (h) switching to a second program. The somatosensory detection falling method according to claim 2, wherein when the judgment result of the step (g) is negative, the step (g) is re-executed, and when the judgment result of the step (g) is YES, Perform step (h). The method for detecting a fall of a somatosensory detection according to claim 3, wherein the step (e) further comprises the step of: (i) determining whether the angle of the first vector and the second vector changes with time is greater than 120 degrees. The method for detecting a fall of a somatosensory detection according to claim 4, wherein when the judgment result of the step (i) is no, the step (j) is performed to determine whether the variation of the included angle with time is between 100 degrees per second. And between 120 degrees per second, and when the judgment result of the step (i) is YES, the execution step (k) is recorded as a fall. The somatosensory detection falling method according to claim 5, wherein when the judgment result of the step (j) is YES, the performing step (1) is recorded as a high fall risk, and when the judgment result of the step (j) is If no, perform step (m) to record as sitting down or lying down naturally. The method for detecting a fall of a somatosensory detection according to item 6 of the patent application, wherein after step (k) and step (l), the method further comprises the steps of:
(n) controlling the color image capturing module of the detecting device to continuously capture a continuous image;
(o) converting the continuous image into a movie and uploading it to a network; and (p) issuing an email or a message for notification. The method for detecting a fall of a somatosensory detection according to claim 7, wherein after the step (p) and the step (m), the step (q) is further included to determine whether the angle is less than 10 degrees. The somatosensory detection falling method according to claim 8, wherein when the judgment result of the step (q) is no, the step (q) is re-executed, and when the judgment result of the step (q) is YES, Re-execute step (f). A somatosensory detection fall method includes at least the steps:
(a) providing an electronic device and a detecting device, the detecting device being connected to the electronic device;
(b) a color image capturing module and a depth image capturing module of the detecting device respectively capture a color image and a depth image;
(c) the electronic device receives the color image and the depth image;
(f) initiating a first program;
(g) determining, according to the color image and the depth image, whether an angle between a first vector and a second vector is greater than 70 degrees;
(h) switching to a second program;
(i) determining whether the angle of the angle between the first vector and the second vector changes over time is greater than 120 degrees per second;
(j) determining whether the variation of the included angle with time is between 100 degrees per second and 120 degrees per second;
(k) recorded as a fall;
(l) recorded as a high fall risk;
(m) record as sitting or lying down naturally;
(n) controlling the color image capturing module of the detecting device to continuously capture a continuous image;
(o) converting the continuous image into a movie and uploading it to a network;
(p) issuing an email or a message for notification; and (q) determining whether the angle is less than 10 degrees;
Wherein, when the judgment result of the step (g) is no, the step (g) is re-executed, and when the judgment result of the step (g) is YES, the step (h) is performed; when the judgment result of the step (i) is no When the step (j) is performed, when the judgment result of the step (i) is YES, the step (k) is performed; when the judgment result of the step (j) is YES, the step (1) is performed, and when the step (j) is performed, When the judgment result is no, step (m) is performed; and when the judgment result of step (q) is no, step (q) is re-executed, and when the judgment result of step (q) is YES, the step is re-executed (f).