TW202403664A - A fall and posture identifying method with safety caring and high identification handling - Google Patents

Abstract

This invention relating to a fall and posture identifying method with safety caring and high identification handling, is for a fall and posture identify identification system. This includes an image capture unit, an image processing unit and a fall identification unit. Firstly, the image capturing unit captures a body images of care recipients to generate a body image data corresponding to each care recipient. Then the image capture unit the image processing unit receives the body image data and preprocesses the body image data for image quality and noise. Finally, the fall identification unit receives the body image data after image preprocessing, and judges whether the care recipient falls, and to issued instant notification message if the care recipient falls.

Description

A fall and posture recognition method with safe care and high privacy processing

The present invention relates to a fall and posture recognition method with safe care and high privacy processing, especially a fall, posture, behavior, etc. identification method with safe care and high privacy processing that uses smart care and can protect the privacy of the person being cared for.

With the advent of the aging era, safety care issues such as safe care for the elderly at home are constantly emerging. In particular, reporting of high-risk falls is a need to minimize injuries and allow the care recipients to receive proper care in real time. However, in the past, The nursing method requires huge medical labor costs. In order to reduce labor costs, the most common method is to monitor in real time through a camera. However, during the long-term care process, the privacy of the person being cared for is not easy to be protected.

The current existing technology is to let the person being cared for wear a wearable smart detection bracelet, and use the internal gyroscope to analyze and determine whether the person being cared for has fallen, and send a notification when the person falls. However, this type of The detection method can easily cause misjudgments due to the daily activities and body movements or behaviors of the care recipient. On the other hand, wearing the sensor can also easily cause physical discomfort, and the care recipient may forget to carry or wear it.

In addition, referring to Figures 22 and 23 together, the commonly known US patent case (US9600993) has mentioned in the specification (Col. 19, Line. 1-52) that through Fourier transforms and Laplace transforms (Fourier transforms , Laplace transforms) to perform shape or outline processing on human images for privacy protection. In addition, it is also mentioned in the instructions (Col.20, Line.11-34) to remove unwanted backgrounds (such as desk lamps). ; For example, if the position of a person's head drops rapidly in a short period of time until it reaches the bottom edge of the field of view, it is judged that a person may fall.

However, in the conventional technology, if it is necessary to analyze the person's image in real time and perform Fourier transformation or Laplace transformation, it will cause a burden on the system's calculation and processing. If there are many people to be cared for, the system will be difficult to Burdening multiple people with real-time image processing will cause the system to crash. On the other hand, when the unwanted background is removed, and when the person being cared for stumbles over a chair, since the system has already removed the background, it will be difficult to determine the cause of the person being cared for. Causes a fall, and then, if only the position movement change is used as the basis for judging a fall, it is easy to cause a misjudgment.

Therefore, how to design a smart nursing care method that can protect the safety and privacy of the care recipient without requiring the care recipient to wear any detection device, and also increase the accuracy of judging the fall of the care recipient, all of which are This is the issue and focus of this case.

One object of the present invention is to provide a fall and posture recognition method with safe care and high privacy processing that allows the care recipient to be protected in safety and privacy.

Another object of the present invention is to provide a fall and posture recognition method with safe care and high privacy processing that increases the accuracy of judgment of falls of the person being cared for.

The present invention provides a fall and posture recognition method with safe care and high privacy processing, which is used in a fall, posture, and behavior recognition system, and provides identification and differentiation of the physical state of at least one person being cared for in at least one area. The fall and posture recognition system includes an image capture unit, an image processing unit and a fall recognition unit. The image capture unit is configured corresponding to the area and is used for shooting toward the person being cared for. The image processing unit is signal-connected to the image capture unit, and the fall recognition unit is signal-connected to the image processing unit. The fall and posture recognition method includes the following steps: a) The image capturing unit captures body images of the care recipients to generate body image data corresponding to each of the care recipients; b) The image processing unit receives the body image data and performs image quality and noise preprocessing on the body image data; and c) The fall identification unit receives the body image data after image preprocessing, and determines whether the person being cared for has fallen through changes in human body posture. If the person being cared for has fallen or has other abnormal conditions, Post an instant notification message.

As mentioned above, the fall and posture recognition method, wherein the image processing unit includes an illumination change analysis module and an ambient light source description module, wherein the step b) includes the following steps: b1) The illumination change analysis module detects changes in ambient light sources in the area to generate a pair of real-time information describing the ambient light sources in the field; b2) The ambient light source description module promptly corrects and establishes a surrounding ambient light source model based on the real-time information of the light source; and b3) The image processing unit searches and detects and identifies multiple care recipients existing in the environment in the area according to the surrounding environment light source model situation.

As mentioned above, the fall and posture recognition method, wherein the fall recognition unit includes a care recipient classification module and a care recipient analysis module, wherein the step c) includes the following steps: c1) The care recipient classification module identifies the care recipients based on the image processing unit's search and detection of the area and distinguishes the environment in the area, and establishes a correspondence between each of the environments. Identify the corresponding skeleton points of the care recipient and a target rectangular tracking frame; c2) The care recipient classification module determines whether the identified care recipients in the environment match one of the characteristics of the care recipient, that is, the identification information, and if so, establishes a corresponding candidate box of possible objects; and c3) The care receiver analysis module retrieves and identifies information based on the characteristics, and analyzes and identifies the posture, gait and/or behavior of each care receiver.

As mentioned above, the fall and posture recognition method, wherein the care recipient classification module further includes a spatio-temporal filter sub-module, wherein the step c1) includes the spatio-temporal filter sub-module of step c11) The module provides appropriate logic and mechanisms, mask values and feature thresholds to calculate the specific features of the identified care recipients in the environment.

The above fall and posture recognition method, wherein the care recipient analysis module includes a geometric analysis sub-module, a curvature algorithm sub-module, a shape algorithm sub-module, and a human body local configuration algorithm sub-module Group (Body Part Algorithm), wherein the step c3) includes a step c31) the geometric analysis sub-module analyzes and identifies the shape, size and/or size change of the identified care recipient in the environment, and the curvature algorithm The sub-module identifies and calculates the posture changes of the specific curvature of each of the care recipients, the shape algorithm sub-module calculates and compares the specific body posture of each of the care recipients, and the human body local configuration algorithm sub-module The system calculates the various human body parts of each person being cared for. On the other hand, it can also establish a multi-person tracking algorithm model at the same time to facilitate simultaneous fall and posture recognition when there are multiple people in the scene.

As mentioned above, the fall and posture recognition method, wherein the care recipient analysis module further includes an optical flow algorithm sub-module, a bag of words algorithm sub-module (Bag of Word), a human skeleton and joint point sub-module, A machine learning sub-module, a fall detection algorithm sub-module, and a scale-invariant feature transformation algorithm module, wherein step c3) includes step c32) of the optical flow algorithm after step c31) The module performs motion detection, object cutting, calculation of collision time and object expansion, motion compensation coding and/or row stereometric measurement of the images of each of the care recipients. The bag-of-words algorithm sub-module is a comprehensive analysis of the fall process. Multiple feature vector groups corresponding to the posture are encoded and classified. The human body skeleton and joint point sub-module retrieves the curvature algorithm sub-module, the shape algorithm sub-module and the human body local configuration algorithm sub-module. Each calculation result generates a state detailed information corresponding to all activities and postures of each care recipient. The machine learning sub-module uses a neural network to perform deep learning to train, adjust and optimize the fall. Identification unit, the fall detection algorithm sub-module retrieves the curvature algorithm sub-module, the shape algorithm sub-module, the human body local configuration algorithm sub-module, and the optical flow algorithm sub-module and each calculation result of the bag-of-words algorithm module to establish a fall detection and judgment model. The scale-invariant feature transformation algorithm module performs calculations on the distance changes of each of the care recipients in the area. Image compensation. In addition, this case can integrate and use various AI algorithms and data sets (Data Sets) training methods to improve the recognition rate and reduce the chance of false misjudgment.

In addition to the fall and posture recognition method mentioned above, the present invention relates to a fall and posture recognition system with safe care and high privacy processing. It also includes an intelligent visual electronic fence function that can prevent other accidents of the person being cared for. detection report. Intelligent visual electronic fence module, wherein the step b) includes a step b1'), the intelligent visual electronic fence module is each of the images captured by the image capture unit (IP CAM) 1 Establish a virtual electronic enclosure area in the area, and perform image superposition with the body image data of the person under care, wherein step c) includes a step c1') to determine whether the person under care has entered the area. If the care recipient enters within or leaves the electronic containment zone, or breaks into a restricted area restricted areas, etc., it will be determined that the person under care may enter the restricted dangerous area, stay for too long, or abnormally leave the safe area set by the electronic fence day and night, etc., and if confirmed, the real-time notification will be released immediately. Notify the message.

The fall and posture recognition method as mentioned above, wherein the image processing unit includes a visual privacy leakage prevention algorithm module, wherein the step b) includes a visual privacy leakage prevention algorithm module of step b1″) The group performs feature contour processing and dynamic image algorithm masking processing on the body image data of the care recipient to generate a human body synthetic mask that corresponds to the care recipient and has secure privacy masking.

As mentioned above, the fall and posture recognition method, wherein the step b1") includes a step b11") of performing feature contour processing and dynamic image algorithm masking processing on multiple object images in the area to generate multiple corresponding objects. A composite mask of the object described above with a secure privacy mask.

As mentioned above, the fall and posture recognition method, wherein the image processing unit further includes a night-time fall and posture model, wherein the step a) includes a step a1") and the night-time fall and posture model through fuzzy technology (Fuzzy Theory) analyzes and intelligently identifies and enhances the body image data at night.

As mentioned above, the fall and posture recognition method further includes a communication unit, and the communication unit is signal-connected to the fall recognition unit. The fall and posture recognition method further includes step d). The communication unit is used to send a caring inquiry message to each of the persons being cared for. It can also understand the on-site situation of the persons being cared for in an emergency and conduct two-way calls.

As mentioned above, the fall and posture recognition method, wherein the care recipient analysis module includes an abnormality detection unit for people in the house, an abnormality detection unit for a specific state for a long time, a work and rest statistics unit and a history browsing unit , wherein step c3) includes the following steps: c31') The abnormality detection unit for people in the house determines whether the person under care is in the predetermined area at a first predetermined time; c32') The abnormality detection unit in which the specific state is too long determines whether the person under care has remained motionless outside a second predetermined time, or whether it has been in the area for too long; The work and rest statistics unit described in c33’) continuously and automatically records the home life and rest status of each of the care recipients; and The history browsing unit described in c34’) is used to browse the home life and rest status of each of the persons under care.

This case is a fall and posture recognition method with safe care and high privacy processing. It uses an image capture unit to provide intelligent image care for each care recipient. It can improve the accuracy of the fall judgment of the care recipient and enable the care recipient to While security and privacy are protected, medical labor costs can also be reduced, and all the above purposes can be achieved.

A fall and posture recognition method with safety care and high privacy processing according to the first embodiment of the present invention, as shown in FIG. 1 , is used in a fall and posture recognition system and is used to identify and distinguish at least one area. The physical state of at least one caregiver (for example, the elderly), in which the fall and posture recognition system includes an image capture unit (IP CAM) 1, an image processing unit (pre-processing) 2 and a fall recognition unit ( detected human) 3, the image capture unit (IP CAM) 1 is configured corresponding to the above-mentioned area and is used to capture images toward the person being cared for, and the image processing unit (pre-processing) 2 is in conjunction with the image capture unit (IP CAM) )1 signal connection, the fall recognition unit (detected human) 3 is connected with the image processing unit (pre-processing) 2 signal. The image capture unit 1 in this case uses a visible light camera that is commonly used by the general public and is low-cost, but uses image destructive processing technology to achieve a high privacy design.

Referring to FIG. 2 , initially in step S101 , the image capture unit 1 captures body images of the care recipients to generate body image data corresponding to each care recipient. Then in step S102 , the image processing unit 2 Receive body image data, and perform image quality and noise preprocessing on the body image data. Then in step S103, the fall identification unit 3 receives the body image data after image preprocessing, and determines that the person is being cared for through changes in human body posture. Whether the person fell down, if not, then return to step S101 and continue to collect images of each person being cared for. If it is determined that the person has fallen, then in step S104, an immediate notification message is issued and the relevant caregivers are notified to come to the scene immediately for treatment. Emergency assistance.

According to a second embodiment of the present invention, a fall and posture recognition method with safety care and high privacy processing is shown in FIG. 3. In this example, the image processing unit 2 includes an illumination change analysis module (illumination change estimation). ) 21 and an ambient light source tuning model 22, as shown in Figure 4 together. In this example, step S102 is followed by step S1021, where the illumination change analysis module 21 detects the environment of the area. The light source changes to generate a pair of real-time information describing the field environment light source, thereby adjusting and compensating the environment light source to provide a stable light source image with better quality. In step S1022, the environment light source description module 22 uses the real-time information of the light source. , timely correct and establish a surrounding environment light source model to facilitate effective processing of the activity status images of the care recipients in the area collected by the image capture unit 1. In step 1023, the image processing unit 2 searches the area based on the surrounding environment light source model. and detecting multiple identified care recipients in the environment.

In addition, the fall identification unit 3 may also include a care recipient classification module (Object Classification) 31 and a care recipient analysis module 32, as shown in Figure 5. In this example, the care recipient classification module 31 It further includes a spatio-temporal filter sub-module (Spatio-Temporal Filter) 311, and the care recipient analysis module 32 includes a geometric analysis sub-module (Geometric Analysis) 321, a curvature algorithm sub-module (Curvature Algorithm) 322, and a shape Algorithm sub-module (Shape-based Algorithm) 323 and a human body partial configuration algorithm sub-module (Body Part Algorithm) 324.

Step S1023 is followed by step S1031. Based on the image processing unit's search and detection of the area, the care recipient classification module 31 identifies and distinguishes the identified care recipients in each environment of the area, and establishes corresponding identifications in the environment. The corresponding skeleton point of the person under care and a target rectangular tracking frame. There is a step S10311 after step S1031. The spatio-temporal filter sub-module (Spatio-Temporal Filter) 311 provides appropriate logic and mechanism, mask value (Mask) and features. The threshold value (Threshold) is used to calculate the specific characteristics of the environment to identify the care recipient, so as to filter certain specific objects or filter interference or noise to the object to be detected, and then step S1032, classifying the care recipient The module 31 determines whether the identified care recipients in the environment match one of the characteristics of the care recipients, that is, the identification information. If not, return to step S1031 to continue searching and detection. If they match, proceed to step S1033. , establish a corresponding candidate box (Bounding Box) of possible objects, and in step S1034, the care recipient analysis module 32 analyzes and identifies the posture, gait and/or behavior of each care recipient based on the feature extraction, that is, the recognition information. By combining the above modules with other algorithms, it is possible to avoid the situation where people overlap but are misjudged as falling. There is a step S10341 after step S1034, in which the geometric analysis sub-module 321 analyzes and identifies the environment. The shape, size and/or size changes of the identified care recipients are identified and calculated by the curvature algorithm sub-module (Curvature Algorithm) 322. The specific curvature posture changes of each care recipient are used to assist in finding the care recipient target and posture recognition. , gait recognition, behavior recognition and other tasks, the shape-based algorithm sub-module (Shape-based Algorithm) 323 calculates and compares the specific body posture of each care recipient, and the human body partial configuration algorithm sub-module (Body Part Algorithm) 324 calculates the various parts of the human body of each person being cared for.

Assume that when a human body falls backward or forward, the body trunk should have specific curvature changes when specific posture changes (as shown in Figures 6 and 7). Therefore, the curvature algorithm sub-module 322 can change the human body posture through , and changes in body movements, etc., to help understand whether the elderly or cared for who are currently concerned may have a tendency to fall, and the changes in the curvature of the human body are undoubtedly important information, and through the human body Continuous changes in curvature can help understand the real-time changes in the human body's body shape and skeleton, and can help to instantly grasp the necessary specific process conditions and specific real-time changes that may meet or satisfy a fall when the human body is walking or moving in the environment. The main function of the shape algorithm sub-module 323 is to collect corresponding "shape descriptions" that keep in mind the different behaviors of the human body (as shown in Figure 8) and complement and complement the aforementioned curvature algorithm sub-module 322), and together provide a method for identifying different postures of the human body. It is part of the detection standards and appearance basics to provide feature recognition algorithms to assist in detecting specific behavioral appearance changes (such as walking, falling, etc.) when human body posture changes.

Referring to Figure 9, in this example, the care recipient analysis module 32 further includes an optical flow algorithm sub-module (Optical Flow Algorithm) 325, a bag of word algorithm sub-module (Bag of Word) 326, Human skeleton and joint point Algorihm (Bone & articulation point Algorihm) 327, a machine learning sub-module (Artificial Intelligent & machine learning) 328, a fall detection algorithm sub-module (Fall Detection AI Algorithm) 329, a scale-varying Scale-invariant feature transform (SIFT) 320.

In addition, as shown in Figure 10, there is a step S10342 after step S10341, in which the optical flow algorithm sub-module (Optical Flow Algorithm) 325 performs motion detection, object cutting, collision time and object expansion on the image of the person being cared for. Calculation, motion compensation coding and/or row stereometrics, the bag-of-words algorithm sub-module 326 establishes multiple corresponding feature vector group encoding and classification for the posture during the fall process, and the human skeleton and joint point sub-module 327 captures the curvature algorithm Each calculation result of the sub-module 322, the shape algorithm sub-module 323 and the human body local configuration algorithm sub-module 324 generates a state detailed information corresponding to all activities and postures of each care recipient, which is processed by the machine learning sub-module 328 Deep learning is performed using a neural network to train, adjust and optimize the judgment of the fall identification unit 3. The fall detection algorithm sub-module 329 captures the curvature algorithm sub-module 322, the shape algorithm sub-module 323, and the human body's local structure. The calculation results of the type algorithm sub-module 324, the optical flow algorithm sub-module 325 and the bag-of-words algorithm sub-module 326 are used to establish a fall detection and judgment model, and the scale-invariant feature transformation algorithm module 320 performs the respective calculations. Image compensation for changes in the distance of the caregiver in the area. On the other hand, it is also possible to establish a multi-person tracking algorithm model at the same time to facilitate simultaneous fall and posture recognition when there are multiple people in the scene.

Among them, the optical flow algorithm sub-module 325 performs calculations through optical flow (Optical flow or optic flow). Optical flow refers to the speed at which an image of an object expresses motion. The reason why an object can be detected by the human eye when it is moving is because when the object moves, a series of continuously changing images will be formed on the human retina. These changing information will continue to flow through the retina of the eye at different times. , just like a kind of light flowing through, so it is called optical flow. The optical flow method can be used to describe the movement of human bodies or objects in the environment relative to machine vision (equivalent to the observer), which can be regarded as the movement of the surface or edge of the elderly or the person being cared for (the object being observed). In other words, optical flow is caused by the movement of the foreground object itself in the scene, the movement of the camera, or the mutual movement of the two. To do this, you can first assign a velocity vector (optical flow, including size and direction) to each pixel (Pixels) of each image in the environment, thus forming an optical flow field. If there are no moving objects in the image in the environment, the optical flow field will be continuous and uniform. However, if there are moving objects, the optical flow of the moving objects and the optical flow of the image will be different, and the optical flow field will no longer be continuous and uniform, thus It can detect moving objects and their positions. The optical flow method is very useful in graphic recognition, computer vision and other image processing fields. It can be used for motion detection, calculation of collision time and object expansion, or for specific three-dimensional measurements through the surface and edges of objects. During the continuous process of falling, the human body will produce changes in the optical flow field of the limbs, skeleton, and joints at the same time. By detecting and identifying the relevant optical flow field changes, the information on the changes in the human body posture can be obtained at the same time, and combined with the aforementioned algorithms to Assist fall recognition and associated posture recognition and analysis.

The bag-of-words algorithm submodule 326 is a probabilistic model of Bayesian multi-layer architecture. It can provide a classification method that needs to establish an intelligent learning mechanism for various objects in the field. It can be used to help determine and evaluate the possibility of falling. and auxiliary mechanisms, and then collect these various clips (Video Clip) related to the fall, and map these various clips related to the fall to their corresponding codebooks (Codebook). Therefore, in the elderly or cared for in their fields and activities, the distribution curve of these code words (Codeword) that may appear can be obtained and statistically calculated. For example, the physical meaning shown in Figure 11 can represent the occurrence of specific gestures. Distribution curve of times. Curve graphs are often used to analyze image content characteristics in image processing and computer vision. From the concept of a curve graph, we can use it to analyze the continuous changes in a specific posture and the judgment factors or principles that may lead to falls.

The human skeleton and joint point sub-module 327 captures the aforementioned curvature algorithm sub-module 322, human body local configuration algorithm sub-module 324, shape algorithm sub-module 323, optical flow algorithm sub-module 325 and bag-of-words algorithm sub-module Relevant data of module 326 is used to obtain all activities and posture details of the elderly or cared for in the environment, and algorithms are designed and integrated with these detailed information to facilitate the construction of the human body's skeleton and joints in the environment. Points and their key parameters. When all real-time important parameters such as human skeletons and joint points in these environments are generated, it is equivalent to understanding the relationship between various objects and human bodies in the scene at any time, and the existence of specific human bodies and their personalized activity postures. Details, as well as the real-time interactive relationship between the human body and the human body. Based on the real-time parameters of these skeletons and joint points, the activities and process information of the people of concern in the environment can be judged based on the real-time information grasped, so as to facilitate the implementation of specific behaviors and actions. Judgment, such as standing, sitting, getting up, lying down, squatting, falling,... various actions. However, because the specific behavior of the human body is derived from a series of postures, it is actually very complicated. In other words, different specific behaviors (Behavior) may often have the same specific posture (Pose). For example, for the specific action of "falling forward" (as shown in Figures 5 and 6), some continuous posture actions are actually similar to "bending over", "hunchback", "picking up something and leaning forward", etc. , are very similar, we can call it "pseudo-fall" here. It is easier for human beings, who are the spirits of all things, to quickly distinguish the subtle differences in these movements because they have intelligence. But if you want computers or machines to be able to distinguish such problems at the same time, you must use artificial intelligence methods such as machine learning and deep learning to design these algorithms and build machines. The learning rules and models enable the machine to have artificial intelligence designated with specific abilities.

Referring to Figure 12, which is a flow chart of the machine learning sub-module 328, you must first select a framework (or platform) for designing this machine learning algorithm. Commonly used frameworks such as Tensorflow, Poytch, etc. or fast feature embedding volumes Product structure CNN, etc. For example, in the case of Tensorflow, this framework can be regarded as a machine learning design tool, which contains many available algorithms and functions; in the case of Tensorflow, it mainly provides some methods of deep learning model (Deep Learning model).

The fall detection algorithm sub-module 329 captures the aforementioned curvature algorithm sub-module 322, shape algorithm sub-module 323, human body local configuration algorithm sub-module 324, optical flow algorithm sub-module 325 and bag-of-words algorithm. Relevant data of module 326 is used to obtain detailed information on all the activities and postures of the elders or people being cared for in the environment, and algorithms are designed to facilitate the integration of these detailed information to construct the skeleton, joints and joints of the human body in the environment. Its key parameters, when all the real-time important parameters of the human body skeleton and joint points in these environments are generated, it is equivalent to knowing at any time the relationship between various objects and the human body in the scene, the existence of a specific human body and its personalized activity posture details, and The interactive relationship between the human body and the human body. In addition, the fall detection algorithm sub-module 329 module has a built-in AI model established by deep learning (Deep Learning) and other methods. Its input parameters are combined with information from the aforementioned modules (such as human skeleton and joint point modules). etc.), in addition to capturing the information output by the aforementioned modules, it also considers the changes in parameters such as the current position, speed, and acceleration of each joint point of the human body, and uses the established AI model to conduct a series of fall detection. Test and judge.

The scale-invariant feature transformation algorithm module 320 needs to perform scale space extreme value detection, key point positioning, direction determination and key point description. The scale space extreme value detection is to search image positions on all scales. Potential interest points that are invariant to scale and rotation are identified through the Gaussian differential function; key point positioning is to determine the position and scale at each candidate position through a well-fitted model. Key points are selected based on their degree of stability; direction determination is based on the local gradient direction of the image, assigning one or more directions to each key point position. All subsequent operations on the image data are transformed relative to the direction, scale and position of the key points, thereby providing invariance to these transformations; the key point description is in the neighborhood around each key point, in the selected Measure the local gradient of the image on a scale. These gradients are transformed into a representation that allows relatively large local shape deformations and illumination changes.

According to another third embodiment of the present invention, a method for combining fall and posture recognition with safe care and high privacy processing with an intelligent visual electronic fence module is shown in Figures 13 and 14. In the embodiment of the intelligent visual electronic fence, the image capture unit captures the image of the person being cared for in the environment. After passing through the image processing unit, the highly private streaming image is removed. In addition to being sent to the fall identification unit for fall identification, another identical stream image will be sent to the smart visual electronic fence module to determine whether the smart visual electronic fence module is triggered to prevent various possible obstructions. Dangerous situation. In the intelligent visual electronic fence module (AI Electrical Fence Algorithm) 23, as in step S101' before step S101, the intelligent visual electronic fence module 23 establishes in each area captured by the image capture unit (IP CAM) 1 An electronic isolation block 230 is superimposed with the body image data of the person being cared for; refer to Figure 15 and determine whether the person being cared for 4 is within the range of the electronic isolation block or in step S103'. Stay away from the electronic isolation block 230, determine whether the person under care has abnormal behavior, and if so, issue an immediate notification message.

Among them, the intelligent visual electronic fencing module 23 is an integration of AI smart algorithm and security electronic fencing technology to construct an "AI electronic fencing algorithm" to expand its application in smart homes, smart long-term care, or Used for safe care of human-centered areas. This case can integrate the range setting of electronic fences according to the special concerns and application needs. In addition to assisting in fall determination and other multi-identification applications, for example, the range of electronic fences can be set to places where the care recipients rarely go (for example, (vacant space where debris is stored). If the person being cared for is within the range of the electronic fence, a danger is likely to occur, so report immediately. Therefore, computer vision and image processing technology can be used to establish an interactive interference relationship between the target of concern (the person being cared for) and the "visual electronic fence". When the interactive interference relationship is identified and established, the system will immediately issue a notification .

According to another fourth embodiment of the present invention, a fall and gesture recognition method with safety care and high privacy processing is shown in FIG. 16 and FIG. 17 together, in which the image processing unit 2 includes an algorithm module to prevent visual privacy leakage. Group (Privacy Protection Algorithm) 24, and as in step S1011 after step S101, the anti-visual privacy leakage algorithm module 24 performs feature contour processing and dynamic image algorithm masking processing on the body image data of the care recipient to generate A pair of destructive masking of human body images corresponding to the care recipient and with secure privacy masking. Furthermore, after step S1021", step S1012" can also be used to perform feature contour processing and dynamic image algorithm masking processing on multiple object images in the area. To generate a plurality of object image destructive masks corresponding to the object and having a secure privacy mask.

Among them, the anti-visual privacy leakage algorithm module 24 provides the integrated use of "image processing" and "AI algorithm" technology to perform feature contour masking processing on images from visible light cameras, as well as dynamic image algorithm superposition, and finally synthesize the image. Achieve high recognition rate while taking into account the safety and privacy of people-centered care. On the other hand, each care recipient target in the environment of concern can be similarly masked to avoid leaking the detailed characteristics and outline of individual objects. , to achieve further privacy protection.

According to another fifth embodiment of the present invention, a fall and posture recognition method with safety care and high privacy processing is shown in Figures 18 and 19, in which the image processing unit 2 further includes a night fall and posture model 24, and in After step S101, as in step S1011, the nighttime fall and posture model 24 enhances the body image data through fuzzy theory analysis and intelligent identification.

The fall and posture recognition system further includes a communication unit 4 (such as mobile App/Line), which is connected to the fall recognition unit 3. After step S104, as in step S1041, the caregiver sends a care inquiry message to Each person being cared for and the caregiver can also understand the on-site situation of the person being cared for through the App or Line at the same time in an emergency situation, and conduct two-way calls.

Referring to Figures 20 and 21 together, in this example, the care recipient analysis module 32 includes an abnormality detection unit 331 for people in the house, an abnormality detection unit 332 for a specific state that has been too long, a work and rest statistics unit 333, and A history browsing unit 334 may also include the following steps after step S1023. S1301 The abnormality detection unit 331 of someone in the house determines whether the person under care is in a predetermined area at a first predetermined time. If someone enters the home of the person under care, Space, the patent of this invention further integrates the previously disclosed related technologies such as human body posture recognition and electronic fencing, automatically displays on the system when someone enters the home space state of the person being cared for, and pushes the prompt message to the caregiver. on the reader’s mobile App and Line.

As in step S1302, the abnormality detection unit 332 of the prolonged specific state determines whether the person being cared for has remained motionless outside a second predetermined time (such as sleeping time), or whether the person has been in the area for too long, in order to further pay attention to the person being cared for. The caregiver is active in various situations in the living space, and can ensure that the person being cared for is in a stable and good state of activity. The patent of this invention further uses the aforementioned human body posture recognition related technology to understand whether the person being cared for is in a specific state or whether the fixed movements are correct. There is anomaly detection that lasts too long.

In step S1303, the work and rest statistics unit 333 continues to automatically record the home and life work and rest status of each person being cared for. In order to further pay attention to the home and life work and rest status of the care recipient, the patent of the present invention further integrates the human body posture recognition and other related information disclosed above. Technology that automatically and continuously records the home life and rest status of the person being cared for (activity status and analysis of postures such as standing/sitting/lying, etc.), and displays it on the system to facilitate caregivers to understand and track and observe at any time, and to facilitate evaluation and measurement of the person being cared for. The caregiver's daily healthy physiological status.

Finally, in step S1304, the history browsing unit 334 is used to browse the home life and rest status of each person being cared for. In order to simultaneously track and pay attention to the long-term work and rest status of the person being cared for, the patent of the present invention further integrates the related technologies such as human body posture recognition disclosed above. , and automatically and long-term recording of the daily routine of the person being cared for (for example, it can store and browse 30 days of historical activities). This case is a fall and posture recognition method with safe care and high privacy processing. It uses an image capture unit to perform smart image care on each care recipient, and increases the accuracy of the care recipient's fall judgment through smart learning. It can also use intelligent The recognition of the visual electronic fence module can set the range of the electronic fence to a place where the care recipient rarely goes (such as an idle space where debris is stored). If the care recipient is within the range of the electronic fence, it is easy to If danger may occur, report it immediately. On the other hand, it can also prevent visual privacy leakage algorithm modules, perform privacy shielding on care recipients and environmental objects, and reduce medical labor costs, achieving all the above purposes.

The present invention shows that it is completely different from the conventional technology in terms of purpose, means and efficacy, and is a major breakthrough. However, it should be noted that the above embodiments are only illustrative to illustrate the principles and effects of the present invention, and are not intended to limit the scope of the present invention. Anyone skilled in the art can make modifications and changes to the embodiments without violating the technical principles and spirit of the present invention. The protection scope of the present invention shall be as described in the patent application scope described below.

1:Image capture unit 2:Image processing unit 21: Illumination change analysis module 22:Ambient light source description module 23: Intelligent visual electronic fence module 24: Anti-visual privacy leakage algorithm module 3: Fall identification unit 4: Communication unit 31:Care recipient classification module 311: Spatiotemporal filter sub-module 32:Care recipient analysis module 321:Geometric analysis submodule 322: Curvature algorithm submodule 323: Shape algorithm submodule 324: Human body local configuration algorithm submodule 325: Optical flow algorithm sub-module 326: Bag of words algorithm submodule 327:Human skeleton and joint idea module 328:Machine learning sub-module 329: Fall detection algorithm module 320: Scale-invariant feature transformation algorithm module 331: Anomaly detection unit for people in the house 332: Anomaly detection unit in a specific state for too long 333: Work and rest statistics unit 334: Historical browsing unit S101~S104, S1021~S1023, S1031~S1034, S10341, S10342, S101’, S1011”, S1012”: steps ,

Figure 1 is a block diagram of a fall and posture recognition system with safety care and high privacy processing of a fall and posture recognition method according to the first embodiment of this case; Figure 2 is a flow chart of a fall and gesture recognition method with safety care and high privacy processing in Figure 1; Figure 3 is a block diagram of a fall and posture recognition system with safety care and high privacy processing of a fall and posture recognition method according to the second embodiment of the present case; Figure 4 is a flow chart of a fall and posture recognition method with safety care and high privacy processing in Figure 3; Figure 5 is a block diagram of the care recipient analysis module of the fall and posture recognition system of Figure 3; Figures 6 and 7 are schematic diagrams of curvature changes of the curvature algorithm sub-module of the fall and posture recognition system in Figure 3; Figure 8 is a schematic diagram of the shape description of the shape algorithm sub-module of the fall and posture recognition system in Figure 3; Figure 9 is a block diagram of another extension of the care recipient analysis module of the fall and posture recognition system in Figure 3; Figure 10 is an extended flow chart of a fall and posture recognition method with safety care and high privacy processing of Figure 4;; Figure 11 is a distribution curve diagram of the bag-of-word algorithm submodule of the shape algorithm submodule of the fall and posture recognition system in Figure 3; Figure 12 is a schematic flow chart of the machine learning sub-module of the fall and posture recognition system in Figure 3; Figure 13 is a block diagram of a fall and posture recognition system with safety care and high privacy processing of a fall and posture recognition method according to the third embodiment of this case; Figure 14 is a flow chart of a fall and gesture recognition method with safety care and high privacy processing in Figure 13; Fig. 15 is a schematic diagram of the electronic fencing block deployed by the intelligent visual electronic fencing module of the fall and posture recognition system in Fig. 13; Figure 16 is a block diagram of a fall and posture recognition method and posture recognition system with safety care and high privacy processing according to the fourth embodiment of the present case; Figure 17 is a flow chart of a fall and posture recognition method with safety care and high privacy processing in Figure 16; Figure 18 is a block diagram of a fall and posture recognition method and posture recognition system with safety care and high privacy processing according to the fifth embodiment of this case; Figure 19 is a partial flow chart of a fall and posture recognition method with safety care and high privacy processing in Figure 18; Figure 20 is a block diagram of the care recipient analysis module of the fall and posture recognition system of Figure 17; Figure 21 is a subsequent flow chart of the continued care recipient analysis module of Figure 20; and 22 and 23 are schematic diagrams of conventional fall detection.

S101~S104: Steps

Claims (11)

A fall and posture recognition method with safe care and high privacy processing, which is used in a fall and posture recognition system and is used to identify and distinguish the physical state of at least one care recipient in at least one area. The fall and posture recognition system includes An image capture unit, an image processing unit and a fall recognition unit. The image capture unit is configured corresponding to the areas and is used to shoot toward the care recipients. The image processing unit is connected to the image capture unit. Signal connection, the fall recognition unit is signal connected with the image processing unit, wherein the fall and posture recognition method includes the following steps: a) The image capture unit captures the body images of the care recipients to generate body image data corresponding to each care recipient; b) The image processing unit receives the body image data and performs image quality and noise preprocessing on the body image data; and c) The fall identification unit receives the body image data after image preprocessing and determines whether the care recipients have fallen through body posture changes. If the care recipients fall or have other abnormal conditions, a real-time notification is issued. Notify the message. For the fall and posture recognition method described in item 1 of the patent application, the image processing unit includes an illumination change analysis module and an ambient light source description module, and step b) includes the following steps: b1) The lighting change analysis module detects changes in the ambient light source in the area to generate a pair of real-time information describing the ambient light source in the field; b2) The ambient light source description module promptly corrects and establishes a surrounding ambient light source model based on the real-time information of the light source; and b3) The image processing unit searches and detects and identifies multiple care recipients existing in the environment in the area based on the surrounding environment light source model. For the fall and posture recognition method described in item 2 of the patent application, the fall recognition unit includes a care recipient classification module and a care recipient analysis module, and the step c) includes the following steps: c1) The care recipient classification module identifies the care recipients in the environments in the area based on the image processing unit's search, detection and distinction in the area, and establishes a correspondence corresponding to the identified care recipients in each environment. Skeleton points and a target rectangular tracking frame; c2) The care recipient classification module determines whether one of the environmentally recognized care recipients matches one of the characteristics of the care recipients, that is, the identification information, and if so, creates a corresponding candidate frame of possible objects; c3) The care recipient analysis module extracts and recognizes information based on the characteristics to analyze and identify the posture, gait and/or behavior of each of the care recipients. For the fall and posture recognition method described in item 3 of the patent application, the care recipient classification module further includes a spatio-temporal filter sub-module, wherein the step c1) includes a step c11) and the spatio-temporal filter sub-module provides Appropriate logic and mechanisms, mask values and feature thresholds to calculate the specific features of the identified care recipients in those environments. For the fall and posture recognition method described in item 3 of the patent application, the care recipient analysis module includes a geometric analysis sub-module, a curvature algorithm sub-module, a shape algorithm sub-module and a human body local structure type algorithm sub-module, wherein the step c3) includes a step c31) the geometric analysis sub-module analyzes and identifies the shapes, sizes and/or size changes of the identified care recipients in the environments, the curvature algorithm The sub-module identifies and calculates the posture changes of the specific curvature of each person being cared for, the shape algorithm sub-module calculates and compares the specific body posture of each person being cared for, and the human body local configuration algorithm sub-module calculates the specific shape and posture of each person being cared for. Various parts of the body of the person being cared for. For the fall and posture recognition method described in item 5 of the patent application, the care recipient analysis module further includes an optical flow algorithm sub-module, a bag-of-word algorithm sub-module, and a human skeleton and joint point sub-module. , a machine learning sub-module, a fall detection algorithm sub-module, and a scale-invariant feature transformation algorithm module, wherein step c3) includes a step c32) after step c31), the optical flow algorithm sub-module The group performs motion detection, collision time and object expansion calculation on each image of the person being cared for. The bag-of-words algorithm sub-module establishes multiple feature vectors corresponding to the posture during the fall process, codes and classifies the human skeleton and joint points. The module retrieves each calculation result of the curvature algorithm sub-module, the shape algorithm sub-module and the human body local configuration algorithm sub-module, and generates a state detailed information corresponding to all activities and postures of the care recipient, The machine learning sub-module uses a neural network to perform deep learning to train, adjust and optimize the judgment of the fall identification unit. The fall detection algorithm sub-module captures the curvature algorithm sub-module, the shape algorithm sub-module module, the human body local configuration algorithm sub-module, the optical flow algorithm sub-module and the bag-of-words algorithm sub-module to establish a fall detection and judgment model, the scale-invariant feature transformation algorithm The module performs image compensation for changes in the distance of each care recipient in the area. The fall and posture recognition method as described in item 1 of the patent application, wherein the image processing unit includes an intelligent visual electronic fence module, wherein the fall and posture recognition method It further includes a step a') before step a), the intelligent visual electronic fence module establishes an electronic fence block in each area captured by the image capture unit, wherein the step c) includes a step c1') Determine whether the care recipients are within the range of the electronic containment block or away from the electronic containment block. If the care recipients enter the range of the electronic containment block or leave the electronic containment block, From the block, it is determined that the care recipients have abnormal phenomena or behaviors, and the real-time notification message is issued. The fall and gesture recognition method described in item 1 of the patent application, wherein the image processing unit includes an algorithm module for preventing visual privacy leakage, wherein step b) includes a step a1") of preventing visual privacy leakage The algorithm module performs feature contour masking processing and dynamic image algorithm masking processing on the body image data of the care recipient to generate a pair of destructive masks of the body image of the care recipient with secure privacy masking. For the fall and posture recognition method described in item 1 of the patent application, the step b1") includes a step a11") that performs feature contour processing and dynamic image algorithm masking processing on multiple object images in the area to generate Multiple object image destructive masks corresponding to those objects with secure privacy masks. As for the fall and posture recognition method described in item 1 of the patent application, the image processing unit 2 further includes a night-time fall and posture model 24, wherein the step a) includes a step a1") of the night-time fall and posture model 24 The body image data is enhanced through fuzzy technology analysis and intelligent identification. The fall and posture recognition method described in item 1 of the patent application, wherein the fall and posture recognition system further includes a communication unit, the communication unit is signal-connected to the fall recognition unit, and the fall and posture recognition method further includes a Step d) The communication unit is used to send a caring inquiry message to each person being cared for. It can also understand the on-site situation of the person being cared for in an emergency and conduct two-way conversations. As for the fall and posture recognition method described in item 3 of the patent application, the care recipient analysis module 32 includes an abnormality detection unit 331 for people in the house, an abnormality detection unit 332 for a specific state for too long, and a work and rest routine. Statistics unit 333 and a history browsing unit 334, wherein step c3) includes the following steps: c31') The abnormality detection unit 331 of the person in the house determines whether the care recipients are in the predetermined area at a first predetermined time; c32') The abnormality detection unit 332 in which the specific state is too long determines whether the care recipients have remained motionless outside a second predetermined time, or whether they have been in the area for too long; c33’) The work and rest statistics unit 333 continuously and automatically records the home life and rest status of each person being cared for; and c34') The history browsing unit 334 is used to browse the home life and rest status of each person being cared for.

Images