TWI391881B - Dodge image processing capacity of the fall detection and care system - Google Patents

Description

Fall detection and care system with multi-image processing capability

The invention relates to the technical field of a fall detection and care system, in particular to a fall detection and care system with multi-image processing capability which is more efficient in computing.

The conventional fall detection and care system generally requires the user to wear a micro-sensing instrument such as an accelerator on the body, and then use the signal of the human activity to identify, which is easy to cause the user to carry, and the lack of on-site image assistance is likely to cause misjudgment. Secondly, the system that uses the camera to perform fall detection and care is mostly based on the image processing of a single camera. If it is directly expanded into multiple cameras, and the accelerated calculation algorithm is not designed for multi-image stream, the system performance and image The quality will be very low.

Moreover, the current security surveillance system commonly used in the market is to use a hardware capture card to connect multiple cameras for screen monitoring, but only the image transmission, or simply detect the foreign object invasion function, fail to perform object recognition function, resulting in Can not be applied to falls detection and care related services, there are still many limitations and problems in the conventional fall detection and care system, which is necessary for improvement.

Conventional fall detection and care system The user is required to wear a micro-sensing instrument such as an accelerator on the body, and then use the signal of the human activity to identify, which is easy to cause the user to carry, and the lack of on-site image assistance is likely to cause misjudgment. Secondly, the system that uses the camera to perform fall detection and care is mostly based on the image processing of a single camera. If you expand directly to multiple cameras and you do not design an accelerated calculation algorithm for multi-image streaming, system performance and image quality will be very low. Moreover, the current security surveillance system commonly used in the market is to use a hardware capture card to connect multiple cameras for screen monitoring, but only the image transmission, or simply detect the foreign object invasion function, fail to perform object recognition function, resulting in Can not be applied to the services related to fall detection and care, and there are still many limitations and problems in the conventional fall detection and care system.

Provide a multi-image image processing capability The fall detection and care system comprises: a plurality of cameras, a multi-frame image processing server and at least one full-end warning device. Each of the cameras is configured to capture video stream data. The multi-frame image processing server is connected to each camera through a network. The multi-frame image processing server includes a memory structure, and the multi-frame image processing servo The device can determine whether there is a fall accident by performing image processing and a fall-like recognition algorithm through the image stream data captured by each camera. The image processing and fall-type recognition algorithm includes four threads. : a. Image capture thread, the image capture thread is saved before the system is started, and an unmanned scene map is saved for later comparison. When the camera processing device is started, the camera shoots at a fixed rate. The live screen and the live screen are stored in the memory structure; b. the image processing thread, which filters out the difference between the scene image and the original unmanned scene map by using the image processing action So that the object image will not be distorted; c. Humanoid contour extraction thread, the humanoid contour extraction thread will The outline of the object in the image is depicted as a humanoid judgment; d. The fall type recognition thread is to compare the humanoid contour of the live scene with a plurality of fallen patterns The fall type is a pre-extracted contour feature point and stored in the system. Therefore, after the human-shaped contour feature of the live shooting picture is compared with the characteristics of the fall-like type, the fall accident can be interpreted according to the comparison result. Occurs; the multi-image image processing server can perform multi-image image stream computing, and the multi-image image stream computing operation is multithreading and pipelining operation, and each thread is responsible for a single step of work. All the threads are operated according to the pipelined schedule, so that when the first video stream data of the first camera enters the system, the image capture thread executes the image capture operation, and after the operation is completed, The captured image is stored in the memory structure, and then the image processing thread performs image processing operations. At the same time, the image capture thread is immediately connected. The second video stream data of the second camera repeats the image capturing work, and after the image capturing thread is paralleled with the image processing thread, the first image stream data wheel is executed by the human contour extraction The second image stream data is transferred to the image processing thread operation, and so on, the system can have up to four parallel operation threads at the same time, thereby achieving multiplex processing at the same time, thereby accelerating the image operation. The security end warning device includes a display screen, and the security end warning device and the multi-image processing server are communicably connected to each other; thereby, when a community person has a fall accident, the camera 撷Sending the image stream data to the multi-image processing server for the image The decision is made by the fall and fall recognition algorithm operation, and a warning is sent to the display screen of the security warning device; after the security guard or the medical staff receives the warning, the immediate rescue and security actions can be performed.

Wherein, the security end warning device can be a handheld device.

Wherein, the handheld device can be a PDA.

Wherein, the handheld device can be a mobile phone.

Wherein, the handheld device can be a palmtop computer.

Wherein, the security end warning device can be a desktop computer.

Among them, the network can be wired or wireless.

1. The present invention utilizes image processing and pattern recognition technology to complete the service of fall detection and care, and the user does not need to wear any micro sensor to the body.

Second, the present invention develops a fall image recognition algorithm, which uses multiple cameras to take different scene images, instantly captures the contour points of the human figure and compares them with the "fall-like" type, which greatly simplifies the image processing calculation, and Does not affect the recognition accuracy rate or false positive rate.

Third, the present invention develops a multi-image image stream acceleration calculation algorithm, which can greatly improve the system image processing performance.

4. The present invention immediately transmits a warning message to a guard or medical staff to view when a person may fall.

5. The present invention is developed using a personalized computer standard architecture without the need to attach a specific software or hardware module.

The above-mentioned embodiments of the present invention will be described with reference to a preferred embodiment and a detailed description with reference to the drawings. , structure and its characteristics, when you can get a deep and specific understanding.

Referring to FIG. 1 to FIG. 7 , the present invention provides a fall detection and care system with multi-image processing capability, which comprises: a plurality of cameras (10) and a multi-frame image processing server (20). And at least one full-end warning device (30). Each camera (10) is configured to capture video stream data; the multi-frame image processing server (20) is connected to each camera (10) through a network, and the multi-frame image processing server (20) Including a memory structure, the multi-image processing server (20) can perform image processing and a fall-like recognition algorithm through the image stream data captured from each camera (10) to determine whether there is a fall accident. Occurs, the image processing and fall-type recognition algorithm includes four threads: a. image capture thread, the image capture thread system is saved before the system is started, and an unmanned scene map (60) is saved. For later comparison, when the photographic processing device is activated, the camera (10) captures the live picture (70) at a fixed rate, and stores the live picture (70) into the memory structure; b. image processing execution The image processing thread is an object (80) that distinguishes between the live screen (70) and the original unmanned scene map (60), and uses the image processing action to filter out the object so that the object is not distorted; c. Contour extraction thread, the contour extraction Thread system The contour (90) of the object (80) in the image is depicted as a humanoid judgment; d. the fall-type recognition thread is the humanoid contour of the live image (70) ( 90) Comparing with a plurality of falling patterns, which are pre-extracted contour feature points and stored in the system, so that the human-shaped contour (90) feature of the live shooting picture and the characteristics of the falling type are performed one by one. After the comparison, the occurrence of the fall accident can be judged according to the comparison result; the multi-image processing server (20) can perform multi-image video stream computing, and the multi-image video stream operation is multi-threading (Multithreading) Parallel (pipelining) operation, each thread is responsible for a single step of work, all threads are operated according to pipelined schedule, whereby when the first video stream data of the first camera enters the system, the image The operation of capturing the image is performed by capturing the thread. After the operation is completed, the captured image is stored in the memory structure, and then the image processing thread performs the image processing operation. At the same time, the image is captured. The thread receives the second image stream data of the second camera and repeats the image capturing work. After the image capturing thread is paralleled with the image processing thread, the first image stream data wheel is The humanoid contour extraction thread, the second image stream data is transferred to the image processing thread operation, and so on, the system can have four parallel computing threads at the same time, thereby achieving multiplex processing at the same time. The security warning device (30) includes a display screen, and the security warning device (30) and the multi-image processing server (20) can communicate with each other. The ground connection; thereby, when the community personnel has a fall accident, the camera (10) retrieves the image stream data and transmits the image stream data to the multi-image image processing server (20) for performing the image processing and the fall type recognition algorithm operation. A judgment is made and a warning is sent to the display screen of the security warning device (30); after the security guard (40) or the medical staff (50) receives the warning, the immediate rescue and security actions can be performed.

The security end warning device (30) can be a handheld device.

Wherein, the handheld device can be a PDA.

Wherein, the handheld device can be a mobile phone.

Wherein, the handheld device can be a palmtop computer.

The security end warning device (30) can be a desktop computer.

Among them, the network can be a wired network or a wireless network.

Further, a detailed description of the embodiments of the present invention is as follows, and the usage scenario of the present invention is as shown in the first figure. In the first picture, assuming that the elderly are active and prone to fall accidents, such as sidewalks, playgrounds, parks, track and field, etc., we set up cameras (10) in each place and transmit the images back to the multi-image processing servo. Falling pattern recognition is performed in the device (20). When the multi-image processing server (20) discriminates that a fall accident occurs in the screen, the mobile phone message is immediately transmitted to the guard (40) or the medical staff (50).

The system screen on the multi-image processing server (20) is as shown in the second figure. We arrange the multi-screens above the screen and number them sequentially. It is a camera No. 0, a camera No. 1, etc. At the bottom of the screen are nine types of “fall-like” patterns for identification. The similarity between the current picture and the fall model is listed next to each image. The smaller the number, the more similar it is. When the continuous shooting pictures are similar to the same fall type, the system will issue a mobile phone warning message, as shown in the third figure.

The following describes the fall recognition algorithm for the single-frame image of this system and the multi-image image stream acceleration operation algorithm.

First, the fall image recognition algorithm:

Falling image recognition is divided into four steps, namely:

Step 1: Image capture thread

Before the system starts, first save an unmanned scene map (60), as shown in Figure 4A, for later comparison. When the system is started, the live picture (70) is taken at a fixed rate, as shown in FIG. 4B, and the live picture is stored into the memory structure;

Step 2: Image Processing Thread

The main task of this step is to filter the object (80) between the live screen (70) and the original unmanned scene map (60) by image processing, and ensure that the object is not distorted (such as the fifth A). Figure shows).

Step 3: Humanoid outline extraction thread

The work of this step is to depict the outline (90) of the object (80) in the image as a humanoid judgment (as shown in Figure 5B). The more complete and unbroken this part of the contour (90), the more accurate the judgment.

Step 4: Falling the sample recognition thread

The human figure (90) of the live picture (70) is mainly compared with nine similar fall patterns. These nine similar fall types can be divided into three categories: lying, kneeling, and sitting. These samples will be taken out of the contour feature points in advance through the same processing steps described above and stored in the system. Therefore, the human figure (90) of the live picture (70) is compared with these feature points one by one, and the fall or not can be judged according to the comparison result.

The above four steps will be repeated. If there is a fall message on the continuous screen and the cumulative number of times exceeds a threshold, the probability of a fall accident on the screen is high, so a mobile phone newsletter is sent to notify the medical staff (50). The entire program can be shown in the sixth picture.

Second, multi-image video stream acceleration calculation algorithm:

When the camera is connected to multiple cameras (10) for fall recognition in different scenes, each camera (10) will generate video streams, and each stream will go through the above four operation steps: Image Fetch Thread, Image Processing thread, Sequence Generation thread, and Patten Recognition thread. For the convenience of the following description, we will refer to these four steps as IF, IP, SG, and PR.

In order to speed up the operation of multi-image stream, we use the principle of multithreading and pipelining. Each thread is responsible for a single step, and all threads are based on pipelined scheduling. To operate, as shown in the seventh picture. When the first stream image data of the first camera enters the system, the image capture thread (IF) That is, the image capture operation is performed, and after the operation is completed, the captured image is written into a specific data structure in the memory structure. Then, the image processing thread (IP) takes over the work of image processing. At the same time, the image capture thread (IF) immediately receives the second image stream data of the second camera, and repeats the image capture operation. After the image capture thread (IF) and the image processing thread (IP) are parallelized, the first image stream data wheel is operated by the humanoid contour extraction thread (SG), and the second image stream data is The image processing thread (IP) operation is performed, and the first image stream data is retrieved by another newly received first image (IF) operation, so at this point of time, the system is equal to three executions. The parallel operation is performed. By analogy, there are up to four threads of parallel operations in the system. As a result of this scheduling, the system will have a stable throughput (Throughput) to process the video stream from each camera, and achieve multiplex processing to accelerate the image calculation.

The fall recognition algorithm and the multi-image stream acceleration operation algorithm covered by the present invention can be easily implemented on a standard architecture of a personal computer and obtain good performance. Moreover, the above algorithm can be applied not only to the fall care of the silver-haired family but also to the daily life of the elderly, and can be developed into an industrial application related to intelligent home preservation and detection of foreign body invasion.

In summary, the system uses image processing and pattern recognition technology to complete the service of fall detection and care, users do not need to wear any micro-sensor on the body. Secondly, the present invention utilizes the principle of multi-threading and pipelined scheduling to complete the parallel computing operation of the single-machine four-frame image processing. Furthermore, the present invention immediately transmits a warning message to the guard when it is recognized that a person may fall. Or medical staff to go to see, making care more efficient. In addition, the present invention is developed using a personal computer standard architecture without the need to attach a specific software or hardware module. In addition, the fall image recognition algorithm developed by the invention utilizes multiple cameras to capture different scene images, instantly captures the contour points of the human body and compares them with the "fall-like" type, which greatly simplifies the image processing calculation. It does not affect the recognition accuracy rate or false positive rate. Moreover, the system has developed a multi-image image stream acceleration calculation algorithm, which can greatly improve various effects such as system image processing performance.

The foregoing is a description of the technical features of the present invention, and the present invention can be modified and modified without departing from the spirit and scope of the present invention. And modifications should be covered in the scope defined by the scope of the patent application below.

(10) ‧‧‧ camera

(20)‧‧‧Multi-image processing server

(30) ‧ ‧ full-end warning device

(40) ‧‧‧Guards

(50) ‧ ‧ medical staff

(60) ‧‧‧No-man scene map

(70)‧‧‧ Live screen

(80) ‧ ‧ objects

(90) ‧ ‧ outline

(IF)‧‧‧Image Capture Thread

(IP)‧‧‧Image Processing Thread

(SG)‧‧‧ Humanoid Profile Extraction Thread

(PR)‧‧‧ Falling sample recognition thread

First Figure: A schematic diagram of an implementation architecture of a possible embodiment of the present invention.

Second Figure: A schematic diagram of the actual operation of a possible embodiment of the present invention.

The third figure is a schematic diagram of a transmission mobile phone warning message according to an embodiment of the present invention.

Figure 4A is a schematic illustration of an unmanned background of a possible embodiment of the present invention.

Figure 4B is a schematic diagram of the actual shooting of the scene in the embodiment of the present invention.

Figure 5A is a schematic diagram of a human figure after the background of the embodiment of the present invention.

Figure 5B is a schematic diagram of the contour of a human figure according to an embodiment of the present invention.

Figure 6 is a schematic diagram of the fall detection identification process in the embodiment of the present invention.

Figure 7 is a schematic diagram of multiple threads and pipeline parallel operations in a possible embodiment of the present invention.

(10) ‧‧‧ camera

(20)‧‧‧Multi-image processing server

(30) ‧ ‧ full-end warning device

(40) ‧‧‧Guards

(50) ‧ ‧ medical staff

Claims (8)

A fall detection and care system with multi-image processing capability includes: a plurality of cameras, each camera is used to capture video stream data; and a multi-frame image processing server, the multi-frame image processing server is transmitted through The network is connected to each camera, and the multi-image processing server includes a memory structure, and the multi-image processing server can perform image processing and fall-like recognition through the image stream data captured by each camera. The algorithm determines whether there is a fall accident. The image processing and fall-type recognition algorithm includes four threads: a. image capture thread, the image capture thread is saved before the system is started. An unmanned scene map for later comparison. When the photographic processing device is activated, the camera is used to capture a live scene and stored in the memory structure; b. an image processing thread, the image processing thread An object that differs from the original unmanned scene map in the live screen is filtered out by the action of the image processing so that the object is not distorted; c. human contour Taking the thread, the humanoid contour extraction thread draws the outline of the object in the image as a humanoid judgment; d. falls the sample recognition thread, and the fall type recognition thread is the live scene The contour of the human figure is compared with a plurality of falling patterns, which are pre-extracted contour feature points and stored in the system, so that the human contour features of the live shooting image are compared with the features of the falling pattern one by one. , according to the comparison result to determine whether a fall accident occurs; The multi-image image processing server can perform multi-image image stream computing, the multi-image image stream operation is multi-thread and pipeline parallel operation, each thread is responsible for a single step of work, all threads are based on pipelined The operation of the program, after the first video stream data of the first camera enters the system, the image capture thread executes the image capture operation, and after the operation is completed, the captured image is stored to the program. In the memory structure, the image processing thread performs image processing operations. At the same time, the image capturing thread immediately receives the second image stream data of the second camera, repeating the image capturing work, and waiting for the image. After the thread is paralleled with the image processing thread, the first image stream data wheel is subjected to the humanoid contour extraction thread operation, the second image stream data is subjected to the image processing thread operation, and so on. The system can have up to four parallel computing threads at the same time, so that the multiplex processing can be performed at the same time, thereby accelerating the effect of the image computing; and, at least one The full-end warning device includes a display screen, and the security-end warning device and the multi-image processing server are communicably connected to each other; thereby, when a community person has a fall accident, the camera The image stream data is transmitted to the multi-image image processing server to perform the image processing and the fall-type recognition algorithm operation to make a judgment, and send a warning to the display screen of the security warning device; the guard or After receiving the warning, the medical staff can perform immediate rescue and preservation actions. The fall detection and care system with multi-image processing capability as described in claim 1, wherein the security warning device can be one Handheld device. The fall detection and care system with multi-image processing capability as described in claim 2, wherein the handheld device can be a PDA. The fall detection and care system with multi-image processing capability as described in claim 2, wherein the handheld device can be a mobile phone. The fall detection and care system with multi-image processing capability as described in claim 2, wherein the handheld device can be a palmtop computer. The fall detection and care system with multi-image processing capability as described in claim 1, wherein the security warning device can be a desktop computer. A fall detection and care system having multiple image processing capabilities as described in claim 1 wherein the network can be a wired network. A fall detection and care system having multiple image processing capabilities as described in claim 1 wherein the network can be a wireless network.