KR20100056711A - Fall dection method using perspective image - Google Patents

Abstract

PURPOSE: A method for recognizing a faint action using a penetration image is provided to convert a fish-eye lens image of a traced person into a penetration image to recognize a faint action, thereby improving recognizing performance. CONSTITUTION: An input image including a target person is obtained using a camera using a fish-eye lens(S10). A mobile object is generated as a foreground image using the difference between the obtained input image and the background image(S11). The detected foreground image is elliptically mapped(S13). An input image including an elliptically mapped foreground image is converted into a penetration image(S14). A faint action is determined from the converted penetration image(S15).

Description

Stall motion recognition method using perspective image {Fall Dection Method using Perspective Image}

The present invention relates to a method for recognizing stunning motion, and more particularly, to a recognition technology for improving recognition performance by converting a fisheye lens image of a tracked person into a perspective image to recognize a stunning motion.

Falls are often found in older people and can lead to injuries or deaths, making them an important threat to the health of single people. The result of medical treatment for the stun, which is an important factor for health, is greatly influenced by the time until the stun is detected, the rescue request is made, and the emergency treatment is performed.

For this reason, stun motion detectors have been proposed to make emergency calls by automatically detecting stun motions using acceleration sensors and tilt sensors. However, according to this proposal, there is a problem that the user must always carry the device (see [1] and [2]).

In order to solve this problem, there is a method of detecting a stunning motion by detecting and tracking a person in real time from a camera image. Among the methods using the image, a method of reducing the necessary camera using a fish-eye lens camera or an omni-directional camera with a large field of view has been proposed [3], [4], [5], [6]). According to these methods, a person is detected in the fisheye lens image itself and a stunning motion is recognized by using a change in the size of the human area. However, this method has a problem that the recognition rate of the stunning operation is lowered.

Here, when the recognition rate of the stun movement is lowered, it means that even if the stun movement occurs, it is highly unlikely to recognize it as a stun movement, which may cause a fatal result in the health care of the single person.

[1] Y.B. Lee, J.K. Kim, M. T. Son and M.H. Lee, "Implementation of Accelerometer Sensor Module and Fall Detection Monitoring System based on Wireless Sensor Network," Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 2315-2318, 2007.

[2] M.R. Narayanan, S.R. Lord, M.M. Budge, B.G. Celler and N.H. Lovell, "Falls Management: Detection and Prevention, using a Waist-mounted Triaxial Accelerometer," Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 4037-4040, 2007.

[3] H. Nait-Charif and S. McKenna. "Activity summarisation and fall detection in a supportive home environment," In Proceedings of the 17th International Conference on Pattern Recognition (ICPR), volume 4, pp. 323-326, 2004.

[4] S.-G. Miaou, P.-H. Sung, and C.-Y. Huang, "A Customized Human Fall Detection System Using Omni-Camera Images and Personal Information," Proc. of Distributed Diagnosis and Home Healthcare (D2H2) Conference, pp. 39-42, 2006.

[5] M.-L. Wang, C.-C. Huang, H.-Y. Lin, "An Intelligent Surveillance System Based on an Omnidirectional Vision Sensor" IEEE Conference on Cybernetics and Intelligent Systems, pp. 1-6, 2006.

[6] Shaou-Gang Miaou, Pei-Hsu Sung and Chia-Yuan Huang, "A Customized Human Fall Detection System Using Omni-Camera Images and Personal Information", Proceedings of the 1st Distributed Diagnosis and Home Healthcare (D2H2) Conference, pp . 39-42, 2006.

Accordingly, an object of the present invention is to provide a stun gesture recognition method using a perspective image to improve recognition rate.

The above object is to obtain an input image including a subject using a camera to which the fisheye lens is applied; Generating a moving object as a foreground image by using a difference between the obtained input image and a background image; Mapping the detected foreground image to an ellipse; Converting an input image including the foreground image mapped to the ellipse into a perspective image; And extracting the size change, the position change, and the moving speed of the mapped ellipse from the converted perspective image to determine the stunning motion based on the movement, the stop, and the movement. Is achieved by

Preferably, in the generating the foreground image, the background image is dynamically updated using a Gaussian mixed model based adaptive background modeling method, and the RGB color model of the input image is converted into a YCbCr color model. In addition, the average brightness may be corrected so that the average brightness does not change suddenly, and the shadow of the subject may be removed by determining whether the shadow area is a shadow area using the brightness change and the color change amount.

In addition, preferably, after generating the foreground image, the method may further include removing noise in an area not included in the foreground image. More preferably, the noise removal may be performed by removing a region consisting of a predetermined number or less pixels from the connected foreground pixel region.

Preferably, in the elliptic mapping step, contour points of the connected foreground pixel areas may be tracked using an edge detection method, and mapped to the ellipses surrounding the outer points.

According to the above arrangement, by using the subject's perspective image, the recognition rate of the stunning motion can be significantly improved, which can greatly contribute to the health care of the lone.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart illustrating a stunning motion recognition method according to the present invention.

1. Input image acquisition

An input image photographed is acquired by using a camera installed at a corner of a ceiling or a wall of a room or a living room (step S10). As described above, in order to secure a wide viewing angle, a fisheye lens having a large viewing angle is applied to the camera.

FIG. 2 is an image representing an input image acquired through a fisheye lens, respectively, FIG. 2 (a) is an image in which the subject is standing, and FIG. 2 (b) is an image in which the subject is lying.

2. Create foreground image

A foreground image is generated from the obtained input image (step S11). In detail, a moving object is generated as a foreground image by using a difference between a background image and an input image. Here, two sub-steps may be performed to generate the foreground image.

Background detection and removal

Since the background image continuously changes as the light changes or the object moves, the background image is dynamically updated by using an adaptive background modeling method based on a Gaussian mixture model. In order to generate a background image, the system is initially limited to when there is no moving object in the input image, and receives an image of at least 300 frames.

Here, since the RGB color image is largely changed according to the influence of illumination, in the present invention, the RGB color model of the input image is converted into a YCbCr color model to separate the illumination from the color to reduce the change of the image due to the illumination change.

Also, in general, the camera automatically adjusts the exposure according to a change in lighting to automatically adjust the brightness of the captured image. When the brightness of the image changes drastically, the area may be mistaken as a moving object. Therefore, in the present invention, after converting the RGB color model to the YCbCr color model, the average brightness of the input image is obtained, and when the value changes from the average brightness for the last 10 frames to more than the threshold, the average brightness is corrected by correcting the image pixel value. Does not change abruptly.

Shadow detection and removal

As shown in FIG. 3 (a), when a moving object appears in the input image, the corresponding pixel becomes different from the background and is classified as a foreground pixel, leaving only the foreground pixel and removing the remaining pixels. Create

In addition, as shown in FIG. 3 (b), since the shadow of the moving object can be recognized as an object, the detection and removal of the shadow is necessary. In this case, since the brightness is dark in the shadow area and the original color is maintained, the shadow area is determined using the brightness change and the color change amount.

3. Noise Reduction

After generating the foreground image through the above process, the noise of the generated foreground image may be removed (step S12).

Noise may need to be removed because pixels may be classified as foreground pixels by noise even in areas not included in a moving object. To this end, the connected foreground pixel area is removed from the area consisting of four pixels or less. 4 illustrates an image in which the shadow and the noise are removed from the foreground image of FIG. 3 (b).

Also, preferably, the Morphology Close operation is applied to merge regions that are separated from each other. In other words, when a part of the moving object has a color similar to the background, the part of the object may be classified as a background and the object may be separated, thereby applying the operation to merge parts of the separated object.

4. elliptical mapping

In order to accurately know the size change, the position change, and the moving speed of the moving object, it is necessary to shape the moving object in a certain pattern. In the present invention, the moving object is mapped to an ellipse (step S13).

In order to map the moving object to an ellipse, the contour detection technique is used to track the outer points of the connected foreground pixel areas and to simplify the shape of the moving object area by mapping the outer points to the ellipse that best wraps them.

By tracking the ellipses detected from images continuously input over time, the size change, position change, and movement speed information of the ellipse are extracted.

At this time, the foreground pixel area may exist due to noise, but the area corresponding to the person exists in a certain range. In particular, since the subject is aware of the stunning motion, it is sufficient to select the largest one among the foreground pixel areas corresponding to the preset range.

FIG. 5 illustrates a result of elliptic mapping of a moving object, that is, a target, of the input image of FIG.

5. Perspective Image Conversion

Typically, when a stunner occurs, a person is in a lying position after a rapid gesture, and then a state of inactivity is continued. In operation S13, the stunning motion may be recognized using a change in the shape of the ellipse mapped to the subject, a movement speed, and the like.

According to the inventor of the present invention, since the change in the shape of the subject's body appears more clearly in the see-through image than the input image by the fisheye lens, the input image by the fisheye lens is converted into the see-through image in order to improve the recognition performance of the stunning motion. (Step S14).

Zimmermann-derived methods can be used to convert fisheye images into perspective images (S. Zimmermann and D. Kuban, "A video pan / tilt / magnify / rotate system with no moving parts," Proceedings of IEEE / AIA Digital Avionics Systems Conference, pp. 523-531, 1992.).

This will be described in detail below.

Referring to FIG. 6, a fisheye lens image coordinate system and a perspective image coordinate system are shown. In FIG. 5, DOV [x, y, z] is a vector representing a viewing direction, and the perspective image obtained when the viewing is observed in this direction at the origin of the fisheye lens image coordinate system. In this case, it is assumed that perspective image coordinate system the position UV plane the DOV [x, y, z] is perpendicular to the origin of the image coordinate system perspective DOV [x, y, z] in which the through.

The formula for mapping the coordinates (u, v) of the perspective image to the coordinates (x, y) of the fisheye lens image is shown in [Equation 1].

In Equation 1, R represents the radius of the fisheye lens image circle, m represents the enlargement ratio, φ represents the rotation angle in the perspective image, β represents the latitude angle between the axis and the line of sight direction, δ is X The hardness angle between the axis and the line of sight direction is shown. In the present invention, R , m , φ was used as a constant and the value was set to R = 160 , m = 1, φ = 0.

FIG. 7 illustrates a result of converting an elliptical mapped input image of FIG. 5 into a perspective image.

6. Determination of stunning action

The moving object is mapped to the ellipse from the converted perspective image in step S15 to extract the size change, the position change, and the moving speed of the ellipse to determine whether the movement, the stop, and the movement are similar to the stunning motion (step S15).

8 shows a process recognized as a stunning action. As shown in the result image, the size and position of the ellipse changes according to the behavior of the subject, and the width and height of the ellipse changes when the subject stands, walks, or lies down. do.

Table 1 shows the frame, horizontal and vertical size changes of the subject standing out and stunning.

frame Horizontal size Vertical size 652 51 96 662 46 77 668 38 54 674 44 45 681 43 49 687 53 54 700 74 65 704 89 37 740 84 37

As the results show, when the user is standing (652 frames), the height of the ellipse is larger than the width of the ellipse. However, it can be seen that the horizontal size is larger than the vertical size when the user is lying down (704 frames). This size change can be seen that the frame (frame) changes differently depending on the user's motion. The method of distinguishing between the simple lying down motion and the stunning motion can also be determined based on a characteristic in which the speed changes according to this motion.

Experiment example

As shown in FIG. 9, 27 stun motions were photographed through 10 testers according to 9 positions, and stun motion recognition and abnormal situation detection experiments were performed. Table 2 below shows whether or not the 10 experimenters recognized the stun movement along the three directions performed at nine positions. As can be seen from Table 2, the recognition rate of 86% was obtained by recognizing 233 stunning motions among 270 stunning motion images.

As a result, in the case of recognizing the stunning motion using the fisheye lens image as it is in Reference [6], as compared with the point where the recognition rate is 79.8%, the stunning movement is performed by converting the fisheye lens image into a perspective image as in the present invention. As a result of recognition, the recognition rate is improved. This is because when the recognition is performed by converting into a perspective image, the size of the ellipse according to the motion of the subject can be accurately known, so that the stunning motion can be recognized better.

According to the present invention, the subject's body is mapped to an ellipse for real-time tracking, the fisheye lens input image is converted into a perspective image, and the ellipses such as size change, position change, and movement speed of the ellipse mapped to the moving object are By using the shape change, the recognition rate of the stunning action can be increased. The adaptive background model is also robust against light changes and more robust in human detection and tracking through color model conversion and average brightness correction.

In the above description, the embodiment of the present invention has been described, but various changes can be made at the level of those skilled in the art. Therefore, the scope of the present invention should not be construed as being limited to the above embodiment, but should be interpreted by the claims described below.

1 is a flowchart illustrating a stunning motion recognition method according to the present invention.

FIG. 2 is an image representing an input image acquired through a fisheye lens, respectively, FIG. 2 (a) is an image in which the subject is standing, and FIG. 2 (b) is an image in which the subject is lying.

3 (a) is an input image, and FIG. 3 (b) is a foreground image obtained by removing only the foreground pixels and removing the remaining pixels from the input image.

4 illustrates an image in which the shadow and the noise are removed from the foreground image of FIG. 3 (b).

FIG. 5 illustrates a result of elliptic mapping of a moving object, that is, a target, of the input image of FIG.

6 shows a fisheye lens image coordinate system and a perspective image coordinate system.

FIG. 7 illustrates a result of converting an elliptical mapped input image of FIG. 5 into a perspective image.

8 shows a process recognized as a stunning action.

9 shows the placement of the stun positions for the experiment.

Claims (6)

Obtaining an input image including the subject using a camera to which the fisheye lens is applied; Generating a moving object as a foreground image by using a difference between the obtained input image and a background image; Mapping the detected foreground image to an ellipse; Converting an input image including the foreground image mapped to the ellipse into a perspective image; And And extracting the magnitude change, the position change, and the movement speed of the mapped ellipse from the converted perspective image to determine the stunning motion based on the movement, the stop, and the movement. Recognition method. The method according to claim 1, In generating the foreground image, Adaptive background modeling based on Gaussian mixed model dynamically updates background image, converts RGB color model of input image into YCbCr color model, and corrects average brightness so that average brightness does not change suddenly , And A method of recognizing stunning motion using a perspective image, characterized in that the shadow of the subject is removed by determining whether the shadow is a shadow area using a brightness change and a color change amount. The method according to claim 1, And after the generating of the foreground image, removing the noise in an area not included in the foreground image. The method according to claim 3, The noise elimination method is performed by removing a region consisting of a predetermined number of pixels or less from a connected foreground pixel region. The method according to claim 1, In the elliptic mapping step, A method of recognizing stunning motion using a perspective image, characterized by tracking the outer points of connected foreground pixel areas using an edge detection technique and mapping the outer points to an ellipse surrounding the outer points. The method according to claim 1, The perspective image conversion is a stun gesture recognition method using a perspective image, characterized in that through the following equation.

Here, (u, v) is the coordinate of the perspective image, (x, y) is the coordinate of the input image, R represents the radius of the fisheye lens image circle, m represents the magnification ratio, φ represents the perspective image Denotes a rotation angle at, β denotes a latitude angle between the axis and the visual direction, and δ denotes a longitudinal angle between the X axis and the visual direction.

Images