KR100885418B1 - System and method for detecting and tracking people from overhead camera video - Google Patents

Abstract

A system and a method for detecting and tracking people from an overhead camera video are provided to detect and track a person, who exists with another object or other people, accurately through an image of an overhead camera. An image analyzer receives a new video frame from a camera(S101), and extracts a region where motion occurs from the received image(S102). The image analyzer detects an edge pixel from the received image(S103), and extracts an oval from the edge pixels(S104,S105). The image analyzer carries out an oval filtering(S106), and matches the detected ovals with the previously tracked oval objects(S107). The tracking information of the tracking object is updated when a specific tracking object corresponds to the detected oval(S108). When there is no tracking object corresponding to the detected oval, the tracking object registers the detected oval as a new track object(S110).

Description

System and method for detecting and tracking people in overhead camera images {SYSTEM AND METHOD FOR DETECTING AND TRACKING PEOPLE FROM OVERHEAD CAMERA VIDEO}

The present invention relates to a system for detecting and tracking a person from a CCTV camera image, and more particularly, to a system for extracting and tracking an individual person using a model of a human head from an overhead camera image. And to a method thereof.

Existing CCTV system is a passive surveillance system, which depends entirely on the interpretation of the images from CCTV cameras. However, in recent years, with the improvement of computer processing capability and the development of computer vision technology, the computer analyzes CCTV camera images in real time. Intelligent video surveillance systems that detect and track moving objects and detect meaningful events are being introduced. The intelligent video surveillance system can be used for various tasks depending on the internal video analysis function. It can count the number of visitors or traffic in certain areas as well as security / surveillance tasks such as intrusion detection and detection of unattended objects. It can also be used for statistical analysis tasks such as measuring data.

In particular, the image-based People Counting System, which counts the number of people passing through or occupying a certain area, has many advantages over existing counting systems using infrared sensors or mechanical gate devices. Is getting.

A typical image-based person counting system is configured to detect and track moving objects through image analysis by receiving images from an overhead camera and count the number of people passing or occupying a specific area based on the image. . Here, the overhead camera refers to a camera which is installed to face the ground vertically from the ceiling, and the reason why the overhead camera image is preferred in the conventional image-based human counting system is that the overlap of the objects in the image is minimized, This is because detection / tracking and counting accuracy can be improved.

Various known technologies exist for detecting and tracking moving objects through image analysis or for image-based object counting systems. For example, using template and Kalman filters in video images with Korean Patent Registration No. 10-0312427. Moving object tracking method and apparatus "," Multiple moving object tracking / monitoring system "with Korean Patent Registration No. 10-0364582," Passing / Outgoing Counting Device by Video Camera Image and Korean Patent Registration No. 10-0625676 and its Control Method "," Automatic Classification and / or Counting System "with International Patent Publication No. WO 02/097713," System and Method for Counting People near External Windowed Doors "with International Patent Publication No. WO 2007/067721, International Patent Publication "Target Detection and Tracking from Overhead Video Streams", WO 2007/002404, and "Human Detection and Track, International Patent Publication No. WO 2007/086826." ing for Security Applications "," People Counting Systems and Methods ", US 2006/0067456, and" System and Method for Counting People, "US 2008/0008360.

As a limitation of the known techniques, there is a lack of a mechanism that separates a person by using the shape characteristics of a person in an image, so that counting errors may occur when several people are crowded or when a person moves with an object such as a shopping cart. Some of the known techniques, which include a mechanism for separating a person from an image, have a problem in that they do not work properly in an overhead camera image.

According to an embodiment of the present invention, a computer receives an image from an overhead camera and analyzes it in real time, extracts and tracks individual people present in the image, and counts the number of people based on the tracking result. It provides a system and method for performing the same.

A system for detecting and tracking a person in an overhead camera image according to an aspect of the present invention,

A camera fixed on the overhead to photograph a person; Analyze the image taken by the camera to detect edge pixels in the motion region, detect ellipse objects from the detected edge pixels, track the detected ellipse objects, detect a specified event from the tracking results of the ellipse objects, An image analyzer for counting the tracking objects that generated the detected event; And receive and process the tracking result of the ellipse objects, the event detection result, the counting result of the tracking objects, etc. from the image analyzer, and display or store the processed result. And a control / monitoring terminal.

The image analyzer may detect and track a person in the overhead camera image, wherein the ellipse object has an attribute value of a preset range in detecting the ellipse object.

The setting of the range of the property value of the ellipse object to be detected is based on the property value of the ellipse that approximates the head circumference of the person in the obtained overhead camera image after photographing the person on the overhead using the camera. Set the range of the attribute value of the ellipse object to be detected.

Method for detecting and tracking a person in an overhead camera image according to a feature of the present invention

Acquiring an input image from a fixed camera; Detecting a motion from the input image to extract a motion region; Detecting edge pixels of the input image within the motion region; Extracting curve segments from the edge pixels; Grouping the curve segments to detect elliptic objects having attribute values in a preset range; Tracking the ellipse objects by matching the detected ellipse objects with the ellipse objects being tracked; Detecting a specified event using a tracking result of the ellipse objects; And counting ellipse objects that caused the detected event.

Extracting curve segments from the edge pixels comprises: grouping neighboring ones through tracking and concatenation of the edge pixels to extract primary curve segments and write branch points; With respect to the first curve segments, moving from the start point to the end point of each curve segment, find a point where the curvature value of each point is larger than the curvature value of the surrounding points within a predetermined interval about the point, and the curvature at the found point Extracting secondary curve segments by dividing the curve segment at that point if the value is greater than or equal to the set value; And extracting tertiary curve segments for the secondary curve segments by merging a pair of curve segments having a curvature value at or below a set value into a single curve segment among the pairs of curve segments that meet at the recorded branch point. Making; And extracting the final curve segments by removing curve segments that are not suitable for extracting an ellipse object having a specified attribute range for the cubic curve segments.

Setting a range of attribute values of the elliptic object to be detected may include: photographing a person on an overhead using the camera; Designating, by the user, an ellipse approximating a head circumference of a person in the captured image; And setting a range of attribute values of the elliptic object to be detected using the attribute values of the designated ellipse.

Setting a range of attribute values of the elliptic object to be detected may include: photographing a person on an overhead using the camera; Detecting edge pixels in the captured image; Extracting ellipses from the edge pixels; Selecting an ellipse from the extracted ellipses to approximate a human head circumference; And setting a range of attribute values of the elliptic object to be detected using the attribute values of the selected ellipse.

The matching of the detected ellipse objects and the ellipse objects being tracked is determined by the area of an area shared by the detected ellipse object and the ellipse object being tracked.

Detecting a specified event using the tracking result of the ellipse objects,

Detect an event where a specific ellipse object exists within a user-defined area for a certain time,

And detecting content of a specific ellipse object passing through a virtual line designated by a user.

According to one of the problem solving means of the present invention described above, even if a number of people in a group through an overhead camera image, even if a moving object other than the person coexists accurately detects and tracks each person, and various tracking results Implement a system that can be used for applications.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

Throughout the specification, when a part is "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element in between. . In addition, when a part is said to "include" a certain component, this means that it may further include other components, except to exclude other components unless otherwise stated.

Hereinafter, a system and method for detecting and tracking a person in an overhead camera image according to an embodiment of the present invention will be described with reference to FIGS. 1 to 5.

1 is a diagram illustrating a configuration of an overhead camera image-based person detection / tracking system according to an embodiment of the present invention.

Referring to FIG. 1, the configuration of an overhead camera image-based person detection / tracking system 100 according to an embodiment of the present invention includes an overhead camera 101, an image analyzer 102, and a control / monitoring terminal 103. It includes.

The overhead camera 101 is installed to face the ground vertically from the ceiling. 3 is a typical example of photographing a person underneath with an overhead camera.

The image analyzer 102 acquires a series of video frame data from the overhead camera 101, analyzes it in real time to detect and track a person, detects a designated event based on the tracking result, Perform the specified task when detecting the specified event. In addition, the image analyzer 102 communicates with the control / monitoring terminal 103 through various communication means. The image analyzer 102 transmits various data such as the acquired video frame data, the detection / tracking result of the person, the detected event information, the counting information of the person being tracked, and the like to the control / monitoring terminal 103 when necessary, and controls / monitors the data. Various settings and control data are received from the terminal 103 and applied.

The control / monitoring terminal 103 may display or store various data received from the image analyzer 102 on the screen, search and play the stored data, and receive the setting and control information of the image analyzer 102 from a user. Passes configuration and control data to analyzer 102.

The image analyzer 102 may be connected to various I / O devices 104 in addition to the control / monitoring terminal 103. For example, sensing results may be input from various sensors, and the results obtained through image analysis such as coefficient information of a person may be output to a dedicated display device.

The configuration of an overhead camera image-based person detection / tracking system 100 according to an embodiment of the present invention is not limited to that shown in FIG. 1. For example, the image analyzer 102 may be integrated into the overhead camera 101 or the control / monitoring terminal 103. It may also be used in connection with simple I / O devices without dedicated control / monitoring terminal 103, and these I / O devices may be integrated into the image analyzer 102.

2 is a flowchart illustrating an overhead camera image-based person detection / tracking method according to an embodiment of the present invention.

The image analyzer 102 repeatedly performs steps S101 to S112 of FIG. 2 to perform detection / tracking of a person in the overhead camera image.

In step S101, the image analyzer 102 receives a new image, that is, a new video frame, from the camera.

In operation S102, the image analyzer 102 extracts a region where motion occurs from the input image. The area in which the motion occurs can be said to include the area of the moving person potentially, and the efficiency of image processing can be increased by considering only the motion area without considering the premise area when extracting the human from the image.

There are several well-known techniques for extracting a motion region, and the Background Subtraction method using a difference between a background image and an input image is representative. Background Subtraction has various types of methods depending on a method of obtaining a background image (or a background model), a method of obtaining a difference between a background image / model and an input image, and a method of updating the background image / model. See, eg, C. Wren, A. Azarbayejani, T. Darrell, A. Pentland, "Pfinder: Real-Time Tracking of the Human Body" (IEEE Trans. On PAMI, Vol. 19, No. 7, 1997), C Stauffer, W. Grimson, "Learning Patterns of Activity Using Real-Time Tracking" (IEEE Trans. On PAMI, Vol. 22, No. 8, 2000), A. Elgammal, D. Harwood, L. Davis, "Non -parametric Model for Background Subtraction "(Proc. of ICCV '99 FRAME-RATE Workshop, 1999).

In operation S103, the image analyzer 102 detects edge pixels from the input image. At this time, detection of the edge pixel is performed only within the motion region obtained in step S102. There are several known techniques for obtaining edge pixels in an image, and the present invention uses a method for obtaining edges of 1-pixel thickness. For example, after detecting the primary edge pixels using Sobel Edge Detector, we can create 1-pixel thick edge using thinning method or use Canny Edge Detector proposed by J. Canny. For Canny Edge Detector, see "A Computational Approach to Edge Detection" (IEEE Trans.on PAMI, Vol. 8, No. 6, 1986). FIG. 4 is a result of extracting edge pixels from the image of FIG. 3.

The image analyzer 102 extracts an ellipse from the edge pixels through steps S104 and S105. As shown in FIG. 4, the human head may be modeled in an ellipse shape in the overhead camera image. Once the fixing of the camera is completed, the human head in the image has a substantially similar size and shape. Therefore, it is possible to isolate a person by finding an ellipse having an appropriate size and shape in the overhead camera image.

In order to extract the ellipse, first, in step S104, the image analyzer 102 extracts meaningful curve segments from the edge pixels obtained in step S103.

5 is a flowchart illustrating a method of extracting curve segments from edge pixels.

In step S201, the image analyzer 102 is given an edge pixel tracking and linking method when an edge map (binary image having an edge pixel having a value of 1 and other pixels having a value of 0) is given. By using the 8-Connected Neighbor format, edge pixels connected to each other are grouped into one curve segment. When the branch point (the point where two or more progress paths occur) is reached during the tracking of the edge pixels, further tracking is stopped and the edge pixels tracked so far are grouped into one curve segment. The branch points are recorded separately and used in merging the curve segments in step S203.

라고 하고, 벡터

와 벡터

사이의 각(angle)을

라고 하면, 픽셀

에서의 곡률 값은

로 주어진다. 찾은 지점에서의 곡률 값이 설정 값 이상이면 그 지점에서 커브 세그먼트를 분할함으로써 하나의 커브 세그먼트 내에서 급격하게 꺾기는 부분이 없도록 만든다.In step S202, the image analyzer 102 moves for each curve segment from the start point to the end point of each curve segment while the curvature value of each point is greater than the curvature value of the surrounding points within a predetermined interval about the point. When the point is found and the curvature value of the found point is more than the set value, the curve segments are extracted by dividing the curve segment at the point. At this time, the coordinates of the i th edge pixel among the edge pixels constituting the curve segment

Say, vector

Vector with

The angle between

Speaking of pixels

The curvature value at is

Is given by If the curvature value at the found point is more than the set value, the curve segment is divided at that point so that there is no sharp bend in one curve segment.

In operation S203, the image analyzer 102 merges a pair of curve segments satisfying the condition into one curve segment with respect to the curve segments that meet at the branch point found in operation S201. The merge condition is a case where the curvature value at the branch point is a certain value or less when a pair of specific curve segments are connected. This makes the curve segments that run smoothly at the branch point into one curve segment.

In step S204, the image analyzer 102 performs curve segment filtering to remove unnecessary curve segments. As described above, the reason for obtaining the curve segments is to extract the ellipse by grouping the curve segments in step S105 of FIG. 2. Since the ellipse to be extracted approximates the shape of the human head, once the camera is fixed, it is within a certain range. Size and shape. Therefore, not all curve segments obtained through steps S201 to S203 are suitable for constructing an ellipse to be obtained in step S105 of FIG. 2. Therefore, by eliminating unnecessary curve segments in advance in step S204, it is possible to extract desired ellipses with a smaller amount of calculation in step S105 of FIG. Unnecessary curve segments are removed, for example, if the length of the curve segment is greater than the maximum of the perimeter of the permissible ellipse, or if the distance between both ends of the curve segment is greater than the maximum of the length of the long axis of the permissible ellipse.

Returning to FIG. 2, in step S105, the image analyzer 102 extracts an ellipse by grouping the curve segments extracted in step S104. There are several known techniques for grouping curve segments to extract ellipses. See, for example, "A Robust, Real-Time Ellipse Detector" (Pattern Recognition, Vol. 38, No. 2, 2005) by SC Zhang, ZQ Liu, "Arc-Based Evaluation and Detection of Ellipses by Y. Qiao, SH Ong. '' (Pattern Recognition, Vol. 40, No. 7, 2007), "A Hierarchical Approach for Fast and Robust Ellipse Extraction" by F. Mai, YS Hung, H. Zhong, WF Sze 8, 2008).

In operation S106, the image analyzer 102 performs elliptic filtering. As described above, once the camera is fixed, the area and shape of the ellipse corresponding to the human head in the overhead camera image have a certain range. Therefore, ellipses whose area and shape are out of the specified range are removed from the ellipses obtained in step S105. At this time, the shape of the ellipse is represented by the ratio of the length of the long axis and short axis of the ellipse. In addition, in the case of ellipses having overlapping areas among the obtained ellipses, the other ellipses are removed except an ellipse having an area and a shape closest to a reference ellipse designated by the user.

In order to set a range for the area and shape of the ellipse to be obtained, the user should designate the ellipse as a reference to the image analyzer 102 in advance. As a method of designating a reference ellipse, manual or semi-automatic methods can be considered. In the manual method, a person is placed under the camera in the system setting mode, and then a reference still image is acquired, and then the user draws an ellipse surrounding the area corresponding to the human head with the mouse in the reference still image. The semi-automatic method is a method in which the user selects an appropriate ellipse when the image analyzer 102 extracts ellipses from the reference still image and presents them to the user.

In operation S107, the image analyzer 102 performs matching between the detected ellipses and the ellipse objects that are being tracked. There may be many matching methods, but in the simplest method, the degree of overlap between the ellipsoid object being tracked in the current frame and the detected ellipse is measured, and the overlapping area is considered to match each other. The position of the tracking object in the current frame simply uses the same value as the position in the previous frame, or based on the position of the tracking object in past frames, using the expected value. There is a way. Kalman Filter can be used to estimate the position in the current frame.

Based on the matching result in step S107, the image analyzer 102 performs steps S108 to S110 for each tracking object or detected ellipse.

In operation S108, when the specific tracking object corresponds 1: 1 with the detected ellipse, the image analyzer 102 updates the tracking information of the tracking object. The tracking information of the tracking object includes the current position, shape, and movement trajectory of the object.

In operation S109, if there is no detected ellipse corresponding to the specific tracking object, the image analyzer 102 destroys the tracking object.

In operation S110, when there is no tracking object corresponding to the detected specific ellipse, the image analyzer 102 registers the detected ellipse as a new tracking object.

In operation S111, the image analyzer 102 checks whether a specific tracking object has generated a user-designated event based on the tracking information of each tracking object. Examples of useful event detection may include detection of an object passing through a virtual line designated by a user in a reference direction in a reference image, or detection of an object staying for a predetermined time or more in a region designated by a user in a reference image.

In step S112, the image analyzer 102 performs a designated task if the event detection is successful in step S111. For example, the image analyzer 102 may transmit the event alarm signal and the information about the detected event to the control / monitoring terminal 103 or various I / O devices 104. Alternatively, the image analyzer 102 may count the number of objects that generated a specific event and transmit the coefficient value to the control / monitoring terminal 103 or the I / O device 104.

The method for detecting and tracking a person in an overhead camera image according to an embodiment of the present invention may be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by the computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transmission mechanism, and includes any information delivery media.

The foregoing description of the present invention is intended for illustration, and it will be understood by those skilled in the art that the present invention may be easily modified in other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is shown by the following claims rather than the above description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present invention. do.

1 is a diagram illustrating a configuration of an overhead camera image-based person detection / tracking system according to an embodiment of the present invention.

2 is a flowchart illustrating an overhead camera image-based person detection / tracking method according to an embodiment of the present invention.

3 is a typical example of photographing a person underneath with an overhead camera.

FIG. 4 is a result of extracting edge pixels from the image of FIG. 3.

5 is a flowchart illustrating a method of extracting curve segments from edge pixels.

Claims (9)

A camera fixed on the overhead to photograph a person; Analyze the image taken by the camera to detect edge pixels in the motion region, detect ellipse objects from the detected edge pixels, track the detected ellipse objects, detect a specified event from the tracking results of the ellipse objects, An image analyzer for counting the tracking objects that generated the detected event; And It receives and processes the tracking results of the ellipse objects, the event detection results, the counting results of the tracking objects, and displays or stores them, and receives the setting and control information for the image analyzer from the user and delivers them to the image analyzer. A system for detecting and tracking a person in an overhead camera image comprising a control / monitoring terminal. The method of claim 1, The image analyzer, And detecting an ellipse object having an attribute value within a preset range in detecting the ellipse object. The method of claim 2, Setting a range of attribute values of the ellipse object to be detected is After photographing a person on the overhead using the camera, setting the range of the attribute value of the ellipse object to be detected based on the attribute value of the ellipse approximating the head circumference of the person in the obtained overhead camera image. A system for detecting and tracking people in overhead camera images. Acquiring an input image from a fixed camera; Detecting a motion from the input image to extract a motion region; Detecting edge pixels of the input image within the motion region; Extracting curve segments from the edge pixels; Grouping the curve segments to detect elliptic objects having attribute values in a preset range; Tracking the ellipse objects by matching the detected ellipse objects with the ellipse objects being tracked; Detecting a specified event using a tracking result of the ellipse objects; And And counting ellipse objects that caused the detected event. The method of claim 4, wherein Extracting curve segments from the edge pixels, Grouping neighboring ones through tracking and concatenation of the edge pixels to extract primary curve segments and write branch points; With respect to the first curve segments, moving from the start point to the end point of each curve segment, find a point where the curvature value of each point is larger than the curvature value of the surrounding points within a predetermined interval about the point, and the curvature at the found point Extracting secondary curve segments by dividing the curve segment at that point if the value is greater than or equal to the set value; For the secondary curve segments, tertiary curve segments are extracted by merging a pair of curve segments having a curvature value at or below a set value among the pairs of curve segments meeting at the recorded branch point into one curve segment. step; And Extracting the final curve segments by removing curve segments not suitable for extracting an ellipse object having a specified property range, for the third curve segments. And how to track. The method of claim 4, wherein Setting a range of attribute values of the ellipse object to be detected is Photographing a person on overhead using the camera; Designating, by the user, an ellipse approximating a head circumference of a person in the captured image; And And setting a range of property values of the ellipse object to be detected using the property values of the specified ellipse. The method of claim 4, wherein Setting a range of attribute values of the ellipse object to be detected is Photographing a person on overhead using the camera; Detecting edge pixels in the captured image; Extracting ellipses from the edge pixels; Selecting an ellipse from the extracted ellipses to approximate a human head circumference; And And setting a range of attribute values of the elliptic object to be detected using the attribute values of the selected ellipse. The method of claim 4, wherein The matching of the detected ellipse objects and the ellipse objects being tracked is And detecting a person in the overhead camera image, wherein the detected ellipse object and the ellipse object being previously tracked are determined by an area of a shared area. The method of claim 4, wherein Detecting a specified event using the tracking result of the ellipse objects, Detect an event where a specific ellipse object exists within a user-defined area for a certain time, A method for detecting and tracking a person in an overhead camera image, comprising: detecting an event that a specific ellipse object passes through a virtual line designated by a user.

Images