KR101061012B1 - Moving object tracking method using wide angle image - Google Patents

Abstract

The present invention relates to a moving object tracking system using a wide angle image and a method thereof, and more particularly, when a moving object is set in a wide angle lens camera serving as a sensor, a PTZ camera tracks the moving object to track a plurality of moving objects. The present invention relates to a moving object tracking system and a method using a wide-angle image capable of accurately tracking a moving object being tracked.

Wide angle camera, PTZ camera, block, moving object, sensor

Description

Method of the mobile object tracking which uses a wide angle image}

The present invention relates to a moving object tracking method using a wide-angle image, and more particularly, when a moving object is set in a wide-angle lens camera serving as a sensor, a PTZ camera can track a plurality of moving objects. The present invention relates to a moving object tracking method using a wide-angle image that can accurately track a moving object being tracked.

Human vision can respond quickly to changes in surroundings and can easily detect and track movement and moving objects. On the other hand, image processing technology that analyzes and infers images continuously inputted with time difference in the field of image recognition using computer is still insufficient to replace the human eye despite the rapid development of hardware. There is a trend. In fact, machine vision systems using computer vision have been used in industrial fields and are being utilized.

As the importance of security has risen in many fields, the digitization in the existing analog CCTV field has made a quantum leap from VCR to DVR, and recently, it has come to establish a complete digital security system utilizing the network infrastructure. . The network video security system is commonly referred to as IP Surveillance, which not only enables monitoring via the Internet, but also allows for multiple remote accesses and installation and cost efficiency, making the majority of the existing analog CCTV market digitized in 2008. This means that consumers' dispositions are changing, from where they need 24-hour security to simple surveillance that wasn't needed before.

To this end, various inventions have been developed using a PTZ camera capable of tracking a moving object, but when only one PTZ camera is configured, a problem of losing the moving object occurs when a large number of moving objects appear in the surveillance area. In addition, the PTZ camera has a problem that its life is shortened by excessive driving to track the moving object.

Accordingly, the present invention was created to solve the above problems, the object of the present invention is as follows.

First, the present invention is to form a wide angle lens camera and a PTZ camera to serve as a sensor to track a plurality of moving objects at the same time.

Second, the present invention is to form a wide angle lens camera and a PTZ camera as a sensor to accurately track the moving object.

Third, an object of the present invention is to provide a block-based motion detection and tracking method using the same.

Fourth, the object of the present invention is to provide a moving object tracking method using the feature point of the object.

As described above, technical means for achieving the object of the present invention is as follows.

delete

delete

delete

delete

delete

delete

Receiving, by the controller, the original image provided by the wide-angle lens camera; Pre-processing the received original video by the video server; Processing the preprocessed image as a background image by the video server; Comparing the processed background image with the updated image by the video server; Storing the compared update image as a binary difference image in a video server; Labeling, by the video server, the block with motion among the processed binary difference images; Identifying, by the video server, the boundary of the movement of the labeled binary image as the boundary of the object; Detecting, by the video server, a feature point of a block identified as an object boundary; Comparing, by the video server, feature information and current object information; Selecting, by the video server, a tracking target among the compared moving object information; Tracking, by the video server, a tracking object by controlling a PTZ camera; And photographing the tracking target with a PTZ camera.

The received original image preprocessing step includes converting the original image into a monochrome image; And filtering the original image converted into the monochrome image.

The filler processing step is characterized by performing at least one of averaging colors among pixels in the vicinity, averaging temporal colors, and changing the overall color tone.

The background image processing step may include updating the background image to a background for registering a background image in which no moving object is found for a predetermined time; And partially updating the partial background only to the partial background even if the entire object is not updated.

The image comparing step may include determining registration of a background image; Block-based processing of the background image and the updated image; Comparing the block-based processed background image with the updated image; And acquiring the updated image processed based on the compared block as a binary difference image.

In the block-based processing step, each pixel of the background image and the update image is set to one block in units of 4 * 4.

The background image and the update comparison step may determine that the pixel is changed if the color difference of each pixel in the block among the background image and the update image processed based on the block is higher than a predetermined level, and if the ratio of the changed pixels is higher than or equal to a predetermined level within the block. It is characterized in that the entire block is determined to be a changed block.

In the binary image acquisition step, the changed block is recorded as 1 and the unchanged block is recorded as 0.

The labeling step may include size filtering the binary difference image; And labeling the size-filtered binary image.

The size filtering step is characterized in that if the size of the object determined as one object is smaller than a certain size, the object is not judged as an object, and the threshold value of the object is set to the size of the person seen by the camera. .

In the labeling step, when the changed blocks are adjacent to each other, the blocks are determined as one moving object, and the adjacent blocks are scanned one or more times to collect adjacent blocks and recognized as one object.

The feature point detection step may include: extracting feature points of each object by identifying a boundary with motion in a binary difference image as a boundary of an object; Storing moving object data of the extracted feature point; Processing changes in the object, such as inflow of the object, extinction of the object, superposition of the object, and separation of the object at stored feature points; And comparing the change of the object with the moving object data.

In the moving object change processing step, the inflow of the object is registered as a new object if a similar moving object is not found in the moving object data, and the extinction of the object is determined to be out of the monitoring range when the disappearing moving object is at the edge of the monitoring range. In this case, when the moving object is in the center, the overlapping object is judged to be overlapped with other moving objects around, and the separation of the object overlaps when the moving object finds similar moving object in the moving object data. It is characterized by determining that is separated.

The tracking target selection step may include: evaluating a moving object according to a tracking standard; And selecting a tracking object among the evaluated moving objects.

Tracking criteria are characterized in that the entry order, the moving object and the moving speed close to the designated position.

The PTZ camera tracking step may include calculating a center coordinate of the object to be estimated from the moving object data; Converting the calculated center coordinates into PTZ camera coordinates; And moving the PTZ camera by coordinates.

The tracking target photographing step may include storing the tracking target as a picture file, recording the number and time of the tracking target, and capturing the tracking target every cycle.

As mentioned above, the effect of this invention is as follows.

First, the present invention has the effect of tracking a plurality of moving objects at the same time by forming a wide angle lens camera and a PTZ camera as a sensor.

Second, the present invention has the effect of accurately tracking a moving object by forming a wide angle lens camera and a PTZ camera as a sensor.

Third, the present invention provides a method for detecting and tracking a moving object by converting an image provided by a wide-angle lens camera into block-based motion, thereby more accurately detecting a moving object.

Fourth, the present invention provides a moving object tracking method using the feature point of the object, there is an effect that can be variously tracked the moving object.

Fifth, there is an effect that the tracking of the moving object can be performed on the equipment itself without the help of the remote client.

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the present invention.

< Example  Configuration according to

As shown in Fig. 1, the present invention is composed of a PTZ camera 1, a wide-angle lens camera 2, a video server 3, and a client 4. The PTZ camera 1 tracks a moving object on the top or one side of the surveillance area. Such a PTZ camera 1 is a camera having ordinary PAN, TILT, and ZOOM functions. At this time, the PTZ camera is installed in the place where the field of view is wide from the top to the bottom so as to secure the entire area to be monitored, and in the case of a building, it is installed on the indoor ceiling.

The wide-angle lens camera 2 is provided below the PTZ camera 1 to detect a surveillance area. At this time, as shown in Figure 2, the wide-angle lens camera 1 forms a support 21 on one side under the PTZ camera 2, the hinge 22 is coupled to the support 21 wide-angle lens camera (2) It can be configured to freely control the surveillance area of). The wide-angle lens camera 2 serves as a sensor for tracking a moving object. Therefore, the PTZ camera 1 only tracks the moving object of the tracking target processed by the image of the wide-angle lens camera 2, thereby extending the life of the PTZ camera 1.

As shown in Fig. 3, the video server 3 receives the information of the wide-angle lens camera 2 and controls the PTZ camera 1. In this case, the video server 3 includes an encoding unit 31, a control unit 32, and a network communication unit 33. The encoder 31 is connected to the PTZ camera 1 and the wide-angle lens camera 2 to receive an image and convert the image into image data for use by the controller 32. Here, the video server 3 may be manufactured integrally with the PTZ camera 1 or installed inside the client 4.

The control unit 32 is connected to the encoding unit 31 is connected to the PTZ camera 1 to receive the image data converted by the encoding unit 31 to detect the moving object, and to track the detected moving object. At this time, the control unit 32 uses an image comparison technique that analyzes the converted image data transmitted from the encoding unit 31 and compares the binary difference image using a programmed block to select a moving object among the image data.

The network communication unit 33 connects with the client 4 and provides the remote client 4 with the image data received by the control unit 32.

The client 4 is connected to the video server 3 to control the video server 3 or to receive the video information from the video server (3). At this time, the connection between the video server 3 and the client 4 is preferably wired communication to the Internet communication network (5). It is also possible to use wireless communication instead of wired communication. In this case, the wireless communication may use wireless LAN, WiBro, Bluetooth, and the like. In addition, the client 4 may be a remote computer, a remote server, a mobile phone, a PDA, or the like. In this case, the client 4 receives the image data from a remote site by using the manager and sets the tracking target among the detected moving objects.

The video server 3 includes a data converter 34 and a data communication unit 35 for controlling the operation of the PTZ camera 1. At this time, the data conversion unit 34 is connected to the control unit 32 to convert the moving object data to be transferred from the control unit 32 to serial data. Here, in order to control the PTZ camera, since the moving object data, that is, the image information including the moving object, is the moving object data, the position of the PTZ camera 1 must be converted into serial data, that is, position data. In this way, the image data of the moving object is converted into position data of the moving object is transmitted to the PTZ camera through the data communication unit 35. At this time, the data communication unit 35 transmits serial data using the PTZ camera 1 through serial communication.

In addition, the video server 3 includes a storage unit 36 connected to the control unit 32 to store an image of the moving object and using at least one of an SD memory, a flash memory, and a hard disk. Features, background image of surveillance area and image of tracking object can be saved as photo file to prevent accidents occurring in surveillance area and when the accident occurs, it is possible to cope quickly with accident by acquiring important information such as face of tracking object.

4 shows that the wide-angle lens camera 2 according to the present invention detects the surveillance area 6, and the PTZ camera 1 tracks the moving object 7.

< Example  Method followed>

As shown in FIG. 5, the present invention receives the original image of the surveillance region provided by the wide-angle lens camera in the step S100 of receiving the original image.

Original image preprocessing step (S200) consists of a monochrome image conversion step (S210) and filter processing step (S220) in the video server. In this case, the monochromatic image conversion step (S210) converts the original image of the received surveillance area into a monochromatic image. Since the expression method of colors such as RGB, HSL, and CMYK uses 3 to 4 types of combinations, image processing is performed. Because it is complicated, it converts into a monochrome image that can express one number. Monochromatic image is usually converted to black and white image, and moving object is based on human, so to take advantage of red tone of skin color, take only red color of RGB and use only H value that represents brightness value in HSL. Use as.

The filter processing step S220 is to filter the original image processed in monochrome after the monochrome image conversion step in the video server. In this case, the original monochrome image is mixed with various noises such as rolling of light and instantaneous flickering, and there is a risk of misidentifying it as a moving object, so each pixel of the original monochrome image is 2 * 2 to remove the noise. Group them together and average the colors between the pixels around them. It also averages temporal hues and handles changes in overall hues. The temporal color here represents external factors such as the brightness of light over time.

Background image processing step (S300) consists of a full update step (S310) and a partial update step (S320) in the video server. The entire update step S310 is registered as a background for registering a background image in which a moving object is not found for a certain time. In the partial update step (S320), even if the entire update is not performed, only the portion where the moving object does not exist is registered as the partial background.

The background image and the update image comparing step S400 are performed by the video server in a determination step S410, a processing step S420, a comparison step S430, and an acquisition step S440. The determination step S410 determines whether the background image is registered, and if the background image is registered, moves to processing step S420, and if the background image is not registered, moves to comparison step S430.

The processing step S420 processes the background image and the update image on a block basis. At this time, each pixel of the background image and the update image is set to one block in units of 4 * 4.

The comparison step S430 compares the background image and the updated image processed based on the block. In this case, if the color difference of each pixel in the block among the block-processed background image and the updated image is greater than or equal to a predetermined level, it is determined as a changed pixel. If the ratio of changed pixels is greater than or equal to a predetermined level within the block, the entire block is determined as a changed block. do.

As shown in FIG. 6, the obtaining step S440 obtains a binary difference image from the processed update image based on the compared block. In this case, the binary difference image is recorded as 1 for the changed block and 0 for the unchanged block.

The compared update image is stored as a binary difference image in the video server in the binary difference image storing step (S500). At this time, the binary difference image is stored in the storage unit 36 of the video server (3).

The labeling step (S600) is to label the block with a movement of the binary image processed by the video server, wherein the labeling step (S600) is composed of a size filtering step (S610) and a labeling step (S620).

Size filtering step (S610) is to filter the binary difference image, if the size of the object determined as one object is less than a certain size is not determined as an object, the threshold value of the object is set to the size of the person visible from the camera Smaller moving objects are ignored. This is to obtain accurate moving object information.

As shown in FIG. 7 and FIG. 8, the labeling step (S620) is to label the size-filtered binary image. When the blocks changed in the binary image are adjacent to each other, one label is identified as a moving object, and the binary image is determined. Scan one or more times to collect adjacent blocks and recognize them as one object. At this time, the labeling step (S620) it is preferable to label the binary difference image by scanning twice to obtain more accurate moving object information.

As shown in FIGS. 9 to 12, in step S700, the boundary of the object is identified as the boundary of the object. This is to accurately obtain the position of the moving object in the surveillance area.

Feature point detection step (S800) detects the feature point of the block identified as the boundary of the object in the video server. At this time, the feature point detection step includes a feature point extraction step (S810), the storage of moving object data (S820), the change processing step of the object (S830) and the change of the object and the moving object data comparison step (S840).

The feature point extraction step (S810) is to extract the feature point of each object identified as the boundary of the object in the boundary of the binary image. Here, the feature point means characteristic data for identifying the identity of each object, and the judgment element of the feature point is determined by the position and size under color. In this case, the position is newly updated every 0.1 to 0.1 seconds, and there is a limit to the distance that a person can move between these times. When the location is close, the color is distinguished by the color of clothes. In this case, the color image itself is used instead of the monochrome image. The size is used by the difference in size when a person is partially obscured or posture is different. The priority for extracting the above feature points is preferably in the order of location> color> size. Since the feature points are partially changed for each image, it is desirable to update them gradually.

The moving object data storing step (S820) registers the moving object data of the extracted feature point and stores it in the storage unit 36.

The change processing step (S830) of the moving object processes the change of the object such as the inflow of the object, the extinction of the object, the superposition of the object, and the separation of the object at the stored feature point in the storing of the moving object data of the extracted feature point (S820). It is. At this time, the inflow of the object is registered as a new object if a similar moving object is not found in the moving object data, and the extinction of the object is determined to be out of the monitoring range when the disappearing moving object is at the edge of the monitoring range. If the object is in the center, it is determined that it overlaps with other moving objects around. If the object is found to be similar in the moving object data, the overlapping moving object is separated.

The change of the object and the moving object data comparison step (S840) is to determine the change of the moving object by confirming the change of the object in the moving object data. At this time, the feature point information of the moving object detected from the beginning is stored in the memory, and the feature point information of the object found in each image is matched with the algorithm to find out which objects were previously found in the feature point information of the object found in each image. Search from the feature point information of the moving object detected from the beginning. Here, the existing moving object is recognized as the moving object, and the newly displayed moving object is identified and registered as the new moving object. At this time, the newly acquired feature point information of each moving object is registered in the feature point information of the moving object detected from the beginning to maintain the latest image information.

The moving object information comparison step of the feature point and the present point (S900) summarizes the moving object information before selecting the moving object for tracking in the video server.

Selecting the tracking target step (S1000) is to select the tracking target from the moving object information compared in the video server, the moving object evaluation step (S1010) and the tracking object selection step (S1020). Moving object evaluation step (S1010) is to evaluate the moving object according to the tracking criteria. At this time, tracking criteria are entry order, moving object and moving speed near the designated position. Here, the tracking criteria may be assigned a weight.

Tracking object selection step (S1020) selects the tracking object of the moving object evaluated in the video server. Here, the selection of the tracking object can be performed by alternately tracking all objects, continuously tracking until the object under tracking disappears from the surveillance area image, and continuously tracking until the captured image of the object under tracking is normally taken. One can be used.

The PTZ camera tracking step (S1100) is to track a tracking object with a PTZ camera, and includes a center coordinate calculation step (S1110), a PTZ camera coordinate conversion step (1120), and a PTZ camera moving step (S1130). At this time, the center coordinate calculation step (S1110) calculates the center coordinate of the object to be estimated from the moving object data. As shown in FIG. 14, the X-Y coordinate system, which is a Cartesian coordinate system, is converted into an r-θ coordinate system, which is a circular coordinate system. Equation 1 used at this time is as follows.

Here, in order to set the coordinate value of the PAN of the PTZ camera, a circular image transmitted from the wide-angle lens camera is set to 0 to 36000 in the counterclockwise direction from 12 o'clock, and is formed into quadrants.

The coordinates of the PAN of the first quadrant are shown in Equation 2 below.

The coordinates of the PAN of the second quadrant are as shown in Equation 3 below.

The coordinate of the PAN of the third quadrant is expressed by Equation 4 below.

The coordinates of the PAN of the fourth quadrant are as shown in Equation 4 below.

Here, to set the coordinate value of the TILT of the PTZ camera, 0 to 36000 values of the circular image transmitted from the wide-angle lens camera are set from the outside to the center, and the coordinate value of the TILT is expressed by Equation 6 below.

PTZ camera coordinate conversion step (S1120) converts the center coordinates calculated by the above-described equation to PTZ camera coordinates.

The PTZ camera moving step (S1130) transmits the converted PTZ to the PTZ camera to move the PTZ camera to the center coordinate of the surveillance area.

As shown in FIG. 13, the tracking target photographing step (S1200) captures the tracking target by moving the PTZ camera to the center coordinate of the moving object in the surveillance area. At this time, the shooting element of the tracking target is used to store the tracking target as a picture file, record the tracking target number and time, and take the tracking target every cycle. The captured image of the tracking target is stored in the storage unit 36 of the video server. This is to reinforce security by remembering the information to be tracked.

As described above, those skilled in the art will understand that the present invention can be implemented in other specific forms without changing the technical spirit or essential features. It is therefore to be understood that the above-described embodiments are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included within the scope of the present invention.

The following drawings, which are attached in this specification, illustrate the preferred embodiments of the present invention, and together with the detailed description thereof, serve to further understand the technical spirit of the present invention, and therefore, the present invention is limited only to the matters described in the drawings. It should not be interpreted.

1 is a schematic diagram showing a moving object tracking system using a wide-angle image according to the present invention.

2 is a perspective view showing a PTZ camera and a wide-angle lens camera according to the present invention.

3 is a block diagram showing a video server according to the present invention.

4 is an operational state diagram showing a moving object tracking system using a wide-angle image according to the present invention.

5 is a flowchart illustrating a moving object tracking method using a wide-angle image according to the present invention.

6 is a block diagram illustrating a changed block of a binary difference image according to the present invention.

7 is a first labeling view showing a labeling step according to the present invention.

8 is a second labeling diagram showing a labeling step according to the present invention;

9 is a first image processing diagram illustrating a boundary detecting step of an object according to the present invention;

10 is a second image processing diagram illustrating a boundary detecting step of an object according to the present invention.

11 is a third image processing diagram illustrating a boundary detecting step of an object according to the present invention.

12 is a fourth image processing diagram illustrating a boundary detecting step of an object according to the present invention;

FIG. 13 is a background image illustrating a surveillance area for setting coordinates according to the present invention. FIG.

14 is a photographic image showing a photographed image according to the present invention.

<Explanation of symbols for the main parts of the drawings>

1: PTZ Camera 2: Wide Angle Camera

3: video server 31: encoding section

32: control unit 33: network communication unit

34: data conversion unit 35: data communication unit

36: storage 4: client

5: Internet communication network 6: Monitoring area

7: moving object

Claims (12)

delete delete delete delete Receiving, by the controller, the original image provided by the wide-angle lens camera; Pre-processing the received original video by a video server; Processing the preprocessed image as a background image by the video server; Comparing, by the video server, whether the background image processed as a block image and the received updater image are block images with motion in units of blocks; Labeling, by the video server, a block with motion among the processed binary image; Size filtering the labeled block image by the video server; Identifying, by the video server, the labeled binary difference image; Detecting, by the video server, a feature point of a block identified as an object boundary; Comparing, by the video server, the moving object information of the feature point and the current point; Selecting, by the video server, a tracking target among the compared moving object information; Tracking the tracking object by controlling the PTZ camera by the video server; And Photographing the tracking object with the PTZ camera; moving object tracking method using a wide-angle image, characterized in that it comprises a. The method of claim 5, The received original image preprocessing step, Converting the original image into a monochrome image; And Filtering the original image converted to the monochrome image; moving object tracking method using a wide-angle image, characterized in that it comprises a. The method of claim 5, The background image processing step, Registering the image for which a moving object is not found for a predetermined time; And Partial updating only a portion of the background where the moving object does not exist even if the entire update does not exist; moving object tracking method using a wide-angle image, characterized in that the. The method of claim 5, The background image and the update image comparison processing step, Determining registration of the background image; Block-based processing of the background image and the updated image; Comparing the updated image with the background image processed based on the block; And Obtaining the updated image processed based on the block as a binary difference image; and moving object tracking method using a wide-angle image. The method of claim 5, The labeling step, Size filtering the binary difference image; And And labeling the size-filtered binary image. 2. The method of claim 5, The feature point detection step, Extracting feature points of the size, color, and position of the binary difference image; And Moving object data of the extracted feature point; moving object tracking method using a wide-angle image comprising a. The method of claim 5, The tracking target selection step, Moving object tracking method using a wide-angle image, characterized in that it comprises the step of evaluating the moving object according to the tracking criteria. The method of claim 5, The PTZ camera tracking step Calculating a center coordinate of the object to be estimated from the moving object data; Converting the calculated center coordinates into the PTZ camera coordinates; And Moving the PTZ camera to the coordinates; moving object tracking method using a wide-angle image, characterized in that it comprises a.

Images