KR20110077465A - The apparatus and method of moving object tracking with shadow removal moudule in camera position and time - Google Patents

Abstract

PURPOSE: An apparatus and method for tracking a moving object with shadow removal module are provided to distinguish a real object from shadows. CONSTITUTION: A video cassette recorder(100) of an image tracking device(10) transmits an image data to a remote control server in real time. An object tracking module(200) periodically receives time information and location information from a control server in a remote place. The object tracking module calculates the direction and the length of the object shadow. The object tracking module extracts a clear object image in which the shadow is removed.

Description

TECHNICAL APPARATUS AND METHOD OF MOVING OBJECT TRACKING WITH SHADOW REMOVAL MOUDULE IN CAMERA POSITION AND TIME}

The present invention provides a camera position and the sun that can move the object in a clear state by removing the shadow effect in the digital image data received by the camera, such as illegal parking control and overspeed driving on the highway. The present invention relates to an image tracking device and a method using a shadow elimination method according to a moving time.

Background-based methods are often used in object extraction for object tracking in systems that require object tracking, such as illegal parking enforcement systems.

In the background-based method, if a stationary vehicle is included in an initial frame captured by the camera, it is difficult to generate a correct background because it is absorbed into the background.

To prevent this, the initial vehicle must be detected. When objects are extracted by the background-based method, shadows may be included in the extracted objects.

In order to track an object in an illegal parking control system, it is necessary to find the position and shape of the object accurately. If the extracted object includes a shadow, the object cannot be accurately tracked.

For this purpose, a method of accurately removing only the shadow part except the real object part is needed. In addition, an objective metric is needed to measure the performance of the shadow removal method.

Among the methods that can be used for initial vehicle detection, stereo imaging obtains image depth information from two cameras and analyzes the depth information three-dimensionally to find a vehicle. There was a problem that was difficult to find.

Another method is the Fuzzy c-Means method, which classifies similar data into a cluster, which can segment the vehicle using clustering in color space. It was difficult to apply to real-time traffic image sequence analysis.

Alternatively, model based segmentation is a method of extracting edges and matching models using prior knowledge of the vehicle's three-dimensional CAD model. However, there is difficulty in model matching when the extracted edge is insufficient to show the 3D CAD model projected in 2D as in the case of a dark vehicle, and also the shape of the vehicle is varied, and the viewing geometry There was a problem that the number of models can increase exponentially.

Alternatively, the Color Information and History Data method classifies all pixels in the extracted blob into vehicle pixels and shadow pixels and calculates the ratio of the blob size to the shadow pixels. Blobs are divided into normal vehicle blobs and dark vehicle blobs. In this case, in the case of the normal vehicle blob (small colored portion), the shadow pixel is removed, and the dark vehicle blob (small colored portion) is forcibly removed using the data stored in the interpolation data or the history array. However, there is a case where even the self shadow is removed because the cast shadow and the self shadow are not distinguished among the shadows.

In addition, the conventional shadow removal methods have a problem that it is difficult to track through a clear object by overlooking the weather disturbance element and the shadow obstruction element related to rain, snow, fog, etc. in tracking the moving object.

In addition, since there is no technology for distinguishing real objects from shadows in real time in the field in real time, since the moving object is always tracked through the conventional shadow removal method in the video data recorded at the remote control server, the object tracking time is increased. There was a problem that took too long.

In order to solve the above problems, in the present invention, the camera itself has a GPS receiver configured to receive the time information of the sun and the location information of the image tracking device periodically from a remote control server, and in real time in the field with the real object in the digital image data. Shadows can be distinguished, and the direction and length of the object's shadow can be predicted in advance to track based on the clear object. The purpose of the present invention is to provide an image tracking apparatus and method.

In order to achieve the above object, the image tracking device through the shadow removal method according to the camera position and the time the sun moves according to the present invention

It consists of an image tracking device 10 for compressing and storing the analog image data coming from the camera into a digital image data on a hard disk after digital conversion processing, and transmitting the video signal for tracking to a remote control server 20,

The video tracking device 10 integrates a multiplexer, a motion detector, a PTZ controller, and a recording function (Time-lapse VCR) to monitor illegal parking and on the highway. Digital video recorder (VCR) 100 for taking a video while monitoring the speed driving, and transmitting the captured video data to a remote control server in real time;

After receiving time information and position information periodically from a remote control server to calculate the direction and length of the object shadow, the object tracking module 200 for detecting and tracking only the clear object with the shadow removed according to the calculated data value This is achieved by constructing.

In addition, the image tracking method through the shadow removal method according to the camera position and the time the sun moves according to the present invention

Detecting an optical object moving in real time from the digital image data captured by the camera (S100);

Classifying a shape of an object based on shape and movement based on the digital image data detected by the optical flow detector (S200);

Dividing the digital image data classified through the object classifying unit into an object and a shadow through object shadow direction calculation and length operation (S310) according to the camera position and time, and removing the shadow (S300);

Determining the last match having object correlation among the shadow-free digital image data through the shadow removal module and tracking the moving object in the image through the movement by the nonlinear voting and the center, model, and size of the object (S400). )Wow,

It monitors the object tracking video screen transmitted from the object tracking module of the remote video tracking device in real time, extracts the position coordinate value of the video tracking device through the GPS base station, and transmits the location information and current time information to the object tracking module. It is achieved by the tracking image remote monitoring step (S600).

As described above, in the present invention, in the present invention, the GPS receiver is configured in the camera itself to receive data from the remote control server periodically and receive the sun information and the location information of the image tracking device to prevent data errors. It can distinguish real objects and shadows in digital image data in real time, and can track based on clear objects by predicting the direction and length of object shadows in advance, resulting in more than 90% better tracking performance than conventional video tracking algorithms. There is a good effect it can provide.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

1 is a block diagram showing the components of the image tracking device through the shadow removal method according to the camera position and the sun moving time according to the present invention, which monitors illegal parking and speeding on the highway, the image It consists of a video tracking device for tracking 10, and the remote control server 20.

First, the image tracking device 10 according to the present invention will be described.

According to the present invention, an image tracking device compresses and stores analog image data coming from a camera as digital image data on a hard disk after digital conversion processing, and transmits the captured image data to a remote control server for tracking. It consists of a digital video recorder (VCR) 100, an object tracking module 200, and a camera 300.

In addition, the image tracking device 10 according to the present invention is configured to detect the primary object by detecting only the clear object in which the shadow is removed in real time according to the remote control of the remote control server through the object tracking module.

The digital video recorder (VCR) 100 integrates a screen splitting function (Multiplexer), a motion detector (Motion Detector), a PTZ controller function (Pan, Tilt, Zoom Controller), a recording function (Time-lapse VCR), It takes an image while monitoring the illegal parking stop and monitoring the speeding on the highway, and transmits the image to a remote control server in real time. As shown in FIG. A monitoring / monitoring unit 110, a recording / backup unit 120 for converting / compressing analog image data into a digital signal, storing the same in an HDD (hard disk driver), and backing up the stored image data using a CD-RW; Search unit 130 to search the file storage information on the HDD, by camera, time, event, multi-channel simultaneous search and panoramic viewer, and TCP / IP, RTP / RTCP (Real Time Transport Protocol / R) Network transmission unit 140 for real-time transmission and recording video remote search to remote control server using eal Time Control Protocol) and UDP (User Datagram Protocol), remote monitoring, remote video recording, remote search, remote recording, system Remote monitoring and control unit 150 to change the setting, the schedule management unit 160 for managing the schedule event for continuous, schedule, motion detection, sensor interlocking, and camera 300 using the EIA-485, EIA-232C serial port The camera driving control unit 170 is configured to control the pan / tilt / zoom.

The object tracking module 200 is installed in a digital video recorder (VCR) programmatically receives the time information and position information from a remote control server periodically to calculate the direction and length of the object shadow, the calculated data It detects and tracks only the clear object from which the shadow is removed according to the value. As shown in FIG. 3, the optical flow detector 210, the object classifier 220, the shadow removal module 230, and the object tracker ( 240).

The object tracking module 200 according to the present invention may receive time information and position information periodically through a remote control server and an IEEE485 wired control line, or may be directly connected to a GPS base station to receive azimuth and time of the sun directly. It is characterized in that it is configured to.

The optical flow detector 210 detects an object moving in real time from the digital image data as an optical flow.

Here, the optical flow refers to a two-dimensional velocity field that is moved when the intensity of the video changes with time.

This is a two-dimensional representation of the velocity of the position change of the pixels in the frame, that is, the moving speed of the image formed in the two-dimensional space when projecting an object moving in the three-dimensional space in the two-dimensional space using the camera 300 Say.

Computing the optical flow is to predict the motion of all the pixels in the image by calculating the amount of change in pixel brightness with respect to the spatial and temporal axes.

In other words, by detecting the optical flow, it is possible to inversely estimate the movement of an object in a three-dimensional space, and implement image encoding by detecting the movement of the object, dividing the object, and compensating for the motion.

In the present invention, as an optical flow detection method, it is detected by an area-based matching technique.

The region-based matching technique uses accurate mathematical derivatives when there are too few frames and aliasing occurs when acquiring images at the time of acquiring images or noise. Because of the inadequate approach, domain-based matching techniques are used.

In the region-based matching technique, the velocity v is defined as the movement d = (dx, dy) in which a constant region in the image most matches with respect to time.

Finding the most accurate match can be done by maximizing similar measures like normalized cross correlation, or distance measures like sum of squared difference (SSD). Minimize the measure.

The object classifying unit 220 serves to classify the shape of an object based on shape and motion based on the digital image data detected by the optical flow detector.

It is composed of a shape-based classifier 221 using two-dimensional spatial information of the object, and a motion-based classifier 222 using the temporally accumulated properties of the object for classification resolution.

The shape-based classification unit 221 serves to classify using a rectangular boundary, an area, a silhouette (contour), and a slope of the detected object area.

That is, we use information of area and shadow contour length to classify the detected objects into three groups (people, vehicles, and other objects).

Dispersedness of the shape-based classification unit is defined as in Equation 1 in view of the area of the objects and the perimeter of the object.

The motion-based classifier 222 classifies based on temporal self-simiarity of the moving object.

Optical flow analysis according to the present invention is usefully used to distinguish between fixed and non-fixed objects.

That is, it uses local optical flow analysis of the detected object regions.

This expects that non-fixed objects, such as humans, will have a high average residual flow, while fixed objects, such as vehicles, will exhibit low residual flows.

In addition, the residual flow generated by human movement becomes periodic. This feature can be used to classify people's movements and people from other objects such as vehicles.

The shadow removing module 230 divides the digital image data classified through the object classifying unit into an object and a shadow through a shadow direction calculation and a length calculation according to the camera position and time, and removes the shadow. As shown in FIG. 4, the GPS receiver 231 includes a shadow operation controller 232, a shadow remover 233, and a data transmitter 234.

The GPS receiver 231 receives the time information of the sun and the location information of the image tracking device periodically from the GPS base station through a remote control server.

The shadow operation controller 232 distinguishes an object from a shadow among digital image data input to the camera 300 through a shadow direction calculation and a length calculation according to the time information of the sun and the location information of the image tracking device received from the GPS receiver. To play a role.

A process of calculating the object shadow direction and calculating the length according to the time information of the sun and the position information of the image tracking device received from the GPS receiver will be described.

First, time information and azimuth information are transmitted from a GPS receiver to a camera 300 remotely located.

That is, the GPS receiver is to give time information and orientation information to the camera 300 at a remote location. When the camera 300 is connected to the outside by wire, the GPS receiver provides time and orientation information to the camera 300 by wire. Do it.

Subsequently, the position of the sun is calculated from the memory unit of the camera 300 stored in advance according to time and azimuth.

That is, the position of the sun is stored in advance in the memory of the camera 300 according to time and orientation.

Subsequently, the direction and the length A of the object shadow occurring in the opposite direction of the moving object through the angle θ of the sun according to the position of the sun and the height B of the moving object are calculated.

In other words, knowing the position of the sun, as shown in Figure 5, the shadow is generated in the opposite direction.

In this case, the length A of the shadow can be calculated through the height and direction information of the sun, as shown in Equation 2 below.

Where A represents the shadow length, B represents the moving object height, and θ represents the angle of the sun.

The angle θ of the sun is automatically calculated by the data stored in the memory of the camera 300 when the sun and the coordinates of the sun are known.

The shadow remover 233 removes only the shadow from the data in which the object and the shadow are separated through the shadow operation controller to extract only the clear object.

The data transmitter 234 transmits only the digital image data of which the shadow is removed through the shadow remover to the object tracker.

The object tracking unit 240 determines the last match having object correlation among the digital image data from which the shadow is removed through the shadow removal module, and moves and objects by nonlinear voting. The moving object in the image is tracked through its center, model, and size, which is configured to track using matching correlation, motion estimation, and position prediction.

In other words, if the area and the objects are separated from the shadow-free digital image data through the shadow removal module, the object tracker is characterized by the area (A), length (L), width (W), Lastly, the object correlation is extracted by extracting the center (Cm), color histogram (Hc), circularity (C), directionality (θ), and silhouettes (contours) of the blobs of objects. Determine the match.

Then, the moving object in the image is tracked through the movement by nonlinear voting (to determine the reference value when comparing each signal) and the center, model, and size of the object.

Next, the remote control server 20 according to the present invention will be described.

The control server 20 sends a remote control signal for object tracking to the object tracking module of the video tracking device used for illegal parking control and overspeed driving on the highway, monitors the object tracking video screen in real time, and tracks each video. It transmits the location information and current time information extracted through the GPS base station to the object tracking module.

It extracts the position coordinate value of each video tracking device through the GPS base station and calculates the current time information according to the position coordinate value and stores it in the internal program memory.

Hereinafter, an image tracking method using a shadow removal method according to a camera position and time according to the present invention will be described.

First, an object moving in real time from the digital image data captured by the camera is detected as an optical flow (S100).

Subsequently, the shape of the object is classified based on the shape and the motion based on the digital image data detected by the optical flow detector (S200).

Subsequently, the digital image data classified through the object classifying unit is classified into an object and a shadow through object shadow direction calculation and length calculation according to the camera position and time, and the shadow is removed (S300).

Here, computing the object shadow direction and length according to the camera position and time is

Transmitting time information and bearing information from a GPS receiver to a camera at a remote location (S310);

Calculating a position of the sun according to time and azimuth from a memory unit of the pre-stored camera (S320);

Comprising a step (S330) for calculating the direction and the length (A) of the object shadow that occurs in the opposite direction of the moving object through the angle (θ) of the sun according to the position of the sun and the height (B) of the moving object.

Subsequently, the shadow elimination module is used to determine the last match with object correlation among the shadow-free digital image data, and to determine the center of gravity, model, and size of the object by nonlinear voting. Tracking the moving object in the image through (S400).

Subsequently, the object tracking image screen transmitted from the object tracking module of the remote image tracking device is monitored in real time, the position coordinate value of the video tracking device is extracted through the GPS base station, and the position information and the current time information are transferred to the object tracking module. To transmit (S500).

[ Of the control server 20  Operation process]

First, the control server 20 sends a remote control signal for object tracking to the object tracking module of the video tracking device used for illegal parking stops and overspeed driving monitoring on the highway.

Subsequently, the object tracking video screen is monitored in real time.

Subsequently, the positional information and the current time information extracted from the position coordinates of each image tracking device through the GPS base station are transmitted to the object tracking module.

1 is a block diagram showing the components of the image tracking device through the shadow removal method according to the camera position and the time the sun moves according to the present invention,

2 is a block diagram showing components of a digital video recorder (VCR) 100 according to the present invention;

3 is a block diagram showing the components of the object tracking module 200 according to the present invention;

4 is a block diagram showing the components of the shadow removal module 230 according to the present invention;

5 is a diagram illustrating an operation of calculating an object shadow direction and calculating a length according to time information of a sun received from a GPS receiver and position information of an image tracking device according to the present invention;

6 is a flowchart illustrating an image tracking method using a shadow removal method according to a camera position and a time at which the sun moves according to the present invention;

7 is a flowchart illustrating a process of calculating an object shadow direction and a length according to a camera position and time according to the present invention.

※ Brief description of reference numerals ※

 10: video tracking device 20: remote control server

100: digital video recorder (VCR) 110: monitoring monitoring unit

130: search unit 140: network transmission unit

150: monitoring control unit 160: schedule management unit

170: camera driving control unit 200: control server

210: optical flow detection unit 220: object classification unit

230: shadow removal module 240: object tracking unit

Claims (5)

In the video tracking device 10 for compressing and storing analog video data coming from the camera 300 as digital video data on a hard disk after digital conversion processing, and transmitting a video signal for tracking to a remote control server 20, The image tracking device 10 integrates a screen splitting function (Multiplexer), a motion detector (Motion Detector), a PTZ controller function (Pan, Tilt, Zoom Controller), recording function (Time-lapse VCR), illegal parking stops A digital video recorder (VCR) 100 for taking a video while monitoring and monitoring a speeding driving on a highway, and transmitting the captured video data to a remote control server in real time; After receiving time information and position information periodically from a remote control server to calculate the direction and length of the object shadow, the object tracking module 200 for detecting and tracking only the clear object with the shadow removed according to the calculated data value Image tracking device through the shadow removal method according to the camera position, the sun moving, characterized in that is configured to include. The method of claim 1, wherein the object tracking module 200 An optical flow detector 210 for detecting an object moving in real time from the digital image data as an optical flow; An object classifying unit 220 classifying the shape of the object based on shape and movement of the digital image data detected by the optical flow detector; A shadow removal module 230 for classifying the digital image data classified through the object classification unit into an object and a shadow through a shadow direction calculation and a length calculation according to a camera position and time, and removing the shadow; An object tracking unit that determines the last match with object correlation among the digital image data from which shadows are removed through the shadow elimination module, and tracks the moving object in the image through the movement by the nonlinear voting and the center, model, and size of the object. Image tracking device through the shadow removal method according to the camera position, the time the sun moves, characterized in that the 240 is configured. The method of claim 2, wherein the shadow removal module 230 A GPS receiver 231 which receives the time information of the sun and the location information of the image tracking device periodically from a remote control server; A shadow calculation controller 232 for distinguishing an object from a shadow among digital image data input to the camera through a shadow direction calculation and a length calculation according to the time information of the sun and the location information of the image tracking device received from the GPS receiver; A shadow removal unit 233 for extracting only a clear object by removing only a shadow from data in which an object and a shadow are separated through a shadow operation control unit; An image tracking device using a shadow removal method according to a camera position and a sun movement, comprising a data transmission unit 234 for transmitting only the digital image data of which the shadow is removed through the shadow removal unit to the object tracking unit. Detecting an optical object moving in real time from the digital image data captured by the camera (S100); Classifying a shape of an object based on shape and movement based on the digital image data detected by the optical flow detector (S200); Dividing the digital image data classified through the object classifying unit into an object and a shadow through object shadow direction calculation and length operation (S310) according to the camera position and time, and removing the shadow (S300); Determining the last match having object correlation among the shadow-free digital image data through the shadow removal module and tracking the moving object in the image through the movement by the nonlinear voting and the center, model, and size of the object (S400). )Wow, It monitors the object tracking video screen transmitted from the object tracking module of the remote video tracking device in real time, extracts the position coordinate value of the video tracking device through the GPS base station, and transmits the location information and the current time information to the object tracking module. Image tracking method through the shadow removal method according to the camera position, the time the sun moves, characterized in that the step (S500). The method of claim 4, wherein the calculating of the object shadow direction and the length according to the camera position and time is performed (S310). Transmitting time information and bearing information from a GPS receiver to a camera at a remote location (S311); Calculating a position of the sun according to time and azimuth from a memory unit of the pre-stored camera (S312); Comprising a step (S313) of calculating the direction and the length (A) of the object shadow that occurs in the opposite direction of the moving object through the angle (θ) of the sun according to the position of the sun and the height (B) of the moving object Image tracking method using shadow removal method according to camera position and time of sun movement.

Images