KR100696725B1 - Shadow casting method using symmetric and distance feature of the object region - Google Patents

Abstract

The present invention relates to a shadow removal method that enables accurate object tracking by removing a shadow area from an object extracted from a real-time video.

To this end, the present invention, (a) extracting the object from the input image, (b) detecting the central axis for the object, (c) detecting the shadow direction through the detected central axis and (d) detecting a shadow candidate region by using the symmetry of the object after detecting the directionality of the shadow, and (e) removing the detected shadow.

Object, symmetry, shadow, distance

Description

Shadow casting method using symmetric and distance feature of the object region}

1 is a flow chart showing a shadow removal process according to an embodiment of the present invention,

2A and 2B are diagrams illustrating a case where an exact center axis is found in the process of selecting a center axis of the extracted object, and a case where the center axis is incorrectly found;

3 is a diagram illustrating a central axis detection process using a blocked histogram according to an embodiment of the present invention;

4 is a view showing a shadow direction detection process according to an embodiment of the present invention;

5 is a view showing a shadow candidate region detection process using symmetry according to an embodiment of the present invention;

6A and 6B are graphs illustrating an application of weights according to deviations of a vertical histogram according to an embodiment of the present invention with respect to an input image and the input image, respectively;

7A to 7D are diagrams showing experimental results comparing the present invention with the conventional art, respectively.

The present invention relates to a method for object extraction and tracking, and more particularly, to a method for removing shadows that enables accurate object tracking by removing a shadow area from an object extracted from a real-time video.

Recently, extracting and tracking moving objects from real-time images has become an important problem in surveillance systems and pattern recognition systems, and related researches are being actively conducted.

In particular, extracting the main feature vector of the detected object in the continuous image and tracking the path becomes an important feature vector during event detection in the surveillance system.

However, if the extracted object is extracted including the shadow area, it may cause incorrect feature vector extraction.

Therefore, many studies have been conducted to solve the problem of incorrect object recognition and path tracking caused by shadows.

Representative methods are statistical method and model based method. The former has the advantage of reducing the sensitivity of the noise (noise), the latter can get a good result, but has the disadvantage of being complicated and time-consuming.

Here, the characteristics mainly used for removing shadows are spectral, spatial and temporal characteristics.

Spectrum information refers to the use of brightness and color information. If spatial and temporal characteristics of region and frame level are used together, improved results can be obtained. .

Recently, many methods for removing shadows using physical properties have been studied.

The present invention has been made to solve the problems described above, after extracting the shadow candidate region using the symmetry of the object, using a weight proportional to the distance from the center axis of the object, the shadow with respect to the extracted object By performing the removal, an object of the present invention is to provide a method for removing shadows that enables accurate object tracking.

In order to achieve the above object, according to an aspect of the present invention,

(a) extracting an object from an input image;

(b) detecting a central axis for the object;

(c) detecting a shadow direction through the detected central axis;

(d) detecting a shadow candidate region by using the symmetry of the object after detecting the directionality of the shadow;

(e) providing a shadow removing method comprising the step of removing the detected shadow.

Also, in the step (b), the central axis obtains a vertical histogram on the object, divides the obtained vertical histogram into an arbitrary number of blocks horizontally, and sets the position of the X axis of the largest histogram for each block in max1 and max2. In the above, the central axis CP is characterized by being obtained by the following equation (1).

CP (x) = (max1 (x) + max2 (x)) / 2 ........ (1)

Further, in the step (c), the direction of the shadow is determined by the following formula (2).

..........(2)

..........(2)

Here, O _s is a shadow direction, Lx and Rx represent right and left end coordinates of the bounding box, and ε means a threshold.

Further, in the step (d), it is characterized by dividing the object region and the shadow candidate region by the following equation (3).

..........(3)

.......... (3)

Here, CP is the central axis of the object and is represented by the X-axis coordinates S _i ^l and S _i ^r of the first pixel included in the object, where S _i ^x is the X-coordinate on the Y axis moving along the central axis CP of the object. Represents the distance of the pixel at.

In the step (e), the shadow is removed when the shadow is inside the object by using a chromaticity value and by applying a weight to the distance from the central axis when the shadow is outside.

In the application of the weights, weights according to distances may be applied to areas in which the vertical histogram is less than or equal to a predetermined value for every frame of the input image.

In addition, the value obtained by subtracting the weight from the difference between the chromaticity average value of the shadow candidate region and the chromaticity average value of the object region is compared with a threshold value, and when the subtracted value is smaller than the threshold value, it is determined as a background image. It features.

The method may further include removing noise from the extracted object and covering a bounding box with respect to each object.

In addition, it is characterized by dividing the area of the bounding box into upper, middle, and lower portions, and obtaining a threshold value for each region.

In addition, when the shadow exists in the lower bound, the shadow is removed only for the lower bound area.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart illustrating a shadow removal process using a symmetry and a distance vector of an object according to an embodiment of the present invention.

First, a subtraction image (object) is extracted by a subtraction operation between a background image generated after background modeling and a continuous input image (S110).

Then, after removing the noise for the extracted object (S120), the central axis for each object is detected (S130).

Thereafter, the direction of the shadow is determined (S140), and if the shadow is outside the object, the shadow is distinguished from the object by checking symmetry (S150).

Thereafter, an adaptive threshold value is extracted for each individual (S160), and the shadow value is removed from the shadow candidate region by comparing the chromaticity value with the threshold value (S170).

As described above, when the shadow is inside the object, chromaticity is used, and when it is outside, the object tracking is performed by removing the shadow by applying a weight to the distance from the central axis (S190).

Here, the shadow removing process according to an embodiment of the present invention, the first process of extracting the object from the input image, the second process of detecting the central axis for the extracted object, and the shadow direction through the detected central axis It includes a third process of detecting the.

The method may further include a fourth process of detecting a shadow candidate region by using the symmetry of the object after detecting the directionality of the shadow, and a fifth process of removing the detected shadow.

Hereinafter, the second to fifth processes will be described in detail.

1. Central axis detection

To remove the shadow, the object's symmetry is first used to distinguish between the shadow and object areas. In this case, the central axis as a reference generally indicates the head position of the extracted object.

2A and 2B are diagrams illustrating a case in which an accurate center axis is found in a process of selecting a center axis of an extracted object, and a case in which the center axis is incorrectly found.

As shown in FIG. 2A, when a correct center axis is found, a vertical histogram is obtained for the object, and has a coordinate value of the X axis having the largest value among them.

However, as shown in FIG. 2B below, the coordinates of the largest X axis in the vertical histogram of the moving object do not always represent the central axis.

As shown in FIG. 2B, incorrect symmetric information can be obtained by detecting a wrong central axis.

3 is a diagram illustrating a process of detecting a central axis through a blocked histogram in a method for removing a shadow according to an embodiment of the present invention.

FIG. 3 illustrates a blocked histogram. When the vertical histogram is divided into any number of blocks horizontally, and the positions of the X-axis of the largest histogram for each block are max1 and max2, the center point CP (center point) is It can obtain | require by Formula (1) as follows.

CP (x) = (max1 (x) + max2 (x)) / 2

FIG. 3B shows a central axis extracted using the blocked histogram of FIG. 3A.

2. Shadow direction detection

After extracting the center point of the object, the direction of the shadow is determined to find out whether the shadow is inside or outside the object.

Using this information, when the shadow is inside, it uses the chromaticity value obtained by the experimental result to remove the shadow by applying it to the lower part of the object.

When the direction of the shadow is set to O _s and the end coordinates of the right and left edges of the bounding box are represented by Lx and Rx, and ε is set as a threshold, it can be expressed as shown in FIG. 4.

The direction of the shadow of the extracted object may be obtained by substituting the variables defined in FIG.

When the shadow is outside, the shadow region is removed using the shadow candidate region detection method using the following symmetry.

3. Shadow candidate region detection using symmetry

5 is a diagram illustrating a shadow candidate region detection process using symmetry in a shadow removing method according to an embodiment of the present invention.

As illustrated in FIG. 5, the center axis of the extracted object and the direction of the shadow are found, and then divided into the object area and the shadow candidate area by using the symmetry of the object.

In general, RGB values did not remove accurate shadows. Therefore, we use chromaticity values that are less sensitive to changes in brightness to remove shadows.

Extracted objects R _k and the left and right of the object's bounding box are called BL and BR. Then, the CP is detected based on the CP detected in FIG. 3, and the object area and the shadow candidate area are substituted by substituting the X-axis coordinates S _i ^l and S _i ^r of the first pixel included in the object R _k into Equation 3. Separate.

Here, S _i ^x represents the distance of the pixel at the X coordinate on the Y axis moving along the central axis CP of R _k .

4. Shadow Removal

The object extracted by using the method using the symmetry of the object 5 in the R _k a R _k ^1, shows a shadow candidate area to R _k ^2.

Chromaticity values with each pixel of the object region R _k to remove the shadow r _c, g _c (red, green), and the average of the shadow candidate region R _k ² in r _c, g _c, and then using the equation (4) Obtain

r _c = r / (r + g + b)

g _c = g / (r + g + b)

6A and 6B are diagrams showing a weighted application area according to a deviation of a vertical histogram in a shadow removing method according to an embodiment of the present invention.

FIG. 6A illustrates an input image, and FIG. 6B is a graph showing averages of histogram deviations by calculating an average of vertical histograms for each frame of the input image of FIG. 6A.

Referring to FIG. 6B, it is often not the object that is far from the central axis of the object. Using this property, weights according to distances are applied from regions where the deviation is equal to or less than an arbitrary value.

That is, the subtracting an arbitrary value to the weighted difference between r _c, g _{c k} of the R ² and the average value, R _k in accordance with the threshold value obtained in the distance to the central axis by the experimental results.

Thereafter, the comparison is determined as a background image when the value is smaller than the threshold. However, applying one threshold across the entire area can result in the loss of objects.

To solve this problem, the area in the bounding box is divided into three parts, Upper, Middle, and Lower Bounds, and a threshold for each area is obtained.

When the shadow is in the lower bound, the shadow is removed only for the region instead of for all the regions.

7A to 7D are diagrams showing experimental results according to a method for removing shadows according to an embodiment of the present invention.

Here, the experiment performs background modeling using a median value of intensity using consecutive image N frames having RGB values.

Thereafter, a difference image is detected with respect to a continuous image coming in after N frames. At this time, the noise is removed from the extracted image, and a bounding box is placed on each extracted object.

In addition, for the region inside the bounding box of each detected object, a method of updating the size of the bounding box after applying the shadow removing method according to the present invention is used.

7A and 7B are conventional 100th frames and 221th frames, respectively, and show object tracking results when only a difference image is used without removing shadows in a continuous image.

7C and 7D are the 100th and 221th frames of the present invention, respectively, and it can be seen that shadows are removed from the same frame so that correct object tracking is performed.

In the above, the method of removing the shadow using the symmetry and the distance vector of the object extracted by the subtraction operation has been described.

In addition, by using the physical properties that distinguish between shadows and objects, accurate shadows can be removed even in similar colors, and it can be extended to remove shadows for objects extracted in groups.

Therefore, the present invention is not limited to the above-described embodiments, and a person having ordinary skill in the art may change the design or avoid the design without departing from the scope of the technical idea of the present invention. Will be in range.

As described above, the present invention extracts the shadow candidate region using the symmetry of the object, and then removes the shadow from the extracted object using a weight proportional to the distance from the center axis of the object.

This removes the shadows in the object, allowing the extraction of the correct feature vector and accurate object tracking.

Claims (10)

(a) extracting an object from an input image; (b) detecting a central axis for the object; (c) detecting a shadow direction through the detected central axis; (d) detecting a shadow candidate region by using the symmetry of the object after detecting the directionality of the shadow; (e) removing the detected shadow. The method of claim 1, In step (b), The central axis is obtained by obtaining a vertical histogram for the object, dividing the obtained vertical histogram into an arbitrary number of blocks horizontally, and setting the positions of the X-axis of the largest histogram for each block as max1 and max2. The method for removing shadows, characterized by the above formula (1). CP (x) = (max1 (x) + max2 (x)) / 2 ........ (1) The method of claim 1, In the step (c), Determination of the direction of the shadow is a shadow removal method, characterized in that obtained by the following formula (2).

..........(2) Here, O _s is a shadow direction, Lx and Rx represent right and left end coordinates of the bounding box, and ε means a threshold. The method of claim 1, In the step (d), the shadow removal method, characterized in that divided into the object region and the shadow candidate region by the following equation (3).

.......... (3) Here, CP is the central axis of the object and is represented by the X-axis coordinates S _i ^l and S _i ^r of the first pixel included in the object, where S _i ^x is the X-coordinate on the Y axis moving along the central axis CP of the object. Represents the distance of the pixel at. The method of claim 1, In the step (e), when the shadow is inside the object using a chromaticity value, when outside the shadow removal method characterized in that to remove the shadow by applying a weight to the distance from the central axis. The method of claim 5, In the applying of the weight, the shadow removal method characterized in that for applying the weight according to the distance from the region where the deviation of the vertical histogram is less than a predetermined value for every frame of the input image. The method of claim 5, The value obtained by subtracting the weight from the difference between the chromaticity average value of the shadow candidate region and the chromaticity average value of the object region is compared with a threshold value, and when the subtracted value is smaller than the threshold value, it is determined as a background image. How to get rid of shadows. The method of claim 1, And removing noise from the extracted object and covering a bounding box with respect to each of the objects. The method of claim 8, And dividing an area of the bounding box into upper, middle, and lower portions, and obtaining a threshold value for each region. The method of claim 9, And when the shadow is in the lower bound, removes the shadow only for the lower bound region.

Images