KR20130013462A - Foreground extraction apparatus and method using ccb and mt lbp - Google Patents

Abstract

PURPOSE: A foreground detection device and a method using a text property with a square codebook and a multiple boundary value are provided to apply a predetermined boundary value to a combination reliability map, thereby rapidly detecting a foreground from a background. CONSTITUTION: A background modeling unit(110) generates a color background model and a texture background model by modeling of a background. A reliability value calculation unit(120) calculates a color reliability value and a texture reliability value for an inputted image. A reliability map combination unit(130) combines a color reliability map with a texture reliability map corresponding to the inputted image. A detection unit(140) detects a foreground from the background by applying a predetermined boundary value to the generated reliability map. [Reference numerals] (110) Background modeling unit; (120) Reliability value calculation unit; (130) Reliability map combination unit; (140) Detection unit

Description

Foreground extraction apparatus and method using CCB and MT LBP using square codebook and texture features with multiple boundary values

The present invention relates to image detection, and more particularly, to a method for detecting a foreground (image) which is a moving object in the background (image) of a screen photographed through a fixed image device such as CCTV.

In order to detect the foreground in the background, background modeling must be performed in advance. If the frame is additionally input in the future and the content of the input frame is significantly different from the content of the background, it may be determined that there is information on the 'foreground' among the contents of the input frame.

The background modeling technique for detecting the foreground is largely divided into the following three methods: probability model based, texture based, and codebook based.

First, the probability model based method models pixel value pattern information of one pixel of a screen using a probability density function. Commonly used probability density functions include Gaussian Mixture Model (GMM) and kernel functions. However, such a technique has a disadvantage in that a computational process is very slow and a learning time for generating a background model is relatively long because the process of predicting and calculating a complex probabilistic model is performed.

Next, the texture-based technique is a method of modeling a relationship between one pixel and neighboring pixels as background information, not the pixel value itself. The most widely used examples include M. Heikkila and M. Pietikainen, "A Texture-Based Method for Modeling the Background and Detecting Moving Objects", Pattern Analysis and Machine Intelligence, vol. Proposed Local Binary Pattern (LBP), X. Tan and B. Triggs, "Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions", IEEE transactions on Image Processing, vol. . 19, pp. Local Ternary Pattern (LTP) as proposed in 1635-1650, 2010, and in S, Liao, G. Zhao, V. Kellokumpu, M. Pietikainen, and S.Z. Li, "Modeling Pixel Process with Scale Invariant Local Patterns for Background Subtraction in Complex Scenes", Computer Vision and Pattern Recognition, pp. Scale Invariant Local Ternary Pattern (SILTP) technique as proposed in 1301-1306, 2010, and the like. However, there are disadvantages in that LBP technique is very sensitive to noise, and LTP technique is sensitive to lighting change. SILPT is robust against both illumination and noise, but has a problem of generating noise when the difference between the center pixel and neighboring pixels does not show a Gaussian distribution.

Finally, codebook based techniques are described in K. Kyungnam and T.H. Chalidabhongse, "Background modeling and subtraction by codebook construction", International Conference on Image Processing, vol. 5, pp. 3061-3064, first proposed in 2004 and recently published in F. Porikli and O. Tuzel, "Baysian Background Modeling for Foreground Detection", VSSN, 2005, Doshi. H, Trivedi. A very active study, as disclosed in M, "Hybrid Cone-Cylinder Codebook Model for Foreground Detection with Shadow and Hilight Supression", Advanced Video and Signal-based Surveillance, pp19-19, 2006. The background model is a special type of codebook modeling the pixel value information that is initially input, and if the pixel value entered during the test is located inside the learned codebook structure, the background is classified as the background and the foreground otherwise. This method does not use any form of probability distribution information, which reduces the amount of computation, and efficiently models the dynamic elements of the background, such as changes in lighting caused by the weather or arbitrary light sources, and shaking tree branches. However, in the area where the brightness of the background is small, excessive code element overlap occurs, causing a large number of fals. se negative alarm (FNA) In other words, a pixel corresponding to the foreground generates a misclassification in the background.

An object of the present invention is to provide an apparatus and method for detecting foreground using a square codebook for automatically and automatically detecting the foreground in a background and a texture feature to which multiple boundary values are applied.

In order to achieve the above object, the foreground detection apparatus using a texture feature applied with a square codebook and multiple boundary values according to at least one embodiment of the present invention, the background is a fixed image of the image photographed for a certain time period of the specific area; A background modeling unit for generating a color background model and a texture background model by modeling the color information and the texture information of each; For each of the input pixels, which are pixels constituting the input image, a value representing the likelihood that the input pixel is included in the foreground which is a moving image in the photographed image is calculated as a color reliability value in consideration of the color background model, and the probability A reliability value calculator which calculates a value representing P as a texture reliability value in consideration of the texture background model; A reliability map combiner which generates a combined reliability map by adaptively combining a color reliability map, which is an array of color reliability values, and a texture reliability map, which is an array of texture reliability values, according to the input image; And a detector configured to detect the foreground from the background by applying a predetermined boundary value to the combined reliability map.

Here, the background modeling unit generates the color background model by modeling as a set of one or more code elements, each of which is one or more cubes of which the size of the pixel values of the background is minimized and each size is constant. In this case, in generating the color background model, the background is a YCbCr image, and the pattern is a three-dimensional pattern on the Y axis, the Cb axis, and the Cr axis.

Here, the background modeling unit uses the pixels of the background as the center pixels, and the pixel value difference during the predetermined time period between the center pixel and each of the peripheral pixels which is a predetermined number of pixels spaced apart from the center pixel in the background by the predetermined pixel. In consideration of this, the texture background model is generated by determining the pixel value difference for each of the peripheral pixels for each of the center pixels. At this time, in generating the texture background model, the background is an image having only a Y component. In this case, the predetermined pixel is two pixels, and the predetermined number of pixels is six pixels.

Here, the reliability value calculator calculates the color reliability value for each of the input pixels by considering whether the input pixel is located inside the pattern and whether the shadow is affected by the shadow.

Here, the reliability value calculator is configured to determine whether the pixel value difference from the peripheral pixel when the input pixel is the center pixel is equal to or less than the determined pixel value difference for each of the input pixels. Computation is performed for each pixel, the calculated number is normalized and the normalized result is determined as the texture reliability value.

The reliability map combiner generates the combined reliability map for each of the input pixels by considering whether a difference between the color reliability value and the texture reliability value is greater than a threshold. In this case, the reliability map combiner further considers whether the color reliability value is greater than the texture reliability value, and whether each of the amount of texture of the input pixel and the amount of texture of the corresponding pixel of the background is equal to or less than a reference value. And generate the combined reliability map.

In order to achieve the above object, the foreground detection method using a texture feature applied with a square codebook and multiple boundary values according to at least one embodiment of the present invention (a) a background of a fixed image of the image photographed for a certain time in a specific area; Generating a color background model and a texture background model by modeling the color information and the texture information of the background, respectively; (b) For each of the input pixels which are pixels constituting the input image, a value representing the possibility of the input pixel being included in the foreground which is a moving image in the photographed image is calculated as a color reliability value in consideration of the color background model. Calculating a value representing the likelihood as a texture reliability value in consideration of the texture background model; (c) adaptively combining a color reliability map, which is an array of color reliability values, and a texture reliability map, which is an array of texture reliability values, according to the input image to generate a combined reliability map; And (d) detecting the foreground in the background by applying a predetermined boundary value to the combined reliability map.

Here, the step (a) generates the color background model by modeling as a set of one or more code elements each of which is one or more cubes each having a constant size and at least wrapping the pattern of pixel values of the background. . In this case, in generating the color background model, the background is a YCbCr image, and the pattern is a three-dimensional pattern on the Y axis, the Cb axis, and the Cr axis.

Here, the step (a) is a pixel during the predetermined time period between the center pixel and each of the peripheral pixels which are a predetermined number of pixels spaced apart from the center pixel in the background by using each pixel of the background as a center pixel. In consideration of the value difference, the texture background model is generated by determining the pixel value difference for each of the peripheral pixels for each of the center pixels. In this case, in generating the texture background model, the background is an image having only Y components, the predetermined pixel is two pixels, and the predetermined number of pixels is six pixels.

Here, the step (b) calculates the color reliability value for each of the input pixels by considering whether the input pixel is located inside the pattern and whether the shadow is affected by the shadow.

In the step (b), it is determined whether the pixel value difference with the peripheral pixel when the input pixel is the center pixel is equal to or less than the determined pixel value difference for each of the input pixels. Computation is performed for each of the neighboring pixels, the calculated number is normalized, and the normalized result is determined as the texture reliability value.

Here, in step (c), the combined reliability map is generated by considering whether the difference between the color reliability value and the texture reliability value is greater than a threshold value for each of the input pixels. In this case, the step (c) further considers whether the color reliability value is greater than the texture reliability value, and whether each of the amount of the texture of the input pixel and the amount of the texture of the corresponding pixel of the background is equal to or less than a reference value. To generate the combined reliability map.

According to at least one embodiment of the present invention, the foreground detection apparatus and method using the texture feature to which the square codebook and the multiple boundary values are applied can quickly and accurately detect the foreground in the background.

1 is a block diagram illustrating a foreground detection apparatus using a texture feature to which a square codebook and multiple boundary values are applied according to at least one embodiment of the present invention.
2 is a reference diagram for describing a color background model according to at least one embodiment of the present invention.
FIG. 3 is a flowchart for describing an operation of the reliability value calculator 120 shown in FIG. 1.
4 is a reference diagram for comparing an existing codebook with a codebook proposed by the present invention.
5 is a reference diagram for describing a texture background model according to at least one embodiment of the present invention.
6A and 6B are flowcharts for describing an operation of the reliability map combiner 130 shown in FIG. 1.
7 is a flowchart illustrating a foreground detection method using a texture feature to which a square codebook and multiple boundary values are applied according to at least one embodiment of the present invention.

In order to fully understand the present invention, the operational advantages of the present invention, and the objects achieved by the practice of the present invention, reference should be made to the accompanying drawings that illustrate preferred embodiments of the present invention and the accompanying drawings.

Hereinafter, an apparatus and method for detecting a foreground using a square codebook and a texture feature to which multiple boundary values are applied according to at least one embodiment of the present invention will be described with reference to the accompanying drawings.

1 is a block diagram illustrating a foreground detection apparatus using a texture feature applied with a square codebook and multiple boundary values according to at least one embodiment of the present invention, and FIG. 2 illustrates a color background model according to at least one embodiment of the present invention. 3 is a flowchart for explaining an operation of the reliability value calculator 120 shown in FIG. 1, and FIG. 4 is a reference diagram for comparing an existing codebook with a codebook proposed by the present invention. 5 is a reference diagram for explaining a texture background model according to at least one embodiment of the present invention.

As shown in FIG. 1, the foreground detection apparatus using a texture feature to which a square codebook and multiple boundary values are applied according to at least one embodiment of the present invention includes a background modeling unit 110, a reliability value calculator 120, and a reliability map. Combination unit 130 and the detection unit 140 is composed of.

The background modeling unit 110 models a 'background', which is a fixed image among 'images taken for a predetermined time (that is, learned)' of a specific region, by using the color information and the texture information of the background, respectively. Create each model and texture background model. Here, generating each means that a color background model is generated using color information, and a texture background model is generated using texture information.

The reliability value calculator 120 calculates a 'reliability value' that is a value indicating a possibility that the input pixel is included in the 'foreground' for each of the input pixels that are pixels constituting the input image. In detail, the reliability value calculator 120 calculates a 'color reliability value' in consideration of the color background model and calculates a 'texture reliability value' in consideration of the texture background model. That is, the color reliability value means a reliability value calculated in consideration of the color background model, and the texture reliability value means a reliability value calculated in consideration of the texture background model. Both the color reliability value and the texture reliability value are probability values having a value of 0 or more and 1 or less.

The operation of the background modeling unit 110 and the reliability value calculator 120 will be described in more detail.

First, an operation in which the background modeling unit 110 generates a color background model and the reliability value calculator 120 calculates a color reliability value will be described in detail.

The background modeling unit 110 wraps the background of the 'pattern of pixel values for the predetermined time' (the identification numbers 210 and 216 are examples of such a 'pattern' in FIG. 2A) to the minimum. By modeling using one or more cubes, a color background model is created. Here, the sizes of each of the 'one or more cubes' are the same (see FIG. 2B). As such, in generating the color background model, the background is a YCbCr image and the pattern is a three-dimensional pattern on the Y-axis, the Cb-axis, and the Cr-axis. In the present specification, the 'code-element' refers to the 'cube' (see FIG. 2B). As already mentioned, the size of each of the code elements is always fixed, which is the traditional technique for changing the size of the cube to represent the color range, such as K. Kyungnam and T.H. Chalidabhongse, "Background modeling and subtraction by codebook construction", International Conference on Image Processing, vol. 5, pp. 3061-3064, 2004, Barna, Doshi. H, Trivedi. M, "Hybrid Cone-Cylinder Codebook Model for Foreground Detection with Shadow and Hilight Supression", Advanced Video and Signal-based Surveillance ", pp19-19, 2006.

The reliability value calculator 120 calculates the color reliability value for each input pixel in consideration of whether the input pixel is located inside the pattern and whether the shadow is influenced by the shadow. In detail, when an input image is input to the reliability value calculator 120 (step 310), the reliability value calculator 120 determines whether the input pixel matches the codebook for each input pixel, that is, the input pixel is It is determined whether the code elements are located inside the code elements (step 312). (In the case of FIG. 2, the identification number 214 is located outside the code elements. On the contrary, since the pixel of identification number 212 is located inside the code element (s), it is evaluated as a pixel included in the background). If it is determined in step 312 that the input pixel is located inside the code elements, Determine that the 'color confidence value' is zero, i.e., determine that the input pixel is 100% likely to be 'background' (step 314), whereas in step 312 the input pixel is not located inside the code elements. If it is determined, the shadow condition of the input pixel is checked (step 316), and the color reliability value of the input pixel is considered in consideration of 'maximum number of channels matched during codebook matching (cnt_CB)' and 'shadow condition value (cnt_SC)'. Calculate (step 318). Herein, 'the maximum number of channels matched during codebook matching' refers to the number of channels located inside the codebook among the Y value (Y channel), Cb value (Cb channel), and Cr value (Cr channel) of the input pixel. For the meaning of shadow condition values, see Doshi. H, Trivedi. M, "Hybrid Cone-Cylinder Codebook Model for Foreground Detection with Shadow and Hilight Supression", Advanced Video and Signal-based Surveillance ", pp19-19, 2006.

As mentioned above, according to the present invention, all the code elements have the same size, and the reason for this is to minimize the number of false positive alarms, ie, the case where a pixel corresponding to the background is misclassified into the foreground. I express it. The left figure of FIG. 4 (a) shows the distribution of the learning samples input during the learning period (= during the above-described 'time'). Here, the learning sample refers to pixel values of images (frames) input during the learning period, and all of the points shown in FIG. 4 are represented in the YCbCr space. K. Kyungnam and T.H. Chalidabhongse, "Background modeling and subtraction by codebook construction", International Conference on Image Processing, vol. 5, pp. 3061-3064, 2004 generate code elements 400, 402, ... as shown on the right side of FIG. 4 (a) to cover this sample distribution. However, this type of codebook can generate a false negative alarm region as indicated by the identification number 410. For example, even if the input pixel 420 is actually a pixel included in the foreground, according to FIG. 4A, the input pixel 420 may be misclassified as a pixel included in the background. On the other hand, if the codebook is a square codebook according to the present invention (see Fig. 4 (b)), such an error area does not occur, and thus more accurate foreground detection is possible. Except for fixing the codebook size of code elements, at least one embodiment of the present invention for the 'codebook generation', 'update' and 'removal' methods is described in K. Kyungnam and T.H. Chalidabhongse, "Background modeling and subtraction by codebook construction", International Conference on Image Processing, vol. 5, pp. 3061-3064, 2004, was used.

Meanwhile, the meaning of codebook matching is further explained. In order to check whether the codebook is matched, that is, to determine whether the input pixel is located inside the code elements, the color range that the 'background' may have in the codebook The information must be stored. For example, assuming that the range of color values observed during learning is 0 <Y <100, 75 <Cr <100, 125 <Cb <200, if 'Y = 120, Cr = 80, Cb = 0' When an input pixel comes in, it must be classified as 'foreground' because it does not satisfy the observed Y range.

Up to now, the background modeling unit 110 generates a color background model, and the reliability value calculator 120 calculates the color reliability value in detail. Hereinafter, the background modeling unit 110 An operation of generating a texture background model and calculating the texture reliability value by the reliability value calculator 120 will be described in detail.

The background modeling unit 110 uses a predetermined number of pixels spaced apart from the center pixel in the background by a predetermined pixel (for example, two pixels (see FIG. 5)) with each pixel of the background as the center pixel. A texture background model is generated by determining the pixel value difference for each of the neighboring pixels for each center pixel in consideration of the pixel value difference during the predetermined time between each of the neighboring pixels and the center pixel. Herein, the 'determined pixel value difference' means 'back_range' described in FIG. 5. Referring to the case of FIG. 5 as an example, the background modeling unit 110 is provided for each of the peripheral pixels of the central pixel C (six pixels represented by 0, 1, 2, 3, 4, and 5 in FIG. 5). The pixel value difference from the center pixel C is determined as 'back_range_i' (i is an integer of 0 ≦ i ≦ 5). As such, in generating the texture background model, the background is an image having only Y components. Generating the texture background model means generating boundary value information for generating binary codes. Unlike the color background model consisting of one codebook, the texture background model may be composed of six codebooks CB_i. The input for generating CB_i is a Y component difference value between the center pixel C and the i-th peripheral pixel. The specific codebook generation method is almost identical to that of the color model. The difference is that the color background model uses the (Y, Cr, Cb) vector of the pixel as an input for codebook generation. In the case of CB_i, the Y component difference value between the center pixel and the i-th peripheral pixel is used. .

The reliability value calculator 120 determines, for each of the input pixels, whether the pixel value difference from the peripheral pixel when the input pixel is the center pixel is equal to or smaller than the corresponding 'determined pixel difference'. Calculate, normalize the calculated number and determine the normalized result as texture confidence value.

The operation of the reliability value calculator 120 will be described with reference to FIG. 5. Once CB_i is generated, the Y component difference value pattern between the center pixel C and the surrounding pixels (pixels denoted by 0, 1, 2, 3, 4, and 5 in FIG. It can be expressed as shown in FIG. In FIG. 5C, each of the identification numbers 510 to 518 represents a code element, and a back_range between dotted lines indicates a range of possible Y component difference values. If the input pixel has entered and the Y component difference value between the input pixel and the i-th peripheral pixel is equal to or smaller than the 'determined pixel value difference (back_range_i)', the input pixel is denoted as 1, otherwise 0 is indicated (FIG. 5). (C) of). If this method is performed for six CB_i, a six-dimensional 0-1 vector can be obtained as a response value for the input pixel. This vector is useful for predicting a texture reliability map to be described later. The reliability value calculator 120 calculates a texture reliability value according to Equation 1 below.

[Equation 1]

conf_t_ (p) = (number of binary codes '0') / 6

Here, conf_t_ (p) denotes a texture reliability value for the input pixel p, and the number of binary codes 0 refers to the number of 0s in the 0-1 vectors of the above 6-dimensional, and dividing by 6 means normalization.

The reliability map combiner 130 adaptively combines a 'color reliability map', which is an array of color reliability values, and a texture reliability map, which is an array of texture reliability values, according to the input image. Generate a joint reliability map.

The reliability map combiner 130 may calculate the combined reliability value of one input pixel using the color reliability value and the texture reliability value according to Equation 2 below. The joint reliability map is an array of joint reliability values calculated as described above.

[Equation 2]

conf_com = α * conf_t + (1-α) * conf_c

Here, conf_com means a coupling reliability value, α (where α is a real number of 0≤α≤1) means a weight, conf_t denotes a texture reliability value, and conf_c denotes a color reliability value.

α is the difference between the color confidence value and the texture confidence value (ConDiff), the amount of texture the background pixel has (amt_back), the amount of texture the input pixel has (amt_ins), the difference between the two texture quantities (amtDiff) and the color of the pixel. It is determined by (Lab value). Specifically, the relationship of Equation 3 below is followed.

[Equation 3]

Here, i denotes an index of each of all peripheral pixels, j denotes an index of each of all code elements, elem_mean_ij denotes an i-th peripheral pixel and a center coordinate of the j-th code element, and k is a corresponding pixel. The total number of code elements the texture background model of.

The detection unit 140 detects the 'foreground' in the 'background' by applying a predetermined boundary value to the combined reliability map.

6A and 6B are flowcharts for describing an operation of the reliability map combiner 130 shown in FIG. 1.

The reliability map combiner 130 generates a combined reliability map for each input pixel in consideration of whether the difference between the color reliability value and the texture reliability value is greater than the threshold (operation 610).

Further, the reliability map combiner 130 further adds whether the color reliability value is greater than the texture reliability value and whether the amount of the texture of the input pixel and the amount of the texture of the corresponding pixel of the background is less than or equal to the reference value. In consideration, a combined reliability map is generated (steps 612 to 622).

Specifically, steps 610 to 622 are performed as follows.

First, the reliability map combiner 130 calculates ConDiff (step 600), determines whether the ConDiff is greater than a threshold (eg, 0.3), and determines whether both the color reliability value and the texture reliability value are reliable. Step 610.

If it is determined in step 610 that the ConDiff is not greater than the threshold, the reliability map combiner 130 determines that the color reliability value and the texture reliability value are both reliable and determines the weight value α as 0.5.

On the other hand, if it is determined in step 610 that the ConDiff is greater than the threshold value, the reliability map combiner 130 determines that one of the color reliability value and the texture reliability value is wrong, thereby finding which reliability value is wrong (step 612). To step 622).

The reliability map combiner 130 determines whether the color reliability value is greater than the texture reliability value (step 612), and if it is determined that the color reliability value is greater than the texture reliability value, the reliability map combiner 130 checks the texture amount of the background and the texture amount of the input pixel. (Step 614). This means checking whether amt_ins and amt_back are both greater than eg 15 and amtDiff is less than 10 for example. If the conditions checked in step 614 are satisfied, this means that there is almost no texture in the input and the background, so the reliability map combiner 130 determines that the texture reliability value is unreliable and the weight α is 0. In other cases, the weight α is determined to be 0.5.

On the other hand, if it is determined in step 612 that the color reliability value is less than or equal to the texture reliability value, the reliability map combiner 130 performs a texture pattern analysis module (step 616) to determine whether the corresponding texture is caused by a shadow. To judge. If it is determined that the texture is not caused by the shadow, the reliability map combining unit 130 ignores the color reliability value by setting the weight α to 1, and otherwise sets the texture reliability value by setting the weight α to 0. Ignore it. However, if a texture change occurs in the input image when the amount of background texture is large, the weight α is set to 0.5 to consider both the texture reliability value and the background reliability value.

6B is a flowchart for describing operation 616 in detail.

The texture pattern analysis module is used to determine whether the texture change in the input is caused by foreground or shadow. To this end, the reliability map combiner 130 first checks the texture amount (amt_ins) of the input pixel and the texture amount (amt_back) of the background pixel (step 618).

If it is determined in step 618 that both the size of amt_ins and amt_back are small, this means that the amount of texture detected in the background and input is small. In this case, the reliability map combiner 130 determines that the reliability of the texture reliability value by the texture background model is very low, and sets the weight α to 0 so that the influence of the texture background model is ignored.

On the other hand, if amt_back is small in step 618 but the size of amt_ins is determined to be large, this means that the background pattern is covered by the foreground or shadow. In this case, the reliability map combiner 130 must finally determine whether the pixel is the foreground or the shadow by checking the cnt_SC value and the color information of the input pixel. If the pixel is a shadow, the weight α is 0. In the case of the foreground, the weight α is set to 1 (steps 620 and 622).

On the other hand, if it is determined in step 618 that the size of amt_back is large enough, since no special feature can be expected, the reliability map combiner 130 sets the weight α to 0.5 to contribute to the color background model and the texture background model. To be the same.

7 is a flowchart illustrating a foreground detection method using a texture feature to which a square codebook and multiple boundary values are applied according to at least one embodiment of the present invention.

First, the foreground detection apparatus using the texture feature applied with the square codebook and the multi-boundary value according to at least one embodiment of the present invention includes a color of the background of 'background' which is a fixed image of the image photographed for a certain time. In operation 710, the color background model and the texture background model are generated by modeling the information and the texture information.

After operation 710, the foreground detection apparatus using the texture feature to which the square codebook and the multi-boundary value are applied according to at least one embodiment of the present invention may display a value indicating the possibility that the input pixel is included in the 'foreground' for each input pixel. In consideration of the model, it is calculated as the color reliability value, and in consideration of the texture background model, it is calculated as the texture reliability value (step 720).

After operation 720, the foreground detection apparatus using the texture feature to which the square codebook and the multiple boundary values are applied according to at least one embodiment of the present invention may include a color reliability map and an array of color reliability values for the input image. In operation 730, a combined reliability map is generated by adaptively combining the 'texture reliability map', which is an array of texture reliability values, according to the input image.

After operation 730, the foreground detection apparatus using the texture feature to which the square codebook and the multiple boundary values are applied according to at least one embodiment of the present invention detects the 'foreground' in the 'background' by applying a preset boundary value to the combined reliability map. (Step 740).

A program for executing a foreground detection method using a square codebook and a texture feature applied with multiple boundary values according to the present invention as described above on a computer may be stored in a computer-readable recording medium.

Here, the computer-readable recording medium may be a magnetic storage medium (for example, a ROM, a floppy disk, a hard disk, etc.), and an optical reading medium (for example, a CD-ROM, a DVD). : Digital Versatile Disc).

The present invention has been described above with reference to preferred embodiments. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

110: background modeling unit 120: reliability value calculation unit
130: reliability map coupling unit 140: detection unit

Claims (20)

A background modeling unit for generating a color background model and a texture background model by modeling a background, which is a fixed image, among the images photographed for a predetermined time in a specific region by using the color information and the texture information of the background, respectively;
For each of the input pixels, which are pixels constituting the input image, a value representing the likelihood that the input pixel is included in the foreground which is a moving image in the photographed image is calculated as a color reliability value in consideration of the color background model, and the probability A reliability value calculator which calculates a value representing P as a texture reliability value in consideration of the texture background model;
A reliability map combiner which generates a combined reliability map by adaptively combining a color reliability map, which is an array of color reliability values, and a texture reliability map, which is an array of texture reliability values, according to the input image; And
And a detection unit for detecting the foreground in the background by applying a predetermined boundary value to the combined reliability map. The method of claim 1, wherein the background modeling unit
Square codebooks and multiples are generated by modeling as a set of one or more code elements that are at least one cubes of each size having a constant wrap pattern of pixel values of the background at least. A foreground detection device using a texture feature to which a boundary value is applied. 3. The texture of claim 2, wherein the background is a YCbCr image and the pattern is a three-dimensional pattern on a Y-axis, a Cb-axis, and a Cr-axis. Foreground detection device using the feature. The method of claim 1, wherein the background modeling unit
Considering the pixel value difference during the predetermined time between each of the peripheral pixels, which are a predetermined number of pixels spaced apart from the center pixel by a predetermined pixel in the background, with each pixel of the background as the center pixel, And the texture background model is generated by determining the difference in pixel values for each of the neighboring pixels for each pixel, and the foreground detection apparatus using the texture feature to which the multiple boundary values are applied. 5. The foreground detection apparatus of claim 4, wherein the background is an image having only a Y component in generating the texture background model. 6. 5. The foreground detection apparatus of claim 4, wherein the predetermined pixel is two pixels and the predetermined number of pixels is six pixels. 6. The method of claim 2, wherein the reliability value calculator
For each of the input pixels, the color reliability value is calculated based on whether the input pixel is located inside the pattern and whether the shadow is affected by the shadow. Foreground detection apparatus using a. The method of claim 4, wherein the reliability value calculator
For each of the input pixels,
When the input pixel is the center pixel, whether the pixel value difference with the peripheral pixel is equal to or smaller than the determined pixel value difference is calculated for each of the peripheral pixels with respect to the input pixel, and the calculated number is normalized. And a normalized result is determined as the texture reliability value, and the foreground detection apparatus using the texture feature to which the multi-boundary value is applied to the square codebook. The method of claim 1, wherein the reliability map combiner
Foreground detection apparatus using a texture feature applied to the square codebook and the multiple boundary values, characterized in that for generating the combined reliability map, considering whether the difference between the color reliability value and the texture reliability value is greater than a threshold value for each input pixel . 10. The method of claim 9, wherein the reliability map combiner
Generating the combined reliability map by further considering whether the color reliability value is greater than the texture reliability value and whether each of the amount of texture of the input pixel and the amount of texture of the corresponding pixel of the background is equal to or less than a reference value. An apparatus for detecting foreground using a square codebook and a texture feature to which multiple boundary values are applied. (a) generating a color background model and a texture background model by modeling a background, which is a fixed image of the image photographed for a certain time period, using the color information and the texture information of the background, respectively;
(b) For each of the input pixels which are pixels constituting the input image, a value representing the possibility of the input pixel being included in the foreground which is a moving image in the photographed image is calculated as a color reliability value in consideration of the color background model. Calculating a value representing the likelihood as a texture reliability value in consideration of the texture background model;
(c) adaptively combining a color reliability map, which is an array of color reliability values, and a texture reliability map, which is an array of texture reliability values, according to the input image to generate a combined reliability map; And
and (d) detecting the foreground in the background by applying a predetermined boundary value to the combined reliability map. 12. The method of claim 11, wherein step (a)
Square codebooks and multiples are generated by modeling as a set of one or more code elements that are at least one cubes of each size having a constant wrap pattern of pixel values of the background at least. A foreground detection method using texture features with edge values applied. The method of claim 12,
In generating the color background model, the background is a YCbCr image and the pattern is a three-dimensional pattern on a Y-axis, a Cb-axis, and a Cr-axis. . 12. The method of claim 11, wherein step (a)
Considering the pixel value difference during the predetermined time between each of the peripheral pixels, which are a predetermined number of pixels spaced apart from the center pixel by a predetermined pixel in the background, with each pixel of the background as the center pixel, And the texture background model is generated by determining the pixel value difference for each of the peripheral pixels for each pixel, and the foreground detection method using the texture feature to which the multiple boundary values are applied. 15. The method of claim 14,
In generating the texture background model, the background is an image having only a Y component, and the foreground detection method using a texture feature to which a multi-boundary value is applied. 15. The method of claim 14,
The predetermined pixel is two pixels, and the predetermined number of pixels is six pixels, the foreground detection method using a texture feature to which a square codebook and multiple boundary values are applied. The method of claim 12, wherein step (b)
For each of the input pixels, the color reliability value is calculated based on whether the input pixel is located inside the pattern and whether the shadow is affected by the shadow. Foreground detection method using. The method of claim 14, wherein step (b)
For each of the input pixels,
When the input pixel is the center pixel, whether the pixel value difference with the peripheral pixel is equal to or smaller than the determined pixel value difference is calculated for each of the peripheral pixels with respect to the input pixel, and the calculated number is normalized. And a normalized result is determined as the texture reliability value, and the foreground detection method using the texture feature to which the multi-boundary value and the square codebook are applied. The method of claim 11, wherein step (c)
For each input pixel, the combined reliability map is generated by considering whether the difference between the color reliability value and the texture reliability value is greater than a threshold value. . The method of claim 19, wherein step (c)
Generating the combined reliability map by further considering whether the color reliability value is greater than the texture reliability value and whether each of the amount of texture of the input pixel and the amount of texture of the corresponding pixel of the background is equal to or less than a reference value. A foreground detection method using a texture feature to which a square codebook and multiple boundary values are applied.

Images