KR101468861B1 - Active attentional sampling method for accelerating foreground detection from video, and computer-readable recording medium for the same - Google Patents

Abstract

The present invention applies a foreground probability map and a sampling mask according to temporal characteristics, spatial characteristics, and frequency characteristics of an input image to a foreground detection algorithm that performs operations on a pixel-by-pixel basis to remove a background from an input image, To an active focused sampling technique capable of improving detection speed. According to the present invention, when an active focused sampling mask is generated for an input image, only the portion corresponding to the sampling mask is processed for the corresponding frame, so that foreground detection is achieved, and the foreground detection speed is greatly improved. In other words, compared with the conventional foreground detection method performed on a pixel-by-pixel basis, the processing speed can be improved to about 6.6 times without deteriorating the detection performance, and the full-HD image can be detected in real time .

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an active focused sampling method for improving a foreground detection speed from an image, and a computer readable recording medium therefor. 2. Description of the Related Art [0002]

The present invention relates to an active focused sampling method for improving the foreground detection rate from an input image. More particularly, the present invention relates to a foreground detection algorithm for performing foreground detection on a pixel-by-pixel basis in order to remove a background from an input image and to detect a foreground, and a foreground probability map and a sampling mask To an active focused sampling technique that can improve the foreground detection rate.

The foreground detection algorithm is the process of separating the actual moving parts from the input image, not the background parts. As computer vision technology develops, foreground detection algorithms are recognized as preprocessing steps for high - level complex algorithms and are required to be processed very quickly. In addition, the problem of increasing the operation speed due to the tendency of the image size to become larger is becoming increasingly important.

Recently, the foreground detection technique based on the probability model based on the pixel has been attracting much attention because of its good performance. Although these techniques show good detection performance in various image situations, there is a problem that it is difficult to apply in real time because calculation time is too long because of a large amount of computation.

Accordingly, various researches for increasing the computation speed of the foreground detection algorithm have been made as follows.

The first approach is to optimize the algorithm. The mixed Gaussian model (GMM) detects the foreground in various video environments, but it is slow to learn and requires many operations per frame. Thus, D.-S. Lee. "Effective Gaussian mixture learning for video background subtraction," TPAMI, 2005 accelerated the convergence of learning models using a learning schedule method that gradually changes between the two learning methods. Z. Zivkovic and F. van der Heijden. "Patient Recognition Letters, 2006, by Bayesian method, determines the number of Gaussian modes required by each pixel for the image, which greatly improves the speed. P. Gorur and B. Amrutur. "Speeded up gaussian mixture model algorithm for background subtraction," AVSS 2011 further improved Zivkovic's method by minimizing real number operations.

The second approach uses parallel processing, and a method of increasing the operation speed by using a multicore processor such as OpenMP or GPU has been proposed. V. Pham et al. "IEEE RIVF, 2010 shows that full-HD video can be detected in real time using full-HD video using the GPU. The second approach has the disadvantage of requiring parallel processing hardware to improve detection speed.

The third approach is selective sampling. J. Park et al. IEEE ICIP, 2006, proposed a hierarchical decomposition method of input image using a quad-tree structure. The computation time is reduced, while a small object is missed. Disadvantages existed. H.-K. Kim et al. IEEE ITC-CSCC 2008 proposed a sampling mask generation method that can be combined with the conventional pixel-based foreground detection method, and D.-Y. Lee et al. "Fast background subtraction algorithm using two-level sampling and silhouette detection," IEEE ICIP, 2009 also proposed two-stage pixel sampling. These algorithms result in relatively accurate detection of small objects, but they are inefficiently performing unnecessary operations because they use tightly designed structured sampling patterns.

It is an object of the present invention to provide an active focused sampling method for improving the foreground detection rate from an input image.

More particularly, the present invention relates to a foreground detection algorithm that performs operations on a pixel-by-pixel basis in order to remove a background from an input image and detect a foreground, and provides a foreground probability map and a foreground probability map according to temporal characteristics, spatial characteristics, And to provide an active focused sampling technique that can improve the foreground detection rate by applying a sampling mask.

)를 생성하는 제 2 단계; 전경 확률지도(

)와 전경탐지 결과인 탐지 마스크(D)를 기초로 샘플링 마스크(

)를 프레임별로 생성한 뒤, 샘플링 마스크(

)에 대해서 선택적인 픽셀단위 전경탐지를 수행하는 제 3 단계;를 포함하여 구성된다.According to an aspect of the present invention, there is provided a method for acquiring an active sampling mask for improving foreground detection speed, the method comprising: a first step of acquiring temporal characteristics, spatial characteristics, and frequency characteristics of three foreground objects; Based on temporal, spatial, and frequency characteristics,

); Foreground probability map (

) And a detection mask D as a result of foreground detection

) Is generated for each frame, and a sampling mask (

And a third step of performing selective pixel-based foreground detection on the pixel-by-pixel basis.

와 샘플링 마스크

를 이용하여 탐지 마스크의 시퀀스

를 찾는 것이 바람직하다.In the present invention, the foreground detection for generating the detection mask (D) in the third step is a sequence of video frames

And a sampling mask

A sequence of detection masks

.

)를 생성한 후 랜덤산발형 샘플링(C3), 중요도 공간확장형 샘플링(C4), 깜짝픽셀 샘플링(C5)을 적용하여 구성되는 것이 바람직하다.Also, in the present invention, the third step is a foreground probability map

, Random scattering type sampling (C3), importance space expansion sampling (C4), and surprise pixel sampling (C5) are preferably applied.

) 생성을 위해 전경 특성의 추정치를 연산하며, 시간특성 척도(

)를 탐지 결과의 이력에서 추측하며, 공간특성 척도(

)를 각 픽셀 주변의 전경 픽셀 수로 추정하며, 주파수특성 척도(

)를 탐지 결과가 일정 시간 동안 바뀌는 비율을 통해 추정하는 것이 바람직하다.Further, in the present invention,

), And calculates a time property measure ("

) Is estimated from the history of the detection result, and the spatial characteristic measure

) Is estimated as the number of foreground pixels around each pixel, and the frequency characteristic measure (

) Is estimated based on the rate at which the detection result changes over a certain period of time.

Meanwhile, a computer-readable recording medium according to the present invention records a program for performing an active focused sampling method for improving the foreground detection speed from the above image.

According to the present invention, when an active focused sampling mask is generated for an input image, only the portion corresponding to the sampling mask is processed for the corresponding frame, so that foreground detection is achieved, and the foreground detection speed is greatly improved. In other words, compared with the conventional foreground detection method performed on a pixel-by-pixel basis, the processing speed can be improved to about 6.6 times without deteriorating the detection performance, and the full-HD image can be detected in real time .

1 shows an overall algorithm of an active focused sampling method according to the present invention.
FIG. 2 is a diagram showing a result of foreground detection performed using an active focused sampling mask according to the present invention. FIG.
FIG. 3 is a diagram for explaining the concept of spatial scalable sampling in the present invention; FIG.
FIG. 4 is a graph showing the influence of a parameter ω _s and a variable k in critical space expansion sampling in the present invention. FIG.
FIG. 5 is a diagram illustrating the average of F ₁ - measures of each frame during the entire sequence for various foreground detection methods for explaining the effect of the present invention. FIG.
FIG. 6 is a view showing a calculation time velocity increase result for explaining the effect of the present invention; FIG.
FIG. 7 is a view showing a calculation time change during an entire frame of an image for explaining the effect of the present invention; FIG.
FIG. 8 is a diagram showing the degree of decrease in average calculation amount for explaining the effect of the present invention. FIG.

Hereinafter, the present invention will be described in detail with reference to the drawings.

The active focused sampling method of the present invention is designed based on a human selective attention mechanism in which previous experience has an effect on the current attention area. For example, when a guard watches a CCTV camera, he does not focus on the entire image. Instead, he empirically classifies the area of the image into ① the background part (the non-moving part), ② the moving part, but not the important part, and ③ the moving part that needs attention. The present invention models this selective attention system.

For example, in an image obtained from surveillance video, most of the pixels are background portions, and the foreground portions occupy a very small portion both temporally and spatially. The ratio of the foreground part in the data set generally used in the conventional literature is measured as a percentage as shown in the following [Table 1].

Dataset Number of frames Average (%) Standard Deviation Wallflower 7553 5.03 6.25 VSSN2006 16074 2.30 1.13 PETS2006 41194 1.04 0.26 AVSS2007 33000 3.36 1.02 PETS2009 2581 5.48 1.58 SABS 6400 2.42 1.83 Average 2.42 1.18

The data set in [Table 1] is Wallflower, VSSN2006, PETS2006, AVSS2007 i-LIDS challenge, PETS2009, SABS, and the pixel ratio of the foreground part is about 2.42% in general input image. Therefore, if the foreground detection operation can concentrate only on the foreground area, the computation amount is drastically reduced. The present invention has a technical point that focuses on a foreground region detected in a previous frame and concentrates on a current frame in the input image.

Hereinafter, the technical idea of the present invention will be described in detail with reference to the drawings.

1 is a diagram showing an overall algorithm of the active focused sampling method of the present invention. Referring to FIG. 1, in order to obtain an active sampling mask for improving the foreground detection speed, three characteristics of the foreground distinguishable from the background, that is, temporal, spatial, and frequency characteristics are used (C1).

The temporal property is expected to be part of the foreground region in the current frame if the pixel was previously the foreground pixel. The spatial property is that if the surrounding pixels are foreground, then the pixel is likely to be a foreground pixel, which is proportional to the number of surrounding foreground pixels. The frequency characteristic is that if the foreground / background label changes too often, the pixel is likely to be noise or a dynamic background region, and is less likely to be a stable foreground region.

)를 생성한다(C2). 즉, 본 발명에서는 바람직하게는 모든 프레임에서 전경 확률지도

에 따라 업데이트를 시행한다.Then, based on these temporal, spatial, and frequency characteristics,

(C2). That is, in the present invention, preferably, the foreground probability map

The update will be carried out according to.

In the present invention, random random scattering sampling (C3), spatially expanding importance sampling (C4), surprise pixel sampling (Surprise Pixel Sampling (C5) is used.

)를 이용하여 n이 픽셀 인덱스라고 할 때

인 픽셀에 대하여 선택적인 픽셀단위 전경탐지가 수행된다. 샘플링 마스크는 이후에 수행되는 픽셀단위 전경탐지 기술에 별다른 제한을 가하지 않으므로 임의의 픽셀단위 전경탐지 기술과 조합될 수 있다.Thereafter, in the present invention, a sampling mask is preferably generated in all the frames (C6). Sampling mask (

) And n is a pixel index

An optional pixel-by-pixel foreground detection is performed. The sampling mask can be combined with any pixel-based foreground detection technique since it does not impose any restrictions on the pixel-by-foreground detection technique to be performed subsequently.

와 샘플링 마스크

를 이용하여 탐지 마스크의 시퀀스

를 찾는 것이다. 탐지 마스크 D(n)은 픽셀 n이 배경이면 D(n) = 0이고 전경이면 D(n) = 1의 값을 갖는다. 비디오 이미지(

), 샘플링 마스크(

), 탐지 마스크(

)는 각각 N 픽셀에 해당하는 집합인

,

,

로 개별적으로 이루어져 있다. 여기서 모든 마스크는 이진 마스크이다. 본 발명에 따른 픽셀단위 전경탐지는 샘플링 마스크인

인 픽셀에서만 선택적으로 수행되는 점에 특징이 있다.
On the other hand, foreground detection is a sequence of video frames

And a sampling mask

A sequence of detection masks

. The detection mask D (n) has a value of D (n) = 1 if pixel n is background and D (n) = 1 if it is foreground. Video image (

), A sampling mask (

), A detection mask (

) Is a set of N pixels

,

,

Respectively. Where all masks are binary masks. The pixel unit foreground detection according to the present invention is a sampling mask

In which only a pixel is selected.

Hereinafter, with reference to FIG. 1 to FIG. 8, a detailed description will be given of each of the processes (C1 to C6) as to the design of the active focused sampling for improving the foreground detection speed from the image using the characteristics of the foreground.

) 생성을 위해 전경 특성의 추정치를 연산한다. 보다 구체적으로, 본 발명에서는 각 픽셀마다 시간, 공간, 주파수 특성을 측정하기 위해 추정 모델을 제안하며, 세가지 특성 척도는 {

,

,

}로 나타낸다. 시간특성 척도(

)는 탐지 결과의 최근 이력에서 추측되고, 공간특성 척도(

)는 각 픽셀 주변의 전경 픽셀 수로 추정되며, 주파수특성 척도(

)는 탐지 결과가 일정 시간 동안 바뀌는 비율을 통해 추정된다.First,

) To calculate the foreground property estimate. More specifically, in the present invention, an estimation model is proposed to measure time, space and frequency characteristics for each pixel, and three characteristic scales are {

,

,

}. Time characteristic measure (

) Is estimated from the recent history of the detection result, and the spatial characteristic measure (

) Is estimated as the number of foreground pixels around each pixel, and the frequency characteristic measure (

) Is estimated through the rate at which the detection results change over a period of time.

All estimation models are updated by activating the moving average methodology with learning rates of α _T , α _F , α _S , respectively. Here, all learning rates are between 0 and 1. Estimation model for each characteristic scale is as follows.

)는 전경의 시간 특성을 추정하기 위해 각각의 위치 n에서 그 위치의 탐지 마스크 결과의 이력을 하기의 [수학식 1]과 같이 연산하여 평균하여 구한다.First, a time characteristic measure (

) Is obtained by calculating the history of the detection mask result at the position n at each position n by calculating the following equation (1) so as to estimate the temporal characteristic of the foreground.

값이 1에 가까워 질수록 그 픽셀에서 전경이 출현할 확률이 상승한다.here

The closer the value is to 1, the higher the probability that the foreground appears at that pixel.

)는 하기의 [수학식 2]와 같이 연산되며, 주변 픽셀의 탐지 결과를 이용하여 각 픽셀 n의 공간적 연관성을 측정한다.Next, the spatial characteristic measure (

) Is calculated according to the following equation (2), and the spatial correlation of each pixel n is measured using the detection result of the neighboring pixels.

은 픽셀 n을 중심으로 주위의 w×w 정사각형 공간으로 이루어지는 공간적 이웃을 나타내며,

가 1에 가깝다는 것은 전경의 일부가 될 확률이 상승하는 것을 나타낸다.here,

Represents a spatial neighbor composed of a w x w square space around the pixel n,

Is close to 1 means that the probability of becoming part of the foreground is increased.

)는 하기의 [수학식 3]과 같이 연산되며, 탐지 결과가 예컨대 이전 세 프레임 동안 두 번이상 바뀌었다면 본 발명에서는 그것을 동적 배경의 증거로 간주한다.Next, the frequency characteristic measure (

) Is computed as in Equation (3) below, and if the detection result is changed more than once for the previous three frames, for example, the present invention regards it as proof of the dynamic background.

은 n에서의 주파수 변화 특성을 나타낸다. 시간특성 척도(

) 및 공간특성 척도(

)와는 다르게 픽셀 n 은

의 값이 0에 가까울수록 안정적인 전경이 될 가능성이 증가한다.
here,

Represents the frequency change characteristic at n. Time characteristic measure (

) And spatial characteristics (

), Pixel n is

The closer the value of " 0 " is, the more likely it is to become a stable foreground.

)를 생성하는 과정(C2)에 대해 살펴본다. 앞서의 과정을 통해 배경과 구별되는 전경 특성 척도(

,

,

)에 대한 추정 값으로서

,

,

를 획득하였다. 이들 추정 값은 0 내지 1사이의 값을 가지고 있는데, 해당 픽셀이 전경이 가능성을 표시한다. 이를 이용하여 픽셀 n, 프레임 t에서의 전경확률을 다음과 같이 [수학식 4]로 정의한다.Next,

(C2) will be described. Through the above process, the foreground feature scale

,

,

) ≪ / RTI >

,

,

. These estimated values have a value between 0 and 1, and the pixel indicates the possibility of the foreground. Using this, the foreground probability at pixel n, frame t is defined as: " (4) "

Next, a process (C6) of generating an active sampling mask will be described.

)는 픽셀 단위 OR 연산(

)으로 하기의 [수학식 5]에 의해 3가지 마스크들의 조합으로 얻어진다.In the present invention,

) Is a pixel-by-pixel OR operation

) Is obtained by a combination of three masks according to the following equation (5).

,

,

는 각각 랜덤산발형 샘플링(C3), 중요도 공간확장형 샘플링(C4), 깜짝픽셀 샘플링(C5)에 대한 샘플링 마스크이다. 각 샘플링 단계에서 샘플링 마스크는 전경 확률지도(

)와 전경탐지 결과인 탐지 마스크(D)를 기초로 생성된다. 본 발명에서 능동적 집중 샘플링을 위한 샘플링 마스크는 다음과 같은 [수학식 6]으로 표현가능하다.

,

,

Are sampling masks for random scatter sampling (C3), significance spatial scalable sampling (C4), and surprise pixel sampling (C5), respectively. At each sampling step, the sampling mask includes a foreground probability map (

And a detection mask D as a result of foreground detection. In the present invention, the sampling mask for active focused sampling can be expressed by the following Equation (6).

FIG. 2 is a diagram showing a result of foreground detection performed using an active focused sampling mask according to the present invention.

이고, 파란색 픽셀은 중요도 공간확장형 샘플링 마스크

이며, 빨간색 픽셀은 깜짝픽셀 샘플링 마스크

이다. [도 2(b)]를 참조하면 대부분의 공간이 검은 색으로 표시되어 있는데, 대부분의 공간에서 마스크는 0의 값을 갖는다. 이처럼 리던던시(redundancy)를 제거함으로써 이후의 과정에서 연산량이 최적화된다.First, FIG. 2 (a) shows an image of an input image. 2 (b) shows an active focused sampling mask used for foreground detection in the present invention, wherein the white pixels are a random scattering type sampling mask

, And the blue pixels are the significance spatial extended sampling mask

, And the red pixel is a surprise pixel sampling mask

to be. Referring to FIG. 2 (b), most of the space is displayed in black. In most spaces, the mask has a value of zero. This eliminates redundancy and optimizes the computation in the process.

), 공간적 특성(

), 주파수 특성(

)을 각각 나타내며, [도 2(h)]는 이러한 시간적, 공간적, 주파수 특성으로부터 얻어지는 전경 확률지도(

)를 나타낸다. [도 2(a)]와 [도 2(h)]를 비교하면 전경 확률지도(

)가 입력 영상에 포함된 전경 대상(차량, 사람)을 잘 반영하고 있음을 확인할 수 있다.(FIG. 2 (c)) is a result of foreground detection in which the active focused sampling mask according to the present invention is combined with the above-described GMM algorithm, and FIG. 2 (d) This is the result of GMM algorithm for all the pixels of the input image. On the other hand, FIG. 2 (e) to FIG. 2 (g) show the temporal characteristics obtained from the input image of FIG. 2 (a)

), Spatial characteristics

), Frequency characteristics

(Fig. 2 (h)) shows a foreground probability map obtained from such temporal, spatial and frequency characteristics

). [Fig. 2 (a)] and Fig. 2 (h)

) Is well reflected in the foreground object (vehicle, person) included in the input image.

Next, the random scattering type sampling (C3), the significance spatial expansion type sampling (C4), and the surprise pixel sampling (C5) will be described in detail.

% 픽셀은 랜덤산발형 샘플링에 의해 선택된다. 이때, ρ의 값은 0.05에서 1 사이의 값으로 설정하는 것이 바람직하다. 균일한 랜덤 샘플링은 모든 픽셀이 평균적으로 프레임

마다 확인되는 것과 유사하다. 무작위 표본의 수(N_S)는 ρN이고, 이 숫자는 모든 프레임에서 일정하다.First, a random scattering type sampling (C3) will be described. Of all pixels

% Pixels are selected by random scatter sampling. At this time, the value of p is preferably set to a value between 0.05 and 1. The uniform random sampling ensures that all the pixels

Is similar to what is confirmed every time. The number of random samples (N _S ) is ρN, and this number is constant in all frames.

)로 측정된 정보의 양에 기초하는데,

인 픽셀 n은 현재 프레임에서 다시 샘플링된다(

). 이렇게 재사용되는 픽셀의 숫자

는 적응적으로 변경되는데, 재사용을 제외한 나머지 개수

의 표본들이 전체이미지에서 무작위로 다시 샘플링 된다.
On the other hand, it is preferable to reuse a part of the pixels generated in the previous frame. The determination of the pixel to be reused is based on the foreground probability (

), ≪ / RTI >

Pixel n is resampled in the current frame (

). The number of pixels to be reused in this way

Is adaptively changed, and the remainder

Are randomly resampled from the entire image.

)는 빈 공간이 없는 온전한 전경 영역을 만들기엔 너무 듬성듬성하고 작은 물체를 놓칠 수 있다. 그러므로 전경으로 탐지된 성긴 픽셀들 사이를 채우는 것이 필요하다. 이를 위해 본 발명에서는 오직 필요한 지역에만 세밀하게 초점을 맞추어 샘플링하는 중요도 샘플링 방식을 사용한다.Next, we consider importance spatial expansion sampling (C4). Random sampling mask (

) May be too sparse to miss a small object to create a perfect foreground area without empty space. Therefore, it is necessary to fill in between the coarse pixels detected in the foreground. To this end, in the present invention, a significance sampling method is used in which only a required region is closely focused and sampled.

Conventional significance sampling high specimen density in regions where importance weight is high. In the present invention, the sampling mask must include all the foreground pixels, and sampling the same pixel multiple times does not fill the empty space, so high density sampling in the foreground portion is not sufficient.

인 모든 점에서의 중요도 비중에 비례하여 확장하는 중요도 공간확장형 샘플링 방법론을 제안한다.To solve this sampling problem, in the present invention, the sampling range is set as shown in FIG. 3

We propose a spatial expansive sampling methodology that expands in proportion to the importance of all points.

)를 이용하여 중요도 공간확장형 샘플링 마스크(

)를 생성하는 과정을 나타낸다. [도 3(a)]와 [도 3(b)]는 각각 전경 확률지도(

)와 랜덤 샘플링 마스크(

)의 각 포인트에 대해 공간확장 영역의 너비(

)가 계산된 모습을 나타낸다. [도 3(c)]는 모든 사각형의 안쪽 포인트를 1로 설정하여 공간확장형 샘플링 마스크(

)를 생성하는 것을 나타낸다.3 is a diagram for explaining the concept of spatial scalable sampling in the present invention,

) Is used to extract the importance spatial expansion sampling mask (

). ≪ / RTI > [Fig. 3 (a)] and [Fig. 3 (b)] show a foreground probability map

) And a random sampling mask (

The width of the space extension area (

) Is calculated. FIG. 3 (c) shows a spatial expansion sampling mask (

).

의 너비를 가진 정사각형이다. 도면을 참조하면 비록 정사각형 지역이 겹치지만 이것들은 [도 3]에 도시된 것처럼

인 한 지역을 구현한다.The shape of the expanded region proportional to the importance proportion depends on the importance weight of the ith random scattering sample

Of the width of the square. Referring to the drawings, although the square areas overlap, these are the same as those shown in Fig. 3

To implement the region.

인 것)의 중요도 가중치가

가 된다. 즉

에 비례하여 픽셀 i를 중심으로 하는

크기의 샘플링 영역 N(i)를 확장시키며 하기의 [수학식 7]와 같이 표현가능하다.Assuming that the importance distribution is uniform, each random scatter type sample i (

Of the importance weight of

. In other words

Lt; RTI ID = 0.0 > i < / RTI >

(I), and can be expressed as shown in the following Equation (7).

는 다음의 [수학식 8]과 같이 결정된다.Space expansion width

Is determined according to the following equation (8).

That is, ω _s is an extension constant with k as a variable, and k in general has a value of √3 or √5.

FIG. 4 is a graph showing the influence of a parameter ω _s and a variable k in importance spatial expansion sampling in the present invention. That is, referring to FIG. 4, it can be seen how ω _s is designed and the effect of the variable k.

= 1일 때 이미지가 N_s 사각형으로 똑같이 분해되었다는 추정하에서 설정된 사각형의 너비를 나타낸다. 그러나 실제 상황에서는 N_s 샘플이 일정하게 분포되어 있지 않고 대부분의

는 1보다 작으며, 그에 따라 [도 4(b)]에서 보여지는 것처럼

가 추측된 전경 영역을 빈틈없이 다루지 못한다. [도 4(c)]를 참조하면 1 보다 큰 매개변수 k(예: √3)에 의해 마스크가 전경 영역을 빈틈없이 다룰 수 있도록 확장시킨다. 그리고, [도 2(b)]에서 볼 수 있는 것처럼 높은 전경 확률 지역은 넓게 샘플링 되고 낮은 확률 지역에서 대부분의

는 0이다.
Referring to FIG. 4 (a), k = 1

= 1 indicates the width of the rectangle set under the assumption that the image is equally decomposed into N _s squares. However, in the actual situation, N _s samples are not uniformly distributed,

Is less than 1, and as shown in Fig. 4 (b)

Can not handle the estimated foreground area. Referring to Fig. 4 (c), the mask extends the foreground area to be seamlessly covered by a parameter k (e.g., √3) larger than 1. And, as can be seen in [Fig. 2 (b)], the high foreground probability area is widely sampled and most of the

Is zero.

Next, a surprise pixel sampling mask (C5) will be described. Even when accurately modeling the foreground probability, foreground detection has inherently statistically unpredictable characteristics. Abnormal scenes are caused by unpredictable suddenness, for example, when a person or car suddenly appears in a new direction or a thief enters a restricted area. It should be possible to successfully detect objects that suddenly appear unexpectedly, and the random scattered samples described above become important when catching these unpredictable cases.

인 곳)에 대해 깜짝픽셀 인덱스

는 다음의 [수학식 9]와 같이 주어진다.A pixel is defined as a surprise pixel when a certain pixel is detected as a foreground even though the foreground probability is small in the previous frame. Observation of the foreground pixel is very surprising because the foreground object is not expected to be at that pixel location. So you can find new foreground pixels by widening the sampling area around the pixels in the current frame. Pixel i (

Surprise pixel index

Is given by the following equation (9).

라면 i를 중심으로 하는

지역)에 대해

으로 생성된다.
The surprise pixel sampling mask is N (i) region (

Centered on ramen i

Area)

.

The performance of the present invention will be described with respect to a plurality of images described above. In order to demonstrate the practical applicability of the active focused sampling method according to the present invention, the inventive algorithm is combined with various detection algorithms in various video scenes with different resolutions and resolutions to compare detection performance and computation amount.

The algorithm according to the active concentrated sampling method of the present invention was implemented in C ++ on a PC with Intel Core i7 2.67GHz processor and 2.97GB RAM. No parallel processing methods such as GPUs, OpenMP, pthread, and SIMD (single instruction multiple data) were used throughout the entire experiment, and they were calculated in a sequential manner on a single core. The parameters are set to be constant, specifically α _T = 0.1, α _F = 0.01, α _S = 0.05, ρ = 0.05, and k = √3.

FIG. 5 is a diagram illustrating the average of F ₁ - measures of each frame during the entire sequence of various foreground detection methods for explaining the effect of the present invention. FIG. It can be seen that the method according to the present invention can be successfully combined with several foreground detection methods and post-processing methods without deteriorating performance.

Referring to FIG. 5, even when the active focused sampling method of the present invention is applied, the foreground detection performance remains the same. SABS images were used to verify the degradation of foreground detection performance when active focused sampling was applied. SABS images are images produced by simulating various situations artificially to evaluate foreground detection results in pixels, and include various scenarios such as light reflection, shadows, traffic lights, and shaking trees. Considering that the maximum value of F ₁ - measure is 0.8 in various algorithms, SABS image is very difficult to detect in the foreground. Even if sampling according to the present invention is applied, foreground detection performance is never degraded .

FIG. 6 is a diagram illustrating a calculation time-rate increase result for explaining the effect of the present invention. Referring to FIG. 6, it can be seen that the operation amount is greatly reduced by applying the active concentrated sampling method of the present invention . As the amount of computation decreases, the detection time can be shortened. On the average, the computation time is shortened by 6.6 times. On the other hand, for a fast foreground detection algorithm, a relatively small speed increase is shown because the mask generation time is relatively large. In order to obtain the comparison result of the computation time shortening according to [FIG. 6], the test was performed with a full HD image. In the foreground detection algorithm having a large amount of computation such as GMM, shadow GMM and KDE, the computation shortening ratio is approximately 8.5 times, and Zivkovic, Gorur In the fast foreground detection algorithm, the rate of reduction is about three times.

FIG. 7 is a diagram illustrating a calculation time change during an entire frame of an image for explaining the effect of the present invention. The GMM method is applied to the SABS image to test it. The computation time when applying the present invention rises as the ratio of the foreground area increases, and the original GMM method also takes more time when the foreground area rises, resulting in a constant rate of rate increase . That is, as shown in FIG. 7, the foreground region of the input image changes from 0% to 10%.

FIG. 8 is a graph showing the degree of reduction in the average calculation amount for explaining the effect of the present invention, and the degree of reduction in the amount of calculation is compared with the sampling methods of the related art. The comparison values are based on the optimized values. Referring to FIG. 8, it can be seen that the effect of reducing the calculation amount is remarkably greater than that of the prior art. The prior art methods add unnecessary sampling and unnecessary computation by applying a predetermined normalized sampling pattern regardless of the situation of an image. The present invention is more efficient because it eliminates unnecessary calculations depending on the image.

Accordingly, real-time detection can be performed on a full HD image by applying the active concentrated sampling method of the present invention. In the prior art, using the GPU was the only method of real-time foreground detection in full HD video. However, referring to Table 2, according to the present invention, it becomes possible to apply the conventional pixel unit foreground detection method to high-resolution images in real-time applications. The GPU version included GeForce GTS250 (128 CUDA cores) and other experiments with a single core processor. All detection methods were applied to full HD video (1920 × 1080) as optimal parameters and the detection time was measured separately with and without applying the method according to the present invention.

Detection method Original Speed (FPS) The method of the present invention
(FPS) GPU 78.9 - GMM 1.6 18.6 KDE 3.5 31.5 Efficient GMM 3.4 23.5 Shadow GMM 2.2 23.5 Zivkovic 9.7 29.7 Gorur 11.8 33.7

The present invention can also be embodied in the form of computer readable code on a computer readable recording medium. At this time, the computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored.

Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage, and the like, and may be implemented in the form of a carrier wave . The computer-readable recording medium can also be stored and executed by a computer-readable code in a distributed manner on a networked computer system. And functional programs, codes, and code segments for implementing the present invention can be easily deduced by programmers skilled in the art to which the present invention belongs.

As described above, the embodiments of the present invention have been disclosed in the present specification and drawings, and although specific terms have been used, they have been used only in a general sense to easily describe the technical contents of the present invention and to facilitate understanding of the invention. And is not intended to limit the scope of the invention. It will be apparent to those skilled in the art that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein.

Claims (12)

In order to obtain an active sampling mask for improving the foreground detection speed, the first step is to acquire three characteristics of the foreground: temporal, spatial, and frequency characteristics;
Based on the temporal characteristic, the spatial characteristic, and the frequency characteristic,

);
The foreground probability map (

) And a detection mask D as a result of foreground detection

) Is generated for each frame, and the sampling mask (

A third step of performing selective pixel-based foreground detection on the pixel-by-pixel basis;
And an active focused sampling method for enhancing the foreground detection rate.
The method according to claim 1,
The foreground detection for the generation of the detection mask (D) in the third step is a sequence of video frames

And a sampling mask

A sequence of detection masks

The method comprising the steps of: (a) extracting a plurality of images;
The method of claim 2,
The third step includes the steps of:

), Random scattering sampling (C3), significance spatial expansion sampling (C4), and surprise pixel sampling (C5) are applied to the active sampling method.
The method of claim 3,
In the first step,
The foreground probability map (

) To calculate the foreground property estimate,
Time characteristic measure (

) From the history of the detection result,
Spatial Characteristics Scale (

) Is estimated as the number of foreground pixels around each pixel,
Frequency Characteristics Scale (

) Is estimated based on a ratio of the detection result changing for a predetermined time. The method of active focused sampling for improving the foreground detection speed from the image.
The method of claim 4,
Wherein the first step updates the temporal characteristic, the spatial characteristic, and the frequency characteristic by operating a moving average whose learning rate is α _T , α _F , and α _S , respectively. Intensive sampling method.
The method of claim 5,
In the first step,
The time characteristic measure (

), The history of the detection mask result at that position in each position n

Lt; RTI ID = 0.0 >

And the probability that the foreground appears at the pixel increases as the value approaches 1. The active focused sampling method for improving the foreground detection speed from the image.
The method of claim 6,
In the first step,

The spatial characteristic measure (

), And calculates the sum of squares in a w x w square space with center n

Represents a spatial neighbor around pixel n,

Is closer to 1 indicates that the probability of becoming a part of the foreground is increased. The active focused sampling method for improving the foreground detection speed from the image.
The method of claim 7,
In the first step,

The frequency characteristic measure (

),

Represents the frequency change characteristic at n, and the pixel n

And the probability that the foreground becomes larger increases as the value of " 0 " is closer to 0. The active focused sampling method for improving the foreground detection speed from the image.
delete The method of claim 8,
In the third step, the sampling mask (which is an active sampling mask)

)

, And

,

,

Characterized in that each of the random sampling sampling (C3), the significance spatial expansion sampling (C4) and the sampling of the surprise pixel sampling (C5) represents the sampling mask of the random scatter sampling (C5).
delete A computer-readable recording medium having recorded thereon a program for causing a computer to execute an active focused sampling method for improving foreground detection speed from an image according to any one of claims 1 to 8 or 10. A computer-

Images