KR20180111150A - Apparatus for extracting foreground from video and method for the same - Google Patents

Abstract

According to an embodiment of the present invention provides an apparatus for extracting a foreground from an image, including: a database for storing input images photographed by a photographing apparatus installed at a fixed position; a reference background model managing unit for generating a reference background model corresponding to a target image to extract the foreground using one or more input images; a learning background model managing unit for generating or updating a learning background model corresponding to the target image through automatic background learning; and a foreground extracting unit for extracting the foreground corresponding to the target image by comparing the target image with the reference background model and the learning background model. Accordingly, the present invention can prevent the wrong detection and non-detection of the foreground.

Description

[0001] APPARATUS FOR EXTRACTING FOREGROUND FROM VIDEO AND METHOD FOR THE SAME [0002]

The present invention relates to an apparatus and method for separating a foreground from an image acquired through a photographing apparatus such as a fixed camera or a CCTV.

In recent years, many CCTVs have been installed around them to monitor incidents and accidents, and more CCTVs are expected to be installed in the future. Currently, however, most CCTVs are performing only simple image monitoring and storage functions. Therefore, there has been an inefficient problem of searching through the accurate time information in order to check a desired event in a stored image or directly searching for a long time.

Recently, intelligent video surveillance (IVS) and intelligent image analysis technology that can detect and annotate events / accidents automatically by recognizing space, objects, and people in CCTV images have been researched / developed There are also products with basic functions now on the market. More and more CCTVs are expected to feature these capabilities.

However, the main reason why the existing intelligent video surveillance (IVS) system is not widely used is due to limitations of the performance of the intelligent video surveillance system, especially high false alarm rate and low detection rate. That is, it is frequently detected that a normal situation is recognized as an abnormal situation and false detection is made or an abnormal situation is missed. Although there are many causes of this performance limitation, the foreground detection and non-detection of the background learning and the background subtraction technique which are used as the core element technologies of the intelligent video surveillance are the biggest problems.

The background learning and foreground separation techniques are used to study the background through the unchanging part of the continuous image, while separating the part having a large difference from the background into the foreground. Since the part separated by the foreground can be recognized as a person or an object, it is used as basic data for judging an abnormal situation.

That is, in the past, foreground extraction was performed by dividing the background and foreground by calculating the difference between the input image and the learning background model, and reflecting the divided background to the learning background model according to the determined learning rate.

However, the automatic background learning has a fatal disadvantage that it can not be separated into the foreground by absorbing a person or an object that has been stopped for a long time as a background. However, if the automatic background learning is not used, the slowly changing illumination intensity is not reflected in the background model, and the whole image can be extracted to the foreground after a certain time.

In addition, the background learning and foreground separation techniques have a problem that foregrounds similar to the background can not be distinguished as shown in FIGS. 1 and 2 below. FIG. 1 is a still view of a video image taken on a train station platform, and FIG. 2 is a still picture of a foreground image extracted from the video of FIG. Using the existing background learning and foreground separation techniques, the person located on the right side of FIG. 1 is white and the foreground is incorrectly extracted as shown in FIG. 2 because it is similar to the background color of the surrounding background.

The above-described background technology is technical information that the inventor holds for the derivation of the present invention or acquired in the process of deriving the present invention, and can not necessarily be a known technology disclosed to the general public prior to the filing of the present invention.

Korean Patent Laid-Open No. 10-2011-0109595

An object of the present invention is to provide an apparatus and method for extracting a foreground from an image having a foreground separation technique with improved performance using a reference background model generated by using a fixed photographing apparatus for a long time.

It is another object of the present invention to provide an apparatus and method for extracting a foreground from an image having a foreground separation technique with improved performance using information such as a focal length, a position and an attitude of a camera.

It is also an object of the present invention to provide an apparatus and method for extracting a foreground from an image having a foreground separation technique with improved performance using a mask having a size correction corresponding to each position in the image.

According to an embodiment of the present invention, there is provided an image processing apparatus comprising: a database for storing input images photographed by a photographing apparatus installed at a fixed position; A reference background model management unit for generating a reference background model corresponding to a target image to extract a foreground using one or more of the input images; A learning background model manager for generating or updating a learning background model corresponding to the target image through automatic background learning; And a foreground extracting unit for extracting a foreground corresponding to the target image by comparing the target image with the reference background model and the learning background model; And extracting the foreground from the image.

According to the present invention, an apparatus for extracting a foreground from an image and a method for extracting foreground from the image provide a foreground separation technique with improved performance by utilizing a reference background model generated by using images taken for a long time in a fixed photographing apparatus, The problem of erroneous detection and non-detection can be solved.

In addition, the present invention utilizes information such as the focal distance, position, and posture of a camera, and thereby, a high-performance intelligent video surveillance system using a foreground separation technique with improved performance is constructed can do.

In addition, according to the apparatus and method for extracting the foreground from an image, a foreground separation technique with improved performance can be realized by using a mask whose size is corrected corresponding to each position in a video image.

1 is a view showing an example of an image taken on a train station platform.
FIG. 2 is a diagram showing an example of a result of extracting a foreground from the image shown in FIG. 1 using a conventional technique.
3 is a diagram illustrating an intelligent video surveillance system 1 according to an embodiment of the present invention.
4 is a flowchart illustrating an operation method of the intelligent video surveillance system 1 according to an embodiment of the present invention.
FIG. 5 is a block diagram illustrating an apparatus 100 for extracting a foreground in an image according to an embodiment of the present invention shown in FIG.
FIG. 6 is a diagram showing an operation relationship of an example of the apparatus 100 for extracting a foreground in the image shown in FIG.
7 is a diagram showing an example of an object mask corresponding to each position in an image.
8 is a diagram for explaining an example of a method of estimating the size of an object in an image using GIS information.
9 is a view showing an example of an image photographed in the situation shown in FIG.
10 is a flowchart illustrating a method of extracting a foreground from an image according to an exemplary embodiment of the present invention.

The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated and described in the drawings. The effects and features of the present invention and methods of achieving them will be apparent with reference to the embodiments described in detail below with reference to the drawings. Hereinafter, a repeated description, a known function that may obscure the gist of the present invention, and a detailed description of the configuration will be omitted. Embodiments of the present invention are provided to more fully describe the present invention to those skilled in the art. Accordingly, the shapes and sizes of the elements in the drawings and the like can be exaggerated for clarity.

However, the present invention is not limited to the embodiments described below, but all or some of the embodiments may be selectively combined and implemented in various forms. In the following embodiments, the terms first, second, and the like are used for the purpose of distinguishing one element from another element, not the limitative meaning. Also, the singular expressions include plural expressions unless the context clearly dictates otherwise. Also, the terms include, including, etc. mean that there is a feature, or element, recited in the specification and does not preclude the possibility that one or more other features or components may be added.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to like or corresponding components throughout the drawings, and a duplicate description thereof will be omitted .

1 is a view showing an example of an image taken on a train station platform.

FIG. 2 is a diagram showing an example of a result of extracting a foreground from the image shown in FIG. 1 using a conventional technique.

In detail, FIG. 1 shows a still image of a video shot by a photographing apparatus fixedly installed on a train station platform. FIG. 2 is a still image of a scene extracted from a foreground image using the conventional background learning and foreground extraction techniques in the image shown in FIG.

1 and 2, a person located on the right side of FIG. 1 has a white top and a white background around the background. That is, the human image and surrounding are in a situation similar in color. As a result of extracting the foreground by applying the conventional background learning and foreground extraction techniques, the person is not accurately extracted into the foreground as shown in FIG. That is, according to the related art, the accuracy of extracting the foreground having a color similar to the background is low.

In addition, due to the perspective, the position and size of the same object in the image are different according to the position even though the same object is present, which is not reflected according to the conventional technique. For example, a far object may be excluded from the foreground, or an object in close proximity may be separated into multiple foregrounds.

3 is a diagram illustrating an intelligent video surveillance system 1 according to an embodiment of the present invention.

3, an apparatus 100 for extracting foreground images from an image in an intelligent video surveillance system 1 according to an embodiment of the present invention includes a photographing apparatus 400, a blob detecting apparatus 200, . The blob detection apparatus 200 is interconnected with the apparatus 100 for extracting the foreground from the image and the event detection apparatus 300.

The apparatus 100 for extracting a foreground from an image according to an exemplary embodiment of the present invention receives an image captured at a fixed location and generates a reference background model corresponding to a target image to extract a foreground through input images And uses the learning background model generated through automatic background learning for the target image. Then, the foreground corresponding to the target image is extracted using the difference images obtained by comparing the target image with each of the reference background model and the learning background model, and transmitted to the blob detection apparatus 200. Here, the foreground is extracted from the target image and the remaining background image is used to learn and update the learning background model.

The learning background model refers to a background model generated by analyzing successive parts of the input image, learning the unchanged part as a background, and learning a part having a large difference from the foreground as a background.

The reference background model refers to a background model that is a reference in the target image. This prevents the frozen object from being absorbed gradually into the background over time. At this time, the input images stored in the database can be used to generate the reference background model.

The difference images mean the difference image between the target image and the learning background model and the difference image between the target image and the reference background model. In this case, the difference image may be a short channel image (eg, 640 × 480 × 1) or a multi-channel image (eg, 640 × 480 × 3) according to the background model and the difference calculation method.

In an alternative embodiment, apparatus 100 for extracting foregrounds in an image may generate a reference background model by generating a reference background model by receiving user input. As a result, the user can clearly specify the part he / she wants to designate as the background, and the user feedback can be reflected.

In an alternative embodiment, apparatus 100 for extracting foreground from an image may extract the foreground by receiving user input in extracting the foreground. Accordingly, the user can clearly specify the object to be designated as the foreground, and the user feedback can be reflected.

In an alternative embodiment, the apparatus 100 for extracting a foreground from an image may remove a wrongly separated background and foreground through a mask that takes into account the size of the object in extracting the foreground. In particular, when considering the size of the object, it can be calculated using the GIS (Geographic Information System) information such as the focal distance, the position and the posture of the camera.

For example, a nearby object may appear larger in the image than in the actual size, so that a larger mask may be set, and a far object may appear smaller in the image than the actual size, thus setting a smaller mask.

In an alternative embodiment, apparatus 100 for extracting a foreground from an image may extract the foreground by pre-processing the subject image. In addition, the apparatus 100 for extracting the foreground from the image may post-process the foreground extracted image to the blob detection apparatus 200.

The blob detecting apparatus 200 refers to a device for detecting blobs in a foreground extracted image. Here, the blob is the region of the connected pixel group. Then, the detected blob information can be transmitted to the event detection device 300. [

The event detecting apparatus 300 detects whether an event stored in the event database is generated through information related to the extracted object recognized in the image. For example, detection of an event such as detection of an entry of a restricted area or detection of an abnormal situation exists.

The photographing apparatus 400 means a device capable of photographing an image installed in a fixed apparatus. For example, the photographing apparatus 400 may be a CCTV, a camcorder, a digital camera, or the like.

4 is a flowchart illustrating an operation method of the intelligent video surveillance system 1 according to an embodiment of the present invention.

Referring to FIG. 4, in the intelligent video surveillance system 1 according to an embodiment of the present invention, a photographing apparatus 400 installed at a fixed position captures an image (S401).

In addition, in the intelligent video surveillance system 1 according to the embodiment of the present invention, the photographing apparatus 400 transmits the photographed image to the foreground extracting apparatus 100 (S403).

In addition, the intelligent video surveillance system 1 according to the embodiment of the present invention preprocesses the foreground extracting apparatus 100 to extract the foreground easily from the input image received from the photographing apparatus 400 (S405) . Here, the preprocessing may include reducing the noise of the image, compensating for the motion, or converting the feature vector or the codebook into a feature vector or codebook other than RGB.

Also, in the intelligent video surveillance system 1 according to the embodiment of the present invention, the foreground extracting apparatus 100 extracts the foreground of the preprocessed input image (S407).

Also, in the intelligent video surveillance system 1 according to the embodiment of the present invention, the foreground extracting apparatus 100 performs a post-process on the foreground extracted image so as to easily detect the blob (S409).

In the intelligent video surveillance system 1 according to the embodiment of the present invention, the foreground extracting apparatus 100 transmits the foreground extracted image to the blob detecting apparatus 200 (S411).

In the intelligent video surveillance system 1 according to the embodiment of the present invention, the blob detection apparatus 200 detects a blob from the received foreground extracted image (S413).

In the intelligent video surveillance system 1 according to the embodiment of the present invention, the blob detection apparatus 200 performs filtering on the detected blobs (S415).

Also, in the intelligent video surveillance system 1 according to the embodiment of the present invention, the blob detection apparatus 200 transmits the generated blob information to the event detection apparatus 300 (S417).

Also, in the intelligent video surveillance system 1 according to the embodiment of the present invention, the event detection apparatus 300 detects an event using the event information stored in the event database with respect to the received blob information (S419).

Accordingly, it is possible to extract the foreground more accurately and accurately grasp the object in the target image and the motion of the object, thereby improving the event detection accuracy.

FIG. 5 is a block diagram illustrating an apparatus 100 for extracting a foreground in an image according to an embodiment of the present invention shown in FIG.

5, an apparatus 100 for extracting a foreground from an image according to an exemplary embodiment of the present invention includes a controller 110, a communication unit 120, a memory 130, a database 150, a reference background model manager 160, a learning background model management unit 170, a foreground extraction unit 180, and the like.

In addition, the apparatus 100 for extracting a foreground from an image according to an exemplary embodiment of the present invention may further include a preprocessor 140 or a post-processing unit 190.

In detail, the control unit 110 controls the entire process of extracting the foreground from the image as a kind of central processing unit. That is, the control unit 110 controls the pre-processing unit 140, the reference background model managing unit 160, the learning background model managing unit 170, the foreground extracting unit 180, and the post-processing unit 190 to provide various functions .

Here, the control unit 110 may include any kind of device capable of processing data, such as a processor. Herein, the term " processor " may refer to a data processing apparatus embedded in hardware, for example, having a circuit physically structured to perform a function represented by a code or an instruction contained in the program. As an example of the data processing apparatus built in hardware, a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC) circuit, and a field programmable gate array (FPGA), but the scope of the present invention is not limited thereto.

The communication unit 120 provides a communication interface necessary for transmitting and receiving signals between the apparatus 100 for extracting the foreground from the image and the imaging apparatus 400 and the blob extracting apparatus 200.

Here, the communication unit 120 may be a device including hardware and software necessary to transmit / receive a signal such as a control signal or a data signal through a wired / wireless connection with another network device.

The memory 130 performs a function of temporarily or permanently storing data processed by the controller 110. Here, the memory 130 may include magnetic storage media or flash storage media, but the scope of the present invention is not limited thereto.

The preprocessing unit 140 performs processing for easily extracting the foreground from the input image. This may include reducing noise in the image, compensating for motion, or converting a feature vector or codebook other than RGB.

The database 150 includes input images, reference background models, and learning background models.

The reference background model management unit 160 generates a reference background model corresponding to a target image to extract a foreground through the input images stored in the database 150.

Here, the reference background model means a background model that is a reference in the target image. This prevents the frozen object from being absorbed gradually into the background over time. At this time, the input images stored in the database can be used to generate the reference background model.

In this case, as a first method of generating a reference background model, a user may designate a reference background model directly from input images stored in a database, or designate an image at a specific time point in which there is no object additionally input by a user as a reference background model .

A second method for generating a reference background model is a method for generating a reference background model by calculating an average image of a pre-set T ₁ time from a time when a target image is captured, It can be specified as a model. At this time, the average image can be calculated through arithmetic mean, weighted average of images, or principal component after image matrix transformation. By using the average of the images, the objects in the images can be canceled by the background with high frequency.

Further, as a third method of generating a reference background model, of the inputs is stored in a database image, and D until days after the subject image choelyoung among average image for a preset T ₂ hours before and after from the object image, the recording time , The average image most similar to the target image can be used as the reference background model. In this case, the degree of similarity of images can be used as a measure of the degree of similarity, such as a sum of simple differences between images or a likelihood for a probability model such as a mixed Gaussian model. Accordingly, illuminance, weather, or overall brightness of the image can be considered.

Furthermore, when generating the reference background model, the first to third methods for generating the three reference background models may be mixed and used.

The learning background model management unit 170 generates a learning background model through automatic background learning on the input images and extracts the foreground images from the foreground image extracting unit 180 for the existing learning background model, And updates it.

Here, the learning background model means a background model generated by analyzing consecutive portions of the input image, learning the unchanged portion as a background, and learning a portion having a large difference from the foreground as a foreground.

At this time, as a first method for generating the first learning background model, the first input image may be a learning background model. This is very useful when there is no foreground object in the first input image, and even if there is foreground object, it can be canceled through learning over time. It is also advantageous that the first image can be generated as soon as it is input.

As a second method of generating the first learning background model, an average image of the N input images preset from the first input image may be used as the initial learning background model. This is advantageous in that the effect is greatly reduced through averaging even if there are foreground objects in the initial input images.

The foreground extracting unit 180 extracts the foreground corresponding to the target image using the difference images by comparing the target image to extract the foreground with each of the reference background model and the learning background model. This can increase the accuracy of the distinction between background and foreground.

Here, the difference images mean a difference image between the target image and the learning background model, and a difference image between the target image and the reference background model. In this case, the difference image may be a short channel image (eg, 640 × 480 × 1) or a multi-channel image (eg, 640 × 480 × 3) according to the background model and the difference calculation method.

In this case, the background model and the preprocessed input image may have the following attributes (1) to (3) in addition to RGB or intensity. These attributes can be expressed as Gaussian probability distributions (mean and standard deviation) or histograms.

(1) Image components: RGB or converted color space, intensity,

(2) Spatial gradient of image: Direction and intensity of edge and edge of image

(3) temporal difference of image: optical flow.

In addition, the method of calculating the difference for the short channel or the multi-channel property also includes a method of simply subtracting each channel's value, Euclidean distance, Mahalanobis distance, L1-norm, L2-norm and the like can be used.

As a first method for distinguishing the background from the foreground, if the minimum value (min) of each pixel value of the two difference images exceeds the predetermined threshold value Th ₁ , the foreground can be divided into foregrounds.

As a second method of distinguishing the background from the foreground, if the maximum value (max) of each pixel value of the two difference images exceeds the preset threshold value Th ₂ , it is divided into foregrounds, .

As a third method for distinguishing the background from the foreground, a weight sum according to predetermined weight w and (1-w) is calculated for each pixel value of two difference images, and when the threshold value Th ₃ is exceeded, If not, it can be classified as background.

Further, as a fourth method of distinguishing the background from the foreground, it is possible to calculate variable weights considering the values of i) pixels, ii) values of adjacent pixels, and iii) If the weighted sum exceeds the preset threshold value Th ₄ , it is divided into foregrounds. Otherwise, it can be classified into backgrounds.

Here, the input image, the background model, and the difference image can be multi-channel, and different variable weights can be used for each channel (or attribute). The predetermined threshold values Th ₁ , Th ₂ , Th ₃ and Th ₄ and the weight w can be set by the user or can be obtained by using the optimum value derived from the conventional machine learning methodology and the data set acquired in advance have.

In addition, the foreground extracting unit 180 may distinguish the background from the foreground using a mask having a size correction corresponding to each position in the image. This is because even the same object is photographed at different sizes depending on positions in the image.

As a first method for determining the mask size, a method of receiving a user's input and setting the object size S _c at the center of the photographing apparatus (see 400 in FIG. 3) and the increase / decrease rate S _d when moving up and down The mask size can be determined.

)를 이용하여, 객체의 높이(H)와 영상 속에서의 객체의 크기의 관계식을 세울 수 있다.As a second method for determining the mask size, the mask size can be determined using the GIS information corresponding to the photographing apparatus (see 400 in FIG. 3). In detail, the focal length f, the central coordinates (c _x , c _y ), the installed height L and the tilted angle ( _x , _y ) of the photographing apparatus

), The relation between the height H of the object and the size of the object in the image can be established.

The post-processing unit 190 performs image processing for easily detecting the blob for the foreground extracted image generated by extracting the foreground in the foreground extracting unit 180. [

FIG. 6 is a diagram showing an operation relationship of an example of the apparatus 100 for extracting a foreground in the image shown in FIG.

Referring to FIG. 6, an apparatus 100 for extracting a foreground in the image shown in FIG. 5 may store an input image in a database and generate a reference background model using the stored images. Then, the difference between the learning background model and the reference background model can be calculated using the difference images for the target image to extract the foreground. Then, the foreground can be extracted by separating the foreground and background from the calculated difference.

At this time, if the background is divided into foreground, it can be reflected in the existing learning background model using the divided background information according to the predetermined learning rate.

Here, the reference background model can be generated using images stored in the database. In particular, the reference background model may be automatically generated by determining similarity of images, or may be manually selected / generated by receiving a user input. Here, the generated reference background model can be used to calculate a difference corresponding to a subsequent input image.

In addition, when the background and the foreground are distinguished, they may be automatically classified using difference images, or the user's input may be received and manually classified.

Furthermore, when distinguishing the background from the foreground, a mask having a size correction corresponding to the position in the image can be used using GIS information. As a result, when you distinguish between foreground and background, you can correct the wrong background and foreground.

Accordingly, the limit of the learning background model can be supplemented and overcome by using the reference background model generated from the various input images stored in the database, as in the conventional art.

7 is a diagram showing an example of an object mask corresponding to each position in an image.

Referring to FIG. 7, the sizes of objects photographed at respective positions in an image are different from each other. This is basically due to the perspective, objects far away from the photographing apparatus (see 400 in FIG. 3) appear smaller in the image than the actual size. However, if we do not use masks with size corrections corresponding to each position in the image, distant objects may be separated from foreground objects, and nearby objects may be separated into multiple foreground objects do.

Therefore, as shown in FIG. 7, a mask having a size correction corresponding to each position in the image can be used by utilizing GIS information. Thus, more accurate foreground extraction is possible.

8 is a view for explaining an example of a method of estimating the size of an object in an image using GIS information.

9 is a view showing an example of an image photographed in the situation shown in FIG.

9 shows an object in an image. The origin of the image is the left upper end, the right direction is the x-axis, and the downward direction is the y-axis. The portion corresponding to the '+' shape at the center corresponds to the central coordinate in the image.

이다. 그리고, 영상 속에서의 중점 좌표는 (c_x, c_y)이다. 또한, 영상 속에서 객체의 저점이 위치한 좌표는 P'(x, y), 영상 속에서 객체의 고점이 위치한 좌표는 Q'(x, y'), 객체의 크기는 H미터라 한다. 이 경우, 영상 속에서 객체의 크기는 (y-y')픽셀이라 할 수 있다. 이는, 아래 방향이 양의 y축에 해당하여, y가 y'보다 크기 때문이다.Referring to Figs. 8 and 9, the photographing apparatus (see 400 in Fig. 3) is located at point C, the focal length is f, the installed height is L meters,

to be. The center coordinates in the image are (c _x , c _y ). The coordinates of the bottom point of the object in the image are P '(x, y), the coordinates of the object in the image are Q' (x, y '), and the size of the object is H meters. In this case, the size of the object in the image may be (y-y ') pixels. This is because the downward direction corresponds to the positive y axis and y is greater than y '.

를 획득할 수 있다.Here, point C is closest to the ground, point P _- is the point at which the object touches the ground, point Q is the point at which the object is highest, point R is the point where the straight line passing through point C and point Q touches the ground Quot; The distance between the point C 'and the point P is Z, and the distance between the point C' and the point R is Z '. Further, an angle formed by the focus direction of the photographing apparatus (refer to 400 in Fig. 3) and the point P by the following equation (1) using the GIS information given in advance

Can be obtained.

[Equation 1]

Then, a relational expression of Z and L can be obtained as shown in the following equation (2).

&Quot; (2) "

Then, the peak Q '(x, y') of the object in the image can be obtained through the following equation (3).

&Quot; (3) "

As another method, the following equation (4) may be used.

&Quot; (4) "

The coordinates Q '(x, y') corresponding to the object size (y-y ') in the image or the high point of the object in the image can be predicted through the above equations, Mask size can be used.

Also, in the case where the camera direction is inclined in three directions (degrees of freedom 3), the relationship can be derived as in the above equation to predict the size.

10 is a flowchart illustrating a method of extracting a foreground from an image according to an exemplary embodiment of the present invention.

Referring to FIG. 10, an apparatus for extracting a foreground in an image according to an embodiment of the present invention receives an input image captured at a fixed position (see 100 in FIG. 3) S1001).

In addition, in the method of extracting the foreground in the image according to the embodiment of the present invention, a device for extracting the foreground from the image (see 100 in FIG. 3) performs a preprocessing process so that the foreground can be easily extracted from the input image S1003).

The preprocessing process may include reducing the noise of the image, compensating for the motion, or converting the feature vector or codebook to a feature vector other than RGB.

In addition, in the method of extracting foreground from an image according to an embodiment of the present invention, an apparatus for extracting a foreground from an image (see 100 in FIG. 3) stores an input image in a database (see 150 in FIG. 3) ).

In addition, a method of extracting a foreground from an image according to an embodiment of the present invention is a method of extracting a foreground from an image (see 100 in FIG. 3) by using an input image stored in a database (see 150 in FIG. 3) And creates a background model (S1007).

Here, the reference background model refers to a background model that is a reference in a target image to extract a foreground. This prevents the frozen object from being absorbed gradually into the background over time.

In this case, as a first method of generating a reference background model, a user may designate a reference background model directly from input images stored in a database, or designate an image at a specific time point in which there is no object additionally input by a user as a reference background model .

A second method for generating a reference background model is a method for generating a reference background model by calculating an average image of a pre-set T ₁ time from a time when a target image is captured, It can be specified as a model. At this time, the average image can be calculated through arithmetic mean, weighted average of images, or principal component after image matrix transformation. By using the average of the images, the objects in the images can be canceled by the background with high frequency.

Further, as a third method of generating a reference background model, of the inputs is stored in a database image, and D until days after the subject image choelyoung among average image for a preset T ₂ hours before and after from the object image, the recording time , The average image most similar to the target image can be used as the reference background model. In this case, the degree of similarity of images can be used as a measure of the degree of similarity, such as a sum of simple differences between images or a likelihood for a probability model such as a mixed Gaussian model. Accordingly, illuminance, weather, or overall brightness of the image can be considered.

Furthermore, when generating the reference background model, the first to third methods for generating the three reference background models may be mixed and used.

In addition, in a method of extracting foreground from an image according to an embodiment of the present invention, an apparatus for extracting a foreground from an image (see 100 in FIG. 3) calculates a difference between a target image and a reference background model (S1009).

Here, the difference image between the target image and the reference background model can be generated and used to calculate the difference.

In addition, in the method of extracting foreground from an image according to an embodiment of the present invention, an apparatus for extracting foreground from an image (see 100 in FIG. 3) calculates a difference between a target image and a learning background model (S1011).

Here, the learning background model means a background model generated by analyzing consecutive portions of the input image, learning the unchanged portion as a background, and learning a portion having a large difference from the foreground as a foreground.

At this time, as a first method for generating the first learning background model, the first input image may be a learning background model. This is very useful when there is no foreground object in the first input image, and even if there is foreground object, it can be canceled through learning over time. It is also advantageous that the first image can be generated as soon as it is input.

As a second method of generating the first learning background model, an average image of the N input images preset from the first input image may be used as the initial learning background model. This is advantageous in that the effect is greatly reduced through averaging even if there are foreground objects in the initial input images.

Here, the difference image between the target image and the learning background model can be generated and used to calculate the difference.

In addition, in the method of extracting foreground from an image according to an embodiment of the present invention, an apparatus for extracting a foreground in an image (see 100 in FIG. 3) extracts a difference between a reference background model calculated for a target image and a learning background model To extract the foreground (S1013).

Here, as a first method for distinguishing the background from the foreground, if the minimum value (min) of each pixel value of the two difference images exceeds the predetermined threshold Th ₁ , the foreground can be divided into foreground and background.

As a second method of distinguishing the background from the foreground, if the maximum value (max) of each pixel value of the two difference images exceeds the preset threshold value Th ₂ , it is divided into foregrounds, .

As a third method for distinguishing the background from the foreground, a weight sum according to predetermined weight w and (1-w) is calculated for each pixel value of two difference images, and when the threshold value Th ₃ is exceeded, If not, it can be classified as background.

Further, as a fourth method of distinguishing the background from the foreground, it is possible to calculate variable weights considering the values of i) pixels, ii) values of adjacent pixels, and iii) If the weighted sum exceeds the preset threshold value Th ₄ , it is divided into foregrounds. Otherwise, it can be classified into backgrounds.

Here, the input image, the background model, and the difference image can be multi-channel, and different variable weights can be used for each channel (or attribute). The predetermined threshold values Th ₁ , Th ₂ , Th ₃ and Th ₄ and the weight w can be set by the user or can be obtained by using the optimum value derived from the conventional machine learning methodology and the data set acquired in advance have.

In particular, when separating the background from the foreground in the target image, it is possible to distinguish the background from the foreground by using a mask having a size correction corresponding to each position in the image. This is because even the same object is photographed at different sizes depending on positions in the image.

As a first method for determining the mask size, a method of receiving a user's input and setting the object size S _c at the center of the photographing apparatus (see 400 in FIG. 3) and the increase / decrease rate S _d when moving up and down The mask size can be determined.

)를 이용하여, 객체의 높이(H)와 영상 속에서의 객체의 크기의 관계식을 세울 수 있다.As a second method for determining the mask size, the mask size can be determined using the GIS information corresponding to the photographing apparatus (see 400 in FIG. 3). In detail, the focal length f, the central coordinates (c _x , c _y ), the installed height L and the tilted angle ( _x , _y ) of the photographing apparatus

), The relation between the height H of the object and the size of the object in the image can be established.

In addition, a method of extracting foreground from an image according to an embodiment of the present invention is a method of extracting a foreground from an image (see 100 in Fig. 3) by learning using a background and foreground information The background model is updated (S1015).

In addition, in the method of extracting foreground from an image according to an embodiment of the present invention, an apparatus for extracting a foreground from an image (see 100 in FIG. 3) (S1017).

The post-processing is an image processing that is performed so that the blob can be detected easily from the foreground extracted image.

In this manner, unlike the conventional foreground extraction technique, not only the learning background model but also a comparison with the reference background model can be additionally performed to extract the foreground with higher accuracy.

In an alternative embodiment, a step S1009 of calculating a difference between the target image and the reference background model in the steps S1001, S1003, S1005, S1007, S1009, S1011, S1013, S1015, and S1017, (S1011) of calculating the difference from the learning background model may be performed in parallel.

In an alternative embodiment, the step of calculating the difference from the learning background model for the target image (S1011) is performed first in the steps (S1001, S1003, S1005, S1007, S1009, S1011, S1013, S1015 and S1017) And step S1009 of calculating the difference from the reference background model with respect to the target image may be performed.

The embodiments of the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention or may be those known and used by those skilled in the computer software arts. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROM and DVD, magneto-optical media such as floptical disks, medium, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code, such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be modified into one or more software modules for performing the processing according to the present invention, and vice versa.

The specific acts described in the present invention are, by way of example, not intended to limit the scope of the invention in any way. For brevity of description, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of such systems may be omitted. Also, the connections or connecting members of the lines between the components shown in the figures are illustrative of functional connections and / or physical or circuit connections, which may be replaced or additionally provided by a variety of functional connections, physical Connection, or circuit connections. Also, unless explicitly mentioned, such as " essential ", " importantly ", etc., it may not be a necessary component for application of the present invention.

Accordingly, the spirit of the present invention should not be construed as being limited to the above-described embodiments, and all ranges that are equivalent to or equivalent to the claims of the present invention as well as the claims .

1: Intelligent video surveillance system
100: Device for extracting foreground from image
110: control unit 120: communication unit
130: memory 140: preprocessing section
150: Database 160: Reference background model manager
170: learning background model managing unit 180: foreground extracting unit
190: Post-
200: blob detection device 300: event detection device

Claims (1)

A database for storing input images photographed by a photographing apparatus installed at a fixed position;
A reference background model management unit for generating a reference background model corresponding to a target image to extract a foreground using one or more of the input images;
A learning background model manager for generating or updating a learning background model corresponding to the target image through automatic background learning; And
A foreground extracting unit for extracting a foreground corresponding to the target image by comparing the target image with the reference background model and the learning background model;
And extracting the foreground from the image.

Images