JP3037495B2 - Object image extraction processing method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to extraction of an object image for separating a background including a moving target object from a fixed moving image by threshold processing into a background area, a moving object area, and a shadow area. It relates to a processing method.

[0002]

2. Description of the Related Art In a binarization method for obtaining a moving object region in a moving image, the configuration of a reference background image and the determination of a threshold value are important. In Japanese Patent Application No. Hei 3-187981, a single frame background image is disclosed. Instead of using a fixed threshold or a fixed threshold, constructing a background model that reflects the fluctuation of the background from the feature amount of the background equivalent image of multiple frames, performing adaptive threshold processing and model updating sequentially to extract the moving object region doing.

[0003]

However, in this method, the configuration of the background image that follows the change in the sunshine condition and the adaptive threshold processing for the imaging noise are possible, but the shadow of an object having a different shape depending on the illumination condition is possible. Is also extracted at the same time, and it is difficult to extract only the true animal.

[0004] It is an object of the present invention to stably separate a moving object from which a shadow is removed from a background in a moving image.

[0005]

FIG. 1 is a block diagram showing the principle of the present invention. Reference numeral 1-1 in the figure denotes a feature extracting unit, which extracts features for each frame. That is, a moving image input from an imaging device such as a TV camera in a frame unit is extracted for each pixel from a plurality of feature amounts required for each frame.

Reference numeral 1-2 denotes a background model constituting unit,
A background determination is performed on a pixel of interest in a target frame, and a parameter representing a slight change in the background obtained from the series information of a plurality of pixels of interest regarded as the background so far, and a parameter corresponding to the plurality of backgrounds A pixel group is extracted over all pixels.

Reference numeral 1-3 denotes a threshold processing unit which sets a moving object extraction area and a shadow area, and sets an animal when the pixel value of the target frame is within the moving object area and does not fall within the shadow area. Extract as body.

[0008]

According to the present invention, a background model reflecting the fluctuation of the background is constructed from image characteristics of a plurality of frames, a background removal region and a shadow separation region based on the background removal region are set, and threshold processing is performed to follow a change in shooting conditions, and An adaptive moving object is extracted for each pixel.

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment will be specifically described below with reference to FIG. In the figure, reference numeral 1-1 denotes a feature extraction unit for each frame, which converts a moving image input from an imaging apparatus such as a TV camera in units of frames into a plurality of feature amounts required for each frame for each pixel. Extract. The feature amount may be any one of luminance, hue, saturation, density gradient between adjacent pixels, an amount represented by other color system, and density information obtained from an infrared camera, which can be extracted from the color information of the image. One or a combination of two or more can be used.

Reference numeral 1-2 denotes a background model constructing unit which constructs a background model in a target frame for each pixel by using one or more of the above-mentioned feature amounts in series. The background model performs a background determination on a pixel of interest in a target frame, and a parameter representing a slight change in the background obtained from the series information of a plurality of pixels of interest regarded as the background so far. It is configured by extracting a background-corresponding pixel group over all pixels.

For example, when certain two features v ₁ and v ₂ are used, pixel values on a certain pixel are distributed as shown in FIG. In the figure, 2-1 is a background equivalent pixel, 2-2 is a non-background pixel, 2-3 is a background region _Rb , 2-4 is the center of the background distribution, and 2-5 and 2-6 are the degree of spread of the distribution. Represent. The background model includes a center (a position (m ₁ , m ₂ ) of the distribution center, shape parameters A and B) representing the center of the spread of the distribution of a plurality of pixels corresponding to the background and the distribution shape, and a background region R _b. And a group of pixel values within.

[0012] BACKGROUND determination and background update, as shown in Figure 3, to set the background update region R _n as to cover the gradual change from the past background value based on the background region R _b to date,
The pixel values of the current frame corresponding to the background if the update region, to reconstruct a new background model interchanged with pixels of the oldest frame (the background area becomes R _'b). If the model is outside the update area, the model is not updated. The initial background model is composed of a plurality of background images that do not include a clear moving object and includes a parameter indicating the distribution and an image group. In FIG. 3, 3-1 is a background equivalent pixel, 3
-2 is the pixel of the oldest frame, 3-3 is the pixel of the current frame, 3-4 is the background area, 3-5 is the background update area R _n ,
3-6 represents the updated background region R _'b.

[0013] 1-3 is a threshold processing unit, as shown in FIG. 4, is moved, rotated within the feature space in proportion to the background region R _b moving object is proportional to the extraction region R _m, and the background region R _b set the shadow region R _s were located pixel value of the frame of interest is the moving object region and regarded as a moving object if not applied to the shadow area, and extracts. In FIG. 4, the background corresponds pixel 4-1, the pixels of the current frame is 4-2, 4-3 background region R _b, 4-4 animal body extraction region R _m, 4-5 shadow region R represents _s .

Further, the above-described updating of the background model and the determination of the moving object image are sequentially performed for each pixel of each frame, so that adaptive threshold processing can be performed while following a change in imaging conditions. The image of the animal body that is robust to the fluctuation of the image can be extracted.

As an embodiment of the present invention, it is assumed that three-dimensional RGB image information is used as a feature amount obtained from an image, and that a shadow region has the same shape as the background distribution and is darkened and reduced in the direction of the origin of the feature space. Next, a method of extracting a moving object region from a moving image including a moving object image will be described.

FIG. 5 shows the structure of the moving object image extraction processing in this embodiment. First, a moving image is input via an image input system such as a television camera, and the target moving image is stored in the image storage unit 5-1 for each frame. In order of accumulation,
Each frame image is transferred from the image storage unit 5-1 to the image memory 5-
Send to 2. The feature extraction unit 5-3 calculates each RGB information of the image in the image memory 5-2, and
-4.

[0017] In initial model configuration unit 5-5, the frame t ₀ specifying regarded past N frames and background equivalent image, the pixel values X _{i, j} of the background corresponding image from the feature image memory 5-4 for each pixel N RGB values (in order x _{1 (i, j)} ,
x2 _{(i, j)} and x3 _{(i, j)} ) and read the principal component analysis of the pixel value set (covariance matrix Σ obtained from equations (1) to (3)). Solving the eigenvalue problem of equation (4) of

[0018]

(Equation 1)

And the eigenvalues λ ₁ , λ ₂ , λ ₃ and the unit eigenvector obtained from equation (5)

[0020]

(Equation 2)

A background model is defined using as a parameter. That is, as shown in Expression (6), the background value distribution is approximated by an ellipsoid having an axis in the direction of the eigenvector centering on the average value vector and an intersection with each axis at the average root of the corresponding eigenvalue.

[0022]

(Equation 3)

[0023]

(Equation 4)

[0024]

(Equation 5)

[0025]

(Equation 6)

[0026]

(Equation 7)

[0027]

(Equation 8)

Next, the determined parameters of the background model and the image values of the background-equivalent image group are stored in a background model storage buffer 5-6, and the threshold processing parameters used for model updating and extraction processing are stored in a threshold processing parameter storage buffer 5-6. 8
To be stored.

In the background model calculation section 5-7, the designated
Frame t₀Subsequent frames (t ₁The characteristic image of
Pixel value retrieved from memory 5-4

[0030]

(Equation 9)

However, using the parameters stored in the background model storage buffer 5-6 and the threshold processing parameter storage buffer 5-8, it is checked whether or not Expression (7) is satisfied. Is determined.

[0032]

(Equation 10)

If it is determined that the background is the background, t _{0 is set} to t ₁ based on the equations (1) to (5) from the background equivalent values of the latest N frames including the current pixel value. Replacement,
The model parameters are recalculated and the background equivalent image group is exchanged to update the background model. Here, the updating of the model is performed by replacing a new pixel value with the oldest pixel value of the pixel for image data independent for each pixel of N frames stored in the background model storage buffer 5-6. Performed by Since this operation is performed independently for each pixel, in the model that has been updated to some extent from the initial state, the time at which data for each pixel is collected does not always match.

In the extraction processing unit 5-9, the pixel values from the feature image memory 5-4 are read.

[0035]

[Equation 11]

However, the one satisfying the expression (8) using the parameters stored in the background model storage buffer 5-6 and the threshold processing parameter storage buffer 5-8 is determined as the background region, and the expression (8) And the (9) is not satisfied
Those satisfying the expression are determined as a moving object region, and the others are determined as shadow regions.

[0037]

(Equation 12)

[0038]

(Equation 13)

Further, the determination result of the moving object, the background or the shadow is stored in the moving object image memory 5-10 together with the position information,
It is output by the moving object output unit 5-11. In the present embodiment, it is assumed that the distribution of the shadow region has the same shape as the background distribution and is darkened and reduced in the direction of the origin of the feature space, and each parameter is uniformly multiplied by a constant. However, depending on lighting conditions, etc.,
Non-uniform movement / reduction and rotation of the distribution shape may be performed.

Although the features of the image have been described as three-dimensional image information, when one multi-dimensional feature is used, model construction / update and threshold processing are performed based on simple average and variance values. By doing so, a similar effect can be obtained. When a two-dimensional feature value is used, an approximation of the distribution shape obtained from the result of the principal component analysis of the feature value distribution is called an equal probability density ellipse (concentrated ellipse). Often used to represent

[0041]

According to the present invention, it is possible to not only stably separate and extract a moving object having a follow-up property to a change in photographing conditions, but also taking local background characteristics into consideration and excluding shadows. In addition, moving image processing that effectively uses a plurality of frames can be performed, and can be a useful tool in motion analysis of a human image or motion analysis of a car or the like.

In the prior art, the configuration of the background image that follows the change in the sunshine condition and the adaptive threshold processing for the imaging noise are possible, but the shadow of the object having a different shape depending on the illumination condition is also extracted at the same time. The animal could not be completely extracted.

[Brief description of the drawings]

FIG. 1 is a diagram showing the principle configuration of the present invention.

FIG. 2 is a diagram illustrating a distribution of two features of a background equivalent image of a plurality of frames.

FIG. 3 is a diagram illustrating a technique for determining a background update.

FIG. 4 is a diagram illustrating a determination method for extracting a moving object.

FIG. 5 is a diagram showing a configuration of an embodiment of the present invention.

[Explanation of symbols]

 1-1 Feature extraction unit 1-2 Background model construction unit 1-3 Threshold processing unit 5-1 Image storage unit 5-2 Image memory 5-3 Feature extraction unit 5-4 Feature image memory 5-5 Initial model construction unit 5 -6 Background model storage buffer 5-7 Background model calculation unit 5-8 Threshold processing parameter storage buffer 5-9 Extraction processing unit 5-10 Animal image memory 5-11 Animal output unit

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-64-48181 (JP, A) JP-A-63-194477 (JP, A) Proceedings of the IEICE Autumn Meeting, Volume 6, D-198 , “Robust object extraction method from xt spatiotemporal image” (58) Fields investigated (Int. Cl. ⁷ , DB name) G06T 7/20 JICST file (JOIS)

Claims (1)

(57) [Claims] 1. An object image extraction processing method for extracting an area of a moving object from a moving image, comprising: extracting one or more features from each frame image in the moving image; A background model is composed of a group of feature values and a statistic calculated from them, a model of a shadow region obtained by deforming and moving the model in a feature value space is extracted, and an extraction calculated from both the constructed models is performed. By separating the moving object region and the background region using the threshold value, reconstructing both models from the feature amount group including the current frame, separating the moving object region and updating the model sequentially for each pixel, An object image extraction processing method characterized by extracting only a moving object region that does not include a shadow by optimal threshold processing while following a change in the background.