JP3036349B2 - Background image generation method and apparatus - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for automatically generating a background image by removing a moving object from a moving image composed of a moving object and a background.

[0002]

2. Description of the Related Art With respect to generation of a background image, there is a demand for a method which has high stability against image fluctuation due to the passage of a moving object and can quickly follow fluctuation of the background itself due to lighting conditions and the like.

[0003] Mitsubishi Electric Technical Report Vol. 67, no. 7 (1
In the background image generation method described as a proposed method in “Highly Reliable Intrusion Monitoring Device Using Motion Information” described in 993), the difference between the background image one frame before and the current input image is calculated for each pixel. The region having a small difference is determined as a region close to the background, and the region having a large difference is determined as a region far from the background, and the background is updated with a value corresponding to the difference. This method can quickly adapt to a gradual change in the background due to time, but cannot adapt to a sudden change in the background due to a sudden change in the sunshine condition. There is also a disadvantage that a background image must be prepared in advance.

In the background image generation method and apparatus described in Japanese Patent Application Laid-Open No. 3-112282, a difference between a time-series input image and a reference image with a certain time delay is obtained for each pixel, and the difference value is calculated. Then, the pixel belonging to the moving object region and the pixel belonging to the background region are identified, and only the background region is updated according to the difference value. In this method, when the speed of the moving object is slow as shown in FIG. 5 and the regions of the same pixel value included in the moving object overlap in the input image and the reference image, the overlapping portion is erroneously determined as the background region. There is a problem of taking in the background. However, in FIG. 5, black indicates the moving area and white indicates the background area.

[0005]

When a background image is generated, the two conditions that the stability of the image is high with respect to the fluctuation of the image due to the passage of a moving object and that it can quickly follow the fluctuation of the background itself due to lighting conditions and the like are simultaneously satisfied. There is a need. Also,
In many cases, a moving image to be actually generated as a background image is taken outdoors or when moving objects at various speeds are taken.

However, in the conventional method, when the background area and the moving object area are not determined, there is a problem that the following of the fluctuation of the background itself is slow and a background image is required in advance. Is highly likely to be erroneously determined for a low-speed moving object, and these methods alone cannot sufficiently cope.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a method and an apparatus for generating a background image which can quickly follow a change in the background caused by a change in a photographing environment and which is stable against a moving object entering at various speeds.

[0008]

[Means for Solving the Problems]

[0009]

The background image generating apparatus according to the present invention comprises: an image input means for inputting a moving image composed of a background and a moving object; a first input image storage means for storing an input image at the time; A second input image storage unit for storing an input image one frame before the time, and an image stored in the first input image storage unit copied to the second input image storage unit and input by the image input unit Input image updating means for storing the input image obtained in the first input image storage means,
A background mask in which the difference between the two images stored in the first and second input image storage means is binarized, and the value of a pixel that has changed is 0 and the value of a pixel that has not changed is 1 is set as a background mask. The background mask generating means to be generated and the number of frames n having a value of 2 or more are prepared in advance, and for 1 ≦ i ≦ n, i
An i-th background mask storing means for storing a background mask one frame before, and an operation of copying and storing the background mask stored in the i-th background mask storing means in the (i + 1) -th background mask storing means Is performed in descending power order for 1 ≦ i ≦ n−1, and the background mask generated by the background mask generating means is stored in the first background mask storage means; The logical product of n background masks stored in the storage means is calculated, and an area composed of pixels having a value of 1 is determined as a background area, and an area composed of pixels having a value of 0 is determined as a moving object area. The background image stored in the background image storage unit is updated by updating the pixels corresponding to the background area according to the difference value from the first input image and making the other pixels unchanged. And the background update means for generating a scene image,
The image processing apparatus further includes a background image storage unit that stores the background image generated by the background update unit, and an image output unit that outputs the background image stored in the background image storage unit.

Further , the background image generating apparatus of the present invention comprises an image input means for inputting a moving image composed of a background and a moving object, and a first input image storage means for storing the input image at the time. Second input image storage means for storing an input image one frame before the time, and copying the image stored in the first input image storage means to the second input image storage means. Input image updating means for storing the input image input by the input means in the first input image storage means, and binarizing the difference between the two images stored in the first and second input image storage means A background mask generating means for generating a background mask in which the value of a pixel that has changed is 0 and a value of a pixel that has not changed is 1; and the number of frames n having a value of 2 or more are prepared in advance, and 1 ≦ k ≤n
The background mask k-1 frames before the current time to the k-th background
When the scenery mask is used, for 2 ≦ i ≦ n, the first to i-th
The i-th cumulative mask, which is the logical product of the background masks, is stored and
For the i = 1, the i-th storing the first background mask
An accumulative mask storage unit, and a logical product of the accumulative mask stored in the i-th accumulative mask storage unit and the background mask obtained by the background mask generating unit is calculated.
1 ≦ i ≦ n−1
The cumulative mask updating means for storing the background mask generated by the background mask generating means in the first cumulative mask storage means, and the cumulative mask stored in the n-th cumulative mask storage means. A region consisting of pixels having a value of 1 is determined to be a background region, a region consisting of pixels having a value of 0 is determined to be a moving object region, and a background image stored in a background image storage means described below is determined for a pixel corresponding to the background region. A background update unit for generating a new background image by updating according to the value of the difference from the first input image and making the other pixels unchanged, a background image generated by the background update unit , And image output means for outputting the background image stored in the background image storage means.

[0012]

According to the present invention, the variation information of each pixel obtained from the difference between adjacent frames is accumulated for a certain period of time, and from this information, it is determined whether each pixel belongs to the moving object area or the background area. Deal with moving objects. That is, when there is a change even for a moment, it is determined that the pixel belongs to the moving object area. Therefore, even when the moving amount of the moving object is small and the same color portion is likely to be input to the same pixel within a very short time, it can be expected that it will not be erroneously determined as the background area. Is obtained.

Further, since the background image can be updated every frame, it is possible to quickly follow a change in the background due to sunshine or the like.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of a background image generating method according to the first invention will be described with reference to FIG. FIG. 1 shows a flow chart for one frame at time t. In practice, this processing is repeatedly executed. Further, the background mask 106-1 at time t to the background mask 106 at time t-n + 1 in the figure are shown.
All n images up to -n are binary images.

First, when a moving image including a background and a moving object is input, in a background mask generation 101, an input image 105-2 at time t and an input image at time t-1 among the input moving images. Find the difference between 105-1 and
A binary image is generated so that the difference value of each pixel is 0 if the difference value is equal to or greater than the threshold value and 1 if the difference value is less than the threshold value. At time t, the binary image obtained in
Of the background mask 106-1. That is, a pixel to which 0 is given in the background mask at time t indicates that the corresponding pixel of the input image has changed between time t and time t−1,
Pixels given a 1 indicate that no variation has occurred.
Next, in the logical product 102, the background mask 10
A logical product image is generated by taking the logical product of n binary images from 6-1 to the background mask 106-n at time t-n + 1 .
In the background region identification 103, if the value of the logical product of each pixel of the logical product image is 1, it is determined that the pixel belongs to the background region, and if it is 0, it is determined that the pixel belongs to the moving object region. In the background image update 204, the background image is updated by a method described below as an example by referring to the background image 107-1 at the time t-1 and the input image 105-2 at the time t. The image 107-2 is generated. First, in the case of a pixel determined to belong to the background area, a fixed pixel value between the corresponding pixel value of the background image 107-1 at time t-1 and the corresponding pixel value of the input image 105-2 at time t is determined. Background image 1 at time t by calculating a weighted average in percentage
The corresponding pixel value is 07-2. In the case of a pixel determined to belong to the moving object area, the corresponding pixel value of the background image 107-1 at time t-1 is directly used as the corresponding pixel value of the background image at time t. As described above, the time t
Is the background image 107-2 at time t-1.
The new background image is updated from −1 only in the background area.

An embodiment of the background image generating method according to the second invention will be described with reference to FIG. FIG. 2 shows a flow chart for one frame at time t. In practice, this process is repeatedly performed. Further, the cumulative mask 206-1 at time t to the cumulative mask 20 at time t-n + 1 in FIG.
All n images up to 6-n are binary images.

First, when a moving image including a background and a moving object is input, a cumulative mask generation 201 generates, among the input moving images, an input image 205-2 at time t and an input image at time t-1. 205-1 and the difference between
A binary image is generated so that the difference value of each pixel is 0 if the difference value is equal to or greater than the threshold value and 1 if the difference value is less than the threshold value. The binary image obtained in the generation 201 of the cumulative mask is converted to a time t.
Is a cumulative mask 206-1. Next, in the cumulative mask update 202, for i = 2, 3, 4,..., N, the logic of the cumulative mask 206-i at time ti + 1 and the cumulative mask 206-1 at time t is calculated. The sum is obtained and a new cumulative mask 206-i at time t-i + 1 is set. That is, a pixel given 0 in the cumulative mask 206-n at time t-n + 1 indicates that the corresponding pixel of the input image has changed during the past n frames, and a pixel given 1 has changed. Indicates that it did not occur. Next, identification 203 of the background area
At the time t-n + 1, the cumulative mask 206-n
It is determined that the pixel belongs to the background area when the value is 1 and the pixel belongs to the moving object area when the value is 0 for each pixel. In the background image update 204, t-1
The background image 207-1 and the input image 205-2 at time t are referred to, and as an example, the background image is updated by the same method as the background image update 104 described in the embodiment for the background image generation method of the first invention. And the background image 207 at time t
-2 is generated. As described above, the background image 207 at the time t
-2 is a new background image obtained by updating only the background area from the background image at time t-1.

An embodiment of the background image generating apparatus according to the third invention will be described with reference to FIG. In FIG. 3, the n images stored in the first background mask storage unit 306-1 to the n-th background mask storage unit 306-n are 2
It is a value image.

First, a moving image composed of a background and a moving object is input by the image input means 300. Next, the input image updating means 301 performs the following two operations. First, the contents of the first input image storage means 302 are copied and stored in the second input image storage means 303. Second, the image input by the image input means 300 at the time is stored in the first input image storage means 302. Therefore,
The first input image storage means 302 always stores the input image at the time, and the second input image storage means 303 stores the input image at the time.
The input image before the frame is stored. The background mask generating means 304 calculates a difference between the images stored in the first input image storing means 302 and the second input image storing means 303. , A binary image is generated so as to be 1 if the value is less than the threshold value. That is, a pixel to which 0 is given indicates that there is a change between the time and one frame before that, and a pixel to which 1 is given indicates that there is no change. Next, the background mask updating means 305 performs the following n operations.
First, the (i-1) th background mask storage means 306- (i-
The operation of copying the contents of 1) to the i-th background mask storage means 306-i is described as i = n, n-1, n-2,.
・ In order of descending power for 2, Next, the binary image generated by the background mask generation means 304 is stored in the first background mask storage means 306-1. By this operation, for 1 ≦ i ≦ n, the ith background mask storage unit 3
06-i always stores the background mask i-1 frame before the current time. The background updating means 307 calculates the logical product of the n binary images stored in the first background mask storing means 306-1 to the n-th background mask storing means 306-n, and calculates the logical product of each pixel. If the value is 1, it is determined that the pixel belongs to the background area, and if the value is 0, it is determined that the pixel belongs to the moving object area, and the background image stored in the background image storage unit 308 is referred to. The background image generation method of the invention is updated by the same method as the background image update 104 described in the embodiment. Then, the image output unit 309 reads and outputs the background image stored in the background image storage unit.

An embodiment of the background image generating apparatus according to the fourth invention will be described with reference to FIG. In FIG. 4, n images stored in the first cumulative mask storage unit 406-1 to the n-th cumulative mask storage unit 406-n are 2
It is a value image.

First, a moving image including a background and a moving object is input by the image input means 400. Next, the input image updating means 401 performs the following two operations. First, the contents of the first input image storage means 402 are copied and stored in the second input image storage means 403. Second, the image input by the image input unit 400 at the time is stored in the first input image storage unit 402. Therefore,
The first input image storage means 402 always stores the input image at the time, and the second input image storage means 403 stores the input image at the time.
The input image before the frame is stored. The background mask generation unit 404 obtains a difference between the images stored in the first input image storage unit 402 and the second input image storage unit 403. If the difference value is equal to or larger than a threshold value for each pixel, 0 is set. , A binary image is generated so as to be 1 if the value is less than the threshold value. That is, a pixel to which 0 is given indicates that there is a change between the time and one frame before that, and a pixel to which 1 is given indicates that there is no change. Next, the cumulative mask updating means 405 performs the following n operations.
First, the (i-1) th cumulative mask storage means 406- (i-
The operation of reading the contents of 1) and storing the binary image obtained as a logical product with the binary image generated by the background mask generation unit 404 in the i-th cumulative mask storage unit 406-i is i = n. , N-1, n-2,..., 2 in descending power order. Next, the background mask generating means 40
Storing the binary image generated in step 4 in the first cumulative mask storage means 4
06-1. By this operation, 2 ≦ i ≦ n
And the i-th cumulative mask storage means 406-i always has
Background mask at i-1 frame before the time from the time
Is stored . The background update unit 407 reads the binary image stored in the n-th cumulative mask storage unit 406-n, and when the pixel value of each pixel is 1, the pixel belongs to the background area. determines that the pixel belongs to the moving object area, with reference to the background image stored in the background image storage unit 408, an update 104 of the background image as described in example to the background image generation method of the first aspect of the invention by way of example Update in the same way.
Then, the image output unit 409 reads and outputs the background image stored in the background image storage unit.

[0022]

According to the present invention, by accumulating the fluctuation information of each pixel value over the past n frames, it is possible to reduce the number of erroneous determinations of the background region much more than in the past. Therefore, when updating the background image, the moving object is not added due to the erroneous determination, and the stability of the background image against the approach of the moving object is dramatically improved. In addition, since the update can be performed every frame, it is possible to quickly follow the fluctuation of the background.

[Brief description of the drawings]

FIG. 1 is a flowchart showing one embodiment of a background image generation method according to the first invention;

FIG. 2 is a flowchart showing an embodiment of a background image generating method according to the second invention;

FIG. 3 is a block diagram showing an embodiment of a background image generation device according to the third invention;

FIG. 4 is a block diagram showing an embodiment of a background image generation device according to the fourth invention.

FIG. 5 is an explanatory diagram showing a problem of the related art.

[Explanation of symbols]

101 Background mask generation 102 Logical product 103 Identification of background area 104 Update of background image 105-i (i = 1, 2) Input image at time t + i-2 106-i (1 ≦ i ≦ n) Time t-i + 1 Background mask 107-i (i = 1, 2) Background image at time t + i-2 201 Cumulative mask generation 202 Cumulative mask update 203 Background region identification 204 Background image update 205-i (i = 1, 2) Time t + i-2 input image 206-i (1 ≦ i ≦ n) Background mask at time t−i + 1 207-i (i = 1, 2) Background image at time t + i−2 300 Image input means 301 Input image update means 302 First input image storage means 303 Second input image storage means 304 Background mask generation means 305 Background mask update means 306-i (1 ≦ i ≦ n) i-th background mask storage Stage 307 Background update means 308 Background image storage means 309 Image output means 400 Image input means 401 Input image update means 402 First input image storage means 403 Second input image storage means 404 Background mask generation means 405 Cumulative mask update means 406 −i (1 ≦ i ≦ n) i-th cumulative mask storage unit 407 background update unit 408 background image storage unit 409 image output unit

Claims (2)

(57) [Claims] An image input means for inputting a moving image composed of a background and a moving object; a first input image storage means for storing an input image at the time; and an input one frame before the time. Second input image storage means for storing an image,
Input image updating means for copying an image stored in the first input image storage means to the second input image storage means and storing the input image input by the image input means in the first input image storage means; The difference between the two images stored in the first and second input image storage means is binarized, and the value of the pixel having changed is set to 0, and the value of the pixel not changed is set to 1
Background mask generating means for generating a background mask
The number n of frames having the above values is prepared in advance, and 1 ≦ i ≦ n
The i-th background mask storage means for storing the background mask of the (i-1) -th frame before the time, and the background mask stored in the i-th background mask storage means for the i-th background mask.
The operation of copying and storing in the background mask storage means 1 is
.Ltoreq.i.ltoreq.n-1 in descending power order, a background mask updating means for storing the background mask generated by the background mask generating means in the first background mask storing means, and the first to n background mask storing means. Is calculated as a logical product of n background masks stored in the region, a region formed of pixels having a value of 1 is determined as a background region, and a region formed of pixels having a value of 0 is determined as a moving object region. The background image stored in the storage unit is updated by updating pixels corresponding to the background area in accordance with the value of the difference from the first input image and making the other pixels unchanged to newly add the background image. , A background image storage means for storing the background image generated by the background update means, and an image output means for outputting the background image stored in the background image storage means. Background image generation apparatus according to claim and. 2. An image input means for inputting a moving image composed of a background and a moving object; a first input image storage means for storing an input image at the time; and an input one frame before the time. Second input image storage means for storing an image,
An input image update for copying an image stored in the first input image storage means to the second input image storage means and storing the input image input by the image input means in the first input image storage means Means, and the difference between the two images stored in the first and second input image storage means is binarized, and the value of a pixel that has changed is set to 0, and the value of a pixel that does not change is set to 1. a background mask generating means for generating a background mask, prepared in advance the number of frames n to take two or more values, 1
For ≤k≤n, the background map k-1 frames before the current time is
When the mask is the k-th background mask, for 2 ≦ i ≦ n
I-th cumulative mask that is the logical product of the first to i-th background masks.
And store the first background mask for i = 1
An i- th cumulative mask storing means for storing a logical product of a cumulative mask stored in the i-th cumulative mask storing means and a background mask obtained by the background mask generating means; The operation of storing in the storage means is performed in descending power order for 1 ≦ i ≦ n−1, and the cumulative mask updating means for storing the background mask generated by the background mask generating means in the first cumulative mask storage means; The value of the cumulative mask stored in the cumulative mask storage means of n is 1
Is determined as a background region, and a region including 0 pixels is determined as a moving object region.A background image stored in a background image storage unit described later is referred to as a moving object region. A background updating unit that generates a new background image by updating in accordance with a difference value from the first input image and making the other pixels unchanged, and stores the background image generated by the background updating unit; A background image generating apparatus comprising: a background image storage unit; and an image output unit that outputs a background image stored in the background image storage unit.