JP3267107B2 - Background image creation method and image processing apparatus - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of generating a background image and an image processing apparatus, and more particularly to a method of generating a reference background image in image processing and an image processing apparatus used for a moving object detecting device, a monitoring device, and the like.

[0002]

2. Description of the Related Art In recent years, traffic flow measurement and various types of monitoring have been performed using image processing. In many cases, a background subtraction method is used to extract a target object. In this method, an image serving as a reference is captured in advance as a background image, and an object is to be extracted by a difference processing with respect to an input image. The background image is referred to as a background image, and a person determines that there is no moving object in the image, and captures and sets the image. And, in the description of JP-A-3-118677,
There is a technique for stably extracting an object by dynamically updating a reference background image in response to a change in brightness caused by a change in weather in the outside world or the like.

[0003] As this kind of technology, there is a document "Procedures of the 9th Symposium on Image Sensing Technology in Industry, pp.15-20, June 1994". The technique of background image creation disclosed in the literature is a logical product difference image (a logical product of a binary image based on the current image and the image five images before, and a binary image based on the current image and the image ten years ago. A still area and a moving area are determined from the obtained image), and the pixel value of the input image corresponding to the still area is used as the pixel value of the background image.

[0004]

In the above prior art,
As shown in FIGS. 4 and 5, the binary image b (t−1) 13 or 42 and the binary image b (t) 13 ′ or 42 ′ are obtained by a logical product operation (AND process) 35. The moving area 36 or 43 is inverted 37 to create a still area m '(t) 38 or 44 as a mask image. This is the input image f
A new background image g ′ (t) 40 or 45 is created by multiplying 39 with (t−1) 10 ′ or 41 ′. As shown in the figure, there is a problem that a part 50 or 51 of the moving area may be captured in the background image g ′ (t) 40 or 45.

FIG. 4 is a processing function diagram showing a background image creating method according to the prior art. This represents a case where the moving areas between the images overlap in the case where the moving speed of the moving object is slow or the like. FIG. 5 is a processing function diagram showing another background art creating method according to another related art. In the case where the moving speed of the moving object is fast or the image capturing interval is long, the case where the moving regions between the images overlap is generated.

That is, the prior art does not consider that a moving object such as a vehicle enters a background image due to a moving speed of a moving body or an interval at which each image is taken, and obtains a highly accurate reference background image. There was a problem that can not be. Therefore, an object of the present invention relates to creation of a reference background image required by a moving object detection device and a monitoring device,
An object of the present invention is to provide a highly accurate background image creating method and an image processing device.

[0007]

SUMMARY OF THE INVENTION The object of the present invention is to select two input images, "before" and "after", which are successive in time, from images continuously transmitted at arbitrary time intervals. A first step of sequentially obtaining binary images by performing differential processing and binary image processing using the “before” and “after” input images, and obtaining the binary images sequentially; After performing a logical sum operation using the previous moving area and sequentially creating the current moving area, and replacing the current moving area with the previous moving area for the next calculation, and performing a logical OR operation a predetermined number of times A second step of extracting the current moving area and creating a moving area, a third step of inverting the moving area to create a still area, and a step of extracting the “previous” input image corresponding to the still area. , And sequentially perform a collection operation to create an intended background image. It is achieved by the inclusion of a fourth step.

[0008] In other words, one image (one frame) is
It is composed of a stationary area such as a road and a moving area such as a vehicle. In order to create a background image, it is necessary to correctly determine a still region. The still region is determined by subtracting the moving region from the entire image (i.e., the sum of the still region and the moving region).
It is necessary to correctly determine the moving area to be subtracted. Therefore, instead of a logical product operation (AND processing) that can be obtained only partially, a logical OR operation that requires the entire moving area
(OR processing) is performed so that a moving area can be correctly obtained even when moving objects overlap in a plurality of camera fields of view having different times.

Further, the image processing apparatus which achieves the above-mentioned object has two input images, "before" and "after", which are successive in time, from images continuously transmitted at arbitrary time intervals. A binary image creating means for sequentially creating a binary image by performing a difference process and a binary image process using the “before” and “after” input images, and sequentially creating a binary image. Using the value image and the previous moving area, OR operation processing is performed to sequentially create the current moving area, and the current moving area is replaced with the previous moving area for the next operation, and the OR operation is performed a predetermined number of times. The moving area creating means for extracting the current moving area after the processing and creating the moving area, the still area creating means for inverting the moving area to create the still area, and the " The `` before '' input images are sequentially collected and processed, and Configure and a background image producing means for producing an image, and processing the image to be transmitted.

[0010]

Embodiments of the present invention will be described below with reference to embodiments shown in the drawings. First, an outline of the moving object detection device according to the present embodiment will be described as an example. FIG. 2 is a block diagram showing the configuration of the moving object detection device according to one embodiment of the present invention. The moving object detection device according to the present embodiment includes an image processing unit 100 that processes a captured image, and a camera 101 that captures an image.
A monitor 111 for displaying images and various information, and a CPU for controlling the entire apparatus and processing the results of the image processing unit 100
112.

The image processing unit 100 includes an A / D converter 10
2, image memory 103, inter-image calculation circuit 104, binarization circuit 105, inversion circuit 106, histogram processing circuit 1
07, histogram memory 108, linear combination operation circuit 1
09, including the D / A converter 110. The image memory 103 has, for example, k (G1 to Gk) gray image memories of 256 × 256 pixels, and j (B1 to Bk) binary image memories for storing binary images as necessary.
j) Prepare. Hereinafter, the operation will be described.

Based on a command from the CPU 112, the image processing unit 100 takes in an image signal photographed by the camera 101, and the A / D converter 102
The image data is converted into density data of 28 gradations and stored in the image memory 103. Further, the image processing unit 100 includes a CPU 112
, An image inversion process, a binarization process, an image inversion process, a histogram process (density frequency distribution calculation process), a linear combination operation process, etc. Circuit 104, binarization circuit 10
5. The processing is executed by the inversion circuit 106, the histogram processing circuit 107, the linear combination operation circuit 109, and the like. The processing result and the like are converted into a video signal via the D / A converter 110 as necessary, and displayed on the monitor 111. Note that the histogram memory 108 is a memory that stores a density frequency distribution for calculation and determination.

After the reference background image is obtained,
An outline of an example of a process of extracting a moving object such as a vehicle or a person and a process of measuring a flow of the moving object will be described.
FIG. 3 is a diagram illustrating a flow of a process of the moving object detection device in FIG. 2. First, the image processing unit 100 obtains a difference image 4 by a difference process 3 between the input image 1 and the background image 2 as a reference, binarizes the difference image 4 with a predetermined threshold value, and creates a binary image 4. Thereafter, the CPU 112 executes the vehicle extraction processing 6 for each lane, and executes the measurement processing 7 for the number of vehicles, individual vehicle speed, and the like for each lane on this image.

Next, the details of the background image creation processing for creating the reference background image, which is the central part of the present invention, will be described. FIG. 1 is a processing function diagram showing a background image creating method according to one embodiment of the present invention. That is, the function of each process for creating the reference background image is shown. FIG.
, The difference processing 11, the logical sum calculation processing 14, the multiplication processings 18 and 21, the addition processing 23, and the like are performed by the inter-image calculation circuit 104 in FIG.

First, an input image “f (t−2)” of “before” and an input image of “after” of two frames which are continuously transmitted from the camera at a certain time interval and are temporally shifted back and forth. f (t-1) 1
0 ′, and the difference processing 11
The binarization circuit 105 further performs a binarization process 12 to obtain a binary
A value image b (t-1) 13 'is created. Next, the image that was "after" is changed to the "before" input image f (t-1) 10 ', and the "after" image is newly added.
, The input image f (t) 10 ″ is input, that is, the progressive input processing is performed to sequentially create the binary image b (t) 13 ″. That is,
In the first step, two images, “before” and “after”, which are temporally preceding and succeeding, are obtained by “forward input processing” from images continuously transmitted at arbitrary time intervals, "Before"
Then, it can be said that the difference processing and the binary image processing are performed using the “after” input image and the binary image is sequentially generated. Then, the inter-image calculation circuit 104 and the binarization circuit 105 including the progressive input processing and the difference processing correspond to a binary image creating means.

Next, a method of sequentially executing a logical OR operation from the sequentially generated binary images to generate a mask image as a moving area is as follows. Specifically, the binary image b (t-2) 13 And the previous moving area m (t-
Using the 2) and the binary image b (t-1) 13 ', an OR operation 14 is performed by the inter-image operation circuit 104 to obtain a mask image m (t-1) 15 as the current moving area. Next, the mask image m (t-1) 15 is replaced with the previous moving area for the next calculation, and a new binary image b (t) 13 ″ is subjected to a logical OR operation 14 ′ to obtain the current moving area. Mask image m (t) 1 as
Get 5 '. After repeating the above arithmetic processing a predetermined number of times (every time, that is, including once, usually 4 to 5 times),
The extracted moving area is set as the “moving area” as a result of executing the extraction of the moving object. The finally set moving area in the case of this embodiment is:
This is the illustrated mask image m (t) 15 ′.

That is, in the second step, a logical sum operation is performed using the created binary image and the previous moving area to sequentially generate the current moving area, and the current moving area is used for the next calculation. Can be said to be replaced with the previous moving area, and the current moving area after the logical OR operation has been performed a predetermined number of times is extracted to create a moving area. The inter-image calculation circuit 104 including the moving area extraction processing and the logical sum operation processing corresponds to a moving area creating unit. The point of the present invention is that the creation of the mask image of the moving area is performed by the OR operation (OR processing). By repeating the OR process a predetermined number of times, all the objects moving in the moving area are extracted, so that the correct moving area is obtained.

Next, the inversion circuit 106 inverts the mask image m (t) 15 ′ as a moving area, which is obtained by repeating the above-mentioned predetermined number of times, to obtain an inverted mask image m ′ ( t) Obtain 17. That is, in the third step,
It can be said that the moving area is inverted to create a still area.
Then, the inversion circuit 106 corresponds to a still area creating unit.

Finally, as a method of creating an intended background image using an inverted mask image as a still area,
There is a method based on “collective operation processing”. The "collective calculation process" utilizes the afterimage and the density frequency distribution, and is as follows. First, an inverted mask image as a static area
A multiplication process 21 is performed by multiplying m ′ (t) 17 by an input image f (t−1) 10 ′ as a “previous” input image, which is repeatedly inverted a predetermined number of times and synchronized with the time when the still region is obtained. ,background image
m'f (t) 22 is set.

Next, a mask image m (t) as a moving area
15 ′ is multiplied by 18 with the previous background area image g ″ (t−1) 19, and an afterimage background area mg ″ (t) taking into account the previous background area image included in the currently obtained moving area. Ask for 20. Then, the afterimage background area mg "(t) 20 and the above-described background image m'f (t) 22 are added to each other 23 to obtain a current background area image g" (t) 24. The obtained current background region image g "(t) 24 is used for the next calculation.
g "(t-1) 19. Also, at the time of the first calculation, the previous background area image has been cleared.

Then, the current background region image g "(t) 2
It is determined whether there is no area of density 0 (the area indicated by white in No. 24), that is, whether it can be adopted as a background image. More specifically, the histogram processing circuit 107 and the histogram memory 108 calculate a density frequency distribution, and determine whether there is no pixel having a density of 0 and all the pixels are buried to form a reference background image. This is the determination process 25. If all the pixels are not filled, the process returns to the beginning in the continuation process 31, and each process is repeated from the input of the image. Then, the current background region image g ″ (t) 24 at the time when it is determined that all the pixels are filled is output to create an intended background image. In the illustrated embodiment, the background image g ′ (t) 26 is used. Is equivalent to

That is, the fourth step comprises:
It can be said that, by using the still area and the “previous” input image corresponding to the still area, collective arithmetic processing is sequentially performed to create an intended background image. The inter-image calculation circuit 104 including multiplication processing and addition processing, the histogram processing circuit 107 including determination processing and repetition continuation processing, and the histogram memory 108 can be said to correspond to background image creation means.

Next, a method for avoiding the influence of noise, although the processing time is slightly increased, will be described. This method
A linear combination operation is added to the above embodiment. Here, a slightly complicated example in which a linear combination operation is added to the fourth step will be described. In the determination processing 25 of the above embodiment, the current background region image g ″ (t) in which all the pixels are buried
24 is created, this is used as the background image g '(t) 26
Is temporarily set. Then, using the current background image g ′ (t) 26 and the previous background image g ′ (t−1) 27, the linear combination operation circuit 109 uses the linear combination operation circuit 28 of the following equation (1).
Is repeatedly executed to obtain an intended reference background image g (t) 29.

G (t) = Ag ′ (t) + (1−A) g ′ (t−1) (Equation 1) where A: Linear combination coefficient The obtained reference background image g (t) 29 is For the next calculation, it is replaced as the previous background image g '(t-1) 27. In addition,
At the time of the first calculation, the previous background image g '(t-1) 27 has been cleared.

Then, in decision processing 30, it is checked whether or not the processing of equation (1) has been performed a predetermined number of times (at least once, usually two to three times). In the process 32, the reference background image g (t) 29 is output to create an intended background image. On the other hand, if the specified number of times has not been reached, the process returns to the top and the process is repeated.

Therefore, in the above method, the processes from the first step to the fourth step are repeated to obtain the previous background image created in the previous process and the current background image created in the current process. It can be said that a fifth step of performing a linear combination operation process using the background image and the current background image to create a reference background image is added. Also, judgment processing,
Linear combination operation circuit 109 including repetition continuation processing and the like
Corresponds to a linear combination operation means. This additional use of the linear combination operation reduces the influence of random noise, so that a more accurate reference background image can be created, which is an effective method.

[0027]

According to the present invention, even if a moving object exists in the input image, the moving object does not enter the background image, and a highly accurate reference background image can be created.

[Brief description of the drawings]

FIG. 1 is a processing function diagram showing a background image creating method according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating a configuration of a moving object detection device according to an embodiment of the present invention.

FIG. 3 is a diagram showing a flow of processing of the moving object detection device of FIG. 2;

FIG. 4 is a processing function diagram showing a background image creation method according to the related art.

FIG. 5 is a processing function diagram showing another conventional background image creation method.

[Explanation of symbols]

1, 10 input image, 2 background image, 3, 11 difference processing, 4 difference image, 5, 13 binary image, 6 extraction processing, 7 measurement processing, 12 binary processing, 14 ... OR operation processing, 15 mask image, 16 inversion processing, 17 inverted mask image, 18 and 21 multiplication processing, 19 previous background area image, 20 residual image background area, 22 background image, 2
3: addition processing, 24: current background area image, 25, 30 ...
Judgment process 26: current background image, 27: previous background image, 28: linear combination operation process, 29: reference background image, 31: continuation process,
32: Reference background image setting processing, 100: Image processing unit, 1
01: camera, 102: A / D converter, 103: image memory, 104: inter-image calculation circuit, 105: binarization circuit,
106: inverting circuit, 107: histogram processing circuit, 1
08 histogram memory, 109 linear combination operation circuit, 110 D / A converter, 111 monitor, 112
CPU

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-5-205175 (JP, A) JP-A-2-71380 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06T 7/00-7/60 G06T 1/00 H04N 7/18 G08B 13/18

Claims (4)

(57) [Claims] 1. Among images continuously transmitted at an arbitrary time interval, two input images, "before" and "after", which are temporally preceding and succeeding each other, are sequentially obtained by input processing. A first step of performing a difference process and a binary image process using the “before” and “after” input images and sequentially creating a binary image, and using the created binary image and the previous moving area, Sum operation processing to sequentially create a current moving area, and replace the current moving area with the previous moving area for the next operation, extract the current moving area after performing a logical OR operation a predetermined number of times, A second step of creating a moving area, a third step of inverting the moving area and creating a still area, and sequentially collecting and processing the “previous” input image corresponding to the still area; And a fourth step of creating an intended background image. Background image generation method for the butterflies. 2. The method according to claim 1, wherein the processing from the first step to the fourth step is repeated to obtain a previous background image created in the previous processing and a current background image created in the current processing. A fifth step of performing a linear combination operation using the previous background image and the current background image to generate a reference background image. 3. An image which is successively transmitted at an arbitrary time interval and is obtained by sequentially inputting two input images, "before" and "after", which are successively shifted in time. A binary image creating means for sequentially creating a binary image by performing a difference process and a binary image process using the “before” and “after” input images, and using the created binary image and the previous moving area The OR operation is performed to sequentially create the current moving area, and the current moving area is replaced with the previous moving area for the next operation, and the current moving area is extracted after the OR operation is performed a predetermined number of times. A moving area creating means for creating a moving area; a still area creating means for inverting the moving area to create a still area; and the "previous" input image corresponding to the still area sequentially gathered. A background image creating means for performing arithmetic processing and creating a background image. The image processing apparatus characterized by. 4. The apparatus according to claim 3, wherein processing from said binary image creating means to said moving area creating means, said still area creating means, and said background image creating means is repeated, and said processing is performed in the previous processing. A linear combination operation means for obtaining a previous background image and a current background image created in the current process, and performing a linear combination operation process using the previous background image and the current background image to create a reference background image. Characteristic image processing device.