JP3062394B2 - Moving image extraction method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving object image extracting method for stably extracting only a moving object image from an image under various environments. For example, an intruder is extracted in an image monitoring apparatus. Used in some cases.

[0002]

2. Description of the Related Art FIG. 2 is a block diagram showing the configuration of a conventional moving object image extracting method. Hereinafter, a conventional moving object image extraction method will be described with reference to FIG. An absolute value of a density difference is calculated by a difference process 4 from a background image input through the image input unit 1 and stored in the background image memory 2 in advance and an input image stored in the input image memory 3. A difference image is obtained, and the difference image is output to the binarization processing 5.
In the binarization process 5, the difference image is binarized using a preset threshold, and an area of the input image that has changed from the background image is obtained. In the change area extraction processing 6, the change area is extracted from the input image as a region representing the moving object in the input image, that is, a moving object image 7.

[0003]

However, the conventional moving object image extraction method has the following problems.
In the binarization process 5, when a moving object image is extracted by binarizing a difference image using a preset threshold value, it is difficult to set the threshold value. It is difficult to extract the image accurately.
For example, in a bright environment where the shadow of a moving object is clearly visible, the threshold value must be set relatively high to remove the influence of the shadow and accurately extract only the moving object image. Will not be extracted. on the other hand,
In a dim environment where a moving object is visible or invisible, the threshold value must be set relatively low, but if it is too low, noise is extracted. Thus, it is difficult to set an appropriate threshold value in each environment. Still further, if the threshold is fixed when the environment changes, even if the moving image can be extracted well in a bright environment, the moving image cannot be extracted well if the environment changes and becomes dim, or conversely, In a dark environment, even if a moving object image can be successfully extracted, if it becomes bright, a moving object image may not be extracted properly.

[0004]

According to a first aspect of the present invention, in order to solve the above-mentioned problems, a difference image between a previously stored background image when a moving object image does not exist and an input image sequentially input is set as a threshold value. The following processing is executed in a moving object image extracting method for obtaining a changing region by binarizing by comparing the above, and extracting the changing region as a moving object image. That is, a difference process for finding a difference image between the current and past background images, a difference image feature value extraction process for finding the maximum density value of the difference image, and a process of newly inputting the current background image as a past background image. A background image update process that sets the background image to be the current background image, and the difference process, the difference image feature value extraction process, and the background image update process are repeated a predetermined number of times. An initial setting process for obtaining a background difference maximum density value that is an average value and an optimum threshold value setting process for setting the threshold value based on the background difference maximum density value and the density value of the background image are executed. In the second invention, after executing the initial setting process, a maximum density value of a difference image between the background image and the input image is compared with the maximum density value of the background difference to determine whether a moving object image exists. An image change detection process to be performed is executed. Then, when it is determined by the image change detection process that there is no moving object image, based on the maximum density value of the difference image between the background image and the input image, an array of the maximum density values of the plurality of frames and the background difference maximum density A background difference feature array update process for updating a value is executed, the background image update process updates the input image as a background image, and the optimal threshold setting process performs a background difference maximum update performed by the background difference feature array update process. The threshold is updated based on a density value and a density value of the input image.

[0005]

According to the first aspect of the present invention, the moving object image extracting method is configured as described above.
A maximum density value of a difference image between the current and past background images is obtained. This maximum density value is a value representing the influence of noise generated by the imaging location, the image input device, and the like, and this value is a threshold value for separating the region representing the background from other regions. By repeatedly performing the difference processing, the difference image feature value extraction processing, and the background image update processing, an array of the maximum density value of the difference image between the background images of a plurality of frames is obtained. The influence of noise is reduced by calculating the background difference maximum density value, which is the average value, from the maximum density value array by the initial setting process. The optimum threshold value setting processing sets an optimum threshold value based on the maximum background difference density value and the density value of the background image in order to eliminate the influence of a moving object shadow that may occur under a bright background. A binarization process is performed based on the threshold to extract a moving object image. According to the second aspect, the image change detection process compares the maximum density value of the difference image between the background image and the input image with the maximum background difference density value to determine whether a moving object image exists. If the moving object image does not exist, the maximum density value of the difference image between the input image and the background image is added to the array, and the maximum background difference density value is obtained based on the maximum density value. As described above, the background difference maximum density value and the background image are updated according to the change in the environment. In the optimum threshold value setting process, the threshold value is updated based on the maximum density value of the background difference and the density value of the input image, and the threshold value is set according to a change in environment. Therefore, the above problem can be solved.

[0006]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First, the principle of the present embodiment will be described with reference to the drawings. 3 and 4 are diagrams illustrating examples of a background image and an input image in a bright environment, respectively. FIG. 5 is a difference image obtained by calculating the absolute value of the difference between the background image of FIGS. 3 and 4 and the input image. FIG. 6 shows a density histogram of the difference image shown in FIG. 7 and 8 are diagrams illustrating an example of a background image and an example of an input image in a dark environment, respectively. FIG. 9 is a difference image obtained by calculating the absolute value of the difference between the background image of FIGS. 7 and 8 and the input image. FIG.
0 represents the density histogram of the difference image shown in FIG. FIG. 11 shows a density histogram of a difference image between the background image at time t and the background image after Δt.
FIG. 12 illustrates a density histogram of a difference image between the background image at time t and the background image after Δt when there is no noise. In order to extract only a moving object (a person in FIG. 4) using the difference processing between the background image and the input image in a bright environment as shown in FIG. 3, the density value b in FIG. It is best to binarize the difference image such as 5. Hereinafter, the reason will be described. First, the portion A having a small density value will be considered. In an ideal device in which no noise is generated, the difference image between the background images should have all pixel values of “0”, and the density histogram should be as shown in FIG. However, the density histogram of the difference image between the background images becomes as shown in FIG. 11 due to the influence of noise from the image input device or the like. From this figure, it can be said that the portion A in FIG. 6 is a region representing a background portion other than the region changed in the difference image. for that reason,
It can be said that the portions B and C represent the regions that have changed from the background image in the input image. Is an area representing. In a bright environment as shown in FIG. 3, a shadow generally appears on a moving object. In many cases, a portion B representing a small change in density represents a shadow area.
The portion C indicating that the density change is large often indicates the region of the moving object. Therefore, the density value b is an optimum threshold value for extracting only a moving object.

On the other hand, in a dark environment as shown in FIG.
It is appropriate to binarize the difference image using the density value a ′ in 0 as a threshold. Because, in a dark environment, B in FIG.
Since there is no region representing the shadow of the moving object, a 'for separating A' representing the background region and region C 'representing the moving object region is most suitable as the threshold value. Therefore,
In the present invention, the optimum threshold value, that is, the density value b in FIG.
The density value a 'in 0 is obtained by the following procedure. First, in each environment, a density value that separates a region representing a background from other regions, that is, a density value a in FIG. 6 and a density value a ′ in FIG. 10 are obtained. For this purpose, first, a difference image between background images at different times is obtained, and then a maximum density value is obtained from the difference image. Further, in order to reduce the influence of noise, the maximum density value is obtained from a difference image between a plurality of background images, and an average value thereof is obtained.
This value is regarded as the density value a in FIG. 6 or the density value a ′ in FIG. 10 (hereinafter, this density value is referred to as a background difference maximum density value D). However, if the respective maximum density values are not stable due to the influence of noise, the density values of the pixels in which the cumulative frequency of the density histogram of the difference image is almost 100% of all the pixels may be used. The background difference maximum density value D is a value indicating the influence of noise depending on the shooting location, image input device, and the like, and indicates that the influence always appears from the background. Therefore, this value is a threshold for separating the region representing the background from the other region.

Next, in order to eliminate the influence of the shadow of a moving object that may occur under a bright background, a value for further increasing the threshold from the maximum background difference density value D (hereinafter, this value is referred to as a change value α Is called). The density value b in FIG. 6 is obtained from the change value α. The change value α is set to 0 since it is not necessary to consider the influence of the shadow in a dark environment as shown in FIG. On the other hand, in a bright environment as shown in FIG. 3, since the influence of the shadow increases as the brightness increases, the change value α is increased according to the brightness. Therefore, the optimal threshold T is determined using the following equations (1) and (2).

(Equation 1) Here, α = Qx + R (2) where D is the maximum density value of the background difference, x is the average value of the density values of all pixels in the background image, α is the change value, and Q, R, and s are
Represents a constant. Since the average value x of the densities of all the pixels of the background image is considered to represent the brightness of the entire background image, in a dark environment (the average value x is equal to or less than s) as shown in FIG. Becomes the optimum threshold value T as it is. On the other hand, when the background is bright as shown in FIG. 3 (the average value x is equal to or more than s), the change value α is obtained by using a linear function of Expression 2 so that the threshold value is increased according to the magnitude of the average value. Determine the optimal threshold T. At this time, Q, R, and s are empirically determined in advance. For example, in our experiments, Q
= 0.2, R = 0, and s = 60. This makes it possible to automatically set a binarization threshold suitable for each environment, and to stably extract only a moving object. FIG. 1 is a configuration block diagram showing a moving object image extraction method according to an embodiment of the present invention. Hereinafter, referring to FIG.
A moving image extraction method will be described.

Image Input Unit 10 The image input unit 10 takes an image of a certain place using a TV camera or the like, and sends the image to an image memory 20. The image memory 20 has two image memories, a background image memory and an input image memory. Normally, image data sent from the image input processing 10 is input to the input image memory. However, at the start of the processing, the background image obtained by capturing only the background is stored in the background image memory, and the difference processing 11 is executed. Difference processing 11 In the difference processing 11, the absolute value of the difference between the density values of the respective pixels is calculated from the image data stored in the background image memory and the input image memory of the image memory 20 to obtain a difference image. After smoothing the difference image and removing noise components, the data of the difference image is stored in a difference image memory 2.
1 and executes a difference image feature value extraction process 12. In the difference image feature value extraction process 12, the difference image memory 21
Is obtained from the difference image stored in the difference image feature value memory 22. Next, when the processing is started, the initial setting processing 13 is executed, and otherwise, the image change detection processing 15 is executed.

[0010] In the initial setting process 13 initial setting process 13, considers the maximum density value sent from the difference image feature value memory 22 and the maximum density value of the difference image between the background image, the background difference feature until the number of specified frames The value is stored in the array memory 23, and the next background image update processing 14 is executed. When the specified number of frames is reached, the average of the maximum density values stored in the background difference feature array memory 23 is obtained, and the average value is stored in the background difference feature array memory 23 as the background difference maximum density value D. Background Image Update Processing 14 In the background image update processing 14, the contents of the background image memory of the image memory 20 are rewritten to the contents of the input image memory. If the initial setting at the start of the process has been performed, the process returns to the image input unit 10, and the difference process 11, the difference image feature value extraction process 12, the initial setting process 13, and the background image update process 1
Step 4 is repeated for the specified number of frames. In other words, the newly input background image is set as the input image, the previous background image is set as the background image, and the difference processing 11 between the newly input background image and the previous background image, the difference image feature value extraction processing 12,
The initial setting process 13 and the background image updating process 14 are repeated by a specified number of frames to obtain a new maximum density value.
The values are stored in the background difference feature array memory 23 one after another. Then, when the specified number of frames is reached, the average of the maximum density values stored in the background difference feature array memory 23 is obtained,
The average value is stored as the background difference maximum density value D in the background difference feature array memory 23. That is, the background difference feature array memory 23 has a memory for storing the maximum density value of the difference image between the background images of the specified number of frames, and a memory for storing the background difference maximum density value D which is the average value. ing. However, an initial setting process 13 is performed,
Until the background difference maximum density value D is set, a background image obtained by capturing only the background where no intruding object exists is continuously input by a human hand. When the background difference maximum density value D is set,
Subsequently, whether or not a moving object is present in the input image is checked by the image change detection processing 15 using the value.

Image Change Detection Process 15 In the image change detection process 15, if the absolute value of the difference between the current maximum density value of the difference image and the maximum background density difference value D is smaller than a certain reference, the current input image contains moving objects. Is determined to be an image that does not exist, that is, a background image, and the background difference feature array update processing 16 is executed. Background Difference Feature Array Update Process 16 In the background difference feature array update process 16, the oldest one of the maximum density values in the past several frames stored in the background difference feature array memory 23 is replaced with the current maximum density value. Further, the average value is obtained from the updated maximum density values between several frames, and the average value is stored in the background difference feature array memory 23 as the background difference maximum density value D. Further, in the background image update processing 14, the contents of the background image memory of the image memory 20 are rewritten to the contents of the input image memory. By these update processes, it is possible to respond to a gradual change in the environment. In the image change detection process 15, if the absolute value of the difference between the current maximum density value of the difference image and the background difference maximum density value is larger than a certain reference, the optimal threshold setting process 17 is executed on the assumption that a moving object exists. .

[0012] In the optimum threshold value setting processing 17 optimum threshold value setting process 17, first, an average value x of the density values in all the pixels from the background image data in the image memory 20. Next, the average value x and the background difference feature array memory 23
The optimal threshold T is obtained from the maximum background difference density value D stored in the optimal threshold memory T using Expressions (1) and (2), the value is stored in the optimal threshold memory 24, and the moving object output unit 18 is executed. Moving object output processing 18 In the moving object output processing 18, the input image is binarized using the optimum threshold value T stored in the optimum threshold value memory 24 to obtain a change area, and this change area is extracted as the moving object image 19.
As described above, in the present embodiment, the optimum threshold value for extracting the moving object image is automatically set using the maximum density value of the difference image between the background images of a plurality of frames. Accurately only the moving object image from the image, and
There is an advantage that extraction can be performed at high speed.

[0013]

As described in detail above, according to the first and second aspects, the average value of the maximum density value of the difference image between the background images of a plurality of frames and the density value of the background image are used. Since the threshold for extracting the moving object image is automatically set according to the environment, only the moving object image can be accurately and rapidly extracted from the input image under various environments.

[Brief description of the drawings]

FIG. 1 is a configuration block diagram illustrating a moving object image extraction method according to an embodiment of the present invention.

FIG. 2 is a configuration block diagram showing a conventional moving object image extraction method.

FIG. 3 is a diagram illustrating an example of a background image in a bright environment.

FIG. 4 is a diagram illustrating an example of an input image under a bright environment.

FIG. 5 is a diagram showing an example of a difference image between FIGS. 3 and 4;

FIG. 6 is a diagram illustrating an example of a density histogram of FIG. 5;

FIG. 7 is a diagram illustrating an example of a background image in a dark environment.

FIG. 8 is a diagram illustrating an example of an input image in a dark environment.

FIG. 9 is a diagram illustrating an example of a difference image between FIGS. 7 and 8;

FIG. 10 is a diagram showing an example of a density histogram of FIG. 9;

FIG. 11 is a diagram illustrating an example of a density histogram between background images.

FIG. 12 is a diagram illustrating an example of a density histogram between background images when there is no noise.

[Explanation of symbols]

 Reference Signs List 10 Image input unit 11 Difference processing 12 Difference image feature value extraction processing 13 Initial setting processing 14 Background image update processing 15 Image change detection processing 16 Background difference feature array update processing 17 Optimal threshold setting processing 18 Moving object output unit 20 Image memory 21 Difference image Memory 22 Difference image feature value memory 23 Background difference feature array memory 24 Optimal threshold memory

──────────────────────────────────────────────────続 き Continuation of the front page (72) Inventor Yoshinori Shimosakoda 1-7-12 Toranomon, Minato-ku, Tokyo Oki Electric Industry Co., Ltd. (56) References JP-A-6-303485 (JP, A) JP-A-5-120432 (JP, A) JP-A-5-46729 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06T 1/00 G06T ^7/ 00-7/60 H04N 7/18

Claims (2)

(57) [Claims] 1. A change region is obtained by binarizing a difference image between a background image stored in advance when a moving object image does not exist and an input image sequentially input by comparing the difference image with a threshold value, and determining the change region as a moving object image. In the moving object image extraction method, a difference process for finding a difference image between a current and a past background image, a difference image feature value extraction process for finding a maximum density value of the difference image, A background image, a background image update process in which a newly input background image is set as a current background image, and the difference process, the difference image feature value extraction process, and the background image update process are repeated a predetermined number of times to perform a plurality of frames. Initial setting processing for obtaining a background difference maximum density value that is an average value from an array of maximum density values, and setting the threshold value based on the background difference maximum density value and the density value of the background image Optimize the threshold value setting processing, the moving object image extraction method and executes that. 2. After executing the initial setting process, a maximum density value of a difference image between the background image and the input image is compared with the maximum density value of the background difference to determine whether a moving object image exists. Performing an image change detection process, and when it is determined that the moving object image does not exist by the image change detection process, the maximum density value of the plurality of frames is determined based on the maximum density value of the difference image between the background image and the input image. Performing a background difference feature array update process for updating an array and the background difference maximum density value, the background image update process updating the input image as a background image, and the optimal threshold setting process performing the background difference feature array update process 2. The moving object image extraction method according to claim 1, wherein the threshold value is updated based on the background difference maximum density value updated according to (1) and the density value of the input image.