JPS63194477A - Background picture extracting method - Google Patents

Abstract

PURPOSE:To obtain a background picture by using only continuous picture data including moving object by forming the histogram of the representative brightness level of a noticed small block and deciding a background level according to the representative brightness level of a maximum frequency on the histogram. CONSTITUTION:All the continuous picture data is divided for every small block, and a filtering for making the level of the maximum frequency a central value is carried out. Then, as the representative brightness level of the small block, a frame having a level near to a level lmf having the maximum frequency is extracted. In order to select one suitable frame from candidate frames 4 as a background data extracting original picture, a frame the remotest from a frame changing in the brightness level immediately before and immediately after of the frames having the maximum frequency level is used as a frame for extracting the background data.

Description

DETAILED DESCRIPTION OF THE INVENTION "Field of Industrial Application" The present invention relates to a background image extraction method used, for example, in preprocessing when extracting a moving object from continuous image data.

"Prior Art" Generally, in moving image processing, the simplest method for extracting an area where a moving object is present is a method that utilizes the difference between two screens in front and behind each other.

For example, see Reference 1 (Yoshi Asada, "Detection and Tracking of Moving Objects" Journal of the Institute of Electronics and Communication Engineers '79/6, vol, J62-D
.

The method using continuous difference images proposed by A6 (pp. 395-402) uses binarized continuous images and calculates the difference images between one frame of interest and several frames in the vicinity of this frame. By using only the positive (or negative) components of the multiple differential images that are created sequentially and taking the logical OR of the consecutive frames, the area of the moving object that exists in the frame of interest is determined. Detect.

In addition, another method has been proposed in which a difference image between a background image taken in advance and moving image data is obtained, and this is subjected to threshold processing to detect the area of a moving object. There is.

However, among the moving object detection methods in video processing as described above, the method that uses difference images (the former method) is strongly affected by noise when processing is performed using differences in gray levels in the real world. The problem is that it is not possible to accurately detect the position of a moving object, and the amount of movement of the object is set small enough that the same moving object overlaps in adjacent frames, in other words, the time interval between frames is set sufficiently small. The problem is that it has to be kept as is.

Also, there is a method of storing the background as data in advance (
The latter method has a problem in that it is difficult to obtain the background image itself when the moving object is constantly moving.

In view of this situation, it is an object of the present invention to provide a background image extraction method that can obtain a background image necessary for detecting a moving object using only moving image data.

``Means for Solving the Problems'' According to the present invention, image data that is continuously input with the imaging object fixed is used, focusing on small sections in the image, and continuous or A histogram of the representative brightness level of the small section of interest is created for the plurality of sampled frames, and the level of the background in the small section of interest is determined based on the representative brightness level of the maximum frequency on the histogram. In this determination, for example, the representative brightness level that has the highest frequency is set as the brightness level of the background.

Or, the brightness level determined using the brightness level in one frame selected from among the original picture frames having the maximum frequency level, or the brightness level of multiple frames among the original picture frames having the maximum frequency level. brightness level.

In this way, the background image extraction method of the present invention focuses on small sections within each frame of continuous or sampled image data, creates a histogram of the brightness level of each frame for each small section, and calculates the maximum brightness level on the histogram. Continuous or sampled image data can be created by treating the level with the frequency as the brightness level of the background image in that small section and storing it in the output data for the entire screen.
A background image with moving objects removed can be obtained using

``Example'' A background image extraction method as an example of the present invention will be described below with reference to the drawings.

As shown in FIG. 1, a moving object 1 indicated by diagonal lines is moving from left to right within the screen with a wavy pattern 2 as a background.

When focusing on one point A on the screen, point A is the 1st, 4th, 5th
．． In the 6th frame, it is a point in the background, but in the 2nd and 3rd frames, it is a point in the moving object 1.

As shown in FIG. 2, the level d of point A is the first point where point A is the background. 4th. In the fifth and sixth frames, the level is t3, and in the second and third frames, where the point A is a point in the moving object 1, the level is t1 and the level t2, respectively. Here, it is assumed that the brightness level of the background wave pattern 2 can be considered constant regardless of the position of the moving object 1 in the time interval from the first to the sixth frame.
The brightness level of the moving object 1 is non-uniform at different locations within the moving object 1, and can also vary over time.

When a histogram of the brightness level at point A within six frames is taken, as shown in FIG. 3, it is found that the maximum frequency is at level t3 corresponding to the background level. Therefore, the output data of the background image is level t3 at point A.
should be adopted.

By performing the above procedure for all points in the image, a background image 3 containing only the wavy pattern 2 without the moving object 1 is obtained, as shown in FIG.

In actual video data, due to the influence of shadows, backgrounds, and other noises, the brightness level is not the same even in frames where moving object 1 does not exist, as shown in Figure 5. In some cases, the brightness level may be the same even if the frames are the same. In this case, if a histogram of the luminance levels of the point of interest is created, it will become impossible to extract the background based on the level with the maximum frequency, as shown in FIG.

Therefore, before extracting the brightness level of the point of interest in each frame, we perform smoothing for each subsection, filtering using the maximum frequency level within the subsection as a representative value, and processing to reduce brightness gradation. Then, a histogram is created using the brightness levels in the preprocessed image data. In order to make the level of the dense portion corresponding to the background of the histogram created in this way more noticeable, it is effective to perform a histogram correction process such as weighting adjacent levels. Furthermore, pixel diversion may be included in the preprocessing process to shorten processing time.

As mentioned above, when preprocessing is performed before creating a histogram, in order to reproduce the background image, it is necessary to output the level with the highest frequency on the histogram as it is and obtain the thinned-out sampled background data. The following method for outputting background image data may be used. That is, each frame at the highest frequency level on the histogram is extracted as a candidate frame in which the background exists at the point of interest, and the value in the original image data of one frame selected from the candidate frames is assigned as the background. Alternatively, an arbitrary number of frames may be selected from a plurality of candidate frames, and the values in the original image data of those frames may be used to output them as the background.

Hereinafter, a case where a small section is treated as a unit will be explained using an example. When extracting a background image 3' (Fig. 8) containing only the background pattern 2' excluding the moving object 1' using n consecutive images as shown in Fig. 7(a), first, All continuous image data is divided into small sections, and filtering is performed using the most frequent value in each section as the representative value (Figure 7 (b)). At the same time, each section is represented by one point and the array is reduced. (Figure 7(C)). In other words, if the image data is divided into 8 x 5 small sections, the reduced array will have 8 x 5 elements for one frame, as shown in Figure 7(c). . Further, the value of each element of the array is a representative brightness value of each small section.

Now, focusing on the small section A', if the representative luminance of the small section A' from the 1st frame to the nth frame is plotted as a graph of luminance versus frame number, it will be as shown in Fig. 7(d). . In the first frame, the level is low because the moving object 1' with low brightness is included in the subdivision A', but the moving object 1' is not included in the second to nth frames. , is at a more or less constant level. This can be expressed in a histogram as shown in Figure 7 (,), but because the brightness level of the background pattern 2' differs slightly depending on the frame, the histogram shows a distribution that is concentrated at low frequency near high brightness levels. It will be done.

In some cases, even the background may not correspond to the maximum frequency level, or the coarse distribution may show the maximum frequency even though it does not correspond to the background. Therefore, in order to make the densely spaced portions on the histogram more noticeable and increase the reliability of background level detection, it is preferable to correct the histogram to include the contribution of adjacent levels by weighting. For example, let the frequency at level i be (
t), h2(i)=-h, (1-1), (i)+7h 1
Histogram h2 corrected according to the formula (+ + 1)
(i) is shown as in FIG. 7(f). By considering a generalized histogram that has undergone such correction processing, it is possible to ensure that the dense distribution corresponding to the background on the histogram stably provides the maximum frequency level.

Next, a frame having a level close to the level 'mf having the maximum frequency in the histogram obtained as described above (FIG. 7(f)) is extracted as the representative brightness level of the small section. Here, as shown in FIG. 7(a), the second frame to the nth frame are selected as candidates. One frame is selected from among these candidate frames 4 by a method described later, and the small section B in the original image data corresponding to the small section A' of interest is output as the background. For example, the nth
The value in the small section B of the frame is substituted into the corresponding section of the background image data 3' as shown in FIG. 7(g).

If the above-described process of extracting the background of the subdivision A' of interest is performed for all subdivisions, the background pattern 2' of all points will be extracted as shown in FIG. Note that the method is not limited to the method of directly using the value in subdivision B of the nth frame as the background of the focused subdivision A'. A method of outputting the average of arbitrary plurality of frames may be used.

For example, the following method may be used to select an appropriate frame from among the candidate frames 4 as the original image for background data extraction.

Even in frames where moving object 1 does not exist, in the frames immediately before and after the frame where moving object 1 exists, there is a possibility that effects such as shadows and reflections from moving object 1 will be seen within the focused subdivision A'. Therefore, there is a method in which, among the frames having the maximum frequency level, the frame that is the most distant (temporally) from the frame whose brightness level changes immediately before or after is adopted as the frame for background data extraction.

For example, to explain using five examples shown as graphs of brightness levels against frame numbers in Figures 9(a) to (,), 13r is a frame in which the brightness level changes by more than a certain value in the immediately preceding and following frames. (break frame)
Therefore, the frame with the maximum number of frames from each candidate frame to the nearest node break frame can be adopted as the frame for background extraction (indicated by ■ in FIG. 9). Note that, as shown in FIG. 9(,), when the brightness level is constant, a specific frame (for example, the first frame) may be adopted.

Since the background image extraction method of the present invention is configured as described above, a background image excluding only the moving part can be extracted using continuous images including the moving body.

"Effects of the Invention" As detailed above, according to the background image extraction method of the present invention, a small section in the screen in consecutive or sampled frames is focused on, and the representative brightness level of the focused small section is With an extremely simple configuration, a histogram is created, and the background level of the small section of interest is determined based on the representative luminance level of the maximum frequency on the histogram. Moving objects are removed using only continuous Fi image data that includes moving objects. The advantage is that it is possible to obtain a background image with a sharp background image.

In addition, in order to minimize the influence of noise and shorten processing time, even if preprocessing such as smoothing and sampling is applied to the original image, the original image frame with the highest frequency level on the histogram is selected. With a simple configuration in which the brightness level determined using the brightness level in one frame or the brightness level of multiple frames among the original image frames having the maximum frequency level is output as the brightness level of the background, Further, according to the present invention, a stable and highly reliable background image can be obtained, so by taking the difference between a continuous image containing a moving object and the background image, it is possible to extract a moving object with high precision. Since objects can be detected, it is possible to provide image processing technology that is effective for monitoring intruding objects and automatically measuring traffic flow.

[Brief explanation of the drawing]

Figure 1 shows six consecutive images (frames) as moving image data including a moving object taken with the imaging object fixed, and Figure 2 shows the change in brightness level at point A in Figure 1. A diagram shown for each frame, FIG. 3 is a diagram showing a histogram of brightness levels from the first frame to the sixth frame at point A in FIG. 1, and FIG. 4 is a diagram showing output background image data. Figure 5 is a diagram corresponding to Figure 2 showing the change in brightness level at point A when noise is included;
The figure shows the histogram of the brightness level in Figure 5, and Figure 7.
The figure is a diagram for explaining the background image extraction method when dividing the screen into small sections and applying smoothing processing to each small section.
This figure shows a background image that can be extracted by the process explained in FIG. 7, and FIG. 9 is an explanatory diagram of a method for selecting a frame having the maximum frequency level.

Claims (1)

[Claims] (1) In a method of extracting a background image from which only moving objects are removed using image data that is continuously input with the imaging object fixed, sampling is performed continuously or within a specified time by focusing on small sections in the image. Background image extraction is characterized in that a histogram of representative brightness levels of a small section of interest is created for a plurality of frames, and the background level of the small section of interest is determined based on the representative brightness level of maximum frequency on the histogram. Method.