JPH0824350B2 - Background image extraction method - Google Patents

Description

TECHNICAL FIELD The present invention relates to a background image extraction method used for preprocessing when extracting a moving object from continuous image data, for example.

"Prior Art" Generally, in moving image processing, a method of utilizing a difference between two preceding and following screens is used as a simplest method for extracting a region where a moving body exists.

For example, Reference 1 (Asada et al., “Detection and tracking of moving objects”)
IEICE Transactions '79 / 6, vol.J62-D.No.6.395-402
In the method of using a continuous difference image proposed by P., a binary difference image is used to continuously create a difference image between one frame of interest and several frames in the vicinity of this frame. , The area of the moving object existing in the frame of interest is detected by using only the positive (or negative) component of the obtained plurality of difference images and taking the logical sum of successive frames.

As another method, there is also proposed a method of detecting a region of a moving object by obtaining a difference image between a background image and moving image data, which has been captured in advance, and thresholding the difference image. There is.

However, among the moving object detection methods in moving image processing as described above, in the method using the difference image (the former method), when the processing is performed by the difference in the gray level of the real world, the influence of noise is strong, There is a problem that the position of the moving object cannot be detected accurately, and the moving amount of the object is small enough to overlap the same moving object in adjacent frames, in other words, the time interval of the frame should be set to be sufficiently small. There is a problem that it must be kept.

Further, the method of storing the background in advance as the data (the latter method) has a problem that it is difficult to obtain the background image itself when the moving object is constantly moving.

In view of such a situation, it is an object of the present invention to provide a background image extraction method capable of obtaining a background image necessary for detecting a moving object using only moving image data.

[Means for Solving the Problems] According to the present invention, image data continuously input with the image pickup device fixed is used, and images of continuous frames or images of a plurality of frames sampled within a specified time, The image of each frame is divided into small sections, and a histogram is created from the time-series data of the representative brightness level for each small section at the same position in the image of each frame, and the maximum frequency frequency level or representative brightness level is displayed on the histogram. The representative small section is determined for each small section, and the entire screen is configured with the image data of the representative small section as the background level of the small section.

[Embodiment] A background image extracting method as an embodiment of the present invention will be described below with reference to the drawings.

As shown in FIG. 1, a moving object 1 indicated by diagonal lines is moving from left to right in the screen with the corrugated pattern 2 as a background. Focusing on one point A on the screen, the point A is
Although it is a point in the background in 6 frames, it is a point in moving object 1 in the second and third frames.

As shown in FIG. 2, the level of point A is level l _{3 in} the first, fourth, fifth, and sixth frames in which point A is the background, and point A is the point in moving object 1. In a certain 2nd frame and 3rd frame, it becomes level l ₁ and level l ₂ , respectively. Here, it is assumed that the luminance level of the background wave pattern 2 is constant regardless of the position of the moving object 1 in the time section from the first frame to the sixth frame, and the luminance level of the moving object 1 is the same. It is assumed that the locations in No. 1 are non-uniform and that they can change with time.

As shown in FIG. 3, when the histogram of the brightness level at the point A in the 6 frames is taken, it is found that the level l ₃ corresponding to the background level has the maximum frequency. Therefore, the level l ₃ may be adopted at the point A as the output data of the background image.

By performing the above procedure for all points in the image, as shown in FIG. 4, a background image 3 that does not include the moving object 1 but only the corrugated pattern 2 is obtained.

In the actual moving image data, due to the influence of shadows, blurring, and other noises, as shown in FIG. 5, the brightness levels are not the same even in a frame in which the moving object 1 does not exist,
Even in the frame in which the moving object 1 exists, the brightness level may be the same. In this case, when a histogram of the brightness level of the point of interest is created, it becomes as shown in FIG. 6, and it becomes impossible to extract the background by the level having the maximum frequency.

Therefore, prior to extracting the brightness level of the point of interest in each frame, smoothing for each small section, filtering using the maximum frequency level in each small section as a representative value, and processing for reducing the brightness gradation are performed. Then, a histogram is created using the brightness levels in the image data that has been subjected to these preprocessing. In order to make the level of the dense portion corresponding to the background of the histogram thus created conspicuous, it is effective to perform histogram correction processing such as weighting adjacent levels. Further, the pre-processing process may include the pixel spreading for reducing the processing time.

As described above, when the preprocessing is performed prior to the histogram creation, in order to reproduce the background image, instead of outputting the level having the maximum frequency on the histogram as it is and obtaining the thinned and sampled background data, A method of outputting the background image data may be used. That is, each frame having the highest frequency level on the histogram is extracted as a candidate frame having a background at the point of interest, and the value in the original image data of one frame selected from the candidate frames is output as the background. Alternatively, an arbitrary number of frames may be selected from a plurality of candidate frames, and the values in the original image data of those frames may be used to output as the background.

Hereinafter, a case where a small section is treated as a unit will be described using an embodiment. When a background image 3 '(FIG. 8) including only the background pattern 2'excluding the moving object 1'is extracted using n consecutive images as shown in FIG. 7 (a), first, All continuous image data is divided into small blocks, and filtering is performed with each block having the maximum frequency as a representative value (Fig. 7 (b)). At the same time, each block is represented by one point and the array is reduced. (Fig. 7 (c)). That is, if the image data is divided into 8 × 5 small sections, the reduced array has 8 × 5 elements for one frame, as shown in FIG. 7C. . The value of each element of the array is the ice-breaking brightness value of each small section.

Here, focusing on the small section A ′, from the first frame to the nth
FIG. 7 (d) shows the representative brightness of the small section A'up to the frame as a graph of the brightness level with respect to the frame number. In the first frame, the level is low because the moving object 1'having low brightness is included in the small section A ', but the moving object 1'is not included in the second frame to the nth frame. , It is almost constant level. This is shown in a histogram in FIG. 7 (e). However, since the luminance level of the background pattern 2 ′ is slightly different depending on the frame, the histogram shows a distribution that is densely packed near the high luminance level with low frequency. To be

In some cases, the background may not correspond to the maximum frequency level, or the maximum frequency may be shown even though the background does not correspond to the coarse distribution. Therefore, in order to make the dense portion on the histogram noticeable and to improve the reliability of background level detection, by weighting,
The histogram may be corrected to include the contribution of adjacent levels. For example, let the frequency at level i be h
₁ (i), The histogram h ₂ (i) corrected by the equation is shown in FIG. 7 (f). By considering a generalized histogram that has been subjected to such a correction process, it is possible to stably give the maximum frequency level to the dense distribution corresponding to the background on the histogram.

Next, as a representative brightness level of the small section, a frame having a level close to the level l _mf having the maximum frequency in the histogram (FIG. 7 (f)) obtained as described above is extracted. Here, as shown in FIG. 7D, the second frame to the nth frame are candidates. One frame is selected from these candidate frames 4 by a method described later or the like, and the small section B in the original image data corresponding to the target small section A'is output as the background. For example, the nth
The value in the small section B of the frame is substituted into the corresponding section of the background image data 3'as shown in FIG. 7 (g).

If the procedure for extracting the background for the focused small section A'described above is performed for all the small sections, the background pattern 2'of all points is extracted as shown in FIG. Note that the method is not limited to the method of directly using the value in the small section B of the nth frame as the background of the small section of interest A ′.
For each point in the small section B, a method of outputting an average of an arbitrary plurality of frames from the second frame to the nth frame may be used.

To select an appropriate one frame from the candidate frames 4 as the background image extracting original image, for example, the following method may be used.

Even in a frame in which the moving object 1 does not exist, in frames immediately before and after the frame in which the moving object 1 exists,
Focus on the effect of shadows and reflections from moving object 1
Since there is a possibility that it can be seen inside the frame, the frame that is farthest (temporally) from the frame whose brightness level has changed immediately before and after the frame with the highest frequency level is adopted as the frame for background data extraction. There is.

For example, referring to FIGS. 9 (a) to 9 (e), a description will be given using five examples shown as a graph of the luminance level with respect to the frame number. (Break frame), and the frame having the maximum number of frames from each candidate frame to the closest break frame may be adopted as the frame for background extraction (marked in FIG. 9). As shown in FIG. 9 (e), when the brightness level is constant, a specific frame (for example, the first frame) may be adopted.

Since the background image extracting method of the present invention is configured as described above, it is possible to extract the background image excluding only the moving part by using the continuous image including the moving body.

[Advantages of the Invention] As described in detail above, according to the background image extraction method of the present invention, attention is paid to a small segment in an image of a plurality of consecutive or sampled frames, and the representative luminance level of the focused small segment is A histogram is created from time-series data, and the background level of the focused subsection is determined by the representative luminance level of the maximum frequency on the histogram.The moving object is detected using only continuous image data including the moving object. The advantage is that the removed background image can be obtained.

In addition, we will minimize the influence of noise and shorten the processing time,
Even if pre-processing such as smoothing and sampling is performed on the original image, the luminance level in one frame selected from the original image frames having the maximum frequency level on the histogram, or the original image having the maximum frequency level Only the background portion can be extracted with a simple configuration in which the brightness level determined using the brightness levels of a plurality of frames among the frames is output as the background brightness level.

Further, according to the present invention, a stable and highly reliable background image can be obtained. Therefore, the moving object can be accurately detected by taking the difference between the continuous image including the moving object and the background image. It is possible to provide an image processing technology effective for object monitoring technology and automatic measurement of traffic flow.

[Brief description of drawings]

FIG. 1 is a diagram showing six consecutive images (frames) as moving image data including a moving object photographed with the image pickup body fixed, and FIG. 2 shows a change in luminance level at point A in FIG. FIG. 3 is a diagram showing each frame, FIG. 3 is a diagram showing a histogram of luminance levels from a first frame to a sixth frame at point A in FIG. 1, and FIG. 4 is a diagram showing output background image data. FIG. 5 is a diagram corresponding to FIG. 2 showing changes in the luminance level at point A when noise is included, and FIG.
FIG. 7 is a diagram showing a histogram of luminance levels in FIG.
FIG. 8 is a diagram for explaining a background image extraction method when the screen is divided into small sections and smoothing processing is performed for each small section.
FIG. 9 is a diagram showing a background image that can be extracted by the processing described in FIG. 7, and FIG. 9 is an explanatory diagram of a method of selecting a frame having a maximum frequency level.

Claims (1)

[Claims] 1. A method of extracting a background image from which only a moving object is removed using image data continuously input with an image pickup object fixed, wherein images of consecutive frames or a plurality of frames sampled within a specified time The image of each frame is divided into small sections in the same manner, and the process of determining the representative brightness level is performed for each of the small sections, and the representative brightness is calculated for each of the small sections at the same position on the image of each frame. A histogram is created from time-series data of levels, and a representative sub-partition is determined for each sub-partition at the same position on the basis of the maximum frequency brightness level or representative brightness level on the histogram. Is output as a background image.