JPH03112282A - Method and apparatus for generating background picture - Google Patents

Abstract

PURPOSE:To generate a background picture by adding a value proportional to a difference between a picture element value of an input picture signal and a picture element value corresponding to the detected picture element in the back ground picture to a picture element value corresponding to the detected picture element in the background picture so as to revise the background picture. CONSTITUTION:A background generating circuit 106 takes a difference between signals of each processing block segmented by a memory access circuit 105 from an input frame memory 103 and a delay frame memory 104. Then difference signal power is compared with a prescribed threshold level to discriminate whether each picture element of the input picture signal belongs to a moving area or a background area. Moreover, the circuit 106 revises a picture element value corresponding to a picture element discriminated to be belonging to the background area among background pictures stored in a background memory 107 with the a value in response to between the picture element value and the picture element value of the said picture element in the input picture signal. Furthermore, the circuit 106 is provided with a background area discrimination circuit 106a to discriminate the background area and a background memory revision circuit 106b to revise a background picture of the background memory 107.

Description

Detailed Description of the Invention (Field of Industrial Application) The present invention relates to a method of predicting the background area of a moving subject from a moving image and generating an image of that area (background image), and an apparatus suitable for carrying out the method. It is something.

(Prior Art) In recent years, in the field of image coding, predictive coding methods such as intraframe prediction and motion compensated interframe prediction have been studied. The predictive encoding method predicts the next input image signal from past input image signals, calculates the difference, and performs communication by transmitting the obtained difference signal.

One such predictive coding method is called a background prediction method. This background prediction method uses an encoder/decoder to generate a background image in memory for the background area of a person as a subject in video conferences, video calls, etc., and calculates the background that appears and disappears as the person moves. The prediction was made using the generated background image.

According to this, image quality can be improved even when the subject moves rapidly. Such a background image generation method is used not only in the field of communications but also in the field of remote sensing such as monitoring systems.

For example, as a conventional background image generation method of this type, there is a literature (
Institute of Electronics, Information and Communication Engineers Technical Research Report IE83100 (1)
9B4.2.24) pp. 1-5).

According to the method disclosed in this document, a background image is generated using the procedure described below. Note that this explanation will be given with reference to FIG. Here, FIG. 6 is a block diagram schematically showing the configuration of a conventional background image generation device suitable for implementing this method.

First, an original image is converted into an electrical signal representing the density distribution of light by scanning the image pickup tube of a TV (television) camera 601, and then sampled into pixels by an A/D converter 602 and converted into a digital image signal. be done. This input digital image signal for one screen (one frame) is stored in the input frame memory 603.

Next, the background generation circuit 605 uses the input frame memory 60
A background image is generated in the background memory 606 using the input digital image signal stored in 3 and the digital image signal of the previous frame stored in the delayed frame memory 604 according to the procedure described below.

■...1 stored in the delayed frame memory 604
An interframe difference signal is obtained by subtracting the previous frame image signal from the input image signal stored in the input frame memory 603.

■...When the absolute value of this inter-frame difference signal is greater than a certain threshold value, that pixel is considered to be included in the moving area of the subject (area of moving objects), and the contents corresponding to this pixel in the background memory 606 (pixel value) is not updated.

■...When the absolute value of the inter-frame difference signal is less than or equal to L (not including zero), the pixel is considered to be included in a stationary part of the subject (background area), and the background memory 6
06 (the corresponding pixel value is updated to a value that is closer to the pixel value of the input image by M (however, M is a predetermined fixed value).

(2) When the interframe difference signal is zero, the pixel value corresponding to this pixel in the background memory is not updated.

(2) The update in (2) above is performed only once every N frames (N is a predetermined positive integer).

A background image was generated on the background memory 606 by performing the processes ① to ② on consecutive moving images.

(Problem to be solved by the invention) However, in the conventional background image generation method described above,
There were problems as explained below.

(2) Background areas are determined by setting a threshold value for the inter-frame difference signal, but it is difficult to set the optimal threshold value.

(2) When determining a background area, the background area can be determined more accurately if the signal used for determination includes as many characteristic portions that indicate the background as possible. but,
In the conventional method, since the background area is determined pixel by pixel, it is likely that a characteristic portion indicating that the background is "□" is not included, and as a result, the background area is likely to be erroneously detected.

■...In order to prevent the contents of the background memory from being rewritten inadvertently, the amount of update of the background memory per time is set to a fixed value (M above), and the number of updates is roughly 1 every N frames.
Because of the number of times, it is slow to converge to the pixel value of the background memory or the pixel value of the true background image. Roughly speaking, this convergence speed depends on the input image, such that the larger the difference between the pixel values of the input image and the pixel values of the background memory, the slower it becomes.

Note that the "true background image" here refers to a background image that is visually determined to be the true background.

The present invention has been made in view of the above points, and therefore, an object of the present invention is to reduce the number of false detections when detecting a background area, and to reduce the speed at which the pixel value of the background memory converges to the true pixel 0. The object of the present invention is to provide a method that can generate a background image quickly and without depending on the input image, and an apparatus suitable for implementing the method.

(Means for Solving the Problem) In order to achieve this object, according to the first invention of this application, a background area of an image including a moving area and a background area is In a method of generating a background image by predicting using past signals of an image signal, it is determined whether a certain test pixel belongs to a background region or a moving region by using the above-mentioned test pixel in the input image signal and this test pixel. Using the difference between the image signal of each predetermined pixel around the test pixel and the image signal of each pixel corresponding to the test pixel and the predetermined pixel in the past input image signal mentioned above, in a specific example, For example, this is done by comparing the power or amplitude of the difference signal between both signals with a predetermined threshold, and if it is determined that the aforementioned test pixel belongs to the background area, the pixel value corresponding to the test pixel in the background image is , the background image is updated by adding a value proportional to the difference between the pixel value of the input image signal and the pixel value corresponding to the test pixel in the background image.

Further, according to the second invention of this application, the background area of an image including a moving area and a background area is predicted using an input image signal and a past signal of this input image signal, and a background image is generated. In the generating device, an input frame memory stores an input image signal, a delayed frame memory stores past signals of this input image signal, and a predetermined block of image signals is cut out from each of the input frame memory and the delayed frame memory. a memory access circuit, a background memory that stores a background image, a difference between the input image signal described above and the past signal described above, and this difference signal is compared with a predetermined threshold value to determine whether each pixel of the input image signal In addition to determining whether the pixel belongs to the moving area or the background area, the pixel value corresponding to the pixel determined to belong to the background area in the background memory is determined based on the pixel value and the pixel value of the pixel in the input image signal. and a background generation circuit that updates the background with a value corresponding to the difference between the two.

(Function) According to the background image generation method of the first invention, first, it is determined whether or not a test pixel belongs to a background area using the input image signal of the test pixel and the pixels around the test pixel and the history of this signal. This is done using the difference signal between the signals. Therefore, the signal used to determine the background area is likely to include a characteristic portion indicating that the pixel is the background, so that it is possible to determine whether the test pixel belongs to the background area, that is, to detect the background area with high accuracy.

Next, if it is determined that the test pixel belongs to the background area, the pixel value corresponding to this test pixel in the background image is added to the pixel value of the input image signal and the pixel value corresponding to the test pixel in the background image. Since the background image is updated by adding a value proportional to the difference between the background image and the background image, it is possible to appropriately and quickly converge the pixel @ of the background image to the true pixel value of the background image.

Further, according to the background image generation device of the second invention of this application, the background image generation method of the first invention can be easily implemented.

(Example) Hereinafter, an example of the background image generation method of this application and an example of a background image generation device H suitable for implementing this background image generation method will be described with reference to the drawings.

Note that the input image signal referred to in the following description of the embodiment is considered to be a three-dimensional signal of space and time, as shown in FIG.
It will be expressed as (x, y, t).

2' image production °n8 First, an embodiment of the background image generation device M (hereinafter sometimes abbreviated as "generated snowmaking") of the second invention suitable for implementing the background image generation method of the first invention will be explained. do. FIG. 1 is a diagram for explaining this, and is a diagram schematically showing the configuration of the snow-making system of the embodiment.

In FIG. 1, 101 is a TV camera for image input;
2 is an A/2 that samples the image input from the TV camera 101 and generates a digital input image signal I(x, y, t).
D converter 103 is input image signal I(x, y, t) 7
104 indicates the input frame memory for storing the input image signal I (x, y, t) in the past] frame previous signal I (x, y, t-1), respectively. . Delayed frame memory +0
4 is connected to the subsequent stage of the input frame memory 103. In addition, this input frame memory 103 stores the input image signal ■(x
, y, t) are transferred to the delayed frame memory 104 as past signals I (x, y, t-1).

Furthermore, in FIG. 1, 105 is a memory access circuit;
106 represents a background generation circuit, and +07 represents a background memory.

The functions of these constituent components 105, 106 and 1070 will be explained in detail in the section on the background image generation method described later, but briefly described as follows.

The memory access circuit 105 is connected to the input frame memory 10
A certain pixel (
Hereinafter, it will be referred to as a test pixel. ) I (xo, y.

, t) A processing block (part of the image signal) of an arbitrary size centered at the center is cut out and used in the background generation circuit 10 in the subsequent stage.
6 (This is the circuit that outputs this.

Further, the background memory 107 is a memory that stores a background image. Although details will be described later, the background image stored in the background memory 107 is updated by the background generation circuit 106.

Further, the background generation circuit 106 is connected to the input frame memory 103.
The delay frame memory 104 and the memory access circuit 105 calculate the difference between the signals of the respective processing blocks cut out by the memory access circuit 105, and compare the difference signal power with a predetermined threshold value to determine the motion area of each pixel of the input image signal. Determine whether it belongs to the background area or the background area. This background generation circuit 106 generates a pixel value corresponding to a pixel determined to belong to the background area, out of the background image stored in the background memory 107, by combining this pixel value with the input image signal. The pixel value is updated with a value corresponding to the difference between the pixel value and the pixel value of the pixel.

Note that, as shown in FIG. 3, the background generation circuit 106 of this embodiment includes a background area determination circuit 106a for determining a background area, and a background memory 10.
Background memory update circuit 10 for updating the background image of No. 7
6b.

Next, an embodiment of the background image generation method of the first invention will be described using an example using the background image generation apparatus described using FIG. 1.

Input image signal I(x, y, t) generated by TV camera 101 and A/D converter 102 is stored in input frame memory 103.

The background area determination circuit 106a of the background generation circuit 106 selects a pixel I (xo.

yo, t) @A predetermined pixel centered on the test pixel; in this case, a processing block B (xo, yo, t) consisting of NXN (N is a predetermined positive integer) pixels 200
Cut out. Roughly speaking, from the delayed frame memory 104, a pixel I (xo.

yo, t-1) 'l centered processing block 8 (xo
, yo, t-1). Then, the following (1) EM[1(XO, Vo, t)'ffi is calculated using the pixel values of all pixels in both extracted blocks.

According to the formula, the squared error EMe(Xo+)', O+ o=Σ(B(xo, Yo
+ t)-B(xo, Vo+ t-1))2/Σ(B
(xo, yo, t))2・”(1) Furthermore, the background region determination circuit 106a determines whether the squared error EM8(XO, Vo, t) of the test pixel I (xo, yo, t) or a predetermined threshold (aE
For Meth, if EMe(Xo,)'o, t)≦EM[lth, the test pixel is determined to be included in the background area, and if EMII(XO,'io, t)>EMeth, the test pixel is determined to be included in the background area. The pixel is determined to be included in the moving area. and,
The background area determination circuit 106a outputs a determination result of "1" if it is determined to be a background region, and "o" if it is determined to be a moving region, to the background memory update circuit 106b.

According to the background region determination procedure described above, pixels around the test pixel are also taken into consideration when determining whether the test pixel belongs to a background region or a moving region. Characteristic portions indicating the background are more likely to be included in the signal used, and as a result, the effect of improving the accuracy of background area detection can be obtained.

Next, the background memory update circuit +06b of the background generation circuit is
The pixel value I(x, y, t) of the input image signal stored in the input frame memory 103 for the pixel determined to be the background area "1" by the background area determination circuit 106al.
and the pixel value BG (x, y, t-1) of the background image stored in the background memory 107, respectively, and update the pixel value of the background image in the background memory 107 according to equation (2) below. However, α in equation (2) is a value determined depending on the design within the range of 0<α<1.

8G (x, y, t) = BG (x, y, t-1) + a
X (I(x, y, t)-BG(x, y, t-
1))...(2) As is clear from equation (2), the update amount of the background image in the background memory is the value obtained by the second term in equation (2), that is, the input image and The value is proportional to the difference from the background image in the background memory. Therefore, the speed at which the background image converges to the true background image can be kept constant regardless of the difference in input images. Note that α in equation (2) is a parameter that adjusts the speed at which the background image stored in the background memory converges to the true background image. FIG. 5 shows how the convergence speed changes when α is changed. Here, in Figure 5,
The horizontal axis is the convergence time, expressed as the number of frames, and the vertical axis is the degree of convergence (%) (pixel value of the generated background image).
It is expressed as xlOO/(the pixel value of the true background image).

This concludes the description of the embodiments of the background image generation method and background image generation device of this application. However, these inventions are not limited to the above-described embodiments, and can be modified in various ways as described below.

In each of the embodiments described above, determination as to whether a test pixel belongs to a moving region or a background region is made based on the difference signal power between the input image signal and the past signal. However, this determination may also be made based on the amplitude of the difference signal.

(Effects of the Invention) As is clear from the above description, according to the background image generation method of this application, first, it is determined whether or not a test pixel belongs to a background area. This is performed using the input image signal of each pixel and the difference signal between this signal and the past signal.

Therefore, since characteristic portions indicating the background are likely to be included in the signal used to determine the background area, the background area can be detected with high accuracy.

Next, if it is determined that the test pixel belongs to the background area, the pixel value corresponding to this test pixel in the background image is added to the pixel value of the input image signal and the pixel value corresponding to the test pixel in the background image. Since the background image is updated by adding a value proportional to the difference between

Further, according to the background image generation device of the second invention of this application,
It becomes easy to implement the background image generation method of the first invention.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing a device suitable for implementing the background image generation method of the present invention, FIG. 2 is a diagram for explaining the notation method of image signals, and FIG. 3 is the internal configuration of the background generation circuit. FIG. 4 is a diagram for explaining the processing blocks used to determine the background area. FIG. 5 is a diagram for explaining the convergence speed of the background image. FIG. 6 is a diagram for explaining the convergence speed of the background image. It is a figure provided for explanation of a method. 101...TV camera, 102...A/D converter 103...Input frame memory 104...Delayed frame memory 105...Memory access circuit 06...Background generation circuit 106a...Background area Judgment circuit 106b...Background memory update circuit 07...Background memory, 00...Processing block. Patent applicant: Oki Electric Industry Co., Ltd.

Claims (2)

[Claims] (1) In a method of generating a background image by predicting a background area of an image including a moving area and a background area using an input image signal and a past signal of the input image signal, a certain test pixel Judgment as to whether the pixel belongs to a background area or a moving area is made using image signals of each of the test pixel in the input image signal and a predetermined pixel around the test pixel, and the test pixel in a past signal of the input image signal. and the difference between the image signal of each pixel corresponding to the predetermined pixel, and if it is determined that the test pixel belongs to the background area,
The background image is updated by adding a value proportional to the difference between the pixel value of the input image signal and the pixel value corresponding to the test pixel in the background image to the pixel value corresponding to the test pixel in the background image. A background image generation method characterized by: (2) In a device that generates a background image by predicting a background area of an image including a moving area and a background area using an input image signal and a past signal of the input image signal, the input image signal is An input frame memory for storing a past signal of the input image signal, a delayed frame memory for storing a past signal of the input image signal, a memory access circuit for cutting out a predetermined block of image signals from each of the input frame memory and the delayed frame memory, and storing a background image. determining a difference between the input image signal and the past signal;
The difference signal is compared with a predetermined threshold value to determine whether each pixel of the input image signal belongs to the moving area or the background area, and the pixel value of the pixel determined to belong to the background area in the background memory is determined. A background image generation device comprising: a background generation circuit that updates the pixel value with a value corresponding to the difference between the pixel value and the pixel value of the pixel in the input image signal.