JP2621341B2 - Predictive coding method for video using background prediction and its apparatus - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a prediction encoding method and a prediction method using a background image in a series of moving images typified between a scene change and the next scene change. The present invention relates to an encoding device.

(Prior Art) Conventionally, as a high-efficiency coding method for a moving image, a predictive coding method is known. An example of the constituent circuit is shown in FIG. In this method, a prediction value 38 obtained by utilizing inter-frame and intra-frame correlations of an image is subtracted from pixel data 37, and the difference value 39 is encoded. The circuit 32 and the background prediction circuit 33 are connected in parallel, switched, and selectively connected by a switch 34. As the switching selection method, a method of sequentially selecting a mode in which the prediction difference is the smallest for each pixel and a method of estimating the selection mode of the current pixel from the prediction difference minimum mode in the already coded pixel are known. I have. The reason for adopting the multi-mode selection method in this way is that the inter-frame prediction is efficient because the inter-frame correlation is high in a normal television signal, but the inter-frame correlation of the image is reduced due to a scene change or a severe movement. In this case, it is more efficient to perform intra-frame predictive encoding using intra-frame correlation of an image or background predictive encoding. In particular, as for the image portion appearing from behind the moving subject, both the correlation between the frames and within the frame are small. Therefore, the prediction is performed using the background image excluding the moving subject portion as much as possible.

As a method for extracting the background image as described above, Japanese Patent Application No. Sho 62
Some are proposed by US Pat. This method utilizes the fact that the signal level of the pixel in the portion hardly changes and shows a flat region in the time axis direction during a period in which a certain portion in the image displays the background. For a series of moving images between a scene change and the next scene change, a flat region in which the signal level of each pixel lasts for a certain period or more is extracted as background data, and over the entire series of images between scene changes. , A background image is to be generated from the extracted background data. FIG. 5 is an example of the flowchart, and the operation will be briefly described below.

First, in step (03), it is determined whether the signal level of each pixel in the input image frame is in a flat area. For example, when the signal level of the ith pixel in the image frame forms the flat area Ai (m) in the period from the j-th frame to the (k-1) -th frame, the signal in the k-th frame If the difference value δ between the level Xi (k) and the average signal level in the flat area Ai (m) is smaller than a certain critical value, in step (04), Xi (k) is compared with the data of the flat area Ai (m). At the same time, it is added to the accumulated data Si of the i-th pixel. Otherwise, level Xi in step (05)
Let (k) be the initial value of the next flat area. (03) ~
The process of (05) is repeated for all the pixels in the image in a sequence of a series of image frames, and all the flat areas Ai (1) to Ai (n) of each pixel and the accumulated data Si thereof are obtained. . Next, at step (07), the stored data Si
Then, the background level Ai of each pixel is calculated as the average value of, and it is determined in step (08) whether the estimation of the background level Ai is appropriate. As a criterion for the determination here, the magnitude of variance of individual data in the accumulated data Si used for the estimation of Ai is used. If the estimation of Ai is judged to be inappropriate, in step (09), all the data included in the flat area Ai (l) having the largest level difference from Ai is excluded from the accumulated data Si, and the step is repeated. (07) Perform the following processing.

A background image extracted from a series of moving images by the above method is stored in advance in a background memory in a background prediction circuit, and is used for background prediction for the series of moving images.

(Problems to be Solved by the Invention) In a moving image for a fixed period in which the framework is fixed, it is expected from a subjective evaluation that most of the background in the image is well predicted using the background image. You. However, in the conventional predictive coding, prediction between frames and within a frame is used in many cases even in a background portion, and still more information is generated as a prediction difference. This is for selecting a prediction mode based on the magnitude of the prediction difference in each mode. That is, since the background image used for background prediction is of an average character extracted from a series of moving images for a certain period, the background image includes a case where a gradual luminance fluctuation occurs in the background portion between frames. In many cases, the inter-frame prediction has a smaller prediction difference than the background prediction, and generates some prediction difference. In addition, the intra-frame prediction often has a smaller prediction difference than the background prediction even for a background portion where the inter-frame correlation is reduced due to a change due to noise or the like.

An object of the present invention is to reduce the amount of generated information at least subjectively by performing prediction using a background image as much as possible for a background portion in a series of moving images within a certain period. An object of the present invention is to provide a predictive coding method and a predictive coding device.

(Means for Solving the Problems) The moving picture prediction encoding method according to the present invention extracts a background image from each frame of a series of moving pictures within a fixed period in advance, and uses the extracted prediction image for background prediction. In the conversion method, information indicating which pixel of which frame was used to generate the background image at the time of extracting the background image is generated,
At least based on the information, a pixel to which background prediction is applied is determined, and a prediction difference value of the pixel to which background prediction is applied is set to zero. The apparatus is configured to extract and output a background image from each frame of a moving image within a predetermined period, and to output background extraction pixel information indicating which pixel of which frame was used for generation of the background image, and a background image extraction unit, Background prediction mode control means for storing background extraction pixel information, and determining a pixel to which background prediction is applied based on at least this, and outputting a background prediction mode selection signal in synchronization with the prediction encoding of the pixel, A predictor for selecting a background prediction from among a plurality of prediction means based on the background prediction mode selection signal; and a zero-level output circuit for forcibly setting a prediction difference signal to zero based on the signal. It is characterized in that at least comprising: a predictive coding means.

(Effect) In the present invention, in each frame of a moving image within a certain period, even if a portion determined to be a background with little change as judged from the entire series of frames is replaced with a predicted background image, there is little subjective evaluation. This is to take advantage of the fact that deterioration does not occur.

In the present invention, first, a background image is extracted from a series of moving images within a certain period represented by a scene change and an image sequence surrounded by the scene change by a method as shown in the related art (FIG. 5). . Next, when predictive coding of each pixel is performed, at least a pixel to which background prediction is applied based on background extraction pixel information indicating which pixel of which frame was finally used for generation of a background image in the extraction processing To determine. This makes it possible to apply background prediction to most of the background that shows only noise and small signal level fluctuations. Further, by setting the prediction difference value to zero for each pixel to which the background prediction has been applied, the amount of information generated in the background portion in the image can be reduced. The selection of the background prediction in this case is performed during a period in which the fluctuation of the signal level of the pixel can be considered to be small and stable after looking over the entire series of frames as described above. The deterioration of the image quality due to this can be kept low.

(Example) Hereinafter, an example of the present invention will be described with reference to FIGS. FIG. 1 is a basic block diagram of one embodiment for realizing the predictive coding system of the present invention. First, a background image extraction circuit 1 is applied to each frame of a series of input moving images for a given period.
Extracts a background image. The extraction method is performed after the entire frame within the period is overlooked.
A method as shown in the figure is used. The extracted background image 41 is sent to the predictive coding circuit 3 prior to the predictive coding. At the same time, background extraction pixel information 42 indicating which pixel of which frame was used for background image generation at the time of background image extraction
Is sent to the background prediction mode control circuit 2. Next, the prediction encoding circuit 3 encodes the difference between the input image signal 40 and its prediction value. When selecting the prediction mode of the pixel indicated by the background extraction pixel information 42, the background prediction mode control circuit 2 The background prediction mode selection signal 43 is supplied in synchronization with the conversion circuit 3, whereby the background prediction mode is applied. Further, the prediction encoding circuit 3 forcibly sets the prediction difference to zero when the background prediction mode is selected.

FIG. 2 is a block diagram showing one embodiment for realizing a background image extracting circuit according to the present invention. In this embodiment, the background image extraction method shown in FIG. 5 of the related art is used. First, in the flat area detection circuit 4, steps (01) to (01) in FIG.
By the method of (05), all flat regions of each pixel are detected from a series of image frames, and supplied to the background image generation circuit 5. In the background image generation circuit 5, the steps (06) to (09) in FIG.
The background level of each pixel in the signal in the flat area
The background image 41 is calculated from 45 and output. In addition, the portion of each flat region that is not excluded in the step (09), that is, the flat region finally used for calculating the background level, includes which pixel of which frame was included in the background extraction pixel information 42. Output as

FIG. 3 is a block diagram of an embodiment of a moving picture coding / recording / reproducing system using the predictive coding means in the present invention. Here, the background image 41 generated in advance is supplied to the background prediction circuit 11 before encoding a series of image signals. A background prediction mode selection signal 43 is supplied to the prediction mode control circuit 12 from the background prediction mode control circuit 2 in FIG. 1 in accordance with the background extraction pixel information 42 in synchronization with the encoding. First, for the input image signal 40, the predictor 8 uses the intra-frame prediction circuit 9, the inter-frame prediction circuit 10, and the background prediction circuit.
A prediction value 46 is obtained from each of the 11 values. The prediction mode control circuit 12 compares each prediction value 46 with the input image signal 40, selects a prediction mode, and determines the prediction value 47 by the switch circuit 13. However, the background prediction is selected only when the background prediction mode selection signal 43 is supplied. That is, the background prediction mode selection signal 43 is used to force the background prediction to be used for the pixel indicated by the background prediction mode selection signal 43, and the other pixels are selected from among the inter-frame prediction or the intra-frame prediction, whichever has the smaller prediction difference. Next, a difference value between the input image signal 40 and the prediction value 47 is calculated by the subtractor 14. When the background prediction is selected at the time of selecting the prediction mode, the switch circuit 15 is selected.
Is connected to the zero-level output circuit 16 to make the predicted difference value zero.
After the prediction difference value 48 is quantized by the quantization circuit 17, it is encoded by the code conversion circuit 20, while being inversely quantized by the inverse quantization circuit 18 and added to the prediction value by the adder 19 to obtain a frame. It is supplied to the intra prediction circuit 9 and the inter-frame prediction circuit 10. The mode signal 49 is encoded by the code conversion circuit 20, and is supplied to the decoder 7 via the data recording medium 21 as an encoded output 44 together with the encoded quantized prediction difference value. Also background image
41 is also coded by intra-frame predictive coding or the like, supplied to the decoder 7 via the data recording medium 21, restored to the original background image 51, and used by the predictor 22 for background prediction.

Note that the background prediction mode selection signal 43 does not necessarily have to completely correspond to the background extraction pixel information 42, and the pixel configuration to output the background prediction mode selection signal 43 from the distribution of the background extraction pixel information 42 in the same frame. May be determined. For example, it is possible to ignore an isolated background extraction pixel and incorporate a non-background extraction pixel surrounded by the background extraction pixel into the background extraction pixel. In addition, a method of ignoring background extraction pixels isolated between frames and incorporating non-background extraction pixels that are background extraction pixels in the preceding and succeeding frames into background extraction pixels may be combined.

Further, also in the method of selecting a prediction mode in units of blocks composed of a plurality of pixels, if data is extracted in units of blocks in the background image extraction method, background extraction pixel information can be obtained in units of blocks. Further, as described above, the background extraction pixel information obtained in pixel units may be reconstructed so that background prediction is selected in block units. When background prediction is selected for each block, the prediction difference values of all pixels in the block are set to zero.

(Effect of the Invention) In the present invention, the prediction difference is set to zero when the background prediction is selected, so that the amount of information generated by the difference signal can be suppressed. Also, the background prediction selection is applied to the pixels used for generating the background image at the time of extracting the background image in preference to the other modes, so that it is possible to increase the selectivity of the background prediction mode and promote the above effect. it can. Further, for the above reason, since the background prediction is applied to pixels determined to be a background portion having little inter-frame variation in signal level after looking over the entire series of image frames, the prediction difference value is set to zero. The error due to is not cumulatively expanded between frames, does not cause large inter-frame distortion when the prediction mode is switched, and can suppress the image quality deterioration in the subjective evaluation.

[Brief description of the drawings]

FIG. 1 is a basic block diagram of an embodiment of a predictive coding method according to the present invention, FIG. 2 is a block diagram illustrating an embodiment of a background image extracting means according to the present invention, and FIG. FIG. 4 is a block diagram showing an embodiment of a moving picture coding / recording / reproducing system constituted by predictive coding means, FIG. 4 is a block diagram showing an example of a conventional moving picture predictive code decoding system, FIG. 9 is a flowchart illustrating an example of a conventional background image extraction method. In the figure, 1 ... a background image extraction circuit, 2 ... a background prediction mode control circuit, 3 ... a prediction coding circuit, 4 ... a flat area detection circuit,
5 ... background image generation circuit, 6 ... encoder, 7 ... decoder, 8, 22, 30 ... predictor, 9, 31 ... intra-frame prediction circuit, 10, 32 ... inter-frame prediction circuit, 11, 33: background prediction circuit, 12: prediction mode control circuit, 13, 15, 34 ... switch circuit, 14, 35 ... subtractor, 16 ... zero level output circuit, 1
7 quantization circuit, 18 inverse quantization circuit, 19, 36 adder, 20 code conversion circuit, 21 data recording medium.

Claims (2)

(57) [Claims] A background image is extracted in advance from each frame of a series of moving images within a fixed time, and in a moving image predictive encoding method used for background prediction, which frame of which frame is extracted when the background image is extracted. Information indicating whether a pixel signal has been used for background image generation is generated, a pixel to which background prediction is applied is determined based on at least the information, and a prediction difference value of the pixel to which background prediction is applied is set to zero. A predictive coding method. 2. A background image extracting means for extracting and outputting a background image from each frame of a moving image within a predetermined period, and outputting background extraction pixel information indicating which pixel of which frame was used for generating the background image. Background prediction mode control for storing the background extraction pixel information and determining a pixel to which background prediction is applied based on at least the background extraction pixel information, and outputting a background prediction mode selection signal in synchronization with the prediction encoding of the pixel Means, a predictor for selecting a background prediction from among a plurality of prediction means based on the background prediction mode selection signal, and a zero-level output circuit for forcibly setting a prediction difference signal to zero based on the signal. A predictive coding apparatus for a moving image, comprising: