JPH05292485A - Moving image encoding device - Google Patents

Abstract

PURPOSE:To provide a moving image encoding device which can prevent generating high frequency noise after scene changing for the scene changing or the like for the moving images. CONSTITUTION:For the moving images divided into blocks, this device is provided with a statistic calculating means 107 to calculate the statistic based on the information of the block under consideration, statistic calculating means 109 to calculate the statistic based on the result of the orthogonal transformation of the block motion predicted, and discriminating means 110 to discriminate by which method the encoding of the block under consideration is performed between intra-encoding and inter-encoding based on the statistics calculated at the two calculating parts.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture coding device.

[0002]

2. Description of the Related Art With the improvement of moving image coding technology, coding devices have been actively developed from various fields. The background to this is that the international standardization of coding methods is progressing. Above all, the specifications have been determined in the MPEG (Moving Picture Expert Group) system, which is intended for storage media.

In the same system, motion compensation is performed to reduce redundancy in the time axis direction, a difference between images is calculated, and then redundancy is reduced in the spatial axis direction, which is a kind of orthogonal transform.
CT (discrete cosine transform) and variable length coding are introduced.

Regarding the reduction of redundancy in the time axis method, a continuous moving image is displayed between preceding and following frames (or between fields).
The property of having a high correlation is used. That is, a motion vector is detected for each block between the image to be encoded (current image) and the images preceding and succeeding in time, and motion estimation is performed to predict the current image, Calculate the prediction error between Here, as shown in FIG. 9A, each image forming the moving image is an I picture (frame or intra-field coded image), P picture (forward prediction image), B picture (forward direction, After being classified into three types (backward and bidirectional prediction images), they are coded according to the order of FIG. 9B. On the other hand, for the redundancy in the spatial axis direction, variable length coding is performed after the prediction error signal of each image is quantized in DCT coefficients in block units.

The above-mentioned prediction signals are classified into four types: intra, forward prediction, backward prediction, and bidirectional prediction. When performing block coding, an optimal prediction method must be selected, but the details of the selection unit are outside the scope of standardization, and various methods have been proposed in various fields.
For example, there is a method described in JP-A-1-170276, which encodes an intra-block average value of prediction errors.

Further, in Japanese Patent Laid-Open No. 2-29180, there is a system in which a system with a small code amount is adaptively selected for each block from both a block coding system and a motion compensation system.

Alternatively, as one of the model cases proposed in the examination stage of the standardization method described above, SM3 is used.
Method (MPEG Simulation Model Three ISO-IEC / JTC / SC2 /
WG8Documents of Moving Picture Experts Group). The selection unit of the same system is configured by each selection element that determines whether or not motion compensation is performed, determines whether intra is valid or invalid, and determines a prediction direction when motion compensation is adopted.

For example, when the presence / absence of motion compensation is selected, as shown in FIG. 10, motion compensation is performed when the (X, Y) coordinate system is within the shaded area of the characteristic diagram. In the figure, BD is 16 * 1 between the current images (current image and previous image).
This is the sum of absolute differences in the 6-pixel block, and DBD is the sum of absolute differences between the block of the current image and the block after motion compensation.

On the other hand, when selecting the presence / absence of the intra mode, as shown in FIG. 11, (VAR, VAROR)
The intra mode is selected when the coordinate value is within the shaded area of the characteristic diagram. Here, VAR is the sum of squared differences between frames regarding a block composed of 16 * 16 pixels,
VAROR represents the variance of pixel values within a frame for the same block. A prediction signal is created for each block based on the selection method and the like described above.

[0010]

However, in the above SM3 method, an incorrect prediction method may be selected from a visual point of view when a scene change of a moving image is performed. For example, in FIG. 9A, the fourth P image with many high-frequency components (hereinafter abbreviated as P3 image).
When the scene change is performed between the B-picture and the fifth B-picture (hereinafter, abbreviated as B4 picture), the motion compensation mode is selected for the block of the flat picture part during the coding of the B4 picture. It may end up.

This is because the P3 image before the scene change has a small correlation due to the scene change, so the B5 image is encoded from the P3 image by the information of only the image predicted in the forward direction including the error due to the scene change. This is due to the fact that the block having the flat portion is set to a method for making a determination that the non-intra mode is easily selected. As a result, when the prediction error signal created from the current input image and the prediction signal by motion compensation has a value equal to or less than a predetermined threshold value, the information that becomes a flat image in the quantization is lost, and FIG.
As is clear from the comparison between (a) and (b), when decoding, a non-flat image (harmonic noise) occurs in the block. In addition, FIG.
In (b), the horizontal axis represents the macroblock number and the vertical axis represents the variance.

The moving picture coding apparatus of the present invention has been made in view of such a problem, and its purpose is to prevent generation of high frequency noise after a scene change in a scene change in a moving picture. An object of the present invention is to provide a moving picture coding device that can prevent such a situation.

[0013]

In order to achieve the above object, a moving picture coding apparatus according to the present invention uses a current picture to be coded among moving pictures divided into blocks in a frame or field picture. The target block is encoded using the intra-coding unit that performs the coding using only the correlation within the image and the correlation between the blocks predicted from the coded reference image. Any one of the intra coding means and the inter coding means based on an inter coding means, a statistic based on the information of the block of interest, and a statistic based on an orthogonal transformation result of the motion-predicted block. And selecting means for selecting and encoding the block of interest.

[0014]

That is, in the moving picture coding apparatus of the present invention, based on the statistical amount based on the information of the target block and the statistical amount based on the orthogonal transformation result of the motion predicted block in the moving image divided into blocks. Then, either the intra coding or the inter coding is selected and the block of interest is coded.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a moving picture coding apparatus according to the present invention will be described below with reference to the drawings. FIG. 1 shows a first embodiment of the present invention. When a moving image is input to the blocking unit 101 frame by frame, each image is divided into blocks of m * n pixels and sequentially output. The intra / inter switching means 102 is configured to subtract the subtractor 11 for each block.
1 and the difference signal between the predicted image signal from the motion prediction unit 106 and the current image signal, and the blocking unit 1
One of the current image signals output from 01 is selected based on the control signal from the determination means 110. Therefore, the intra / inter switching means 102 and the determination means 110 form a selection means.

The orthogonal transformation means 103 obtains transformation coefficients by orthogonally transforming the image signal of m * n block size from the intra / inter switching means 102, and as a specific example thereof, a KL transformation (Karhunen-Loeve) is used. conversion),
DCT (Discrete Cosine

Transform) and the like.
Thereafter, the orthogonal transform coefficient is input to the compression means 104,
Quantization and Huffman coding as represented by the scene adaptive coding proposed by Chen and Pratt are performed, and the code is output by assigning a code word.

The encoded signal is output to the transmission path as a code word, while the quantized value of the transform coefficient quantized by the compression means 104 is input to the local decoding means 105. In the same section, the orthogonal transform coefficient obtained by dequantizing the quantized value is restored, and further, the orthogonal transform coefficient is inversely transformed, so that the image is finally locally generated in the encoder. It will be reproduced.

The motion predicting means 106 is supplied with the locally decoded image signal for the already processed image and the current image signal from the blocking means 101, detects a motion vector from both signals, and follows the motion vector. Image motion prediction is performed. The motion prediction signal is input to the subtractor 111 described above. The motion prediction signal from the motion prediction means 106 is also supplied to the orthogonal transformation means 108. The transform coefficient converted by the orthogonal transform unit 108 is calculated by the statistic calculation unit 109 to obtain a statistic indicating the property of the image, and the statistic is output to the determination unit 110.

On the other hand, in the statistic calculation means 107,
A statistic indicating the property of the current image that has been blocked is obtained and output to the determination unit 110. Here, each statistic indicates the flatness and fineness of the image.

The decision means 110 decides which of the intra method and the inter method the coding of the target block is based on the statistic amount of the current image signal and the statistic amount of the motion prediction signal which are divided into blocks. , And outputs the control signal to the intra / inter switching means 102 to control the encoding system. That is, in this embodiment,
Only when the image extremely changes to a fine image for each block such as a scene change, it is predicted that the flat portion will be distorted based on the above two statistics, and the control signal is output. As a result, the intra / inter switching means 102
Operates so as to adopt the current image signal from the blocking means 101 without using motion compensation.

As is apparent from the operation of the first embodiment described above, when a scene is changed in a moving image, the noise of the flat portion which is easily perceived by humans and is generated in the image after the scene change is predicted, and the noise is not generated. As described above, since the processing is performed by the intra method, image quality deterioration can be suppressed. In addition, a block with harmonic noise has an AC component as well as a DC component, so the code amount of the block inevitably increases, but if the block is encoded by the intra method, it will be a flat block. For example, the code amount can be small because it has only the DC component. In this way, the amount of information in the flat portion is not increased more than necessary, so that there is also an effect of keeping the code amount low. Although the frame image has been described in the description of the operation of the first embodiment, the same can be said for the field image. A second embodiment of the present invention will be described below with reference to FIG.

The frame image forming the moving image is once 1
It is input to the image memory 201 frame by frame. afterwards,
The subtraction unit 202 calculates the predicted image signal x ′ from the original image signal x.
Is subtracted, and the subtraction unit 202 outputs a prediction error signal ε. The DCT unit 203 performs discrete cosine transform on the signal, and the quantizing unit 204 quantizes the DCT coefficient in a state in which the DCT coefficient is regulated to a certain step size. At this time, the step size is the variable length coding unit (VLC) 20.
5 and information from the buffer memory unit 206 are optimized. After that, the quantized DCT coefficient is assigned a code word by the variable-length coding unit 205 by, for example, Huffman coding, the same code word is input to the buffer memory 206, and the transfer rate on the transmission path becomes constant. Are sequentially output.

Quantized DCT from quantizer 204
The coefficient output signal is input not only to the variable length coding unit 205 but also to the inverse quantization unit 207. After that, in the inverse DCT unit 208, the inverse discrete cosine transform is performed and the prediction error signal ε ′ is output. Furthermore, the addition unit 20
In 9, the same signal ε ′ and the predicted image signal x ′ are added to create local decoding. Here, there is a locally decoded signal as a signal required to obtain a prediction signal, and the current image in the input image to be encoded from now on (hereinafter abbreviated as current image, the image is a frame image or field). Image), a past image (hereinafter abbreviated as a front image) and a future image (hereinafter abbreviated as a rear image) can be considered.

In the vector detection unit 215, the image memory 2
The current image (original image) stored in 01 and the image memory 21
1 between the previous image stored in 1 and the subsequent image stored in the image memory 212 (both locally decoded images), m *
A motion vector between images is detected for each block composed of n pixels by a block matching method or the like. Therefore,
When block matching is performed between the current image and the previous image and between the current image and the subsequent image, two motion vectors are obtained.

The MC unit 213 performs motion compensation by forward prediction, backward prediction, or bidirectional prediction based on the information of the previous image, the latter image, which is the locally decoded image, and the motion vector detected by the vector detection unit 215. The intra block is about to be selected, and is the motion compensation unit based on the SM3 method that has been proposed in the past. Further, the changeover switch 214 is a place for selecting the predicted image information for predicting the current image output from the MC unit 213 or the current image.

In the DCT section 216, for the block information of the pixel value of m * n pixels sent from the image memory 201,
A two-dimensional DCT operation is performed. The output of the calculation unit is
It is input to the determination unit 217, and the flatness of the image is confirmed.
That is, as shown in FIG. 4, after performing zigzag scanning within a block, the variance σ ² regarding the DCT coefficient as shown in FIG. m is compared to a threshold Th2, σ ² When m> Th2, the DCT coefficient has an AC component, so that the image is determined to be non-flat. Therefore, the flag signal Fg2 = 0 is output so that the subsequent processing for noise removal is not performed. Alternatively, σ ² When m ≤ Th2, the DCT coefficient has only a direct current component or an alternating current component that is very close to a direct current component, so that the image is determined to be flat and Fg2 = 1 is output.

The DCT unit 218 operates only when the flag signal output from the determination unit 217 is Fg2 = 1. Under the same conditions, the predicted image created by the MC unit 213 is DCT.
The data is input to the unit 218, the two-dimensional DCT operation is performed, and the same coefficient is output. Further, in the determination unit 219, the determination unit 21
Variance σ ² of the two-dimensional DCT coefficient for the current image from 7 after zigzag scanning m and the variance σ ² of the two-dimensional DCT coefficient regarding the predicted image from the DCT unit 218 after the zigzag scan k is output, and | σ ² m − σ ² k | <Th3

When is true, it is determined that noise does not occur in the flat portion of the predicted image, the changeover switches 214 and 222 are connected to the S1 side, respectively, and the predicted image is sent from the MC unit 213. On the other hand, when the conditional expression is false,
It is determined that noise has occurred, the changeover switches are respectively connected to the S2 side, and the information of the original image is sent from the image memory 201.

As is clear from FIG. 5, when the characteristic A of the MC image having low frequency components to high frequency components shifts to the characteristic B of the current image having direct current or low frequency components, noise is generated. To be judged.

Further, in the judging units 217 and 219,
Although the variance of the pixel value and the variance of the DCT coefficient after the zigzag scan are described, other statistics can be applied. For example, the sum of absolute differences of DCT coefficients in the same unique vector, the comparison by the sum of squared differences, the comparison by the number of non-zero coefficients of the quantized values of DCT coefficients, or the activity (AC component power excluding DC component of image block). The same operation for preventing noise generation can be performed by using a statistical amount such as a comparison based on the definition of the image definition).

As shown in FIG. 6, the DCT value aij (FIG. 6) of the current image is displayed at the pixel position of the m * n size block.
(A)) and the DCT value bij of the MC image (Fig. 6 (b))
Is given. Here, focusing on the sum of absolute differences, when ΣΣ | aij-bij | ≧ Th4 (0 ≦ i ≦ m−1, 0 ≦ j ≦ n−1) is true, it is determined that noise has occurred, The operation is such that encoding is always performed.

According to the present embodiment, when the scene change from the image with many high frequency components in FIG. 8A to the image with almost DC components in FIG.
It should be added that when 6 * 16, Th2 = 5, Th3 = 1, and Th4 = 80, it was possible to prevent noise generation and good experimental results were obtained. Alternatively, if attention is paid to the sum of squared differences, ΣΣ (aij-bij) ² When ≧ Th5 (0 ≦ i ≦ m−1, 0 ≦ j ≦ n−1) is true, or when attention is paid to the activity, ΣΣbij ² −ΣΣaij ² When ≧ Th6 (0 ≦ i ≦ m−1, 0 ≦ j ≦ n−1) is true, it is determined that noise has occurred, and the intra-encoding is performed without fail.

Similarly, when focusing on the number of non-zero coefficients,
As shown in FIG. 7, assuming that the number of non-zero coefficients of the characteristic A in the MC image is C1 and the number of non-zero coefficients of the characteristic B in the current image is C2, noise is generated when C1-C2 ≧ Th7 is true. Is determined, and the operation is performed such that intra coding is always performed.

As described above, according to the present invention, the noise appearing in the flat portion of the image after the scene change is predicted, and when it is judged that the noise is generated, the intra coding system is switched to, so that the image quality deterioration can be suppressed. .. Although the frame image is described in the description of the operation of the present embodiment, the same can be said for the field image.

FIG. 3 shows a third embodiment of the present invention. The moving image (frame image or field image) is once input to the image memory 301. After that, the subtraction unit 302
At, the predicted image signal x'is subtracted from the original image signal x,
The subtraction unit 302 outputs the prediction error signal ε. The DCT unit 303 performs discrete cosine transform on the signal, and the quantizing unit 304 quantizes the DCT coefficient in a state in which the DCT coefficient is regulated to a certain step size. At this time, the step size is optimized by the information from the variable length coding unit 305 and the buffer memory unit 306. afterwards,
The quantized DCT coefficient is the variable length coding unit (VLC) 3
In 05, a code word is assigned by, for example, Huffman coding, the same code word is input to the buffer memory 306, and sequentially output so that the transfer rate on the transmission path becomes constant.

The output signal of the quantized DCT coefficient from the quantization 304 is input not only to the variable length coding unit 305 but also to the dequantization unit 307. afterwards,
The inverse DCT unit 308 outputs the prediction error signal ε'which is subjected to the inverse discrete cosine transform. Furthermore, the adder 309
At, the same sign ε ′ and the prediction error signal x ′ are added to create a locally decoded signal. Here, there is a locally decoded signal as a signal required to obtain a prediction signal, and the current image in the input image to be encoded from now on (hereinafter abbreviated as current image, the image is a frame image or field). Image is shown.)
A front image and a future image (hereinafter abbreviated as a rear image) can be considered.

In the vector detection unit 315, the image memory 3
The current image (original image) stored in 01 and the image memory 31
Between the previous image stored in 1 or the subsequent image stored in the image memory 312 (both locally decoded images),
A motion vector between images is detected for each block composed of m * n pixels by a block matching method or the like. Therefore, if block matching is performed between the current image and the previous image and between the current image and the subsequent image, two motion vectors are obtained.

The MC unit 313, on the basis of the front image, the rear image, which is the local image, and the information of the motion vector detected by the vector detection unit 315, performs forward prediction, backward prediction, motion compensation by intra-directional prediction, or intra-prediction. The block is about to be selected, and the previously proposed S
This is a motion compensation unit based on the M3 formula. Further, the changeover switch 314 is a place for selecting predicted image information for predicting the current image output from the MC unit 313 or current image.

On the other hand, the variance calculation unit 316 receives the information of the current image (original image) from the image memory 301, and the variance σ ^{2 of} pixel values for each block of m * n pixels. Calculate c. Then the variance σ ² c is input to the determination unit 317,
Compared with threshold Th1. That is, σ ² When c> Th1, the block is determined to be a non-flat image, and the flag signal Fg1 = 0 is output. Alternatively, σ ² When c ≦ Th1, the block is determined to be a flat image, and the flag signal Fg1 = 1 is output. Here, Th1 is zero or a value relatively close to zero.

The DCT section 318 operates in response to the result of the flag signal Fg1 from the judging section 317. That is, F
When g1 = 0, the image is not a flat image that requires noise removal, and therefore the DCT calculation shown below is not performed. Further, under the same condition, not only the operation of the DCT unit 318 but also the operations of the determination unit 319, the DCT unit 320, and the determination unit 321 are not performed.

When Fg1 = 1, the result of calculating the variance of pixel values may be a flat image, so confirmation is performed by orthogonal transformation. That is, the block information of the pixel value of m * n pixels is sent from the image memory 301, and the DCT unit 318 outputs the block information.
A dimensional DCT operation is performed. The output of the calculation unit is input to the determination unit 319 and the flatness of the image is confirmed.

That is, as shown in FIG. 4, after performing zigzag scanning within a block, the variance σ ^{2 of the} characteristic B relating to the DCT coefficient as shown in FIG. m is the threshold
Compared with Th2, σ ² m ＞ Th2

At this time, since the DCT coefficient has an AC component, it is determined that the image is non-flat. Therefore, the flag signal Fg2 = 0 is output so that the subsequent processing for noise removal is not performed. Alternatively, σ ² When m ≤ Th2, the DCT coefficient has only a direct current component or an alternating current component that is very close to a direct current component, so that the image is determined to be flat and Fg2 = 1 is output.

The DCT section 320 operates only when the flag signal Fg2 = 1 output from the determination section 319. In the case of the same condition, the predicted image created by the MC unit 313 is input to the DCT unit 320, a two-dimensional DCT operation is performed, and the same coefficient is output. Furthermore, in the determination unit 321, the determination unit 319
Variance σ ² of the two-dimensional DCT coefficient after the zigzag scan for the current image from m and the variance σ ² of the two-dimensional DCT coefficient for the predicted image from the DCT unit 320 after the zigzag scan k is output, and | σ ² m − σ ² k | <Th3

When is true, it is determined that noise does not occur in the flat portion of the predicted image, the changeover switches 314 and 322 are respectively connected to the SI side, and the predicted image from the MC unit 313 is sent. On the other hand, when the conditional expression is false, it is determined that noise has occurred, the changeover switches are respectively connected to the S2 side, and the information of the original image is sent from the image memory 301.

In the decision units 317, 319 and 321, the variance of the pixel values and the variance of the DCT coefficients after the zigzag scan are described, but the point that they can also be applied to other statistics. , As already mentioned in the second embodiment.

As is apparent from the above description of the operation, in the present embodiment, the DCT operation is performed only on the block of m * n pixels which is considered to be the flat portion in the current image (original image), and the DCT operation is performed in the predicted image. About block DC
When the T operation is performed, the operation is limited to the blocks that are considered to be flat in the image related to the original image, so that the processing efficiency is improved as compared with the second embodiment.

According to the embodiment of the present invention described above, the noise appearing in the flat portion of the image after the scene change is predicted,
When it is determined that noise is generated, the intra-coding method is switched to, so that deterioration in image quality can be suppressed. Further, it is needless to say that noise is not generated in the flat portion, so that the code amount can be suppressed.

[0050]

As is apparent from the above, according to the present invention, when a scene change occurs in a moving image, noise of a flat portion which is easily perceived by humans when it occurs in the image after the scene change is predicted and the noise is not generated. Since the intra-coding method is used as described above, it is possible to suppress image quality deterioration. Moreover, since the amount of information in the flat portion is not increased more than necessary, the code amount can be suppressed low.

[Brief description of drawings]

FIG. 1 is a circuit configuration diagram showing a first embodiment of the present invention.

FIG. 2 is a circuit configuration diagram showing a second embodiment of the present invention.

FIG. 3 is a circuit configuration diagram showing a third embodiment of the present invention.

FIG. 4 is a diagram illustrating a zigzag scan within a block.

FIG. 5 is a characteristic diagram showing a relationship between a scan index (SI) and a DCT value.

FIG. 6 is a DCT value aij of the current image and a DCT value of the MC image
It is a figure which shows the pixel position of the block of m * n size to which the value bij was given.

FIG. 7 is a diagram showing characteristics of an MC image and a current image in comparison.

FIG. 8 is a diagram showing an image with many high-frequency components and an image with almost only DC components.

[Fig. 9] Fig. 9 is a diagram showing moving images classified into three types of I pictures, P pictures, and B pictures.

FIG. 10 is a characteristic diagram showing the relationship between the sum of absolute differences of the original image and the sum of absolute differences of the current image.

FIG. 11: Variance of pixel value in frame and difference between frames 2
It is a characteristic view which shows the relationship with a multiplication sum.

FIG. 12 is a diagram for explaining generation of high frequency noise in a conventional device.

[Explanation of symbols]

101 ... Blocking means, 102 ... Intra / inter switching means, 103 ... Orthogonal transformation means, 104 ... Compression means, 105 ... Local decoding means, 106 ... Motion prediction means,
107 ... Statistical amount calculation means, 108 ... Orthogonal transformation means, 10
9 ... Statistics calculating means, 110 ... Judging means.

Claims (1)

[Claims] 1. An intra code for encoding a block of interest in a current image to be encoded among moving images divided into blocks in a frame or field image using only the correlation in the image. Coding means, an inter coding means for coding a block of interest using the correlation between blocks predicted from a coded reference image, a statistic based on the information of the block of interest, and a motion-predicted block And a selecting means for selecting one of the intra-encoding means and the inter-encoding means on the basis of the statistic based on the result of the orthogonal transformation of (1) to encode the block of interest. Video coding device.