JP2013098713A - Video encoding device, video decoding device, video encoding method, and video decoding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video encoding device, a video decoding device, a video encoding method, and a video decoding method that are capable of enhancing image quality by reducing a locally generated prediction error.SOLUTION: An intra-prediction unit 4 selects a predetermined filter from among one or more prepared filters, performs filter processing on a block boundary in a prediction image generated by performing intra-frame prediction processing using the filter, and outputs the prediction image after the filter processing to a subtraction unit 6.

Description

  The present invention relates to a moving image encoding apparatus and moving image encoding method for encoding a moving image with high efficiency, a moving image decoding apparatus and a moving image decoding method for decoding a moving image encoded with high efficiency, and It is about.

  For example, in an international standard video encoding method such as MPEG (Moving Picture Experts Group) or “ITU-T H.26x”, an input video frame is divided into rectangular blocks (encoded blocks), and the encoded blocks are divided. In addition, a prediction image is generated by performing a prediction process using an encoded image signal, and a prediction error signal that is a difference between the encoded block and the prediction image is subjected to orthogonal transform or quantization processing in units of blocks. Therefore, information compression is performed.

For example, MPEG-4 AVC / H. H.264 (ISO / IEC 14496-10 | ITU-T H.264) performs intra prediction processing from encoded neighboring pixels or motion compensation prediction processing between adjacent frames (for example, Non-Patent Document 1). See).
MPEG-4 AVC / H. In H.264, in the luminance intra prediction mode, one prediction mode can be selected from a plurality of prediction modes for each block.
FIG. 16 is an explanatory diagram showing an intra prediction mode in the case where the luminance block size is 4 × 4 pixels.
In FIG. 16, white circles in the block represent pixels to be encoded, and black circles represent encoded pixels that are pixels used for prediction. When the luminance block size is 4 × 4 pixels, nine intra prediction modes from mode 0 to mode 8 are defined.

In FIG. 16, mode 2 is a mode in which average value prediction is performed, and the pixels in the block are predicted by the average value of the adjacent pixels above and to the left of the block.
Modes other than mode 2 are modes in which directionality prediction is performed. Mode 0 is vertical prediction, in which a prediction image is generated by repeating adjacent pixels on the block in the vertical direction. For example, mode 0 is selected for a vertical stripe pattern.
Mode 1 is horizontal prediction, in which a prediction image is generated by repeating the adjacent pixel on the left of the block in the horizontal direction. For example, mode 1 is selected for a horizontal stripe pattern.
In modes 3 to 8, a predicted image is generated by generating interpolation pixels in a predetermined direction (the direction indicated by the arrow) using the encoded pixels on the top or the left of the block.

Here, the luminance block size to which the intra prediction is applied can be selected from 4 × 4 pixels, 8 × 8 pixels, and 16 × 16 pixels, and in the case of 8 × 8 pixels, 4 × 4 pixels. Similarly, nine intra prediction modes are defined. However, the pixels used for prediction are not encoded pixels themselves but those obtained by performing filter processing on these pixels.
On the other hand, in the case of 16 × 16 pixels, in addition to intra prediction modes related to average value prediction, vertical direction prediction, and horizontal direction prediction, four intra prediction modes called “plane prediction” are defined.
The intra prediction mode related to the Plane prediction is a mode in which a pixel generated by interpolating the upper and left encoded adjacent pixels in the block in an oblique direction is a prediction value.

In the intra prediction mode in which the directionality prediction is performed, for example, a prediction value is generated in a direction predetermined by the mode such as 45 degrees. Therefore, the direction of the boundary (edge) of the object in the block is the direction indicated by the prediction mode. If they match, the prediction efficiency increases and the amount of codes can be reduced.
However, if there is a slight deviation between the direction of the edge and the direction indicated by the prediction mode, or the edge in the encoding target block is slightly distorted (fluctuates, bends, etc.) even if the directions match. A large prediction error may occur locally, and the prediction efficiency may be extremely reduced.
In order to prevent such a decrease in prediction efficiency, in the 8 × 8 pixel direction prediction, smoothing is performed by performing a prediction process using a smoothed process performed on an encoded adjacent pixel. The prediction image generated is generated, and the prediction error that occurs when the prediction direction slightly shifts or the edge has a slight distortion is reduced.

MPEG-4 AVC (ISO / IEC 14496-10) / ITU-TH H.264 standard

Since the conventional image encoding apparatus is configured as described above, if a smoothed prediction image is generated, a prediction error that occurs even if a slight shift in the prediction direction or a slight distortion occurs in the edge Can be reduced. However, in Non-Patent Document 1, smoothing processing is not performed for blocks other than the 8 × 8 pixel block, and only one type of smoothing processing is performed for the 8 × 8 pixel block.
Actually, even in a block having a size other than 8 × 8 pixels, even if the pattern of the predicted image and the image to be encoded are similar, a large prediction error occurs locally due to a slight mismatch of edges, There has been a problem that the prediction efficiency may be significantly reduced.
Even in the same size block, if the prediction mode is different, a smoothing process is performed regardless of the location where the prediction error is likely to occur in the block and the processing suitable for reducing the prediction error. Since only the processing is prepared, there is a problem that the prediction error may not be sufficiently reduced.

In particular, in the directionality prediction (direction prediction from the upper right to the lower left direction) as in mode 3 or mode 7 in FIG. 16, the prediction signal of the pixel at the left end of the block whose distance from the adjacent pixel on the block used for prediction is long. And the adjacent encoded pixel signal tend to be discontinuous, and there is a problem that distortion may occur at the block boundary. The distortion at the block boundary also occurs at the block boundary at the upper end of the block in the direction prediction from the lower left to the upper right as in mode 8.
Furthermore, when performing prediction based on average value prediction, the prediction value of the pixel located at the block boundary is used as the surrounding encoded pixel signal so that all the prediction values in the block are the average value of the pixels adjacent to the block. On the other hand, there is a problem that distortion tends to occur at the boundary portion of the block because the signal tends to be discontinuous.

  The present invention has been made to solve the above-described problems, and is a moving image encoding device, a moving image decoding device, and a moving image capable of reducing locally generated prediction errors and improving image quality. It is an object to obtain an encoding method and a moving image decoding method.

  In the moving picture coding apparatus according to the present invention, the intra prediction means generates a filter by selecting a predetermined filter from one or more filters prepared in advance and performing an intra-frame prediction process using the filter. The filter processing is performed on the block boundary of the predicted image being processed, and the predicted image after the filter processing is output to the difference image generation means.

  According to the present invention, the intra prediction unit selects a predetermined filter from one or more filters prepared in advance, and uses the filter to perform an intra-frame prediction process to generate a predicted image Since the filter processing for the block boundary is performed and the prediction image after the filter processing is output to the difference image generation means, the effect of reducing the locally generated prediction error and improving the image quality There is.

It is a block diagram which shows the moving image encoder by Embodiment 1 of this invention. It is a flowchart which shows the processing content (moving image encoding method) of the moving image encoding device by Embodiment 1 of this invention. It is a block diagram which shows the moving image decoding apparatus by Embodiment 1 of this invention. It is a flowchart which shows the processing content (moving image decoding method) of the moving image decoding apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the example by which the largest encoding block is divided | segmented into a some encoding block hierarchically. (A) shows the distribution of the encoding block and prediction block after a division | segmentation, (b) is explanatory drawing which shows the condition where encoding mode m ( _Bn ) is allocated by hierarchy division | segmentation. Is an explanatory diagram showing an example of the prediction block P _i ^n-selectable intra prediction parameter coding block B ⁿ (intra prediction mode). It is an explanatory diagram showing an example of a pixel used for generating the predicted values of the pixels in the prediction block P _i ⁿ in the case of ^{_{l i n = m i n =}} 4. It is explanatory drawing which shows the reference pixel arrangement | positioning of the filter process with respect to intra directionality prediction. is an explanatory view showing the relationship between the distance between the ^{_{l i n = m i n =}} 4 pixel and the reference pixels in the prediction block _P ^{i n} in the case of. It is an explanatory diagram showing a change of the filter coefficients _a 0, _{a 1} in the pixel in the prediction block _P ^{i n} in the case of ^{_{l i n = m i n =}} 4. The upper left pixels in the prediction block P _i ⁿ is an explanatory diagram showing a relative coordinate whose origin. It is explanatory drawing which shows the classification | category of the pixel in a block at the time of performing the filter process with respect to average value prediction. It is explanatory drawing which shows the reference pixel arrangement | positioning of the filter process with respect to average value prediction. It is explanatory drawing which shows an example of the table which determines which filter is used for every combination of an intra prediction mode index and prediction block size. It is explanatory drawing which shows intra prediction mode in case the block size of a brightness | luminance is 4x4 pixel.

Embodiment 1 FIG.
1 is a block diagram showing a moving picture coding apparatus according to Embodiment 1 of the present invention.
In FIG. 1, when a video signal is input as an input image, the block dividing unit 1 divides the input image into maximum encoding blocks which are encoding blocks of the maximum size determined by the encoding control unit 2 and performs encoding. Until the upper limit number of hierarchies determined by the control unit 2 is reached, a process of dividing the maximum coding block hierarchically into each coding block is performed.
In other words, the block dividing unit 1 divides the input image into the respective encoding blocks according to the division determined by the encoding control unit 2, and performs a process of outputting the encoded block. Each coding block is divided into one or a plurality of prediction blocks which are prediction processing units.
The block dividing unit 1 constitutes a block dividing unit.

The encoding control unit 2 determines the maximum size of the encoding block that is a processing unit when the prediction process is performed, and determines the upper limit number of layers when the encoding block of the maximum size is hierarchically divided. As a result, processing for determining the size of each encoded block is performed.
In addition, the encoding control unit 2 selects an encoding block output from the block division unit 1 from one or more selectable encoding modes (one or more intra encoding modes and one or more inter encoding modes). A process for selecting an encoding mode to be applied is performed. As an example of the selection method, there is a method of selecting a coding mode having the highest coding efficiency for the coding block output from the block dividing unit 1 from one or more selectable coding modes.
In addition, when the coding mode having the highest coding efficiency is the intra coding mode, the coding control unit 2 predicts an intra prediction parameter used when performing the intra prediction process on the coding block in the intra coding mode. Inter prediction that is determined for each prediction block that is a processing unit and that is used when performing an inter prediction process on a coding block in the inter coding mode when the coding mode having the highest coding efficiency is the inter coding mode. Processing for determining a parameter for each prediction block which is a prediction processing unit is performed.
Further, the encoding control unit 2 performs a process of determining a prediction difference encoding parameter to be given to the transform / quantization unit 7 and the inverse quantization / inverse transform unit 8.
The encoding control unit 2 constitutes an encoding control unit.

  If the coding mode determined by the coding control unit 2 is the intra coding mode, the changeover switch 3 outputs the coded block output from the block dividing unit 1 to the intra prediction unit 4 and the coding control unit 2. If the coding mode determined by the above is the inter coding mode, a process of outputting the coding block output from the block dividing unit 1 to the motion compensation prediction unit 5 is performed.

When the intra control mode is selected by the encoding control unit 2 as the encoding mode corresponding to the encoded block output from the changeover switch 3, the intra prediction unit 4 performs the prediction process for the encoded block. For each prediction block which is a prediction processing unit, an intra prediction process (intraframe prediction) using an intra prediction parameter determined by the encoding control unit 2 while referring to a locally decoded image stored in the intra prediction memory 10. Process) to generate an intra-predicted image.
However, after the intra prediction image is generated, the intra prediction unit 4 generates the same intra prediction image as the intra prediction image by the video decoding device from one or more filters prepared in advance. A filter is selected according to the state of various parameters that are known at the time, and the filter processing is performed on the block boundary of the intra-predicted image using the filter. The result is output to the adding unit 9.
The intra prediction unit 4 and the intra prediction memory 10 constitute intra prediction means.

Specifically, there are the following four parameters as various parameters, and a filter is uniquely determined according to the state of at least one of the four parameters.
・ Parameter (1)
Block size parameter of the predicted image (2)
Quantization parameter / parameter (3) determined by the encoding control unit 2
Distance and parameter between reference pixel and filter processing target pixel used when generating predicted image (4)
Intra prediction parameters determined by the encoding control unit 2

When the inter coding mode is selected by the coding control unit 2 as the coding mode corresponding to the coding block output from the changeover switch 3, the motion compensation prediction unit 5 and the motion compensation prediction frame memory 12 A motion vector is searched by comparing a locally decoded image of one frame or more stored in the image with a prediction block unit that is a prediction processing unit, and the motion vector and the inter prediction parameter determined by the encoding control unit 2 are used. Then, the inter prediction process (motion compensation prediction process) for the encoded block is performed for each prediction block, and a process of generating an inter prediction image is performed.
The motion compensation prediction unit 5 and the motion compensation prediction frame memory 12 constitute a motion compensation prediction unit.

The subtraction unit 6 subtracts the intra prediction image generated by the intra prediction unit 4 or the inter prediction image generated by the motion compensated prediction unit 5 from the encoded block output from the block division unit 1 and performs the subtraction. The process which outputs the prediction difference signal which shows the difference image which is a result to the conversion and quantization part 7 is implemented. The subtracting unit 6 constitutes a difference image generating unit.
The transform / quantization unit 7 refers to the prediction difference encoding parameter determined by the encoding control unit 2 and performs orthogonal transform processing (for example, DCT (discrete cosine transform)) on the prediction difference signal output from the subtraction unit 6. DST (discrete sine transform), orthogonal transform processing such as KL transform in which a base design is made for a specific learning sequence in advance is performed to calculate transform coefficients and refer to the prediction differential encoding parameters Then, a process of quantizing the transform coefficient and outputting compressed data that is the quantized transform coefficient to the inverse quantization / inverse transform unit 8 and the variable length coding unit 13 is performed. The transform / quantization unit 7 constitutes an image compression unit.

The inverse quantization / inverse transform unit 8 refers to the prediction difference encoding parameter determined by the encoding control unit 2 and inversely quantizes the compressed data output from the transform / quantization unit 7, and the prediction difference With reference to the encoding parameter, inverse orthogonal transform processing is performed on the transform coefficient that is compressed data after inverse quantization, and a local decoded prediction difference signal corresponding to the prediction difference signal output from the subtraction unit 6 is calculated. Perform the process.
The addition unit 9 includes the local decoded prediction difference signal calculated by the inverse quantization / inverse conversion unit 8, the intra prediction image generated by the intra prediction unit 4, or the inter prediction image generated by the motion compensation prediction unit 5. Are added to calculate a locally decoded image corresponding to the encoded block output from the block dividing unit 1.

The intra prediction memory 10 is a recording medium that stores the locally decoded image calculated by the adding unit 9.
The loop filter unit 11 performs a predetermined filtering process on the local decoded image calculated by the adding unit 9 and performs a process of outputting the local decoded image after the filter process.
The motion compensated prediction frame memory 12 is a recording medium for storing the locally decoded image after the filter processing.

  The variable length coding unit 13 outputs the compressed data output from the transform / quantization unit 7 and the output signal of the coding control unit 2 (block division information in the largest coding block, coding mode, prediction difference coding parameter, Intra-prediction parameters or inter-prediction parameters) and a motion vector output from the motion compensation prediction unit 5 (when the encoding mode is the inter encoding mode) are subjected to variable length encoding to generate a bitstream. . The variable length encoding unit 13 constitutes variable length encoding means.

In the example of FIG. 1, a block division unit 1, an encoding control unit 2, a changeover switch 3, an intra prediction unit 4, a motion compensation prediction unit 5, a subtraction unit 6, transform / quantization, which are components of the moving image encoding device. Unit 7, inverse quantization / inverse transform unit 8, addition unit 9, intra prediction memory 10, loop filter unit 11, motion compensated prediction frame memory 12, and variable length coding unit 13, each of which has dedicated hardware (for example, It is assumed that the CPU is configured by a semiconductor integrated circuit or a one-chip microcomputer). However, when the moving image encoding apparatus is configured by a computer, the block dividing unit 1, encoding control Unit 2, changeover switch 3, intra prediction unit 4, motion compensation prediction unit 5, subtraction unit 6, transform / quantization unit 7, inverse quantization / inverse transform unit 8, addition unit 9, loop filter unit 11, and variable length code Chemical unit 1 The processing contents stored programs describing the the memory of the computer, may execute a program that the CPU of the computer is stored in the memory.
FIG. 2 is a flowchart showing the processing contents (moving image coding method) of the moving image coding apparatus according to Embodiment 1 of the present invention.

FIG. 3 is a block diagram showing a moving picture decoding apparatus according to Embodiment 1 of the present invention.
In FIG. 3, when the variable length decoding unit 31 receives the bit stream generated by the moving picture encoding apparatus of FIG. 1, the variable length decoding unit 31 receives compressed data, block division information, encoding mode, intra prediction parameters (encoding mode) from the bit stream. The inter prediction parameter (when the encoding mode is the inter encoding mode), the prediction differential encoding parameter and the motion vector (when the encoding mode is the inter encoding mode). A long decoding process is performed. The variable length decoding unit 31 constitutes a variable length decoding unit.

  The inverse quantization / inverse transform unit 32 refers to the prediction difference encoding parameter variable length decoded by the variable length decoding unit 31 and inversely quantizes the compressed data variable length decoded by the variable length decoding unit 31. With reference to the prediction differential encoding parameter, the inverse orthogonal transform process is performed on the transform coefficient that is the compressed data after the inverse quantization, and the local decoding prediction output from the inverse quantization / inverse transform unit 8 in FIG. A process of calculating the same decoded prediction difference signal as the difference signal is performed. The inverse quantization / inverse transform unit 32 constitutes a difference image generation unit.

  The changeover switch 33 outputs the intra-prediction parameter variable-length decoded by the variable-length decoding unit 31 to the intra-prediction unit 34 if the coding mode variable-length decoded by the variable-length decoding unit 31 is the intra-coding mode. If the encoding mode variable-length decoded by the variable-length decoding unit 31 is an inter-coding mode, a process of outputting the inter prediction parameters and motion vectors variable-length decoded by the variable-length decoding unit 31 to the motion compensation unit 35 carry out.

The intra prediction unit 34 is a coding mode related to a decoding block (a block corresponding to the “coding block” of the moving picture coding apparatus in FIG. 1) identified from the block division information variable-length decoded by the variable length decoding unit 31. Is in the intra coding mode, the changeover switch 33 refers to the decoded image stored in the intra prediction memory 37 for each prediction block which is a prediction processing unit when performing the prediction process of the decoded block. The intra prediction process (intraframe prediction process) using the output intra prediction parameter is performed to generate an intra predicted image.
However, after the intra prediction unit 34 generates the intra prediction image, the intra prediction unit 34 performs filtering according to the state of various parameters known at the time when the intra prediction image is generated from one or more filters prepared in advance. Is used, filter processing is performed on the block boundary of the intra predicted image using the filter, and the intra predicted image after the filter processing is output to the adding unit 36.
The intra prediction unit 34 and the intra prediction memory 37 constitute intra prediction means.

Specifically, there are the following four parameters as various parameters, and a filter is uniquely determined according to the state of at least one of the four parameters.
However, the parameters to be used are determined in advance as the same parameters as those in the moving picture encoding apparatus in FIG. That is, when the intra prediction unit 4 performs filter processing using the parameters (3) and (4) on the video encoding device side, the intra prediction unit 34 similarly sets the parameters (3) and (4) on the video decoding device side. The parameters used in the moving picture coding apparatus and the moving picture decoding apparatus are unified so that the filtering process is performed using (4).
・ Parameter (1)
Block size parameter of the predicted image (2)
Quantization parameter / parameter (3) variable length decoded by the variable length decoding unit 31
Distance and parameter between reference pixel and filter processing target pixel used when generating predicted image (4)
Intra prediction parameters variable length decoded by the variable length decoding unit 31

The motion compensation unit 35 performs prediction when performing the prediction process of the decoded block when the coding mode related to the decoded block specified from the block division information subjected to variable length decoding by the variable length decoding unit 31 is the inter coding mode. For each prediction block that is a processing unit, while referring to the decoded image stored in the motion compensation prediction frame memory 39, an inter prediction process (motion compensation prediction) using the motion vector output from the changeover switch 33 and the inter prediction parameter is used. Process) to generate an inter prediction image.
The motion compensation unit 35 and the motion compensation prediction frame memory 39 constitute motion compensation prediction means.

  The addition unit 36 adds the decoded prediction difference signal calculated by the inverse quantization / inverse conversion unit 32 and the intra prediction image generated by the intra prediction unit 34 or the inter prediction image generated by the motion compensation unit 35. Then, a process of calculating the same decoded image as the local decoded image output from the adding unit 9 in FIG. 1 is performed. The adding unit 36 constitutes a decoded image generating unit.

The intra prediction memory 37 is a recording medium that stores the decoded image calculated by the addition unit 36.
The loop filter unit 38 performs a predetermined filter process on the decoded image calculated by the adding unit 36 and performs a process of outputting the decoded image after the filter process.
The motion compensated prediction frame memory 39 is a recording medium that stores the decoded image after the filter process.

In the example of FIG. 3, the variable length decoding unit 31, the inverse quantization / inverse conversion unit 32, the changeover switch 33, the intra prediction unit 34, the motion compensation unit 35, the addition unit 36, and the intra prediction, which are components of the video decoding device. It is assumed that each of the memory 37, the loop filter unit 38, and the motion compensation prediction frame memory 39 is configured by dedicated hardware (for example, a semiconductor integrated circuit on which a CPU is mounted, a one-chip microcomputer, or the like). However, when the moving picture decoding apparatus is configured by a computer, the variable length decoding unit 31, the inverse quantization / inverse conversion unit 32, the changeover switch 33, the intra prediction unit 34, the motion compensation unit 35, the addition unit 36, and the loop A program describing the processing contents of the filter unit 38 is stored in the memory of a computer, and the CPU of the computer is stored in the memory. It is also possible to run the program.
FIG. 4 is a flowchart showing the processing contents (moving image decoding method) of the moving image decoding apparatus according to Embodiment 1 of the present invention.

Next, the operation will be described.
In the first embodiment, each frame image of a video is used as an input image, intra prediction from encoded neighboring pixels or motion compensation prediction between adjacent frames is performed, and an obtained prediction difference signal is obtained. A video encoding device that performs compression processing by orthogonal transform / quantization and then performs variable length encoding to generate a bitstream, and a video decoding that decodes the bitstream output from the video encoding device The apparatus will be described.

The moving picture coding apparatus in FIG. 1 performs intra-frame / inter-frame adaptive coding by dividing a video signal into blocks of various sizes in response to local changes in the spatial and temporal directions of the video signal. It is characterized by that.
In general, a video signal has a characteristic that the complexity of the signal changes locally in space and time. When viewed spatially, a small image, such as a picture with a uniform signal characteristic in a relatively wide image area such as the sky or a wall, or a picture containing a person or fine texture, on a video frame. A pattern having a complicated texture pattern in the region may be mixed.
Even when viewed temporally, the change in the pattern of the sky and the wall locally in the time direction is small, but because the outline of the moving person or object moves rigidly or non-rigidly in time, the temporal change Is big.

In the encoding process, a prediction difference signal with small signal power and entropy is generated by temporal and spatial prediction to reduce the overall code amount. However, the parameters used for the prediction are set as large as possible in the image signal region. If it can be applied uniformly, the code amount of the parameter can be reduced.
On the other hand, if the same prediction parameter is applied to a large image region with respect to an image signal pattern having a large temporal and spatial change, the number of prediction differential signals increases because prediction errors increase. .
Therefore, in a region where the temporal and spatial changes are large, the block size for performing the prediction process by applying the same prediction parameter is reduced, the amount of parameter data used for prediction is increased, and the power and entropy of the prediction difference signal are increased. It is desirable to reduce

  In the first embodiment, in order to perform coding adapted to the general characteristics of such a video signal, first, prediction processing or the like is started from a predetermined maximum block size, and the video signal region is divided hierarchically. In addition, the prediction process and the encoding process of the prediction difference are adapted for each divided area.

The video signal format to be processed by the moving image encoding apparatus of FIG. 1 is a color video signal in an arbitrary color space such as a YUV signal composed of a luminance signal and two color difference signals, or an RGB signal output from a digital image sensor. In addition to the above, it is assumed that the video frame is an arbitrary video signal including a horizontal / vertical two-dimensional digital sample (pixel) sequence, such as a monochrome image signal or an infrared image signal.
However, the gradation of each pixel may be 8 bits or 10 bits or 12 bits.

In the following description, for convenience, unless otherwise specified, it is assumed that the video signal of the input image is a YUV signal, and the two color difference components U and V are subsampled with respect to the luminance component Y 4: 2: 0. The case of handling format signals will be described.
A processing data unit corresponding to each frame of the video signal is referred to as a “picture”.
In the first embodiment, “picture” is described as a video frame signal that is sequentially scanned (progressive scan). However, when the video signal is an interlaced signal, “picture” is a unit constituting a video frame. It may be a field image signal.

First, the processing contents of the moving picture encoding apparatus in FIG. 1 will be described.
First, the encoding control unit 2 determines the size of the maximum encoding block used for encoding the picture to be encoded (current picture) and the upper limit of the number of hierarchies into which the maximum encoding block is divided (FIG. 2). Step ST1).
As a method of determining the size of the maximum coding block, for example, the same size may be determined for all the pictures according to the resolution of the video signal of the input image, or the local motion of the video signal of the input image The size difference may be quantified as a parameter, and a small size may be determined for a picture with high motion, while a large size may be determined for a picture with little motion.
For example, the upper limit of the number of division layers can be determined by, for example, determining the same number of layers for all pictures according to the resolution of the video signal of the input image, or when the motion of the video signal of the input image is severe There is a method in which the number of hierarchies is increased so that finer movements can be detected, and when there are few movements, the number of hierarchies is set to be suppressed.

Also, the encoding control unit 2 selects an encoding mode corresponding to each encoding block divided hierarchically from one or more available encoding modes (step ST2).
That is, the encoding control unit 2 divides the image area of the maximum encoding block size into encoded blocks having the encoding block size hierarchically until reaching the upper limit of the number of division layers defined above. A coding mode for each coding block is determined.
There are one or more intra coding modes (collectively referred to as “INTRA”) and one or more inter coding modes (collectively referred to as “INTER”). The coding control unit 2 selects a coding mode corresponding to each coding block from all coding modes available for the picture or a subset thereof.

However, each coding block that is hierarchically divided by the block division unit 1 to be described later is further divided into one or a plurality of prediction blocks, which are units for performing prediction processing, and the division state of the prediction block is also coded mode. Is included as information.
Since the encoding mode selection method by the encoding control unit 2 is a known technique, detailed description thereof is omitted. For example, an encoding process for an encoding block is performed using any available encoding mode. There is a method in which coding efficiency is verified by performing and a coding mode having the best coding efficiency is selected from among a plurality of available coding modes.

In addition, the encoding control unit 2 determines a quantization parameter and a transform block size that are used when the difference image is compressed for each encoding block, and a prediction that is used when the prediction process is performed. A parameter (intra prediction parameter or inter prediction parameter) is determined.
However, when the encoded block is further divided into prediction block units for which prediction processing is performed, a prediction parameter (intra prediction parameter or inter prediction parameter) can be selected for each prediction block.
Furthermore, in the coding block whose coding mode is the intra coding mode, the details will be described later. However, since encoded pixels adjacent to the prediction block are used when performing the intra prediction process, the prediction block unit is used. Since it is necessary to perform encoding, the selectable transform block size is limited to the size of the prediction block or less.

The encoding control unit 2 outputs the prediction difference encoding parameter including the quantization parameter and the transform block size to the transform / quantization unit 7, the inverse quantization / inverse transform unit 8, and the variable length coding unit 13.
Also, the encoding control unit 2 outputs intra prediction parameters to the intra prediction unit 4 as necessary.
Also, the encoding control unit 2 outputs inter prediction parameters to the motion compensation prediction unit 5 as necessary.

  When a video signal is input as an input image, the block division unit 1 divides the input image into the maximum coding block size determined by the coding control unit 2, and further, the divided maximum coding block is coded. 2 is hierarchically divided into the encoded blocks determined by 2 and the encoded blocks are output.

Here, FIG. 5 is an explanatory diagram showing an example in which the maximum coding block is hierarchically divided into a plurality of coding blocks.
In FIG. 5, the maximum coding block is a coding block whose luminance component described as “0th layer” has a size of (L ⁰ , M ⁰ ).
Starting from the maximum coding block, a coding block is obtained by performing hierarchical division to a predetermined depth separately defined by a quadtree structure.
At depth n, the coding block is an image area of size (L ⁿ , M ⁿ ).
However, L ⁿ and M ⁿ may be the same or different, but FIG. 5 shows a case of L ⁿ = M ⁿ .

Hereinafter, the coding block size determined by the coding control unit 2 is defined as the size (L ⁿ , M ⁿ ) in the luminance component of the coding block.
Since quadtree partitioning is performed, (L ^{n + 1} , M ^{n + 1} ) = (L ⁿ / 2, M ⁿ / 2) always holds.
Note that in a color video signal (4: 4: 4 format) in which all color components have the same number of samples, such as RGB signals, the size of all color components is (L ⁿ , M ⁿ ), but 4: 2. : When the 0 format is handled, the encoding block size of the corresponding color difference component is (L ⁿ / 2, M ⁿ / 2).

Hereinafter, the coding block of the n hierarchy expressed in B ^n, denote the encoding modes selectable by the coding block B ⁿ with m (B ^n).
In the case of a color video signal composed of a plurality of color components, the encoding mode m (B ⁿ ) may be configured to use an individual mode for each color component, or common to all color components. It may be configured to use a mode. Hereinafter, unless otherwise specified, description will be made assuming that it indicates a coding mode for a luminance component of a coding block of a YUV signal and 4: 2: 0 format.

As shown in FIG. 6, the encoded block B ⁿ is divided by the block dividing unit 1 into one or a plurality of prediction blocks representing a prediction processing unit.
Hereinafter, a prediction block belonging to the coding block B ⁿ is ^denoted as P _i ⁿ (i is a prediction block number in the n-th layer). FIG. 5 shows an example of P ₀ ⁰ and P ₁ ⁰ .
How the prediction block is divided in the coding block ^Bn is included as information in the coding mode m ( ^Bn ).
All the prediction blocks P _i ⁿ are subjected to prediction processing according to the encoding mode m (B ⁿ ), and it is possible to select individual prediction parameters (intra prediction parameters or inter prediction parameters) for each prediction block P _i ^n. it can.

For example, the encoding control unit 2 generates a block division state as illustrated in FIG. 6 for the maximum encoding block, and identifies the encoding block.
A rectangle surrounded by a dotted line in FIG. 6A represents each coding block, and a block painted with diagonal lines in each coding block represents a division state of each prediction block.
FIG. 6B shows, in a quadtree graph, a situation in which the encoding mode m (B ⁿ ) is assigned by hierarchical division in the example of FIG. 6A. Nodes surrounded by squares in FIG. 6B are nodes (encoding blocks) to which the encoding mode m (B ⁿ ) is assigned.
Information of the quadtree graph is output from the encoding control unit 2 to the variable length encoding unit 13 together with the encoding mode m (B ⁿ ), and is multiplexed into the bit stream.

The changeover switch 3 is output from the block dividing unit 1 when the encoding mode m (B ⁿ ) determined by the encoding control unit 2 is an intra encoding mode (when m (B ⁿ ) ∈INTRA). The encoded block B ⁿ is output to the intra prediction unit 4.
On the other hand, when the encoding mode m (B ⁿ ) determined by the encoding control unit 2 is the inter encoding mode (when m (B ⁿ ) εINTER), the encoded block output from the block dividing unit 1 B ⁿ is output to the motion compensation prediction unit 5.

In the intra prediction unit 4, the coding mode m (B ⁿ ) determined by the coding control unit 2 is the intra coding mode (when m (B ⁿ ) ∈INTRA), and the coding block B is changed from the changeover switch 3 to the coding block B. ⁿ (step ST3), using the intra prediction parameters determined by the encoding control unit 2 while referring to the local decoded image stored in the intra prediction memory 10, the encoding block B ⁿ and implementing intra prediction process for each of the prediction block _P ^{i n} in, it generates an intra prediction image _{P INTRAi} ⁿ (step ST4).
Incidentally, since it is necessary to video decoding device generates exactly the same intra prediction image and the intra prediction image P _INTRAi ^n, intra prediction parameters used for generating the intra prediction image P _INTRAi ⁿ is from encoding control unit 2 The data is output to the variable length encoding unit 13 and multiplexed into the bit stream.
Details of processing contents of the intra prediction unit 4 will be described later.

The motion-compensated prediction unit 5 has the coding mode m (B ⁿ ) determined by the coding control unit 2 in the inter coding mode (when m (B ⁿ ) ∈ INTER), and the coding block is switched from the changeover switch 3 to the coding block. Upon receiving the B ⁿ (step ST3), the motion vector by comparing the locally decoded image after the filtering process stored in the prediction block P _i ⁿ and the motion compensated prediction frame memory 12 of the encoding block B ⁿ Using the motion vector and the inter prediction parameter determined by the encoding control unit 2, the inter prediction process for each prediction block P _i ⁿ in the encoding block B ⁿ is performed, and the inter prediction image P generating a _INTERi ⁿ (step ST5).
Incidentally, since it is necessary to video decoding device generates exactly the same inter prediction image and the inter-predicted image P _INTERi ^n, inter prediction parameters used for generating the inter prediction image P _INTERi ⁿ is from encoding control unit 2 The data is output to the variable length encoding unit 13 and multiplexed into the bit stream.
In addition, the motion vector searched by the motion compensation prediction unit 5 is also output to the variable length encoding unit 13 and multiplexed into the bit stream.

Subtraction unit 6, upon receiving the encoded block ^{B n} from the block dividing unit 1 from its prediction block _P ^{i n} the coded block ^{B n,} the intra prediction image _{P INTRAi} ⁿ generated by the intra prediction unit 4 or, , by subtracting one of the inter prediction image P _INTERi ⁿ generated by the motion compensation prediction unit 5, and outputs the prediction difference signal e _i ⁿ representing a difference image is the subtraction result to the transform and quantization unit 7 (Step ST6).

When receiving the prediction difference signal e _i ⁿ from the subtraction unit 6, the transform / quantization unit 7 refers to the prediction difference encoding parameter determined by the encoding control unit 2 and is orthogonal to the prediction difference signal e _i ⁿ . Perform transform processing (for example, DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), orthogonal transform processing such as KL transform in which a base design is made in advance for a specific learning sequence), and calculate transform coefficients To do.
In addition, the transform / quantization unit 7 refers to the prediction difference encoding parameter, quantizes the transform coefficient, and performs the inverse quantization / inverse transform unit 8 and the variable length on the compressed data that is the transform coefficient after quantization. It outputs to the encoding part 13 (step ST7).

When receiving the compressed data from the transform / quantization unit 7, the inverse quantization / inverse transform unit 8 refers to the prediction difference encoding parameter determined by the encoding control unit 2 and dequantizes the compressed data. .
Further, the inverse quantization / inverse transform unit 8 refers to the prediction differential encoding parameter, and performs inverse orthogonal transform processing (for example, inverse DCT, inverse DST, inverse KL) on the transform coefficient that is the compressed data after inverse quantization. conversion, etc.) was performed, and outputs to the adder 9 calculates the local decoded prediction difference signal corresponding to the prediction difference signal e _i ⁿ output from the subtraction unit 6 (step ST8).

Upon receiving the local decoded prediction difference signal from the inverse quantization / inverse transform unit 8, the adding unit 9 receives the local decoded prediction difference signal and the intra predicted image P _INTRAi ⁿ generated by the intra prediction unit 4 or motion compensation. A local decoded image is calculated by adding one of the inter predicted images P _INTERIn ⁿ generated by the prediction unit 5 (step ST9).
The adding unit 9 outputs the locally decoded image to the loop filter unit 11 and stores the locally decoded image in the intra prediction memory 10.
This locally decoded image becomes an encoded image signal used in the subsequent intra prediction processing.

When the loop filter unit 11 receives the local decoded image from the addition unit 9, the loop filter unit 11 performs a predetermined filter process on the local decoded image, and stores the filtered local decoded image in the motion compensated prediction frame memory 12. (Step ST10).
Note that the filter processing by the loop filter unit 11 may be performed in units of the maximum encoded block or individual encoded block of the input local decoded image, or one picture after the local decoded image for one picture is input. You may go together.
In addition, examples of the predetermined filter processing include processing for filtering the block boundary so that the discontinuity (block noise) of the encoded block boundary is not conspicuous, and an error between the input image indicated by the video signal and the locally decoded image For example, a filter process for compensating for distortion of the locally decoded image so as to minimize the error. Two or more types of these filter processes may be performed.
However, when performing filter processing to compensate for distortion of the local decoded image so that an error between the input image indicated by the video signal and the locally decoded image is minimized, the video signal needs to be referred to by the loop filter unit 11. Therefore, it is necessary to change the moving picture encoding apparatus of FIG. 1 so that the video signal is input to the loop filter unit 11.

The processes of steps ST3 to ST9 are repeatedly performed until the processes for all the coding blocks ^Bn divided hierarchically are completed, and when the processes for all the coding blocks ^Bn are completed, the process proceeds to the process of step ST13. (Steps ST11 and ST12).

The variable length encoding unit 13 uses the compressed data output from the transform / quantization unit 7 and the block division information (FIG. 6B) in the maximum encoding block output from the encoding control unit 2 as an example. (Quadrant tree information), encoding mode m (B ⁿ ) and prediction differential encoding parameter, and intra prediction parameter output from the encoding control unit 2 (when the encoding mode is an intra encoding mode) or inter prediction The parameters (when the encoding mode is the inter encoding mode) and the motion vector (when the encoding mode is the inter encoding mode) output from the motion compensated prediction unit 5 are variable-length encoded, A bit stream indicating the encoding result is generated (step ST13).

Next, the processing content of the intra estimation part 4 is demonstrated in detail.
Figure 7 is an explanatory diagram showing an example of the intra prediction mode is an intra prediction parameters each prediction block P _i ⁿ is selectable within a coding block B ^n. However, N _I is the number of intra prediction modes.
FIG. 7 shows an index value of an intra prediction mode and a prediction direction vector indicated by the intra prediction mode. In the example of FIG. 7, as the number of selectable intra prediction modes increases, Designed to reduce the angle.

Intra prediction unit 4, as described above, with reference to the intra prediction parameters of the prediction block P _i ^n, to implement intra prediction processing for the prediction block P _i ^n, but to generate an intra prediction image P _INTRAi ⁿ , it will be described here intra process of generating an intra prediction signal of a prediction block P _i ⁿ in the luminance signal.

The size of the prediction block P _i ⁿ is assumed to be l _i ⁿ × m _i ⁿ pixels.
FIG. 8 is an explanatory diagram illustrating an example of a pixel used when generating a prediction value of a pixel in the prediction block P _i ⁿ when l _i ⁿ = m _i ⁿ = 4.
In FIG. 8, the encoded pixels (2 × l _i ⁿ +1) and the left encoded pixels (2 × m _i ⁿ ) on the prediction block P _i ⁿ are used as prediction pixels. However, the number of pixels used for prediction may be more or less than the pixels shown in FIG.
Further, in FIG. 8, it is used to predict one line or pixel of one column in the vicinity of the predicted block P _i ^n, 2 rows or two columns, or may be used more pixels in the prediction.

Code index value of the intra prediction mode for prediction block P _i ⁿ is the case of 2 (average prediction) is adjacent to the left of the prediction block P _i predicted block P _i ⁿ the encoded pixels adjacent to the upper side of the ⁿ generating a prediction image the mean value of the reduction already pixel as a prediction value of the pixels in the prediction block P _i ^n.
Prediction index value of the intra prediction mode for the block _P ^{i n} is in the case of _N I -1 (planes (Planar) prediction) is the prediction block _P ⁱ predicted block _P ^{i n} the encoded pixels adjacent to the upper side of the ⁿ using encoded pixels adjacent to the left, to generate a prediction image of interpolated values according to the distance of the pixel and the prediction pixel in the prediction block P _i ⁿ as the predicted value.
When the index value in the intra prediction mode is other than 2 (average value prediction) and N _I −1 (plane prediction), the prediction block P is based on the prediction direction vector ν _p = (dx, dy) indicated by the index value. Generate predicted values for pixels in _i ⁿ .
As shown in FIG. 12, as the origin at the upper left pixel of the prediction block P _i ^n, setting the relative coordinates of the prediction block P _i ⁿ as (x, y), the position of the reference pixels used for prediction, the following L And the intersection of adjacent pixels.

When the reference pixel is at the integer pixel position, the integer pixel is set as the prediction value of the prediction target pixel. When the reference pixel is not at the integer pixel position, an interpolation pixel generated from the integer pixel adjacent to the reference pixel is selected. Estimated value.
In the example of FIG. 8, since the reference pixel is not located at the integer pixel position, a value interpolated from two pixels adjacent to the reference pixel is set as the predicted value. Note that an interpolation pixel may be generated not only from two adjacent pixels but also from two or more adjacent pixels, and used as a predicted value.
While increasing the number of pixels used in the interpolation process has the effect of improving the interpolation accuracy of the interpolated pixels, it increases the complexity of the calculation required for the interpolation process, requiring high coding performance even when the calculation load is large. In the case of a video encoding device, it is better to generate interpolation pixels from more pixels.

Then, the predicted image composed of prediction values in the prediction block P _i ⁿ generated by the above procedure was defined as intermediate prediction image, by performing a filtering process to be described later with respect to the intermediate prediction image, finally It acquires an intra prediction image P _i ^n, and outputs the intra prediction image P _i ⁿ to the subtraction unit 6 and the addition section 9.
Further, the intra prediction parameters used for generating the intra prediction image P _i ⁿ outputs to the variable length coding unit 13 for multiplexing the bitstream.

Hereinafter, specific filter processing will be described.
A filter to be used is selected from at least one or more filters prepared in advance by a method described later, and filter processing is performed on each pixel of the intermediate predicted image according to the following equation (2).

In equation (2), a _n (n = 0, 1,..., N) is obtained from the coefficients (a ₀ , a ₁ ,..., A _N−1 ) and offset coefficients a _N applied to the reference pixels. It is a configured filter coefficient.
p _n (n = 0, 1,..., N−1) represents the reference pixel of the filter including the filter processing target pixel p ₀ . N is an arbitrary number of reference pixels.
s (p _n ) represents the luminance value of each reference pixel, and s hat (p ₀ ) represents the luminance value after filtering in the filtering target pixel p ₀ .
However, the filter coefficients may be configured that there is no offset coefficient a _N. Further, the luminance value s (p _n ) of each reference pixel in the prediction block P _i ⁿ may be the luminance value of each pixel of the intermediate prediction image, or the luminance value after filtering only at the filtered pixel position. You may make it.

If the luminance value s (p _n ) of each reference pixel outside the prediction block P _i ⁿ is an encoded region, the encoded luminance value (decoded luminance value) is still encoded. If it is not a region, a predetermined procedure is used to substitute the luminance value s (p _n ) of each reference pixel in the prediction block P _i ⁿ defined above and the luminance value after encoding of the encoded region. The signal value to be selected is selected (for example, the signal value at the closest position among the candidate pixels is selected).

When performing the above filter processing, the filter is selected in consideration of the following four parameters (1) to (4).
(1) Prediction block P _i ⁿ size (l _i ⁿ × m _i ⁿ )
(2) Quantization parameter included in prediction error encoding parameter (3) Distance between reference pixel ("reference pixel" shown in FIG. 8) used at the time of generating an intermediate prediction image and pixel to be filtered (4) Intra prediction mode index value when the intermediate prediction image is generated

Appropriate filter selection according to the above parameters can be realized by associating an appropriate filter from a group of filters prepared in advance with each of the above parameter combinations.
At this time, a case may be prepared in which filter processing is not performed in association with “no filter processing” for a certain combination of parameters.
Further, since the four parameters (1) to (4) are known parameters on the moving picture decoding apparatus side, no additional information to be encoded necessary for performing the above-described filter processing is generated.
In the above description, the filter is switched by preparing and adaptively selecting a necessary number of filters in advance, but the filter is calculated so that the filter is calculated according to the value of the filter selection parameter. The filter switching may be realized by defining the function as a function of the filter selection parameter.

In the above description, the filter is selected in consideration of the four parameters (1) to (4). However, at least one of the four parameters (1) to (4) is shown. A filter may be selected in consideration of the above parameters.
Hereinafter, the case where parameters (3) and (4) are used will be described in detail.

Generally, farther pixel from the prediction block P _i encoded pixels adjacent to ⁿ (8 "pixels used in prediction") used for generating the intermediate prediction image, such as edges between the intermediate predicted image and the input image Deviation is likely to occur.
In particular, in the directional prediction from the upper right to the lower left in which the index value of the intra prediction mode in FIG. 7 is 4, 25, 13, 24, 7, 23, 12, and 22, the coding is adjacent to the pixel at the left end of the block. It tends to be discontinuous with pixels, and distortion often occurs at block boundaries.
Similarly, in the direction prediction from the lower left to the upper right direction in which the index value of the intra prediction mode in FIG. 7 is 5, 33, 17, 32, 9, 31, 16, 30, distortion occurs at the upper boundary of the block. easy.

  Therefore, in the first embodiment, the “(3) distance between the reference pixel used when generating the intermediate predicted image and the pixel to be filtered” is the farthest pixel along the prediction direction (directional prediction from the upper right to the lower left direction). In accordance with “(4) Index value of intra prediction mode when generating intermediate prediction image” for the pixel at the left end of the block at, and the pixel at the top end of the block in the direction prediction from the lower left to the upper right direction) By performing the filtering process, distortion generated at the boundary of the prediction block is reduced.

FIG. 9 is an explanatory diagram showing the reference pixel arrangement of the filter processing.
In FIG. 9, the prediction mode in which the index value of the intra prediction mode is 4, 25, 13, 24, 7, 23, 12, 22 is group (1), and the index value of the intra prediction mode is 5, 33, 17, 32. classifies prediction modes 9,31,16,30 group (2), the vertical direction between the filter processing target pixel _{p 0} and the reference pixel _{p 1} (group 1) or horizontally (group ( The pixel shift of 2)) is represented by d.
In the example of FIG. 9, d = 3 is represented. Then, the filtering process according to the equation (2) is performed on the filtering target pixel p ₀ in FIG. 9 with the reference pixel number N being 2 and the offset coefficient a _N being invalid.

As described above, the filter processing for compensating the distortion of the predicted image with high accuracy by adaptively switching the pixel d of the parameter d and the filter coefficients a ₀ and a ₁ for each filter processing target pixel p _{0 and} each prediction mode. Can be realized.

The following filter coefficients a ₀ and a ₁ are examples.
a ₀ = 3/4, a ₁ = ¼
By performing the smoothing filter process using the reference pixels p ₁ along the prediction direction, improves the continuity of pixel values in the prediction direction, it is possible to reduce the distortion of the block boundary.
An example of the parameter d is as follows.
d = 1:
Prediction mode with index values of 4, 5, 25, 33, 13, 17 in FIG. 7 d = 2:
Prediction mode with index values of 24, 32, 7, and 9 in FIG. 7 d = 3:
Prediction mode with index values 23 and 31 in FIG. 7 d = 6:
Prediction mode with index values 12 and 16 in FIG. 7 d = 16:
Prediction mode with index values of 22 and 30 in FIG.

  In general, high-accuracy filter processing can be realized by providing the parameter d with real number accuracy so as to follow the prediction direction. However, when d is not an integer, the virtual pixel located at the position d from the surrounding encoded pixels Therefore, in the above example, d is approximated to an integer value for the purpose of simplifying the arithmetic processing. By doing so, processing for generating virtual pixels does not occur. In consideration of the fact that the correlation generally decreases as the distance between the pixels increases, the integer value approximation in the above example is rounded off without using rounding so that it is easy to take a small value when performing integer value approximation. Six rounds are used (approximation results are the same even when rounded off and rounded off). However, the integer value approximation method is not limited to this.

In the above filtering process, the filtering process may not be performed in the intra prediction mode in which the value of d is a certain value or more in consideration of the fact that the correlation generally decreases as the distance between pixels increases.
By doing in this way, the prediction mode which performs a filter process can be limited, and the increase in the calculation load by a filter process can be reduced. In the direction prediction of the groups (1) and (2), an example when simplified is as follows.
d = 1:
Prediction mode with index values of 4, 5, 25, 33, 13, 17 in FIG. 7 d = 2:
Prediction mode with index values of 24, 32, 7, and 9 in FIG. 7 No filter processing:
Prediction mode with index values of 23, 31, 12, 16, 22, and 30 in FIG.

Moreover, in said filter process, since the filter process from which parameter d differs with prediction modes is performed, the filter switching process by a prediction mode is needed. Therefore, the same d value filter process may be performed uniformly for the prediction mode in which the filter process is performed.
By doing so, the performance of the filter processing is lowered, but the prepared filter is only one type of filter with the value of d fixed, and simple filter processing can be realized. In the direction prediction of the groups (1) and (2), an example when d = 1 is fixed is as follows.
d = 1:
Prediction mode with index values of 4, 5, 25, 33, 13, 17, 24, 32, 7, 9 in FIG. 7 No filter processing:
Prediction mode with index values of 23, 31, 12, 16, 22, and 30 in FIG.

  In the filter processing described so far, only the pixels adjacent to the block boundary are filtered. However, the pixels in the prediction block around the block boundary other than the pixels adjacent to the block boundary are also included in the adjacent block. You may make it perform the filter process using a pixel. In this way, not only the pixels adjacent to the prediction block boundary but also the pixels around the boundary can increase the correlation with the adjacent block.

Further, as shown in FIG. 10, “(3) the distance between the reference pixel used when generating the intermediate predicted image and the pixel to be filtered” differs for each pixel on which the filtering process is performed.
Accordingly, since the pixel having a longer distance is more likely to be distorted at the block boundary, the filter coefficients a ₀ and a ₁ may be changed depending on the pixel position as shown in FIG.
An example of filter processing for the directionality prediction of group (1) is as follows. However, the coordinates (x, y) are as shown in FIG. 12, and the group (2) is expressed by an expression in which y in the following expressions (3) and (4) is replaced with x.

  Similar to the directional prediction of the groups (1) and (2) above, when performing prediction based on the average value prediction in which the index value of the intra prediction mode is represented by 2, all prediction values in the block are converted into blocks. Since the average value of adjacent pixels is used, the prediction signal of the pixel located at the block boundary is likely to be a discontinuous signal with respect to the surrounding encoded pixels, and distortion is likely to occur at the boundary portion of the block.

Therefore, the following filter processing is performed for each region shown in FIG. However, the reference pixel arrangement of the following filter is as shown in FIG.
· Area A (the upper left pixel of the prediction block _P ^{i n)}
_{a 0 = 1/2, a} 1 = 1/4, a 2 = 1/4 ( the number of reference pixels N = 3)
- region B (the upper end of the pixel of the prediction block _P ^{i n} other than the region A)
a ₀ = 3/4, a ₂ = ¼ (reference pixel number N = 2)
· Area C (the leftmost pixel of the prediction block _P ^{i n} other than the region A)
a ₀ = 3/4, a ₁ = ¼ (reference pixel number N = 2)
- area D (region A, B, pixels of the predicted block _P ^{i n} non-C)
No filtering

It may also be subjected to the same filtering as the average value prediction even plane prediction index value of the intra prediction mode is represented by N _I -1. By doing in this way, the boundary of the block which performed plane prediction is smoothed, and continuity can be improved.

For simplification of processing, the block size for performing filter processing in the directionality prediction, average value prediction, and plane prediction of the groups (1) and (2) may be limited.
Furthermore, the block size to be limited may be determined for each prediction mode. In that case, the filtering process may be switched by referring to a table based on the prediction mode and the block size.

FIG. 15 is an explanatory diagram illustrating an example of a table that determines which filter is used for each combination of an intra prediction mode index and a prediction block size.
In FIG. 15, “0” is no filter processing, “1” is d = 1, a ₀ = 3/4, a ₁ = 1/8 filter processing, and “2” is d = 2. , A ₀ = 3/4, filter processing for directional prediction with a ₁ = 1/8, and “3” is the filtering processing in the above-described average value prediction and plane prediction.
Thus, the parameters (3), as well as (4), the parameter "(1) the size of the prediction block ^{_{P i n (l i n ×}} m i n) " also by performing consideration to filtering, the filter processing Can reduce the processing load.
However, as a mounting in the form of filtering it is not limited to what is implemented in the form of selecting a filter corresponding filter index by table lookup, each index size and the prediction mode of the prediction block P _i ⁿ The filter processing to be executed may be directly mounted.
Thus, even if it is not the form which refers to a table, if the prediction image obtained as a result of performing a filter process is equivalent, the form of mounting will not be ask | required.

Even when the block size is not limited, a filter process based on a table based on the prediction mode may be configured, or a filter process executed for each prediction mode may be directly implemented.
Thus, even if it is not the form which refers to a table, if the prediction image obtained as a result of performing a filter process is equivalent, the form of mounting will not be ask | required.

Furthermore, generally, the larger the quantization parameter (the larger the quantization step size), the more likely that quantization errors occur in the encoded image. Therefore, block distortion at the left end of the prediction block in the directionality prediction of group (1). In addition, block distortion at the upper end of the prediction block in the directionality prediction of the group (2) is likely to occur.
Therefore, in the above-described filter processing, the value of the filter coefficient a ₀ is reduced as the quantization parameter is larger in consideration of “(2) quantization parameter included in the prediction error coding parameter”. a The value of ₁ may be increased.
By doing in this way, even when the quantization parameter which is easy to produce block distortion is large, block distortion can be reduced efficiently.

Note that the MPEG-4 AVC / H. Similar to the smoothing process performed on the reference image at the time of 8 × 8 pixel block intra prediction of the 264, the intra prediction unit 4, the reference pixels in generating the intermediate prediction image predicted block P _i ⁿ even when configured as a prediction block P _i ⁿ pixels smoothed the encoded pixels adjacent to, it is possible to perform the filtering for the same intermediate predicted image and the above example.

_{Through the} above-described processing, prediction pixels for all the pixels of the luminance signal in the prediction block P _i ⁿ are generated, and the intra prediction image P _INTRAin ⁿ is output.
Incidentally, the intra prediction parameters used for generating the intra prediction image P _INTRAi ⁿ (intra prediction mode) is output to the variable length coding unit 13 for multiplexing the bitstream.

Even for the color difference signal of the prediction block P _i ^n, in the same procedure as the luminance signal, the intra prediction processing based on the intra prediction parameters (intra prediction mode) performed, the intra prediction parameters used for generating the intra prediction image Is output to the variable length encoding unit 13.
However, for intra prediction of color difference signals, the filter processing described above may or may not be performed in the same manner as the luminance signal.
Furthermore, the intra prediction parameter (intra prediction mode) that can be selected by the color difference signal may be different from that of the luminance signal. For example, in the case of the YUV signal 4: 2: 0 format, the color difference signal (U, V signal) is a signal obtained by reducing the resolution to 1/2 in both the horizontal direction and the vertical direction with respect to the luminance signal (Y signal). Since the complexity of the image signal is low and the prediction is easy compared to the luminance signal, the number of intra prediction parameters that can be selected is smaller than the luminance signal, and the amount of code required to encode the intra prediction parameter can be reduced. The computation of the prediction process may be reduced.

Next, the processing contents of the moving picture decoding apparatus in FIG. 3 will be specifically described.
When the variable length decoding unit 31 receives the bit stream generated by the moving picture encoding device in FIG. 1, the variable length decoding unit 31 performs variable length decoding processing on the bit stream (step ST21 in FIG. 4), and the picture of one or more frames The frame size information is decoded in sequence units or picture units.

The variable length decoding unit 31 determines the maximum coding block size and the upper limit of the number of divided layers determined by the coding control unit 2 of the moving picture coding apparatus in FIG. 1 in the same procedure as the moving picture coding apparatus ( Step ST22).
For example, when the maximum encoding block size and the upper limit of the number of division layers are determined according to the resolution of the video signal, the maximum encoding is performed in the same procedure as the moving image encoding apparatus based on the decoded frame size information. Determine the block size.
When the maximum encoding block size and the upper limit of the number of division layers are multiplexed in the bit stream on the moving image encoding device side, values decoded from the bit stream are used.
Hereinafter, in the video decoding apparatus, the maximum encoded block size is referred to as a maximum decoded block size, and the maximum encoded block is referred to as a maximum decoded block.
The variable length decoding unit 31 decodes the division state of the maximum decoding block as shown in FIG. 6 for each determined maximum decoding block. Based on the decoded division state, a decoded block (a block corresponding to the “encoded block” of the moving image encoding apparatus in FIG. 1) is identified hierarchically (step ST23).

  Next, the variable length decoding unit 31 decodes the encoding mode assigned to the decoding block. Based on the information included in the decoded coding mode, the decoded block is further divided into one or more prediction blocks which are prediction processing units, and the prediction parameters assigned to the prediction block units are decoded (step ST24).

That is, when the encoding mode assigned to the decoding block is the intra encoding mode, the variable length decoding unit 31 is included in the decoding block and is intra for each of one or more prediction blocks serving as a prediction processing unit. Decode prediction parameters.
On the other hand, when the coding mode assigned to the decoding block is the inter coding mode, the inter prediction parameter and the motion vector are decoded for each one or more prediction blocks included in the decoding block and serving as a prediction processing unit. (Step ST24).

  Further, the variable length decoding unit 31 divides the decoded block into one or a plurality of transform blocks serving as a transform processing unit based on the transform block size information included in the prediction differential coding parameter, and the compressed data for each transform block. (Transformation coefficient after transformation / quantization) is decoded (step ST24).

If the encoding mode m (B ⁿ ) variable-length decoded by the variable-length decoding unit 31 is an intra-encoding mode (when m (B ⁿ ) ∈INTRA), the changeover switch 33 is changed by the variable-length decoding unit 31. The intra-prediction parameter for each prediction block subjected to variable length decoding is output to the intra-prediction unit 34.
On the other hand, (the case of m ^{(B n)} ∈INTER) variable length decoded coding mode m ^{(B n)} is if the inter coding mode by the variable length decoding unit 31, variable length decoding by the variable length decoding unit 31 The predicted inter prediction parameters and motion vectors in units of prediction blocks are output to the motion compensation unit 35.

When the coding mode m (B ⁿ ) variable-length decoded by the variable-length decoding unit 31 is the intra coding mode (m (B ⁿ ) ∈INTRA) (step ST25), the intra prediction unit 34 selects the changeover switch 33. 1 is received, and the intra prediction parameter is obtained by referring to the decoded image stored in the intra prediction memory 37 in the same procedure as the intra prediction unit 4 in FIG. and implementing intra prediction process to generate an intra prediction image _{P INTRAi} ⁿ for each of the prediction block _P ^{i n} of the decoded block ^{B n} using (step ST26).
That is, the intra prediction unit 34, after generating an intra prediction image P _INTRAi ⁿ above, in the same manner as the intra prediction unit 4 in FIG. 1, from among one or more filters prepared in advance, the intra prediction image select a filter according to the state of the various parameters are known at the time of generating the P _INTRAi ^n, by using the filter, and out the filter processing for the intra prediction image P _INTRAi ^n, intra prediction of the filtered the image _{P INTRAi} ⁿ is the final intra-prediction image.
That is, using the same parameter as the parameter used for filter selection in the intra prediction unit 4, a filter is selected by the same method as the filter selection method in the intra prediction unit 4, and the filter process is performed.

When the coding mode m (B ⁿ ) variable-length decoded by the variable-length decoding unit 31 is the inter coding mode (m (B ⁿ ) ∈INTER) (step ST25), the motion compensation unit 35 performs the changeover switch 33. The motion vector and the inter prediction parameter for each prediction block output from the above are received, and the motion vector and the inter prediction parameter are used while referring to the decoded image after filtering stored in the motion compensated prediction frame memory 39. by carrying out inter-prediction processing for each of the prediction block _P ^{i n} of the decoded block ^{B n} to generate an inter prediction image _{P INTERi} ⁿ (step ST27).

When receiving the compressed data and the prediction difference encoding parameter from the variable length decoding unit 31, the inverse quantization / inverse conversion unit 32 performs the prediction difference encoding in the same procedure as the inverse quantization / inverse conversion unit 8 of FIG. The compressed data is inversely quantized with reference to the parameters.
In addition, referring to the prediction difference encoding parameter, the inverse orthogonal transform process is performed on the transform coefficient that is the compressed data after the inverse quantization, and the local output from the inverse quantization / inverse transform unit 8 in FIG. The same decoded prediction difference signal as the decoded prediction difference signal is calculated (step ST28).

Addition unit 36, decodes the prediction difference signal calculated by the inverse quantization and inverse transform unit 32, an intra prediction image P _{INTRAi n} generated by the intra prediction unit 34 ^or, inter prediction generated by the motion compensation unit 35 by adding one of the image P _INTERi ⁿ calculates a decoded image, and outputs the decoded image to the loop filter unit 38, and stores the decoded image to the intra prediction memory 37 (step ST29).
This decoded image becomes a decoded image signal used in the subsequent intra prediction processing.

Loop filter unit 38, the process of step ST23~ST29 to all decoded block ^{B n} is completed (step ST30), the decoded image outputted from the addition unit 36, and performs a predetermined filtering process, the filter The decoded image after processing is stored in the motion compensated prediction frame memory 39 (step ST31).
Note that the filter processing by the loop filter unit 38 may be performed in units of the maximum decoded block of the input decoded image or individual decoding blocks, or is performed for one picture after the decoded image for one picture is input. May be.
Examples of the predetermined filter process include a process of filtering the block boundary so that the discontinuity (block noise) of the encoded block boundary becomes inconspicuous, and a filter process that compensates for distortion of the decoded image. Two or more types of these filter processes may be performed, and the same filter process as that of the loop filter unit 11 is performed.
This decoded image becomes a reference image for motion compensation prediction and also becomes a reproduced image.

As is apparent from the above, according to the first embodiment, the intra prediction unit 4 of the video encoding device performs intra-frame prediction processing using an intra-frame encoded image signal, thereby allowing intra-frame prediction processing. When generating a prediction image, a filter is selected from among one or more filters prepared in advance according to the state of various parameters related to the encoding of the block to be filtered, and the prediction block P is used using the filter. since it is configured to perform the filtering for the boundaries of the _i ^n, an effect that it is possible to reduce the distortion generated at the boundary of the prediction block P _i ^n, increasing the image quality.

Further, according to the first embodiment, a quantum intra prediction unit 4 is included (1) the size of the partition _{^{P i n (l i n ×}} m i n), (2) the prediction error encoding parameters Parameter, (3) the distance between the reference pixel used for generating the intermediate predicted image and the pixel to be filtered, and (4) the index value of the intra prediction mode when the intermediate predicted image is generated. Since the filter is selected in consideration of the above parameters, the continuity with the adjacent encoded signal is lost when the directional prediction, average value prediction, and plane prediction are performed. The effect of suppressing the prediction error in the boundary portion is obtained, and the effect of improving the prediction efficiency is achieved.

  According to the first embodiment, the intra prediction unit 34 of the video decoding device prepares in advance when generating an intra predicted image by performing an intra frame prediction process using a decoded image signal in a frame. Since the filter is selected according to the state of various parameters related to the decoding of the filter processing target block from the one or more filters that are being processed, and the filter process is performed on the prediction image using the filter. In addition, it is possible to reduce the distortion generated at the boundary of the filter processing target block and to generate the same intra-predicted image as the intra-predicted image generated on the video encoding device side on the video decoding device side. .

Further, according to the first embodiment, the intra prediction unit 34 performs (1) the size of the partition P _i ⁿ (l _i ⁿ × m _i ⁿ ) and (2) the quantum included in the prediction error encoding parameter. Parameter, (3) the distance between the reference pixel used for generating the intermediate predicted image and the pixel to be filtered, and (4) the index value of the intra prediction mode when the intermediate predicted image is generated. Since the filter is selected in consideration of the above parameters, the continuity with the adjacent encoded signal is lost when the directional prediction, average value prediction, and plane prediction are performed. The effect of suppressing the prediction error in the boundary portion is obtained, and the same intra prediction image as the intra prediction image generated on the video encoding device side can be generated on the video decoding device side as well. There is an effect that can be done.

  In the present invention, any constituent element of the embodiment can be modified or any constituent element of the embodiment can be omitted within the scope of the invention.

  1 block division unit (block division unit), 2 encoding control unit (encoding control unit), 3 selector switch, 4 intra prediction unit (intra prediction unit), 5 motion compensation prediction unit (motion compensation prediction unit), 6 subtraction Unit (difference image generation unit), 7 transform / quantization unit (image compression unit), 8 inverse quantization / inverse transform unit, 9 addition unit, 10 intra prediction memory (intra prediction unit), 11 loop filter unit, 12 Motion compensation prediction frame memory (motion compensation prediction means), 13 variable length coding section (variable length coding means), 31 variable length decoding section (variable length decoding means), 32 inverse quantization / inverse transform section (difference image generation Means), 33 selector switch, 34 intra prediction unit (intra prediction unit), 35 motion compensation unit (motion compensation prediction unit), 36 addition unit (decoded image generation unit), 37 intra prediction Measurement memory (intra prediction means), 38 loop filter section, 39 motion compensation prediction frame memory (motion compensation prediction means).

Claims (12)

Determine the maximum size of the coding block that is the processing unit when the coding process is performed, and determine the upper limit number of layers when the coding block of the maximum size is hierarchically divided and can be used. Coding control means for selecting a coding mode corresponding to each coding block divided hierarchically from among one or more coding modes; and a maximum size determined by the coding control means for an input image. And a block dividing means for dividing the encoded block hierarchically up to the upper limit number of layers determined by the encoding control means, and a code divided by the block dividing means When the intra coding mode is selected by the coding control means as the coding mode corresponding to the coding block, the coding block is predicted. For each prediction block that is a prediction processing unit, intra prediction means for generating a prediction image by performing intra-frame prediction processing corresponding to the intra coding mode, the coded block divided by the block dividing means, and the above Difference image generation means for generating a difference image from the prediction image generated by the intra prediction means, and image compression means for compressing the difference image generated by the difference image generation means and outputting compressed data of the difference image A bit stream in which the compressed data output from the image compression means and the encoding mode selected by the encoding control means are variable-length encoded, and the compressed data and the encoded data of the encoding mode are multiplexed. Variable length encoding means for generating
The intra prediction means selects a predetermined filter from one or more filters prepared in advance, and performs filter processing on a block boundary of a predicted image generated by performing intra-frame prediction processing using the filter And a prediction image after the filter process is output to the difference image generation means. When the inter coding mode is selected by the coding control unit as the coding mode corresponding to the coding block divided by the block dividing unit, the prediction block that is a prediction processing unit when performing the coding block prediction process A motion compensation prediction unit is provided that generates a prediction image by performing a motion compensation prediction process for the prediction block using a reference image for each
The difference image generation means generates a difference image between the encoded block divided by the block division means and a prediction image generated by the intra prediction means or the motion compensation prediction means. Video encoding device. The encoding control means determines, for each encoding block, a quantization parameter and a transform block size used when the difference image is compressed, and an intra prediction parameter used when the prediction process is performed or Inter prediction parameters are determined for each prediction block of the coding block,
The image compression means performs the conversion process of the difference image generated by the difference image generation means in the unit of the transform block size determined by the encoding control means, and the quantization parameter determined by the encoding control means , And quantize the transform coefficient of the difference image, to output the quantized transform coefficient as compressed data of the difference image,
The variable-length coding unit is configured to perform the variable-length coding on the compressed data output from the image compression unit and the coding mode selected by the coding control unit, and the intra prediction parameter determined by the coding control unit. Alternatively, the inter prediction parameter, the quantization parameter, and the transform block size are subjected to variable length coding, and the compressed data, the coding mode, the intra prediction parameter or the inter prediction parameter, the quantization parameter, and the transform block size are encoded. 3. The moving picture encoding apparatus according to claim 2, wherein a bit stream in which encoded data is multiplexed is generated.   The intra prediction means includes a block size for performing intra prediction, a quantization parameter determined by the encoding control means, a distance between a reference pixel and a filtering target pixel used when generating a predicted image, and the encoding control means. 4. The moving picture coding apparatus according to claim 3, wherein a filter to be used for the filtering process is selected in consideration of at least one of the determined intra prediction parameters. The size of the block for which intra prediction is performed, the quantization parameter determined by the encoding control means, the distance between the reference pixel used for generating the predicted image and the pixel to be filtered, and the intra prediction determined by the encoding control means For each combination to be considered when selecting a filter among the parameters, a filter selection table indicating the filter used for the filter processing is prepared.
5. The moving picture coding apparatus according to claim 4, wherein the intra prediction means selects a filter to be used for the filtering process with reference to the table. Variable-length decoding means for variable-length decoding compressed data and coding modes related to each coding block hierarchically divided from the encoded data multiplexed into the bitstream, and variable-length decoding by the variable-length decoding means When the coding mode related to the coded block is the intra coding mode, an intra-frame corresponding to the intra coding mode is provided for each prediction block serving as a prediction processing unit when performing the prediction processing of the coding block. Intra prediction means for generating a prediction image by performing a prediction process, difference image generation means for generating a difference image before compression from compressed data related to an encoded block variable length decoded by the variable length decoding means, and A decoded image is generated by adding the difference image generated by the difference image generation means and the prediction image generated by the intra prediction means. And a decoded image generation unit that,
The intra prediction means selects a predetermined filter from one or more filters prepared in advance, and performs a filter process on a block boundary of a predicted image generated by performing an intra-frame prediction process using the filter. A moving picture decoding apparatus that performs and outputs a predicted image after filter processing to the decoded image generation means. When the coding mode related to the coding block that has been variable-length decoded by the variable-length decoding means is the inter coding mode, a reference image is provided for each prediction block that is a prediction processing unit when performing the prediction processing of the coding block. Using a motion compensation prediction unit that performs a motion compensation prediction process on the prediction block to generate a prediction image;
The decoded image generation means adds the difference image generated by the difference image generation means and the prediction image generated by the intra prediction means or the motion compensation prediction means to generate a decoded image. The moving picture decoding apparatus described. The variable length decoding means performs variable length decoding on the compressed data, coding mode, intra prediction parameter or inter prediction parameter, quantization parameter, and transform block size related to each coding block from the coded data multiplexed in the bitstream. And
The difference image generation means dequantizes the compressed data related to the coding block using the quantization parameter related to the coding block variable-length decoded by the variable length decoding means, 8. The moving picture decoding apparatus according to claim 7, wherein a difference image before compression is generated by performing an inverse transform process of the compressed data after quantization.   The intra prediction means includes a block size for performing intra prediction, a quantization parameter variable-length decoded by the variable-length decoding means, a distance between a reference pixel used for generating a predicted image and a pixel to be filtered, and the variable-length decoding. 9. The moving picture decoding apparatus according to claim 8, wherein a filter to be used for the filter processing is selected in consideration of at least one of the intra prediction parameters variable-length decoded by the means. The size of the block for which intra prediction is performed, the quantization parameter variable-length decoded by the variable-length decoding means, the distance between the reference pixel used for generating the predicted image and the pixel to be filtered, and the variable-length decoding by the variable-length decoding means For each combination to be considered when selecting a filter from among the intra prediction parameters, a filter selection table indicating a filter used for the filter processing is prepared.
10. The moving picture decoding apparatus according to claim 9, wherein the intra prediction means selects a filter to be used for the filtering process with reference to the table. The encoding control means determines the maximum size of the encoding block that is a processing unit when the encoding process is performed, and sets the upper limit number of layers when the encoding block of the maximum size is hierarchically divided. An encoding control processing step for determining and selecting an encoding mode corresponding to each encoding block to be hierarchically divided from one or more encoding modes that can be used; Is divided into encoded blocks of the maximum size determined in the encoding control processing step, and the encoded blocks are hierarchically divided until the upper limit number of hierarchies determined in the encoding control processing step is reached. The block division processing step and the intra prediction means, the coding mode corresponding to the coding block divided in the block division processing step is used as the coding mode. When the intra coding mode is selected in the control processing step, intra-frame prediction processing corresponding to the intra coding mode is performed for each prediction block that is a prediction processing unit when performing the prediction processing of the coding block. And an intra prediction processing step for generating a prediction image, and a difference in which the difference image generation means generates a difference image between the encoded block divided in the block division processing step and the prediction image generated in the intra prediction processing step. The image generation processing step, the image compression means compresses the difference image generated in the difference image generation processing step, and outputs compressed data of the difference image, and the variable length encoding means The compressed data output in the image compression processing step and the encoding mode selected in the encoding control processing step And-length coding, a variable length coding process steps encoded data of the compressed data and the encoding mode to generate a multiplexed bit stream,
In the intra prediction processing step, a filter for a block boundary of a prediction image generated by selecting a predetermined filter from one or more filters prepared in advance and performing intra-frame prediction processing using the filter is used. A moving picture encoding method comprising: performing processing, and outputting a predicted image after filter processing to the difference image generation unit. Variable length decoding means, variable length decoding processing step for variable length decoding compressed data and coding mode related to each coding block hierarchically divided from the coded data multiplexed into the bitstream, and intra prediction When the coding mode related to the coding block subjected to variable length decoding in the variable length decoding processing step is the intra coding mode, the prediction block serving as a prediction processing unit when performing the prediction processing of the coding block In each case, an intra prediction processing step for generating a predicted image by performing an intra-frame prediction process corresponding to the intra coding mode, and a coding in which the difference image generating means is variable length decoded in the variable length decoding processing step. The difference image generation processing step for generating a difference image before compression from the compressed data relating to the block, and the decoded image generation means include the difference And a decoded image generation processing step of adding the prediction image generated by the differential image and the intra prediction processing steps generated by the image generation processing step of generating a decoded image,
In the intra prediction processing step, a filter for a block boundary of a prediction image generated by selecting a predetermined filter from one or more filters prepared in advance and performing intra-frame prediction processing using the filter is used. A moving picture decoding method comprising: performing processing, and outputting a predicted image after filter processing to the decoded picture generation unit.

Images