JP2011223319A - Moving image encoding apparatus and moving image decoding apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving image encoding apparatus and a moving image decoding apparatus, which enable to generate a prediction image generated by a motion compensation predict from a reference image containing many high-frequency components and to compress and encode at high efficiency even when a video signal containing many high-frequency components is compressed at a high compression rate.SOLUTION: A motion compensation predict unit corrects a high-frequency component of a reference image 15 stored to a motion compensation predict frame memory, has an interpolation image generation unit 43 that generates a reference image 207 of virtual pixel precision, and generates a prediction image by using either the reference image 15 or the reference image of the virtual pixel precision depending on a motion vector.

Description

  The present invention relates to a moving image encoding device that divides a moving image into predetermined regions and performs encoding in units of regions, and a moving image decoding device that decodes encoded moving images in units of predetermined regions.

  Conventionally, in an international standard video coding system such as MPEG (and ITU-T H.26x, block data (8 × 8 pixels corresponding to a luminance signal 16 × 16 pixels) and a color signal corresponding to each frame of a video signal (16 × 16 pixels) A compression method based on a motion compensation technique and an orthogonal transform / transform coefficient quantization technique is used in units of macroblocks).

  In the video coding method, when motion compensation prediction is performed, a sample whose phase is different from the sample of the input image signal, such as half pixel or ¼ pixel, is virtually created, and motion prediction is performed using the virtual sample. It is known that the coding efficiency is improved by the above. Also in the conventional international standard video encoding system, the pixel at the virtual sample position is generated by interpolation processing.

  In the MPEG-4 AVC (Moving Picture Experts Group-4 Advanced Video Coding) standard of Non-Patent Document 1, a virtual pixel at a half pixel position (a phase point in the middle of a sample of an input image signal) is generated in this interpolation process. In order to do so, a 6-tap fixed linear filter is used. By using a filter with a long tap number, a device for improving reproducibility up to higher frequency components has been devised.

ISO / IEC 14496-10 | ITU-T recommendation H. H.264

  However, in the motion compensation prediction process described in Non-Patent Document 1, it is difficult to reproduce a high-frequency component cut by the high compression process, and there is a problem that it does not contribute to performance improvement at the time of high compression. In particular, when a video signal containing a lot of high-frequency components, such as fine edges, is highly compressed, the prediction error in the input video signal containing a lot of high-frequency components increases because the high-frequency components of the predicted image in the frame memory are deleted. There is a problem that the efficiency of motion compensation prediction is reduced.

  The present invention has been made in order to solve the above-described problem, and a predicted image generated by motion compensation prediction even when a video signal containing a large amount of high-frequency components such as fine edges is highly compressed. It is an object of the present invention to obtain a video encoding device and a video decoding device that can generate a video from a reference image containing a large amount of high-frequency components and can efficiently perform compression encoding.

  In the moving image encoding apparatus according to the present invention, the motion compensation prediction unit corrects a high frequency component of the reference image data in the motion compensation prediction frame memory, and generates an interpolated image generation unit that generates reference image data with virtual pixel accuracy. The prediction image is generated by switching between using the reference image data in the motion prediction frame memory or generating and using the reference image data with the virtual pixel accuracy according to the motion vector.

  The moving image decoding apparatus according to the present invention includes an interpolated image generation unit in which the motion compensation prediction unit corrects high frequency components of the reference image data in the motion compensation prediction frame memory to generate reference image data with virtual pixel accuracy. The reference image data in the motion-compensated prediction frame memory is used according to the motion vector included in the prediction parameter or the reference image data with the virtual pixel accuracy is generated and used to generate a prediction image. is there.

  According to the present invention, the prediction image is generated by switching between using the reference image data in the motion compensated prediction frame memory or generating and using the reference image data with virtual pixel accuracy. Even when a video signal containing a large amount of high-frequency components is highly compressed, a prediction image generated by motion compensation prediction can be generated from a reference image containing a large amount of high-frequency components, enabling efficient compression coding. Thus, a moving image encoding device and a moving image decoding device can be obtained.

It is a block diagram which shows the structure of the moving image encoder which concerns on Embodiment 1 of this invention. It is a figure which shows an example of the encoding mode of the picture which performs the prediction encoding of a time direction. It is a figure which shows another example of the encoding mode of the picture which performs the prediction encoding of a time direction. 3 is a block diagram showing an internal configuration of a motion compensation prediction unit of the video encoding apparatus according to Embodiment 1. FIG. It is a figure explaining the determination method of the predicted value of the motion vector according to encoding mode. It is a figure which shows an example of adaptation of the transform block size according to encoding mode. It is a figure which shows another example of adaptation of the transform block size according to encoding mode. 3 is a block diagram showing an internal configuration of a transform / quantization unit of the video encoding apparatus according to Embodiment 1. FIG. It is a block diagram which shows the structure of the moving image decoding apparatus which concerns on Embodiment 1 of this invention. It is a block diagram which shows the internal structure of the variable length encoding part of the moving image encoder which concerns on Embodiment 2 of this invention. It is a figure which shows an example of a binarization table, and shows the state before an update. It is a figure which shows an example of a probability table. It is a figure which shows an example of a state transition table. FIG. 13A is a diagram illustrating a procedure for generating context identification information. FIG. 13A is a diagram illustrating a binarization table in binary tree representation, and FIG. 13B is a diagram illustrating the positional relationship between an encoding target macroblock and peripheral blocks. FIG. It is a figure which shows an example of a binarization table, and shows the state after an update. It is a block diagram which shows the internal structure of the variable length decoding part of the moving image decoding apparatus which concerns on Embodiment 2 of this invention. It is a block diagram which shows the internal structure of the interpolation image generation part with which the motion compensation prediction part of the moving image encoder which concerns on Embodiment 3 of this invention is provided.

Embodiment 1 FIG.
In the first embodiment, each frame image of video is used as an input, motion compensation prediction is performed between adjacent frames, and compression processing by orthogonal transformation / quantization is performed on the obtained prediction difference signal. A moving picture coding apparatus that generates a bit stream by performing variable length coding and a moving picture decoding apparatus that decodes the bit stream will be described.

  FIG. 1 is a block diagram showing a configuration of a moving picture coding apparatus according to Embodiment 1 of the present invention. 1 divides a macroblock image obtained by dividing each frame image of the input video signal 1 into a plurality of blocks having a macroblock size of 4 into one or more sub-blocks according to the encoding mode 7. When the block dividing unit 2 that outputs the macro / sub-block image 5 and the macro / sub-block image 5 are input, the macro / sub-block image 5 is intraframed using the image signal of the intra prediction memory 28. When the intra prediction unit 8 that predicts and generates the predicted image 11 and the macro / subblock image 5 are input, the reference image 15 of the motion compensated prediction frame memory 14 is used for the macro / subblock image 5. A motion compensated prediction unit 9 that performs motion compensated prediction to generate a predicted image 17 and a macro / sub-block image 5 according to the encoding mode 7 From the switching unit 6 that is input to either the tiger prediction unit 8 or the motion compensation prediction unit 9 and the macro / sub-block image 5 that is output from the block division unit 2, either the intra prediction unit 8 or the motion compensation prediction unit 9 is used. The subtracter 12 that generates the prediction difference signal 13 by subtracting the prediction images 11 and 17 output from one of them, and the conversion / quantization that performs the conversion and quantization processing on the prediction difference signal 13 to generate the compressed data 21 Unit 19, variable-length encoding unit 23 that entropy-encodes compressed data 21 and multiplexes it into bit stream 30, and inverse that generates local decoded prediction difference signal 24 by performing inverse quantization and inverse transform processing on compressed data 21 The quantized / inverse transform unit 22 and the prediction images 11 and 17 output from either the intra prediction unit 8 or the motion compensation prediction unit 9 are added to the inverse quantization / inverse transform unit 22. An adder 25 for generating a local decoded image signal 26, an intra prediction memory 28 for storing the local decoded image signal 26, and a loop filter unit 27 for generating a local decoded image 29 by filtering the local decoded image signal 26. And a motion-compensated prediction frame memory 14 that stores the locally decoded image 29.

  The encoding control unit 3 includes information necessary for processing of each unit (macroblock size 4, encoding mode 7, optimal encoding mode 7a, prediction parameter 10, optimal prediction parameters 10a and 18a, compression parameter 20, and optimal compression parameter 20a. ) Is output. Details of the macroblock size 4 and the encoding mode 7 will be described below. Details of other information will be described later.

The encoding control unit 3 specifies the macro block size 4 of each frame image of the input video signal 1 to the block dividing unit 2 and selects all the selectable macro blocks according to the picture type for each encoding target macro block. Indicates encoding mode 7.
The encoding control unit 3 can select a predetermined encoding mode from the set of encoding modes, but this encoding mode set is arbitrary, for example, the set shown in FIG. 2A or FIG. 2B described below. A predetermined encoding mode can be selected from the list.

FIG. 2A is a diagram illustrating an example of a coding mode of a P (Predictive) picture that performs predictive coding in the time direction. In FIG. 2A, mb_modes 0 to 2 are modes (inter) for encoding macroblocks (M × L pixel blocks) by inter-frame prediction. mb_mode0 is a mode in which one motion vector is assigned to the entire macroblock, and mb_mode1 and 2 are modes in which the macroblock is equally divided horizontally or vertically, and different motion vectors are assigned to the divided sub-blocks. .
mb_mode3 is a mode in which a macroblock is divided into four and different coding modes (sub_mb_mode) are assigned to the divided subblocks.

  sub_mb_mode 0 to 4 are coding modes respectively assigned to sub-blocks (m × 1 pixel blocks) obtained by dividing the macro block into four when mb_mode 3 is selected in the macro block coding mode, and sub_mb_mode 0 Is a mode (intra) in which sub-blocks are encoded by intra-frame prediction. The other modes are modes for inter-frame encoding (inter), sub_mb_mode1 is a mode in which one motion vector is assigned to the entire subblock, and sub_mb_mode2 and 3 are subdivided into subblocks horizontally or vertically, respectively. Sub_mb_mode4 is a mode in which a different motion vector is assigned to each divided subblock, and sub_mb_mode4 is a mode in which a subblock is divided into four and a different motion vector is assigned to each divided subblock.

FIG. 2B is a diagram illustrating another example of the coding mode of the P picture that performs predictive coding in the time direction. In FIG. 2B, mb_modes 0 to 6 are modes (inter) for encoding macroblocks (M × L pixel blocks) by inter-frame prediction. mb_mode0 is a mode in which one motion vector is assigned to the entire macroblock, and mb_modes 1 to 6 divide the macroblock in the horizontal, vertical, or diagonal directions, and assign different motion vectors to the divided subblocks. Mode.
mb_mode7 is a mode in which a macroblock is divided into four and different coding modes (sub_mb_mode) are allocated to the divided subblocks.

  sub_mb_mode 0 to 8 are coding modes respectively assigned to sub-blocks (m × 1 pixel blocks) obtained by dividing the macroblock into four when mb_mode7 is selected as the macroblock coding mode, and sub_mb_mode0 Is a mode (intra) in which sub-blocks are encoded by intra-frame prediction. The other modes are modes (inter) for encoding by inter-frame prediction, sub_mb_mode1 is a mode in which one motion vector is assigned to the entire subblock, and sub_mb_modes 2 to 7 are subblocks divided horizontally, vertically, or diagonally, respectively. The sub_block_mode 8 is a mode in which a sub-block is divided into four, and a different motion vector is assigned to each divided sub-block.

  The block dividing unit 2 divides each frame image of the input video signal 1 input to the moving image encoding device into macroblock images having a macroblock size of 4 specified by the encoding control unit 3. Further, the block division unit 2 assigns different coding modes to the sub-blocks in which the coding mode 7 specified by the coding control unit 3 divides the macroblock (sub_mb_mode1 to 4 in FIG. 2A or sub_mb_mode1 in FIG. 2B). When including .about.8), the macroblock image is divided into sub-block images indicated by the encoding mode 7. Therefore, the block image output from the block dividing unit 2 is either a macro block image or a sub block image according to the encoding mode 7. Hereinafter, this block image is referred to as a macro / sub-block image 5.

  When the horizontal or vertical size of each frame of the input video signal 1 is not an integer multiple of the horizontal size or vertical size of the macroblock size 4, the frame size is equal to the macroblock size for each frame of the input video signal 1. A frame (extended frame) in which pixels are expanded in the horizontal direction or the vertical direction until an integral multiple is generated. For example, when expanding a pixel in the vertical direction, the pixel at the bottom of the original frame is repeatedly filled, or a pixel having a fixed pixel value (gray, black, white, etc.) There are methods such as filling. Similarly, when extending the pixels in the horizontal direction, there are methods such as repeatedly filling the rightmost pixel of the original frame, or filling with a pixel having a fixed pixel value (gray, black, white, etc.). An extension frame generated for each frame of the input video signal 1 and having an integer multiple of the macroblock size is input to the block dividing unit 2 in place of each frame image of the input video signal 1.

  Note that the macroblock size 4 and the frame size (horizontal size and vertical size) of each frame of the input video signal 1 are variable because they are multiplexed into a bitstream in units of sequences or pictures consisting of one or more frames. It is output to the long encoding unit 23.

  Note that the macroblock size value may be defined by a profile or the like instead of being directly multiplexed into the bitstream. In this case, identification information for identifying the profile in sequence units is multiplexed into the bit stream.

  The switching unit 6 is a switch that switches the input destination of the macro / sub-block image 5 in accordance with the encoding mode 7. When the encoding mode 7 is a mode for encoding by intra-frame prediction (hereinafter referred to as intra-frame prediction mode), the switching unit 6 inputs the macro / sub-block image 5 intra prediction unit 8 to When the conversion mode 7 is a mode for encoding by inter-frame prediction (hereinafter referred to as inter-frame prediction mode), the macro / sub-block image 5 is input to the motion compensation prediction unit 9.

  The intra prediction unit 8 performs intra-frame prediction on the input macro / subblock image 5 in units of a macroblock to be encoded specified by the macroblock size 4 or a subblock specified by the encoding mode 7. . The intra prediction unit 8 uses the image signals in the frames stored in the intra prediction memory 28 for all intra prediction modes included in the prediction parameter 10 instructed from the encoding control unit 3. A predicted image 11 is generated for each.

  Here, details of the prediction parameter 10 will be described. When the encoding mode 7 is the intra-frame prediction mode, the encoding control unit 3 designates the intra prediction mode as the prediction parameter 10 corresponding to the encoding mode 7. In this intra prediction mode, for example, a macroblock or sub-block is set to a 4 × 4 pixel block unit, and a prediction image is generated using pixels around a unit block of an image signal in the intra prediction memory 28. A mode in which a block or sub-block is made into 8 × 8 pixel block units, and a prediction image is generated using pixels around a unit block of the image signal in the intra prediction memory 28, and a macro block or sub-block is 16 × 16 A mode for generating a prediction image using pixels around a unit block of an image signal in the intra prediction memory 28 in units of pixel blocks, a mode for generating a prediction image from an image obtained by reducing the size of a macro block or sub block, and the like. is there.

  The motion-compensated prediction unit 9 designates a reference image 15 to be used for generating a predicted image from one or more frames of reference image data stored in the motion-compensated prediction frame memory 14, and the reference image 15 and the macro / subblock Using the image 5, motion compensation prediction according to the encoding mode 7 instructed from the encoding control unit 3 is performed, and a prediction parameter 18 and a prediction image 17 are generated.

  Here, details of the prediction parameter 18 will be described. When the coding mode 7 is the inter-frame prediction mode, the motion compensated prediction unit 9 uses the motion vector as the prediction parameter 18 corresponding to the coding mode 7 and the reference image identification number (reference image index) indicated by each motion vector. Etc. Details of the method of generating the prediction parameter 18 will be described later.

  The subtraction unit 12 subtracts either the predicted image 11 or the predicted image 17 from the macro / sub-block image 5 to obtain a predicted difference signal 13. In addition, the prediction difference signal 13 is each produced | generated with respect to all the prediction images 11 which the intra estimation part 8 produces | generates according to all the intra prediction modes which the prediction parameter 10 designates.

The prediction difference signal 13 generated in accordance with all intra prediction modes specified by the prediction parameter 10 is evaluated by the encoding control unit 3, and the optimal prediction parameter 10a including the optimal intra prediction mode is determined. As an evaluation method, for example, the encoding cost J ₂ described later is calculated using the compressed data 20 obtained by transforming and quantizing the prediction difference signal 13, and an intra prediction mode that minimizes the encoding cost J ₂ is selected.

The encoding control unit 3 evaluates the prediction difference signals 13 respectively generated for all modes included in the encoding mode 7 in the intra prediction unit 8 or the motion compensation prediction unit 9, and performs encoding based on the evaluation result. The optimum coding mode 7a that can obtain the optimum coding efficiency is determined from the modes 7. Also, the encoding control unit 3 determines the optimum prediction parameters 10a, 18a and the optimum compression parameter 20a corresponding to the optimum coding mode 7a from the prediction parameters 10, 18 and the compression parameter 20. Each determination procedure will be described later.
As described above, in the case of the intra-frame prediction mode, the intra prediction mode is included in the prediction parameter 10 and the optimal prediction parameter 10a. On the other hand, in the inter-frame prediction mode, the prediction parameter 18 and the optimal prediction parameter 18a include a motion vector, a reference image identification number (reference image index) indicated by each motion vector, and the like.
Further, the compression parameter 20 and the optimum compression parameter 20a include a transform block size, a quantization step size, and the like.

  As a result of this determination procedure, the encoding control unit 3 outputs the optimal encoding mode 7a, the optimal prediction parameters 10a and 18a, and the optimal compression parameter 20a for the macroblock or subblock to be encoded to the variable length encoding unit 23. . Also, the encoding control unit 3 outputs the optimum compression parameter 20 a of the compression parameters 20 to the transform / quantization unit 19 and the inverse quantization / inverse transform unit 22.

  The transform / quantization unit 19 includes the optimal encoding mode 7a determined by the encoding control unit 3 and the optimal prediction among the plurality of prediction difference signals 13 generated corresponding to all modes included in the encoding mode 7. A prediction difference signal 13 (hereinafter referred to as an optimum prediction difference signal 13a) corresponding to the prediction images 11 and 17 generated based on the parameters 10a and 18a is selected, and the optimum prediction difference signal 13a is encoded. A transform coefficient is calculated by performing transform processing such as DCT (Discrete Cosine Transform) based on the transform block size of the optimum compression parameter 20a determined by the control unit 3, and the transform coefficient is encoded by the encoding control unit 3 Is quantized based on the quantization step size of the optimal compression parameter 20a instructed from the above, and the quantized compressed data 21 which is a transformed coefficient is dequantized / inverse transformed. And outputs to the second and the variable length coding unit 23.

  The inverse quantization / inverse transform unit 22 performs inverse transform processing such as inverse DCT by performing inverse quantization on the compressed data 21 input from the transform / quantization unit 19 using the optimal compression parameter 20a. A local decoded prediction difference signal 24 of the difference signal 13 a is generated and output to the adding unit 25.

  The adding unit 25 adds the local decoded prediction difference signal 24 and the predicted image 11 or the predicted image 17 to generate a local decoded image signal 26, and outputs the local decoded image signal 26 to the loop filter unit 27 and intra. Store in the prediction memory 28. This locally decoded image signal 26 becomes an image signal for intra-frame prediction.

  The loop filter unit 27 performs a predetermined filtering process on the local decoded image signal 26 input from the adding unit 25 and stores the local decoded image 29 after the filtering process in the motion compensated prediction frame memory 14. This locally decoded image 29 becomes the reference image 15 for motion compensation prediction. The filtering process by the loop filter unit 27 may be performed in units of macroblocks of the input local decoded image signal 26, or after one local decoded image signal 26 corresponding to one macroblock is input. You may go together.

  The variable length encoding unit 23 includes the compressed data 21 output from the transform / quantization unit 19, the optimal encoding mode 7a output from the encoding control unit 3, the optimal prediction parameters 10a and 18a, and the optimal compression parameter. 20a is entropy-encoded, and a bit stream 30 indicating the encoding result is generated. The optimal prediction parameters 10a and 18a and the optimal compression parameter 20a are encoded in units corresponding to the encoding mode indicated by the optimal encoding mode 7a.

  As described above, the moving picture coding apparatus according to the first embodiment is configured so that the motion compensation prediction unit 9 and the transform / quantization unit 19 operate in cooperation with the coding control unit 3, respectively. The coding mode, the prediction parameter, and the compression parameter (that is, the optimum coding mode 7a, the optimum prediction parameters 10a and 18a, and the optimum compression parameter 20a) are determined.

  Here, a procedure for determining a coding mode, a prediction parameter, and a compression parameter by which the coding control unit 3 can obtain optimum coding efficiency is as follows. Prediction parameters, 2. 2. compression parameters; The description will be made in the order of the encoding mode.

1. Prediction Parameter Determination Procedure Here, when the encoding mode 7 is the inter-frame prediction mode, a prediction parameter including a motion vector related to the inter-frame prediction, a reference image identification number (reference image index) indicated by each motion vector, etc. A procedure for determining 18 will be described.

  In the motion compensation prediction unit 9, in cooperation with the encoding control unit 3, all the encoding modes 7 (for example, the encoding modes shown in FIG. 2A or FIG. 2B) instructed from the encoding control unit 3 to the motion compensation prediction unit 9 are performed. The prediction parameter 18 is determined for each set. The detailed procedure will be described below.

  FIG. 3 is a block diagram showing an internal configuration of the motion compensation prediction unit 9. The motion compensation prediction unit 9 illustrated in FIG. 3 includes a motion compensation region division unit 40, a motion detection unit 42, and an interpolated image generation unit 43. As input data, the encoding mode 7 input from the encoding control unit 3, the macro / sub-block image 5 input from the switching unit 6, and the reference image 15 input from the motion compensated prediction frame memory 14. There is.

  The motion compensation region dividing unit 40 divides the macro / sub-block image 5 input from the switching unit 6 into blocks serving as motion compensation units in accordance with the encoding mode 7 instructed from the encoding control unit 3. The motion compensation region block image 41 is output to the motion detection unit 42.

  The interpolated image generation unit 43 specifies the reference image 15 used for prediction image generation from one or more frames of reference image data stored in the motion compensated prediction frame memory 14, and the motion detection unit 42 specifies the reference image specified. The motion vector 44 is detected within a predetermined motion search range on 15. The motion vector is detected by a motion vector with virtual sample accuracy as in the MPEG-4 AVC standard. This detection method creates a virtual sample (pixel) by interpolation between integer pixels for pixel information (referred to as an integer pixel) of a reference image, and uses it as a predicted image. In the MPEG-4 AVC standard, a virtual sample with 1/8 pixel accuracy can be generated and used. In the MPEG-4 AVC standard, a virtual sample with 1/2 pixel accuracy is generated by an interpolation operation by a 6-tap filter using six integer pixels in the vertical direction or the horizontal direction. The ¼ pixel precision virtual sample is generated by an interpolation operation using an average value filter of adjacent ½ pixels or integer pixels.

  Also in the motion compensation prediction unit 9 in the first embodiment, the interpolation image generation unit 43 generates a predicted image 45 of virtual pixels according to the accuracy of the motion vector 44 instructed from the motion detection unit 42. Hereinafter, an example of a motion vector detection procedure with virtual pixel accuracy will be described.

Motion vector detection procedure I
The interpolated image generation unit 43 generates a predicted image 45 for a motion vector 44 with integer pixel accuracy that is within a predetermined motion search range of the motion compensation region block image 41. The predicted image 45 (predicted image 17) generated with integer pixel accuracy is output to the subtractor 12, and is subtracted from the motion compensation region block image 41 (macro / sub-block image 5) by the subtracter 12, so that the predicted difference signal 13 is subtracted. become. The encoding control unit 3 evaluates the prediction efficiency with respect to the prediction difference signal 13 and the integer pixel precision motion vector 44 (prediction parameter 18). For the evaluation of the prediction efficiency, for example, the prediction cost J ₁ is calculated from the following equation (1), and a motion vector 44 with integer pixel precision that minimizes the prediction cost J ₁ within a predetermined motion search range is determined.
J ₁ = D ₁ + λR ₁ (1)
Here, D ₁ and R ₁ are used as evaluation values. D ₁ is the sum of absolute values (SAD) in the macro block or sub block of the prediction difference signal, R ₁ is the estimated code amount of the identification number of the motion vector and the reference image pointed to by this motion vector, and λ is a positive number.

In determining the evaluation value R ₁ , the code amount of the motion vector is calculated by predicting the motion vector value in each mode of FIG. 2A or FIG. 2B using the value of a nearby motion vector and converting the predicted difference value into a probability distribution. It is obtained by entropy coding based on it or by performing code amount estimation corresponding to it.

FIG. 4 is a diagram for explaining a method for determining a motion vector prediction value (hereinafter referred to as a prediction vector) in each encoding mode 7 shown in FIG. 2B. In a rectangular block such as mb_mode0, sub_mb_mode1, etc. in FIG. The prediction vector PMV of the rectangular block is calculated from the following equation (2). median () corresponds to the median filter process and is a function that outputs the median value of the motion vectors MVa, MVb, and MVc.
PMV = median (MVa, MVb, MVc) (2)

On the other hand, in the case of the diagonal block mb_mode1, sub_mb_mode2, mb_mode2, sub_mb_mode3, mb_mode3, sub_mb_mode4, mb_mode4, sub_mb_mode5 having the diagonal shape, the same processing as the rectangular block can be applied. The positions A, B, and C that take values are changed. As a result, the method for calculating the prediction vector PMV itself can be calculated according to the shape of each motion vector allocation region without changing, and the cost of the evaluation value R ₁ can be kept small.

Motion vector detection procedure II
The interpolated image generation unit 43 generates a predicted image 45 for one or more ½ pixel precision motion vectors 44 positioned around the integer pixel precision motion vector determined in the “motion vector detection procedure I”. . Hereinafter, similarly to the “motion vector detection procedure I”, the predicted image 45 (predicted image 17) generated with the ½ pixel accuracy is converted by the subtractor 12 into the motion compensation region block image 41 (macro / sub-block image 5). ) To obtain the prediction difference signal 13. Subsequently, the encoding control unit 3 evaluates the prediction efficiency with respect to the prediction difference signal 13 and the motion vector 44 with 1/2 pixel accuracy (prediction parameter 18), and is positioned around the motion vector with integer pixel accuracy. A motion vector 44 having a ½ pixel accuracy that minimizes the prediction cost J ₁ is determined from one or more motion vectors having a ½ pixel accuracy.

Motion vector detection procedure III
The encoding control unit 3 and the motion compensation prediction unit 9 similarly apply to the motion vector with a ¼ pixel accuracy around the motion vector with a ½ pixel accuracy determined in the “motion vector detection procedure II”. A motion vector 44 having a 1/4 pixel accuracy that minimizes the prediction cost J ₁ is determined from one or more motion vectors having a 1/4 pixel accuracy positioned at the position.

Motion vector detection procedure IV
Similarly, the encoding control unit 3 and the motion compensation prediction unit 9 detect a motion vector with virtual pixel accuracy until a predetermined accuracy is obtained.

In this embodiment, motion vector detection with virtual pixel accuracy is performed until a predetermined accuracy is reached. For example, a threshold for the prediction cost is determined, and the prediction cost J ₁ is smaller than the predetermined threshold. In such a case, the detection of the motion vector with the virtual pixel accuracy may be stopped before the predetermined accuracy is reached.

  Note that the motion vector may refer to a pixel outside the frame defined by the reference frame size. In that case, it is necessary to generate pixels outside the frame. One method of generating pixels outside the frame is a method of filling in pixels at the screen edge.

  Note that when the frame size of each frame of the input video signal 1 is not an integer multiple of the macroblock size and an extended frame is input instead of each frame of the input video signal 1, the frame size is set to an integer multiple of the macroblock size. The expanded size (extended frame size) becomes the frame size of the reference frame. On the other hand, when only the local decoding part for the original frame is referred to as a pixel in the frame without referring to the local decoding part of the extension region, the frame size of the reference frame is the frame size of the original input video signal.

  As described above, the motion compensation prediction unit 9 determines the predetermined predetermined values for the motion compensation region block images 41 obtained by dividing the macro / sub-block image 5 into blocks each serving as a motion compensation unit indicated by the encoding mode 7. The motion vector of the accurate virtual pixel accuracy and the identification number of the reference image indicated by the motion vector are output as the prediction parameter 18. Further, the motion compensation prediction unit 9 outputs the prediction image 45 (prediction image 17) generated by the prediction parameter 18 to the subtracter 12, and is subtracted from the macro / sub-block image 5 by the subtractor 12, so that the prediction difference signal 13 is subtracted. Get. The prediction difference signal 13 output from the subtracter 12 is output to the transform / quantization unit 19.

2. Compression Parameter Determination Procedure Here, transform / quantization processing is performed on the prediction difference signal 13 generated based on the prediction parameter 18 determined for each encoding mode 7 in the above “1. Prediction Parameter Determination Procedure”. A procedure for determining the compression parameter 20 (transform block size) used in the process will be described.

FIG. 5 is a diagram illustrating an example of adaptation of the transform block size according to the coding mode 7 illustrated in FIG. 2B. In FIG. 5, a 32 × 32 pixel block is used as an example of the M × L pixel block. When the mode designated by the encoding mode 7 is mb_mode 0 to 6, either one of 16 × 16 and 8 × 8 pixels can be adaptively selected as the transform block size. When the encoding mode 7 is mb_mode7, the transform block size can be adaptively selected from 8 × 8 or 4 × 4 pixels for each 16 × 16 pixel sub-block obtained by dividing the macroblock into four.
The set of transform block sizes that can be selected for each encoding mode can be defined from any rectangular block size that is equal to or smaller than the sub-block size that is equally divided by the encoding mode.

  FIG. 6 is a diagram illustrating another example of adaptation of the transform block size according to the coding mode 7 illustrated in FIG. 2B. In the example of FIG. 6, when the coding mode 7 designates the above-described mb_mode 0, 5 and 6, in addition to 16 × 16 and 8 × 8 pixels as selectable transform block sizes, a sub-unit that is a unit of motion compensation A transform block size corresponding to the block shape can be selected. In the case of mb_mode0, it can be adaptively selected from 16 × 16, 8 × 8, and 32 × 32 pixels. In the case of mb_mode5, it is possible to adaptively select from 16 × 16, 8 × 8, and 16 × 32 pixels. In the case of mb_mode6, it is possible to adaptively select from 16 × 16, 8 × 8, and 32 × 16 pixels. Although not shown, in the case of mb_mode7, it is possible to adaptively select from 16 × 16, 8 × 8, and 16 × 32 pixels. In the case of mb_mode1 to 4, it is possible to select a non-rectangular region. May be selected from 16 × 16 and 8 × 8 pixels, and may be adapted to select from 8 × 8 and 4 × 4 pixels for a rectangular region.

The encoding control unit 3 sets a transform block size set corresponding to the encoding mode 7 illustrated in FIGS. 5 and 6 as the compression parameter 20.
In the examples of FIGS. 5 and 6, a set of transform block sizes that can be selected in accordance with the macroblock encoding mode 7 is determined in advance so that it can be adaptively selected in units of macroblocks or subblocks. Similarly, a set of selectable transform block sizes is determined in advance according to the encoding mode 7 (sub_mb_mode 1 to 8 in FIG. 2B) of the sub-block obtained by dividing the macro block. May be adaptively selected for each divided block unit.
Similarly, when the encoding mode 7 shown in FIG. 2A is used, the encoding control unit 3 determines in advance a set of transform block sizes corresponding to the encoding mode 7 so that it can be selected adaptively. Just keep it.

  The transform / quantization unit 19 cooperates with the encoding control unit 3 to make a macroblock unit designated by the macroblock size 4 or a subblock unit obtained by further dividing the macroblock unit according to the encoding mode 7 Then, an optimal conversion block size is determined from the conversion block size 20. The detailed procedure will be described below.

  FIG. 7 is a block diagram showing an internal configuration of the transform / quantization unit 19. The transform / quantization unit 19 illustrated in FIG. 7 includes a transform block size dividing unit 50, a transform unit 52, and a quantization unit 54. Input data includes a compression parameter 20 (transform block size, quantization step size, etc.) input from the encoding control unit 3 and a prediction difference signal 13 input from the encoding control unit 3.

The transform block size dividing unit 50 transforms the prediction difference signal 13 for each macroblock or sub-block for which the transform block size is to be determined into a block corresponding to the transform block size of the compression parameter 20, and serves as a transform target block 51. The data is output to the conversion unit 52.
If a plurality of transform block sizes are selected and specified for one macroblock or sub-block by the compression parameter 20, the transform target blocks 51 of the respective transform block sizes are sequentially output to the transform unit 52.

  The conversion unit 52 performs conversion processing on the input conversion target block 51 in accordance with a conversion method such as integer conversion or Hadamard conversion in which DCT and DCT conversion coefficients are approximated by integers, and the generated conversion coefficient 53 is quantized. To the unit 54.

The quantization unit 54 quantizes the input transform coefficient 53 according to the quantization step size of the compression parameter 20 instructed from the encoding control unit 3, and dequantizes the compressed data 21 that is the quantized transform coefficient. Output to the inverse transform unit 22 and the encoding control unit 3.
Note that, when a plurality of transform block sizes are selected and designated for one macroblock or sub-block by the compression parameter 20, the transform unit 52 and the quantization unit 54 are described above for all transform block sizes. Then, each compressed data 21 is output.

The compressed data 21 output from the quantization unit 54 is input to the encoding control unit 3 and is used for evaluating the encoding efficiency with respect to the transform block size of the compression parameter 20. The encoding control unit 3 uses the compressed data 21 obtained for all the transform block sizes that can be selected for each of the encoding modes included in the encoding mode 7, for example, encoding using the following equation (3): the cost J ₂ calculates, selects a transform block size for the encoding cost J ₂ to a minimum.
J ₂ = D ₂ + λR ₂ (3)
Here, D ₂ and R ₂ are used as evaluation values. As D ₂ , the compressed data 21 obtained for the transform block size is input to the inverse quantization / inverse transform unit 22, and the local decoded prediction difference signal obtained by performing the inverse transform / inverse quantization process on the compressed data 21 The sum of square distortion between the local decoded image signal 26 obtained by adding the predicted image 17 to 24 and the macro / sub-block image 5 is used. R ₂ is a code obtained by actually encoding the compressed data 21 obtained with respect to the transform block size, the encoding mode 7 and the prediction parameters 10 and 18 relating to the compressed data 21 by the variable length encoding unit 23. A quantity (or estimated code quantity) is used.

  The encoding control unit 3 selects an optimum compression parameter by selecting a transform block size corresponding to the determined optimal encoding mode 7a after determining the optimal encoding mode 7a according to “3. Procedure for determining encoding mode” described later. 20a and output to the variable length coding unit 23. The variable length coding unit 23 entropy codes the optimum compression parameter 20a and then multiplexes it into the bit stream 30.

  Here, the transform block size is selected from a transform block size set (illustrated in FIGS. 5 and 6) defined in advance according to the optimal encoding mode 7a of the macroblock or sub-block. For each size set, identification information such as an ID may be assigned to the transform block size included in the set, and the identification information may be entropy-coded as transform block size information and multiplexed into the bitstream 30. In this case, identification information of the transform block size set is also set on the decoding device side. However, when there is one transform block size included in the transform block size set, the transform block size can be automatically determined from the set on the decoding device side. There is no need to multiplex into stream 30.

3. Coding Mode Determination Procedure According to the above “1. Prediction Parameter Determination Procedure” and “2. Compression Parameter Determination Procedure”, the prediction parameters 10, When 18 and the compression parameter 20 are determined, the encoding control unit 3 further transforms and quantizes the prediction difference signal 13 obtained by using each encoding mode 7, the prediction parameters 10 and 18 and the compression parameter 20 at that time. using compressed data 21 obtained Te, determined from the above equation (3) a coding mode 7 encoding cost J ₂ is reduced, it selects the coding mode 7 as an optimal coding mode 7a of the macroblock.

  Note that the optimal encoding mode 7a may be determined from all the encoding modes in which the skip mode is added as the macroblock or sub-block mode to the encoding mode shown in FIG. 2A or 2B. The skip mode is a mode in which a prediction image that has been motion-compensated using a motion vector of an adjacent macroblock or sub-block on the encoding device side is used as a local decoded image signal. Prediction parameters and compression parameters other than the encoding mode are used. Since it is not necessary to calculate and multiplex the bit stream, it is possible to perform encoding with a reduced code amount. On the decoding device side, a predicted image that has been motion-compensated using motion vectors of adjacent macroblocks or sub-blocks is output as a decoded image signal in the same procedure as the coding device side.

  When an extended frame is input instead of each frame of the input video signal 1 when the frame size of each frame of the input video signal 1 is not an integral multiple of the macroblock size, For the sub-block, the encoding mode may be determined so as to control only the skip mode and suppress the amount of code consumed in the extension region.

  The encoding control unit 3 obtains the optimum encoding efficiency determined by the above “1. Prediction parameter determination procedure”, “2. Compression parameter determination procedure”, and “3. Encoding mode determination procedure”. Is output to the variable length coding unit 23, and the prediction parameters 10 and 18 corresponding to the optimum coding mode 7a are selected as the optimum prediction parameters 10a and 18a. The corresponding compression parameter 20 is selected as the optimum compression parameter 20 a and output to the variable length coding unit 23. The variable length encoding unit 23 entropy-encodes the optimal encoding mode 7a, the optimal prediction parameters 10a and 18a, and the optimal compression parameter 20a and multiplexes them into the bit stream 30.

  Further, as described above, the optimal prediction difference signal 13a obtained from the predicted images 11 and 17 based on the determined optimal encoding mode 7a, the optimal prediction parameters 10a and 18a, and the optimal compression parameter 20a is converted into a transform / quantization unit 19 as described above. The compressed data 21 is converted and quantized in the above manner into compressed data 21, which is entropy-coded by the variable length coding unit 23 and multiplexed into the bit stream 30. The compressed data 21 passes through an inverse quantization / inverse transform unit 22 and an addition unit 25 to become a locally decoded image signal 26 and is input to a loop filter unit 27.

Next, the moving picture decoding apparatus according to the first embodiment will be described.
FIG. 8 is a block diagram showing the configuration of the video decoding apparatus according to Embodiment 1 of the present invention. The moving picture decoding apparatus shown in FIG. 8 entropy-decodes the optimal encoding mode 62 from the bitstream 60 in units of macroblocks, and the macroblock or sub-block divided according to the decoded optimal encoding mode 62 When the optimal prediction parameter 63, the compressed data 64, and the variable length decoding unit 61 that entropy decodes the optimal compression parameter 65 and the optimal prediction parameter 63 are input, the intra prediction mode and the intra prediction included in the optimal prediction parameter 63 are input. When the intra prediction unit 69 that generates the predicted image 71 using the decoded image 74a stored in the memory 77 and the optimal prediction parameter 63 are input, the motion vector included in the optimal prediction parameter 63 and the optimal Specified by the reference image index included in the prediction parameter 63 In accordance with the motion compensation prediction unit 70 that performs motion compensation prediction using the reference image 76 in the motion compensated prediction frame memory 75 to generate the prediction image 72, and variable length decoding according to the decoded optimum encoding mode 62 Using the switching unit 68 that inputs the optimal prediction parameter 63 decoded by the unit 61 to either the intra prediction unit 69 or the motion compensated prediction unit 70 and the optimal compression parameter 65, inverse quantization and compression are performed on the compressed data 64. Either the intra-prediction unit 69 or the motion compensation prediction unit 70 outputs the inverse quantization / inverse transformation unit 66 that performs the inverse transformation process and generates the prediction difference signal decoded value 67 and the prediction difference signal decoded value 67. An addition unit 73 that adds the predicted images 71 and 72 to generate a decoded image 74, an intra prediction memory 77 that stores the decoded image 74, and a filter for filtering the decoded image 74 It includes a loop filter unit 78 that sense to generate a reproduced image 79, and a motion compensated prediction frame memory 75 for storing the reproduced image 79.

When the moving picture decoding apparatus according to the first embodiment receives the bit stream 60, the variable length decoding unit 61 performs entropy decoding processing on the bit stream 60 and performs a sequence unit or a picture including one or more frames. Decode macroblock size and frame size in units. When the macroblock size is defined by a profile or the like without being multiplexed directly to the bitstream, the macroblock size is determined based on profile identification information decoded from the bitstream in sequence units. . Based on the decoded macroblock size and decoded frame size of each frame, the number of macroblocks included in each frame is determined, and the optimal encoding mode 62, optimal prediction parameter 63, and compressed data 64 of each macroblock included in the frame are determined. (That is, quantized transform coefficient data), optimum compression parameter 65 (transform block size information, quantization step size), and the like are decoded.
Note that the optimum coding mode 62, optimum prediction parameter 63, compressed data 64, and optimum compression parameter 65 decoded on the decoding device side are the optimum coding mode 7a, optimum prediction parameters 10a, 18a, coded on the coding device side, This corresponds to the compressed data 21 and the optimum compression parameter 20a.

  Here, the transform block size information of the optimum compression parameter 65 is a transform block selected from a transform block size set defined in advance in units of macroblocks or sub-blocks according to the coding mode 7 on the encoding device side. This is identification information for specifying the size, and the decoding apparatus side specifies the conversion block size of the macroblock or sub-block from the optimal encoding mode 62 and the conversion block size information of the optimal compression parameter 65.

  The inverse quantization / inverse transform unit 66 uses the compressed data 64 and the optimum compression parameter 65 input from the variable length decoding unit 61 to perform inverse quantization / inverse transform processing in units of blocks specified by the transform block size information. And a prediction difference signal decoded value 67 is calculated.

  In addition, when decoding the motion vector, the variable length decoding unit 61 determines a prediction vector by the process shown in FIG. 4 with reference to the motion vectors of the peripheral blocks that have already been decoded, and uses the prediction difference value decoded from the bitstream 60. The decoded value of the motion vector is obtained by addition. The variable length decoding unit 61 includes the decoded value of the motion vector in the optimum prediction parameter 63 and outputs it to the switching unit 68.

  The switching unit 68 is a switch that switches the input destination of the optimal prediction parameter 63 according to the optimal encoding mode 62. When the optimum encoding mode 62 input from the variable length decoding unit 61 indicates the intra-frame prediction mode, the switching unit 68 also uses the optimum prediction parameter 63 (intra prediction mode) input from the variable length decoding unit 61. Is output to the intra prediction unit 69, and when the optimal encoding mode 62 indicates the inter-frame prediction mode, the optimal prediction parameter 63 (motion vector, reference image identification number (reference image index) indicated by each motion vector, etc.) Is output to the motion compensation prediction unit 70.

  The intra prediction unit 69 refers to the decoded image (decoded image signal in the frame) 74a in the frame stored in the intra prediction memory 77, and corresponds to the intra prediction mode indicated by the optimal prediction parameter 63. A predicted image 71 is generated and output.

  Note that the method of generating the predicted image 71 by the intra prediction unit 69 is the same as the operation of the intra prediction unit 8 on the encoding device side. However, the intra prediction unit 8 applies to all intra prediction modes instructed in the encoding mode 7. In contrast to the generation of the corresponding prediction image 11, the intra prediction unit 69 is different in that only the prediction image 71 corresponding to the intra prediction mode indicated in the optimum encoding mode 62 is generated.

  The motion compensated prediction unit 70 predicts a predicted image from one or more reference images 76 stored in the motion compensated prediction frame memory 75 based on a motion vector, a reference image index, and the like indicated by the input optimum prediction parameter 63. 72 is generated and output.

  Note that the method of generating the predicted image 72 by the motion compensation prediction unit 70 is a process of searching for a motion vector from a plurality of reference images among the operations of the motion compensation prediction unit 9 on the encoding device side (the motion detection unit 42 shown in FIG. 3). And corresponding to the operation of the interpolated image generating unit 43), and only the process of generating the predicted image 72 is performed according to the optimal prediction parameter 63 given from the variable length decoding unit 61. Similar to the encoding device, the motion compensation prediction unit 70 embeds pixels outside the frame with pixels at the end of the screen when the motion vector refers to a pixel outside the frame defined by the reference frame size. The prediction image 72 is produced | generated by. The reference frame size may be defined by a size obtained by extending the decoded frame size to an integral multiple of the decoded macroblock size, or may be defined by the decoded frame size. To determine the reference frame size.

  The adding unit 73 adds either one of the predicted image 71 or the predicted image 72 and the predicted difference signal decoded value 67 output from the inverse quantization / inverse transform unit 66 to generate a decoded image 74.

  Since this decoded image 74 is used as a reference image (decoded image 74 a) for generating an intra-predicted image of a subsequent macroblock, the decoded image 74 is stored in the intra-prediction memory 77 and input to the loop filter unit 78.

The loop filter unit 78 performs the same operation as the loop filter unit 27 on the encoding device side, generates a reproduced image 79, and outputs it from the moving image decoding device. Also, this reproduced image 79 is
Since it is used as a reference image 76 for subsequent prediction image generation, it is stored in the motion compensated prediction frame memory 75. Note that the size of the reproduced image obtained after decoding all the macroblocks in the frame is an integral multiple of the macroblock size. When the size of the reproduced image is larger than the decoded frame size corresponding to the frame size of each frame of the video signal input to the encoding device, the reproduced image includes an extended region in the horizontal direction or the vertical direction. In this case, a decoded image obtained by removing the decoded image of the extended area portion from the reproduced image is output from the decoding device.

  When the reference frame size is defined by the decoded frame size, the decoded image in the extended region portion of the reproduced image stored in the motion compensated prediction frame memory 75 is not referred to in subsequent prediction image generation. Therefore, the decoded image obtained by removing the decoded image of the extended area portion from the reproduced image may be stored in the motion compensated prediction frame memory 75.

  As described above, according to the moving picture coding apparatus according to Embodiment 1, the macro / subblock image 5 divided according to the macroblock coding mode 7 depends on the size of the macroblock or subblock. A set of transform blocks including a plurality of transform block sizes is determined in advance, and the encoding control unit 3 selects one transform block size with the best coding efficiency from the transform block size set as the optimum compression parameter 20a. And the conversion / quantization unit 19 divides the optimal prediction difference signal 13a into blocks of the conversion block size included in the optimal compression parameter 20a, and performs conversion and quantization processing. Since the compressed data 21 is generated, the transform block size set is a macro block or a sub block. Compared with the conventional method which is irrespective fixed size, with equal code amount, it is possible to improve the quality of encoded video.

  In addition, since the variable length encoding unit 23 is configured to multiplex the transform block size adaptively selected from the set of transform block sizes according to the encoding mode 7 into the bitstream 30, Correspondingly, in the video decoding apparatus according to Embodiment 1, the variable length decoding unit 61 decodes the optimal compression parameter 65 from the bit stream 60 in units of macroblocks or subblocks, and the inverse quantization / inverse conversion unit 66 However, the transform block size is determined based on the transform block size information included in the optimum compression parameter 65, and the compressed data 64 is inversely transformed and inversely quantized in units of the transform block size. Therefore, the video decoding device can decode the compressed data by selecting the transform block size used on the encoding device side from the set of transform block sizes defined in the same way as the video encoding device. It becomes possible to correctly decode the bitstream encoded by the moving picture encoding apparatus according to Embodiment 1.

Embodiment 2. FIG.
In the second embodiment, a modification of the variable length coding unit 23 of the video encoding device according to the first embodiment and the variable length decoding unit 61 of the video decoding device according to the first embodiment are the same. A modification will be described.

First, the variable length coding unit 23 of the moving picture coding apparatus according to the second embodiment will be described.
FIG. 9 is a block diagram showing an internal configuration of the variable length coding unit 23 of the moving picture coding apparatus according to Embodiment 2 of the present invention. In FIG. 9, the same or equivalent parts as in FIG. In addition, the configuration of the video encoding apparatus according to the second embodiment is the same as that of the first embodiment, and the operation of each component except for the variable-length encoding unit 23 is the same as that of the first embodiment. Therefore, FIGS. 1-8 is used. Further, for convenience of explanation, in the second embodiment, the apparatus configuration and the processing method are based on the assumption that the encoding mode set shown in FIG. 2A is used. However, the encoding mode set shown in FIG. 2B is used. Needless to say, the present invention is also applicable to the assumed apparatus configuration and processing method.

  The variable length encoding unit 23 illustrated in FIG. 9 specifies the correspondence between the index value of the multilevel signal representing the encoding mode 7 (or the optimal prediction parameters 10a and 18a and the optimal compression parameter 20a) and the binary signal. A binarization table memory 105 for storing a binarization table, and an optimal encoding mode 7a (or optimal prediction parameters 10a, 18a, optimal) of the multilevel signal selected by the encoding control unit 3 using the binarization table The binarization unit 92 that converts the index value of the multilevel signal of the compression parameter 20a) into the binary signal 103, the context identification information 102 generated by the context generation unit 99, the context information memory 96, the probability table memory 97, and the state transition The binary signal 103 converted by the binarization unit 92 with reference to the table memory 98 is arithmetically encoded and encoded bit string 11 and the frequency of occurrence of the arithmetic coding processing operation unit 104 that multiplexes the coded bit string 111 into the bit stream 30 and the optimum coding mode 7a (or the optimum prediction parameters 10a and 18a, and the optimum compression parameter 20a). A frequency information generation unit 93 that counts and generates frequency information 94, and a binary that updates the correspondence between the multilevel signal and the binary signal in the binarization table of the binarization table memory 105 based on the frequency information 94 Table update unit 95.

  In the following, the variable-length coding procedure of the variable-length coding unit 23 will be described by taking the macroblock optimum coding mode 7a output from the coding control unit 3 as an example of the entropy-coded parameter. Similarly, the optimal prediction parameters 10a and 18a and the optimal compression parameter 20a, which are parameters to be encoded, may be variable-length encoded in the same procedure as in the optimal encoding mode 7a, and the description thereof is omitted.

  Note that the encoding control unit 3 according to the second embodiment outputs a context information initialization flag 91, a type signal 100, peripheral block information 101, and a binarization table update flag 113. Details of each information will be described later.

  The initialization unit 90 initializes the context information 106 stored in the context information memory 96 according to the context information initialization flag 91 instructed by the encoding control unit 3 and sets the initial state. Details of the initialization processing by the initialization unit 90 will be described later.

  The binarization unit 92 refers to the binarization table stored in the binarization table memory 105 and refers to the index of the multilevel signal representing the type of the optimal encoding mode 7a input from the encoding control unit 3 The value is converted into a binary signal 103 and output to the arithmetic coding processing operation unit 104.

FIG. 10 is a diagram illustrating an example of a binarization table held in the binarization table memory 105. The “encoding mode” illustrated in FIG. 10 is a prediction in which motion compensation is performed using a motion vector of a macroblock adjacent to the encoding mode (mb_skip: encoding device side) in the encoding mode (mb_mode 0 to 3) illustrated in FIG. 2A. 5 types of encoding modes 7 to which an image is used as a decoded image on the decoding device side, and an “index” value corresponding to each encoding mode is stored. The index values of these encoding modes are each binarized with 1 to 3 bits and stored as “binary signal”. Here, each bit of the binary signal is referred to as a “bin” number.
Although details will be described later, in the example of FIG. 10, a small index value is assigned to a coding mode having a high occurrence frequency, and the binary signal is also set to be as short as 1 bit.

  The optimum encoding mode 7 a output from the encoding control unit 3 is input to the binarizing unit 92 and also input to the frequency information generating unit 93.

  The frequency information generation unit 93 counts the frequency of occurrence of the index value of the encoding mode included in the optimum encoding mode 7a (the selection frequency of the encoding mode selected by the encoding control unit) to generate the frequency information 94. The data is output to a binarization table update unit 95 described later.

  The probability table memory 97 includes a plurality of symbols (MPS: Most Probable Symbol) having a high occurrence probability among symbol values “0” or “1” of each bin included in the binary signal 103 and a plurality of combinations of the occurrence probabilities. This is a memory for holding a pair of stored tables.

  FIG. 11 is a diagram illustrating an example of a probability table held in the probability table memory 97. In FIG. 11, “probability table numbers” are assigned to discrete probability values (“occurrence probabilities”) between 0.5 and 1.0.

  The state transition table memory 98 is encoded from the “probability table number” stored in the probability table memory 97 and the probability state before the MPS encoding of “0” or “1” indicated by the probability table number. This is a memory for holding a table storing a plurality of combinations of state transitions to the probability state.

FIG. 12 is a diagram illustrating an example of a state transition table held in the state transition table memory 98. “Probability table number”, “Probability transition after LPS encoding”, and “Probability transition after MPS encoding” in FIG. 12 respectively correspond to the probability table numbers shown in FIG.
For example, in the probability state of “probability table number 1” surrounded by a frame in FIG. 12 (MPS occurrence probability 0.527 from FIG. 11), one of “0” or “1” having a low occurrence probability By encoding the symbol (LPS: Last Probable Symbol), the probability state changes from “probability transition after LPS encoding” to probability table number 0 (MPS occurrence probability 0.500 from FIG. 11). To express. That is, the occurrence probability of MPS is reduced due to the occurrence of LPS.
Conversely, when MPS is encoded, the probability state represents a transition from “probability transition after MPS encoding” to probability table number 2 (MPS occurrence probability 0.550 from FIG. 11). That is, the occurrence probability of MPS is increased due to the occurrence of MPS.

The context generation unit 99 includes a type signal 100 indicating the types of parameters to be encoded (the optimal encoding mode 7a, the optimal prediction parameters 10a and 18a, and the optimal compression parameter 20a) input from the encoding control unit 3 and peripheral block information. 101, the context identification information 102 is generated for each bin of the binary signal 103 obtained by binarizing the encoding target parameter. In this description, the type signal 100 is the optimum encoding mode 7a of the encoding target macroblock. The peripheral block information 101 is the optimal encoding mode 7a of the macroblock adjacent to the encoding target macroblock.
Hereinafter, a procedure for generating context identification information by the context generation unit 99 will be described.

FIG. 13A is a diagram showing the binary table shown in FIG. 10 in binary tree representation. Here, a thick-framed encoding target macroblock shown in FIG. 13B and peripheral blocks A and B adjacent to the encoding target macroblock will be described as an example.
In FIG. 13A, a black circle is called a node, and a line connecting the nodes is called a path. An index of a multilevel signal to be binarized is assigned to the end node of the binary tree. Further, from the top to the bottom of the page, the depth of the binary tree corresponds to the bin number, and a bit string obtained by combining symbols (0 or 1) assigned to each path from the root node to the end node is The binary signal 103 corresponding to the index of the multilevel signal assigned to the terminal node is obtained. For each parent node (non-terminal node) of the binary tree, one or more context identification information is prepared according to the information of the peripheral blocks A and B.

For example, in FIG. 13A, when three pieces of context identification information C0, C1, and C2 are prepared for the root node, the context generation unit 99 uses the neighboring block information of neighboring neighboring blocks A and B. Referring to 101, any one of the three pieces of context identification information C0, C1, and C2 is selected from the following equation (4). The context generation unit 99 outputs the selected context identification information as the context identification information 102.

  In the above equation (4), when the peripheral blocks A and B are the macroblock X, if the encoding mode of the peripheral blocks A and B is “0” (mb_skip), the encoding mode of the encoding target macroblock is “ This is an expression prepared under the assumption that the probability of becoming 0 ″ (mb_skip) is high. Therefore, the context identification information 102 selected from the above equation (4) is also based on the same assumption.

  One context identification information (C3, C4, C5) is assigned to each parent node other than the root node.

  The context information identified by the context identification information 102 holds an MPS value (0 or 1) and a probability table number that approximates the occurrence probability, and is in an initial state. This context information is stored in the context information memory 96.

  The arithmetic coding processing operation unit 104 arithmetically codes the 1 to 3 bit binary signal 103 input from the binarizing unit 92 for each bin to generate an encoded bit string 111 and multiplexes the encoded bit sequence 111 into the bit stream 30. Let Hereinafter, an arithmetic coding procedure based on the context information will be described.

  The arithmetic coding processing calculation unit 104 first refers to the context information memory 96 and obtains context information 106 based on the context identification information 102 corresponding to bin 0 of the binary signal 103. Subsequently, the arithmetic coding processing calculation unit 104 refers to the probability table memory 97 and specifies the MPS occurrence probability 108 of bin 0 corresponding to the probability table number 107 held in the context information 106.

  Subsequently, the arithmetic coding processing calculation unit 104 determines the symbol value 109 (0 or 0) of bin 0 based on the MPS value (0 or 1) held in the context information 106 and the identified MPS occurrence probability 108. 1) is arithmetically encoded. Subsequently, the arithmetic coding processing calculation unit 104 refers to the state transition table memory 98 and converts the probability table number 107 held in the context information 106 and the symbol value 109 of bin 0 previously arithmetically coded. Based on this, the probability table number 110 after bin 0 symbol encoding is obtained.

  Subsequently, the arithmetic coding processing calculation unit 104 uses the value of the probability table number (that is, the probability table number 107) of the context information 106 of bin 0 stored in the context information memory 96 as the probability table number after the state transition ( That is, it is updated to the probability table number 110) obtained from the state transition table memory 98 before encoding the symbol of bin 0.

The arithmetic coding processing calculation unit 104 performs arithmetic coding on the bins 1 and 2 based on the context information 106 identified by each context identification information 102 as in the bin 0, and after the symbol coding of each bin, The information 106 is updated.
The arithmetic encoding processing operation unit 104 outputs an encoded bit string 111 obtained by arithmetically encoding all bin symbols, and the variable length encoding unit 23 multiplexes the bit stream 30.

As described above, the context information 106 identified by the context identification information 102 is updated every time a symbol is arithmetically encoded. That is, it means that the probability state of each node transitions for each symbol encoding. The initialization of the context information 106, that is, the resetting of the probability state is performed by the initialization unit 90 described above.
The initialization unit 90 performs initialization in response to an instruction by the context information initialization flag 91 of the encoding control unit 3, and this initialization is performed at the head of the slice or the like. As for the initial state of each context information 106 (MPS value and probability table number initial value approximating the occurrence probability), a plurality of sets are prepared in advance, and which initial state is selected is encoded. The control unit 3 may include the context information initialization flag 91 and instruct the initialization unit 90.

  The binarization table update unit 95 is based on the binarization table update flag 113 instructed by the encoding control unit 3 and is generated by the frequency information generation unit 93. The encoding target parameter (here, the optimal encoding mode 7a). The binarization table memory 105 is updated with reference to the frequency information 94 indicating the frequency of occurrence of the index values. The procedure for updating the binarization table by the binarization table update unit 95 will be described below.

  In this example, binarization is performed so that the coding mode with the highest occurrence frequency can be binarized with a short code word according to the occurrence frequency of the encoding mode specified by the optimum encoding mode 7a that is the encoding target parameter. The correspondence between the table coding mode and the index is updated to reduce the code amount.

  FIG. 14 is a diagram illustrating an example of the binarization table after the update, and is a post-update state when it is assumed that the state of the binarization table before the update is the state illustrated in FIG. For example, when the occurrence frequency of mb_mode3 is the highest, the binarization table update unit 95 assigns the smallest index value so that a binary signal of a short code word is assigned to the mb_mode3.

  Further, the binarization table update unit 95 generates binarization table update identification information 112 for enabling the decoding device to identify the updated binarization table when the binarization table is updated. Thus, it is necessary to multiplex the bit stream 30. For example, when there are a plurality of binarization tables for each encoding target parameter, an ID for identifying each encoding target parameter is assigned in advance to the encoding device side and the decoding device side, respectively, and the binarization table update unit 95 may output the ID of the updated binarization table as the binarization table update identification information 112 and multiplex it with the bitstream 30.

  In the update timing control, the encoding control unit 3 refers to the frequency information 94 of the encoding target parameter at the head of the slice, and determines that the occurrence frequency distribution of the encoding target parameter has greatly changed beyond a predetermined allowable range. In this case, the binarization table update flag 113 is output. The variable length coding unit 23 may multiplex the binarization table update flag 113 into the slice header of the bit stream 30. Further, when the binarization table update flag 113 indicates “the binarization table is updated”, the variable length coding unit 23 stores the coding mode, compression parameter, and prediction parameter binarization table. Among them, the binarization table update identification information 112 indicating which binarization table is updated is multiplexed into the bitstream 30.

  In addition, the encoding control unit 3 may instruct the update of the binarization table at a timing other than the head of the slice. For example, the encoding control unit 3 outputs the binarization table update flag 113 at the head of an arbitrary macroblock. May be. In this case, the binarization table update unit 95 outputs information specifying the macroblock position where the binarization table has been updated, and the variable length encoding unit 23 also multiplexes the information into the bitstream 30. There is a need to.

  When the encoding control unit 3 outputs the binarization table update flag 113 to the binarization table update unit 95 to update the binarization table, the encoding control unit 3 notifies the initialization unit 90 of the context information initialization flag. 91 is output to initialize the context information memory 96.

Next, the variable length decoding unit 61 of the video decoding apparatus according to the second embodiment will be described.
FIG. 15 is a block diagram showing an internal configuration of the variable length decoding unit 61 of the video decoding apparatus according to Embodiment 2 of the present invention. The configuration of the moving picture decoding apparatus according to the second embodiment is the same as that of the first embodiment, and the operation of each component other than the variable length decoding unit 61 is the same as that of the first embodiment. 1 to 8 are incorporated.

  The variable length decoding unit 61 illustrated in FIG. 15 is multiplexed into the bitstream 60 with reference to the context identification information 126, the context information memory 128, the probability table memory 131, and the state transition table memory 135 generated by the context generation unit 122. An arithmetic decoding processing operation unit 127 that arithmetically decodes the encoded bit string 133 representing the optimal encoding mode 62 (or optimal prediction parameter 63, optimal compression parameter 65) to generate a binary signal 137, and a binary signal. A binarization table memory 143 for storing a binarization table 139 designating a correspondence relationship between the optimum coding mode 62 (or the optimum prediction parameter 63, the optimum compression parameter 65) and the multilevel signal, and the binarization table 139 The binary signal 137 generated by the arithmetic decoding processing calculation unit 127 is And a reverse binarizing section 138 for converting the decoded value 140 No..

  Hereinafter, the variable-length decoding procedure of the variable-length decoding unit 61 will be described using the optimal encoding mode 62 of the macroblock included in the bitstream 60 as an example of the entropy-decoded parameter. Similarly, the optimal prediction parameter 63 and the optimal compression parameter 65 that are parameters to be decoded may be variable-length decoded in the same procedure as in the optimal encoding mode 62, and thus description thereof is omitted.

  In the bit stream 60 of the second embodiment, the context initialization information 121, the encoded bit string 133, the binary table update flag 142, and the binary table update identification information multiplexed on the encoding device side are included. 144 is included. Details of each information will be described later.

  The initialization unit 120 initializes the context information stored in the context information memory 128 at the head of the slice or the like. Alternatively, the initialization unit 120 prepares a plurality of sets in advance for the initial state of the context information (the initial value of the MPS value and the probability table number approximating the occurrence probability), and the decoded value of the context initialization information 121 You may make it select the initial state corresponding to to from a set.

  The context generation unit 122 refers to the type signal 123 indicating the type of parameters to be decoded (optimal encoding mode 62, optimal prediction parameter 63, and optimal compression parameter 65) and the peripheral block information 124, and provides context identification information 126. Generate.

  The type signal 123 is a signal representing the type of parameter to be decoded, and it is determined according to the syntax held in the variable length decoding unit 61 what the decoding target parameter is. Therefore, it is necessary to hold the same syntax on the encoding device side and the decoding device side. Here, it is assumed that the encoding control unit 3 on the encoding device side holds the syntax. On the encoding device side, according to the syntax held by the encoding control unit 3, the parameter type to be encoded next and the parameter value (index value), that is, the type signal 100, are sent to the variable length encoding unit 23. It will be output sequentially.

  The peripheral block information 124 is information such as a coding mode obtained by decoding a macroblock or a subblock, and has a variable length for use as peripheral block information 124 for subsequent decoding of the macroblock or subblock. It is stored in a memory (not shown) in the decoding unit 61 and is output to the context generation unit 122 as necessary.

  The procedure for generating the context identification information 126 by the context generator 122 is the same as the operation of the context generator 99 on the encoding device side. The context generation unit 122 on the decoding device side also generates context identification information 126 for each bin of the binarization table 139 referred to by the inverse binarization unit 138.

The context information of each bin holds an MPS value (0 or 1) and a probability table number for specifying the occurrence probability of the MPS as probability information for arithmetic decoding of the bin.
In addition, the probability table memory 131 and the state transition table memory 135 store the same probability table (FIG. 11) and state transition table (FIG. 12) as the coding device side probability table memory 97 and the state transition table memory 98. .

  The arithmetic decoding processing arithmetic unit 127 arithmetically decodes the encoded bit string 133 multiplexed in the bit stream 60 for each bin to generate a binary signal 137 and outputs the binary signal 137 to the inverse binarizing unit 138.

  First, the arithmetic decoding processing calculation unit 127 refers to the context information memory 128 to obtain context information 129 based on the context identification information 126 corresponding to each bin of the encoded bit string 133. Subsequently, the arithmetic decoding processing calculation unit 127 refers to the probability table memory 131 and specifies the MPS occurrence probability 132 of each bin corresponding to the probability table number 130 held in the context information 129.

  Subsequently, the arithmetic decoding process calculation unit 127 is input to the arithmetic decoding process calculation unit 127 based on the MPS value (0 or 1) held in the context information 129 and the identified MPS occurrence probability 132. The encoded bit string 133 is arithmetically decoded to obtain a symbol value 134 (0 or 1) for each bin. After decoding the symbol value of each bin, the arithmetic decoding process calculation unit 127 refers to the state transition table memory 135 and decodes each bin decoded in the same procedure as the arithmetic coding process calculation unit 104 on the encoding device side. The probability table number 136 after the symbol decoding of each bin (after the state transition) is obtained based on the symbol value 134 and the probability table number 130 held in the context information 129.

Subsequently, the arithmetic decoding processing calculation unit 127 uses the value of the probability table number (that is, the probability table number 130) of the context information 129 of each bin stored in the context information memory 128 as the probability table number after the state transition (that is, the probability table number 130). , Update to the probability table number 136) obtained from the state transition table memory 135 before the symbol decoding of each bin.
The arithmetic decoding processing calculation unit 127 outputs a binary signal 137 obtained by combining the bin symbols obtained as a result of the arithmetic decoding to the inverse binarization unit 138.

The inverse binarization unit 138 selects the same binarization table 139 as at the time of encoding from the binarization table prepared for each type of decoding target parameter stored in the binarization table memory 143. The decoded value 140 of the decoding target parameter is output from the binary signal 137 input from the arithmetic decoding processing calculation unit 127.
When the type of decoding target parameter is the macroblock encoding mode (optimal encoding mode 62), the binarization table 139 is the same as the binarization table on the encoding apparatus side shown in FIG.

  The binarization table update unit 141 is based on the binarization table update flag 142 and the binarization table update identification information 144 decoded from the bitstream 60, and the binarization table stored in the binarization table memory 143. Update.

  The binarization table update flag 142 is information corresponding to the binarization table update flag 113 on the encoding device side, and is included in the header information of the bitstream 60 and the like and indicates whether or not the binarization table is updated. It is. When the decoded value of the binarized table update flag 142 indicates “binary table is updated”, the binarized table update identification information 144 is further decoded from the bitstream 60.

  The binarization table update identification information 144 is information corresponding to the binarization table update identification information 112 on the encoding device side, and is information for identifying the binarization table of parameters updated on the encoding device side. is there. For example, as described above, when there are a plurality of binarization tables in advance for each encoding target parameter, the ID for identifying each encoding target parameter and the ID of the binarization table are set on the encoding device side and the decoding device side. The binarization table update unit 141 updates the binarization table corresponding to the ID value in the binarization table update identification information 144 decoded from the bitstream 60. In this example, the binarization table memory 143 is prepared in advance with the two types of binarization tables of FIG. 10 and FIG. 14 and their IDs, and the state of the binarization table before the update is the state shown in FIG. Assuming that the binarization table update unit 141 performs the update process according to the binarization table update flag 142 and the binarization table update identification information 144, it corresponds to the ID included in the binarization table update identification information 144. Since the binarized table is selected, the state of the updated binarized table becomes the state shown in FIG. 14, which is the same as the binarized table after the update on the encoding device side.

  As described above, according to the moving picture coding apparatus according to Embodiment 2, the coding control unit 3 includes the optimum coding mode 7a, the optimum prediction parameters 10a and 18a, and the optimum compression parameter 20a in which the coding efficiency is optimum. The encoding target parameter is selected and output, and the binarization unit 92 of the variable length encoding unit 23 uses the binarization table of the binarization table memory 105 to encode the encoding target represented by a multilevel signal. The parameter is converted into the binary signal 103, the arithmetic encoding processing arithmetic unit 104 arithmetically encodes the binary signal 103 and outputs the encoded bit string 111, and the frequency information generating unit 93 outputs the frequency information 94 of the encoding target parameter. Since the binarization table update unit 95 is configured to update the correspondence between the multilevel signal and the binary signal of the binarization table based on the frequency information 94, the binarization table Always compared with the conventional method is fixed, in the same quality of the coded video, it is possible to reduce the code amount.

  The binarization table update unit 95 also includes binarization table update identification information 112 indicating whether or not the binarization table is updated, and binarization table update identification information 112 for identifying the binarized table after the update. Therefore, the video decoding device according to the second embodiment is configured so that the arithmetic decoding processing operation unit 127 of the variable length decoding unit 61 includes the bit stream 60. The multiplexed coded bit string 133 is arithmetically decoded to generate a binary signal 137, and the inverse binarization unit 138 uses the binarization table 139 of the binarization table memory 143 to convert the binary signal 137. A binarization table update flag that is converted into a multilevel signal to obtain a decoded value 140 and is decoded from the header information multiplexed in the bitstream 60 by the binarization table update unit 141 And configured to update the predetermined binary table of binary table memory 143 based on the 42 and the binary table update identification information 144. Therefore, since the moving picture decoding apparatus can update the binarization table by the same procedure as that of the moving picture encoding apparatus and debinarize the encoding target parameter, the moving picture coding according to Embodiment 2 can be performed. It becomes possible to correctly decode the bitstream encoded by the apparatus.

Embodiment 3 FIG.
In the third embodiment, a modified example of a prediction image generation process by motion compensation prediction of the motion compensation prediction unit 9 in the video encoding device and the video decoding device according to the first and second embodiments will be described.

  First, the motion compensation prediction unit 9 of the video encoding apparatus according to the third embodiment will be described. The configuration of the moving picture coding apparatus according to the third embodiment is the same as that of the first embodiment or the second embodiment, and the operation of each component other than the motion compensation prediction unit 9 is the same. 1 to 15 are incorporated.

  The motion compensation prediction unit 9 according to the third embodiment has the same configuration and operation except that the configuration and operation related to the virtual sample accuracy predicted image generation processing are different from those of the first and second embodiments. That is, in the first and second embodiments, as shown in FIG. 3, the interpolated image generation unit 43 of the motion compensation prediction unit 9 generates reference image data with virtual pixel accuracy such as a half pixel or a quarter pixel, When the predicted image 45 is generated based on the reference image data with the virtual pixel accuracy, the interpolation is performed by a 6-tap filter using six integer pixels in the vertical direction or the horizontal direction as in the MPEG-4 AVC standard. While the predicted image is generated by creating the virtual pixel, the motion compensated prediction unit 9 according to the third embodiment super-resolutions the reference image 15 with integer pixel accuracy stored in the motion compensated prediction frame memory 14. By enlarging by the processing, a reference image 207 with virtual pixel accuracy is generated, and a predicted image is generated based on the reference image 207 with virtual pixel accuracy.

Next, the motion compensation prediction unit 9 according to the third embodiment will be described with reference to FIG.
As in the first and second embodiments, the interpolated image generation unit 43 of the third embodiment also designates one or more frames of the reference image 15 from the motion compensated prediction frame memory 14, and the motion detection unit 42 is designated. A motion vector 44 is detected within a predetermined motion search range on the reference image 15. The motion vector is detected by a motion vector with virtual pixel accuracy, as in the MPEG-4 AVC standard. In this detection method, a virtual sample (pixel) is created by interpolation between integer pixels for pixel information (referred to as integer pixels) possessed by a reference image, and is used as a reference image.

  In order to generate a reference image with virtual pixel accuracy, it is necessary to generate a sample plane composed of virtual pixels by enlarging (higher definition) the reference image with integer pixel accuracy. Therefore, in the interpolated image generating unit 43 of the third embodiment, when a reference image for motion search with virtual pixel accuracy is required, “WT Freeman, EC Pastor and OT Carmichial,“ Learning ”. A reference image with virtual pixel accuracy is generated using the super-resolution technique disclosed in "Low-Level Vision", International Journal of Computer Vision, vol.40, no.1,2000. In the following description, the motion compensation prediction unit 9 generates a super-resolution reference image 207 with virtual pixel accuracy from the reference image data stored in the motion compensation prediction frame memory 14, and the motion detection unit 42 uses it to generate motion. A configuration for performing vector search processing will be described.

  FIG. 16 is a block diagram showing an internal configuration of the interpolated image generation unit 43 of the motion compensated prediction unit 9 of the moving picture coding apparatus according to Embodiment 3 of the present invention. The interpolated image generation unit 43 shown in FIG. 16 includes an image enlargement processing unit 205 that enlarges the reference image 15 in the motion compensated prediction frame memory 14, an image reduction processing unit 200 that reduces the reference image 15, and an image reduction process. A high-frequency feature extraction unit 201a that extracts a feature quantity of a high-frequency region component from the unit 200; a high-frequency feature extraction unit 201b that extracts a feature quantity of a high-frequency region component from the reference image 15; and a correlation calculation that calculates a correlation value between the feature amounts. Unit 202, a high-frequency component estimation unit 203 that estimates a high-frequency component from the correlation value and the pre-learning data of high-frequency component pattern memory 204, and corrects the high-frequency component of the enlarged image using the estimated high-frequency component, thereby improving the virtual pixel accuracy. And an adding unit 206 that generates a reference image 207.

  In FIG. 16, when the reference image 15 in the range used for the motion search process is input to the interpolated image generation unit 43 from the reference image data stored in the motion compensated prediction frame memory 14, the reference image 15 is displayed as an image. The data are input to the reduction processing unit 200, the high frequency feature extraction unit 201b, and the image enlargement processing unit 205, respectively.

  The image reduction processing unit 200 generates a reduced image having a size of 1 / N in the vertical and horizontal directions (N is a power of 2, such as 2, 4) from the reference image 15 and outputs the reduced image to the high frequency feature extraction unit 201a. This reduction processing is realized by a general image reduction filter.

  The high frequency feature extraction unit 201a extracts a first feature amount related to a high frequency component such as an edge component from the reduced image generated by the image reduction processing unit 200. As the first feature amount, for example, a parameter indicating DCT or Wavelet transform coefficient distribution in the local block can be used.

  The high frequency feature extraction unit 201b performs high frequency feature extraction similar to the high frequency feature extraction unit 201a, and extracts, from the reference image 15, a second feature amount having a frequency component region different from the first feature amount. The second feature amount is output to the correlation calculation unit 202 and also output to the high frequency component estimation unit 203.

  When the first feature amount is input from the high-frequency feature extraction unit 201a and the second feature amount is input from the high-frequency feature extraction unit 201b, the correlation calculation unit 202 localizes between the reference image 15 and the reduced image. The correlation value of the high frequency component region on the basis of the feature amount in the block unit is calculated. As this correlation value, for example, there is a distance between the first feature value and the second feature value.

  The high frequency component estimator 203 uses the second feature amount input from the high frequency feature extractor 201b and the correlation value input from the correlation calculator 202 to pre-learn high frequency component patterns from the high frequency component pattern memory 204. Is identified and a high frequency component to be included in the reference image 207 with virtual pixel accuracy is estimated and generated. The generated high frequency component is output to the adding unit 206.

  The image enlargement processing unit 205 performs a 6-tap operation using six integer pixels in the vertical direction or the horizontal direction on the input reference image 15 in the same manner as the half-pixel precision sample generation processing according to the MPEG-4 AVC standard. An enlarged image obtained by enlarging the reference image 15 in the vertical and horizontal N times size is generated by performing an interpolation operation using a filter or an enlargement filter process such as a bilinear filter.

  The adding unit 206 adds the high-frequency component input from the high-frequency component estimation unit 203 to the enlarged image input from the image enlargement processing unit 205, that is, corrects the high-frequency component of the enlarged image, thereby obtaining a vertical and horizontal N-fold size. An enlarged enlarged reference image is generated. The interpolated image generation unit 43 uses the enlarged reference image data as a reference image 207 with virtual pixel accuracy in which 1 / N is 1.

  The interpolated image generation unit 43 generates a reference image 207 with half pixel (1/2 pixel) accuracy with N = 2, and then converts a virtual sample (pixel) with 1/4 pixel accuracy into an adjacent 1/2 pixel. Or you may comprise so that it may produce | generate by the interpolation calculation using the average value filter of an integer pixel.

In addition to the configuration shown in FIG. 16, the interpolation image generation unit 43 switches whether or not to add the high frequency component output from the high frequency component estimation unit 203 to the enlarged image output from the image enlargement processing unit 205, The generation result of the reference image 207 with pixel accuracy may be controlled. In the case of this configuration, when the estimation accuracy by the high-frequency component estimation unit 203 is poor for some reason, such as when the image pattern is unique, there is an effect of suppressing an adverse effect on the encoding efficiency.
When the addition unit 206 selectively determines whether or not the high-frequency component output from the high-frequency component estimation unit 203 is added, a predicted image 45 is generated in both cases of addition and non-addition to generate motion compensation. Prediction is performed, and the result is encoded to determine the more efficient one. And the information of the addition process which shows whether it added is multiplexed to the bit stream 30 as control information.

  Alternatively, the interpolation image generation unit 43 may control the addition process of the addition unit 206 by uniquely determining from other parameters to be multiplexed into the bit stream 30. As an example of determining from other parameters, for example, the type of encoding mode 7 shown in FIG. 2A or 2B may be used. When the coding mode indicating that the motion compensation area block division in the macroblock is fine is selected, there is a high probability that the pattern is intensely moving. Therefore, the interpolation image generation unit 43 considers that the super-resolution effect is low, and controls the addition unit 206 not to add the high frequency component output from the high frequency component estimation unit 203. On the other hand, when the coding mode indicating that the size of the motion compensation region block in the macroblock is large or the intra prediction mode having a large block size is selected, there is a high probability that the image region is a relatively still image region. Therefore, the interpolated image generation unit 43 considers that the super-resolution effect is high, and controls the addition unit 206 to add the high-frequency component output from the high-frequency component estimation unit 203.

  In addition to using the encoding mode 7 as other parameters, parameters such as the magnitude of the motion vector and the variation of the motion vector field in consideration of the peripheral region may be used. The interpolation image generation unit 43 of the motion compensation prediction unit 9 does not have to multiplex the control information of the addition processing directly into the bit stream 30 by determining the type of parameter shared with the decoding device side, and the compression efficiency is improved. Can be increased.

  Note that the reference image 15 stored in the motion compensated prediction frame memory 14 is converted into the virtual pixel precision reference image 207 by the super-resolution processing described above before being stored in the motion compensated prediction frame memory 14 and then stored thereafter. You may comprise. In the case of this configuration, the memory size required as the motion compensated prediction frame memory 14 increases, but it becomes unnecessary to perform the super-resolution processing sequentially during the motion vector search and the prediction image generation, and the motion compensation prediction processing itself. In addition, the frame encoding process and the generation process of the reference image 207 with virtual pixel accuracy can be performed in parallel, and the processing speed can be increased.

  Hereinafter, an example of a motion vector detection procedure with virtual pixel accuracy using the reference image 207 with virtual pixel accuracy will be described with reference to FIG.

Motion vector detection procedure I ′
The interpolated image generation unit 43 generates a predicted image 45 for a motion vector 44 with integer pixel accuracy that is within a predetermined motion search range of the motion compensation region block image 41. The predicted image 45 (predicted image 17) generated with integer pixel accuracy is output to the subtractor 12, and is subtracted from the motion compensation region block image 41 (macro / sub-block image 5) by the subtracter 12, so that the predicted difference signal 13 is subtracted. become. The encoding control unit 3 evaluates the prediction efficiency with respect to the prediction difference signal 13 and the integer pixel precision motion vector 44 (prediction parameter 18). Since this prediction efficiency may be evaluated by the above equation (1) described in the first embodiment, description thereof is omitted.

Motion vector detection procedure II '
The interpolated image generating unit 43 performs the interpolated image generating unit 43 illustrated in FIG. 16 on the motion vector 44 having the ½ pixel accuracy positioned around the motion vector having the integer pixel accuracy determined in the “motion vector detection procedure I”. A predicted image 45 is generated using a reference image 207 with virtual pixel accuracy generated internally. Hereinafter, similarly to the “motion vector detection procedure I”, the predicted image 45 (predicted image 17) generated with the ½ pixel accuracy is converted by the subtractor 12 into the motion compensation region block image 41 (macro / sub-block image 5). ) To obtain the prediction difference signal 13. Subsequently, the encoding control unit 3 evaluates the prediction efficiency with respect to the prediction difference signal 13 and the motion vector 44 with 1/2 pixel accuracy (prediction parameter 18), and is positioned around the motion vector with integer pixel accuracy. A motion vector 44 having a ½ pixel accuracy that minimizes the prediction cost J ₁ is determined from one or more motion vectors having a ½ pixel accuracy.

Motion vector detection procedure III '
The encoding control unit 3 and the motion compensation prediction unit 9 similarly apply to the motion vector with a ¼ pixel accuracy around the motion vector with a ½ pixel accuracy determined in the “motion vector detection procedure II”. A motion vector 44 having a 1/4 pixel accuracy that minimizes the prediction cost J ₁ is determined from one or more motion vectors having a 1/4 pixel accuracy positioned at the position.

Motion vector detection procedure IV '
Similarly, the encoding control unit 3 and the motion compensation prediction unit 9 detect a motion vector with virtual pixel accuracy until a predetermined accuracy is obtained.

  As described above, the motion compensation prediction unit 9 determines the predetermined predetermined values for the motion compensation region block images 41 obtained by dividing the macro / sub-block image 5 into blocks each serving as a motion compensation unit indicated by the encoding mode 7. The motion vector of the accurate virtual pixel accuracy and the identification number of the reference image indicated by the motion vector are output as the prediction parameter 18. Further, the motion compensation prediction unit 9 outputs the prediction image 45 (prediction image 17) generated by the prediction parameter 18 to the subtracter 12, and is subtracted from the macro / sub-block image 5 by the subtractor 12, so that the prediction difference signal 13 is subtracted. Get. The prediction difference signal 13 output from the subtracter 12 is output to the transform / quantization unit 19. Subsequent processes are the same as those described in the first embodiment, and a description thereof will be omitted.

Next, the moving picture decoding apparatus according to the third embodiment will be described.
The configuration of the moving picture decoding apparatus according to the third embodiment is the same as that of the first embodiment except that the configuration and operation related to the predicted image generation processing with virtual pixel accuracy in the motion compensation prediction unit 70 according to the first and second embodiments are different. Since it is the same as the moving image decoding apparatus of form 1 and 2, FIGS. 1-16 is used.

  In Embodiments 1 and 2 described above, when the motion compensation prediction unit 70 generates a predicted image based on a reference image with a virtual pixel accuracy such as a half pixel or a quarter pixel, as in the MPEG-4 AVC standard, The motion compensated prediction unit 70 according to the third embodiment generates a predicted image by generating a virtual pixel by interpolation using a 6-tap filter using six integer pixels in the horizontal direction or the horizontal direction. The reference image 76 with integer pixel accuracy stored in the motion compensated prediction frame memory 75 is enlarged by super-resolution processing, thereby generating a reference image with virtual pixel accuracy.

Similar to the first and second embodiments, the motion compensated prediction unit 70 according to the third embodiment includes the motion vector included in the input optimum prediction parameter 63 and the identification number of the reference image indicated by each motion vector (reference image). The prediction image 72 is generated from the reference image 76 stored in the motion compensated prediction frame memory 75 and output based on the index).
The adder 73 adds the predicted image 72 input from the motion compensation prediction unit 70 to the predicted differential signal decoded value 67 input from the inverse quantization / inverse transform unit 66 to generate a decoded image 74.

  Note that the method of generating the predicted image 72 by the motion compensation prediction unit 70 is a process of searching for a motion vector from a plurality of reference images among the operations of the motion compensation prediction unit 9 on the encoding device side (the motion detection unit 42 shown in FIG. 3). And corresponding to the operation of the interpolated image generating unit 43), and only the process of generating the predicted image 72 is performed according to the optimal prediction parameter 63 given from the variable length decoding unit 61.

  Here, when the predicted image 72 is generated with virtual pixel accuracy, a motion compensated prediction unit for the reference image 76 specified by the reference image identification number (reference image index) on the motion compensated prediction frame memory 75 is used. 70 performs a process similar to the process shown in FIG. 16 to generate a reference image with virtual pixel accuracy, and generates a predicted image 72 using the decoded motion vector. At this time, when the encoding device side selectively determines whether or not to add the high-frequency component output from the high-frequency component estimation unit 203 shown in FIG. 16 to the enlarged image, the decoding device side performs addition processing. The control information indicating the presence or absence is extracted from the bit stream 60 or is uniquely determined from other parameters to control the addition processing in the motion compensation prediction unit 70. When determining from other parameters, the encoding mode 7, the size of the motion vector, the variation of the motion vector field in consideration of the surrounding area, etc. can be used as in the above-described encoding device side, and motion compensation is performed. When the prediction unit 70 determines the parameter type in common with the encoding device side, the encoding device does not have to multiplex the control information of the addition process directly into the bitstream 30, and the compression efficiency can be improved. .

  Note that in the process of generating a reference image with virtual pixel accuracy in the motion compensated prediction unit 70, the motion vector included in the optimal prediction parameter 18a output from the encoding device side (that is, the optimal prediction parameter 63 on the decoding device side) is virtual. You may implement only when indicating pixel accuracy. In the case of this configuration, the motion compensation prediction unit 9 uses the reference image 15 of the motion compensation prediction frame memory 14 according to the motion vector, or the interpolation image generation unit 43 generates the reference image 207 with virtual pixel accuracy. The prediction image 17 is generated from the reference image 15 or the reference image 207 with virtual pixel accuracy by switching whether to use.

  Alternatively, the process shown in FIG. 16 is performed on the reference image before being stored in the motion compensated prediction frame memory 75, and the reference image with virtual pixel accuracy in which the enlargement process and the high frequency component are corrected is stored in the motion compensated prediction frame memory 75. You may comprise so that it may store. In the case of this configuration, the memory size to be prepared as the motion compensated prediction frame memory 75 increases, but when the number of times the motion vector points to the pixel at the same virtual sample position is large, the processing shown in FIG. Since there is no need, the amount of calculation can be reduced. If the range of displacement indicated by the motion vector is known in advance to the decoding device side, the motion compensation prediction unit 70 may be configured to perform the process shown in FIG. The range of displacement indicated by the motion vector is set by, for example, multiplexing and transmitting a value range indicating the range of displacement indicated by the motion vector in the bitstream 60, or by deciding between the encoding device side and the decoding device side in operation. It may be made known to the decoding device side.

  As described above, according to the video encoding apparatus according to Embodiment 3, the motion compensation prediction unit 9 performs the enlargement process on the reference image 15 in the motion compensation prediction frame memory 14 and corrects the high-frequency component thereof to perform virtual processing. An interpolated image generation unit 43 that generates a reference image 207 with pixel accuracy, and switches between whether to use the reference image 15 or to generate and use a reference image 207 with virtual pixel accuracy according to the motion vector. Therefore, even when the input video signal 1 containing a lot of high-frequency components such as fine edges is highly compressed, the predicted image 17 generated by the motion compensation prediction contains a lot of high-frequency components. It becomes possible to generate from a reference image, and compression encoding can be performed efficiently.

  The video decoding apparatus according to Embodiment 3 also includes an interpolation image generation unit in which the motion compensation prediction unit 70 generates a reference image with virtual pixel accuracy in the same procedure as the video encoding apparatus. The prediction image 72 is generated by switching between using the reference image 76 of the motion compensated prediction frame memory 75 or generating and using a reference image with virtual pixel accuracy according to the motion vector multiplexed in the stream 60. Therefore, it is possible to correctly decode the bitstream encoded by the video encoding apparatus according to Embodiment 3.

  In the interpolation image generation unit 43 in the third embodiment, the above-described W.P. T.A. Freeman et al. Although the reference image 207 with virtual pixel accuracy is generated by the super-resolution processing based on the technique disclosed in (2000), the super-resolution processing itself is not limited to this technique, and any other super-resolution An image technique may be applied to generate the reference image 207 with virtual pixel accuracy.

Further, when the moving picture coding apparatus according to the first to third embodiments is configured by a computer, the block dividing unit 2, the coding control unit 3, the switching unit 6, the intra prediction unit 8, the motion compensation prediction unit 9, the motion A moving picture code describing the processing contents of the compensated prediction frame memory 14, the transform / quantization unit 19, the inverse quantization / inverse transform unit 22, the variable length coding unit 23, the loop filter unit 27, and the intra prediction memory 28 The computer program may be stored in the memory of the computer, and the CPU of the computer may execute the moving image encoding program stored in the memory.
Similarly, when the video decoding device according to Embodiments 1 to 3 is configured by a computer, a variable length decoding unit 61, an inverse quantization / inverse transform unit 66, a switching unit 68, an intra prediction unit 69, a motion compensation prediction unit 70, a motion compensation prediction frame memory 75, an intra prediction memory 77, and a moving picture decoding program describing the processing contents of the loop filter unit 78 are stored in the memory of the computer, and the moving image in which the CPU of the computer is stored in the memory An image decoding program may be executed.

  1 input video signal, 2 block division unit, 3 encoding control unit, 4 macro block size, 5 macro / sub-block image, 6 switching unit, 7 encoding mode, 7a optimal encoding mode, 8 intra prediction unit, 9 motion Compensation prediction unit, 10 intra prediction mode, 11 prediction image, 12 subtraction unit, 13 prediction difference signal, 13a optimal prediction difference signal, 14 motion compensation prediction frame memory, 15 reference image, 17 prediction image, 18 prediction parameter, 18a optimal prediction Parameter, 19 transform / quantization unit, 20 compression parameter, 20a optimum compression parameter, 21 compressed data, 22 inverse quantization / inverse transform unit, 23 variable length coding unit, 24 local decoded prediction difference signal, 25 addition unit, 26 Local decoded image signal, 27 loop filter section, 28 intra prediction memory, 29 stations Decoded image, 30 bit stream, 40 motion compensation region dividing unit, 41 motion compensation region block image, 42 motion detecting unit, 43 interpolated image generating unit, 44 motion vector, 45 predicted image, 50 transform block size segmenting unit, 51 transform Target block, 52 transform unit, 53 transform coefficient, 54 quantization unit, 60 bit stream, 61 variable length decoding unit, 62 optimum coding mode, 63 optimum prediction parameter, 64 compressed data, 65 optimum compression parameter, 66 inverse quantization Inverse transform unit, 67 prediction difference signal decoded value, 68 switching unit, 69 intra prediction unit, 70 motion compensation prediction unit, 71 prediction image, 72 prediction image, 73 addition unit, 74, 74a decoded image, 75 motion compensation prediction frame Memory, 76 reference images, 77 Intra prediction memory, 78 Loop fill Unit, 79 playback image, 90 initialization unit, 91 context information initialization flag, 92 binarization unit, 93 frequency information generation unit, 94 frequency information, 95 binarization table update unit, 96 context information memory, 97 probability table Memory, 98 state transition table memory, 99 context generation unit, 100 type signal, 101 peripheral block information, 102 context identification information, 103 binary signal, 104 arithmetic coding processing operation unit, 105 binary table memory, 106 context information , 107 probability table number, 108 MPS occurrence probability, 109 symbol value, 110 probability table number, 111 encoded bit string, 112 binarized table update identification information, 113 binarized table update flag, 120 initialization unit, 121 context initial Information, 122 context generation unit, 123 type signal, 124 peripheral block signal, 126 context identification information, 127 arithmetic decoding processing operation unit, 128 context information memory, 129 context information, 130 probability table number, 131 probability table memory, 132 MPS occurrence probability, 133 coded bit string, 134 symbol values, 135 state transition table memory, 136 probability table number, 137 binary signal, 138 inverse binarization unit, 139 binarization table, 140 decoded value, 141 binarization table update unit, 142 binarization table update flag, 143 binarization table memory, 144 binarization table update identification information, 200 image reduction processing unit, 201a, 201b high frequency feature extraction unit, 202 correlation calculation unit, 203 high frequency component estimation unit, 204 High-frequency component pattern memory, 205 Image enlargement processing unit, 206 Addition unit, 207 Reference image with virtual pixel accuracy.

Claims (2)

An encoding control unit that outputs an encoding mode in which one or more block division types are designated from among a plurality of block division types that are processing units of motion compensated prediction for an input image;
A block division unit for outputting a block image obtained by dividing the input image into one or more blocks according to the encoding mode;
A motion compensated prediction frame memory for storing reference image data used for the motion compensated prediction;
For each block image output by the block dividing unit, motion compensated prediction is performed using reference image data of the motion compensated prediction frame memory to generate a motion vector and a predicted image, and
A subtraction unit that generates a prediction difference signal by subtracting the prediction image output by the motion compensation prediction unit from the block image output by the block division unit;
A transform / quantization unit that performs transform and quantization processing on the prediction difference signal to generate compressed data;
A variable length encoding unit that entropy encodes the encoding mode, the motion vector, and the compressed data into a bitstream;
The motion compensation prediction unit includes an interpolated image generation unit that corrects a high frequency component of reference image data in the motion compensation prediction frame memory to generate reference image data with virtual pixel accuracy according to the motion vector. An image encoding apparatus that generates a prediction image by switching between using reference image data in the motion prediction frame memory or generating and using reference image data of the virtual pixel accuracy. A bitstream that is compression-encoded in units obtained by dividing an image into blocks of a predetermined size is input, and an encoding mode is entropy-decoded from the bitstream in units of the block, and according to the decoded encoding mode A variable length decoding unit that entropy decodes the prediction parameter, the compression parameter, and the compressed data in units of divided blocks;
Using the compression parameter, an inverse quantization and inverse transform process is performed on the compressed data, and a decoded prediction difference signal is generated, and an inverse quantization / inverse transform unit;
A motion compensation prediction frame memory for storing reference image data used for motion compensation prediction;
Motion that generates a predicted image by performing motion compensation prediction using a motion vector included in the prediction parameter and reference image data in the motion compensated prediction frame memory specified by a reference image index included in the prediction parameter A compensation prediction unit;
An addition unit that adds the prediction image generated by the motion compensation prediction unit to the decoded prediction difference signal and outputs a decoded image signal;
The motion compensation prediction unit includes an interpolation image generation unit that corrects a high frequency component of reference image data in the motion compensation prediction frame memory to generate reference image data with virtual pixel accuracy, and is included in the prediction parameter Moving picture decoding characterized in that a predicted picture is generated by switching between using reference picture data in the motion compensated prediction frame memory or generating and using reference picture data of the virtual pixel accuracy in accordance with a motion vector apparatus.

Images