RU2368095C1 - Image coder and image decoder, method of image coding and method of image decoding, program of image coding and program of image decoding and computer-readable record medium, on which image coding program is recorded, and computer-readable record medium, on which image decoding program is recorded - Google Patents

Abstract

FIELD: physics, image processing.

SUBSTANCE: invention is related to coder/decoder of image digital signal. Image coder is suggested, which comprises the following components: unit of predicted image generation, which generates predicted image in compliance with combination of prediction modes, which indicate method of predicted image generation; unit of decision-making relative to prediction mode, which assesses efficiency of prediction of predicted image produced from unit of predicted image generation, to make decision in respect to previously identified prediction mode; and coding unit, which exposes output signal of decision-making unit versus prediction mode to coding with alternating length of word. Unit of decision-making versus prediction mode makes decision, on the basis of previously identified control signal, which out of common prediction mode and separate prediction mode is used for corresponding colour components, which create input image signal, and multiplexes information about control signal into bitstream, multiplexes, when common prediction mode is used, information of common prediction mode into bitstream and multiplexes, when common prediction mode is not used, information of prediction mode for each colour component into bitstream.

EFFECT: improved optimality of moving image signal coding.

63 cl, 86 dwg

Description

FIELD OF TECHNOLOGY

The present invention relates to a digital image signal encoder, a digital image signal decoder, a digital image signal encoding method, and a digital image signal decoding method, which are used to compress and encode an image, transmit compressed image data, and the like.

BACKGROUND

Traditionally, video coding systems based on international standards, such as MPEG and ITU-TH.26x, have been adopted, mainly provided that the input signal is used in a standard format called the "4: 2: 0" format. The 4: 2: 0 format represents a format for converting a color signal of a moving image, such as RGB, into a luminance component (Y) and two color-difference components (Cb and Cr) and halving the number of color-difference component samples in horizontal and vertical directions with respect to the number of component samples brightness. Since the visual perception of the deterioration of the color difference components is small compared to the deterioration of the brightness component, a conventional video coding system that complies with the international standard is adopted, provided that the amount of information about the encoding object is reduced by implementing the above-described downsampling of the color difference components before encoding. On the other hand, in connection with increasing resolution and improving the quality of video displays in recent years, a system has been tested for encoding images with samples identical to the brightness components without down-sampling color difference components. A format in which the number of samples of luminance components and the number of samples of color difference components are the same is called the 4: 4: 4 format. According to the MPEG-4 AVC standard (ISO / IEC 14496-10) /ITU-T_H.264 (hereinafter referred to as AVC), it was decided to introduce a 4: 4: 4 format, "high profile 444" into the encoding system. Although the traditional 4: 2: 0 format is accepted provided that the color-difference components undergo downsampling and are limited to determining the color spaces Y, Cb and Cr, in the 4: 4: 4 format there is no difference in the sampling frequency between the color components, which makes it possible to directly use R, G, and B instead of Y, Cb, and Cr, and use many other color space definitions. In a video coding system using a 4: 2: 0 format, since the color spaces are fixed as Y, Cb, and Cr, there is no need to consider the types of color spaces in the coding processing. However, the high 4: 4: 4 profile AVC is a system in which the definition of color spaces affects the encoding processing itself. On the other hand, this high 4: 4: 4 profile takes into account compatibility with other profiles for encoding the 4: 2: 0 format defined by the Y, Cb, and Cr spaces. Thus, it cannot be said that a true 4: 4: 4 high profile is designed to optimize its compression efficiency.

Non-Patent Document 1: MPEG-4 AVC Standard (ISO / IEC 14496-10) /ITU-TH.264

SUMMARY OF THE INVENTION

The tasks solved by the invention

For example, in a 4: 2: 0 high profile for AVC 4: 2: 0 encoding, in the macroblock area consisting of 16 × 16 pixel luminance components, the color difference components Cb and Cr corresponding to luminance components are 8 × 8 pixel blocks . Spatial prediction (intra-prediction), which uses the value of peripheral samples in an identical image, is adopted for intra-coding macroblocks in a high profile of 4: 2: 0. Separate intra-prediction modes are used for luminance and color-difference components. The mode having the highest prediction efficiency is selected from the nine types shown in FIG. 3 as the intra-prediction mode for luminance components, and the mode having the highest prediction efficiency is selected from the four types shown in FIG. 9 as the intra-prediction mode for color components of Cb and Cr (it is impossible to use separate prediction modes for Cb and Cr). When predicting motion compensation in a high 4: 2: 0 profile, block size information used as a prediction unit with motion compensation, reference image information used for prediction, and motion vector information for each block are multiplexed only for luminance components. Motion compensation prediction is performed for color difference components using information the same as the information used for motion compensation prediction for luminance components. The above system is suitable provided that the contribution of the color-difference components to the definition of color spaces is small compared to the contribution of the brightness components, which make a significant contribution to the representation of the structure (texture) of the image in the 4: 2: 0 format. However, the true 4: 4: 4 high profile is only a system obtained by a simple extension of the intra prediction mode for a color difference signal in the 4: 2: 0 format even when the block size of the color difference signal is increased by 16 × 16 pixels by one macroblock. As in the 4: 2: 0 format for a single component, for example, a luminance component, only information about one component is multiplexed to perform motion compensation prediction using the inter prediction mode, reference image information, and motion vector information common to the three components. Thus, it cannot be said that the true high 4: 4: 4 format is not always the optimal forecasting method in the 4: 4: 4 format, in which the corresponding color components make equal contributions to the structural representation of the image signal.

Thus, it is an object of the present invention to provide an encoder, a decoder, an encoding method, a decoding method, programs for executing these methods and recording media onto which these programs are recorded, with improved coding optimality of a moving image signal in which there is no difference in sampling frequency between color components, such as the 4: 4: 4 format described in the related art.

MEANS OF SOLVING TASKS

An image encoder according to the present invention includes:

a predicted image generation unit that generates a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image;

a decision unit for the prediction mode that evaluates the prediction efficiency of the predicted image output from the predicted image generation unit for deciding on a predetermined prediction mode; and

a coding unit that subjects the output of the decision unit regarding the prediction mode to coding with a variable word length in which

the decision block regarding the prediction mode makes a decision based on a predetermined control signal which of the general prediction mode and the individual prediction mode are used for the respective color components forming the input image signal, and multiplexes the information about the control signal into a bit stream multiplexes when the common mode is used prediction, general prediction mode information into the bitstream and multiplexes when the general prog mode ozirovaniya not used, prediction mode information for each of the color components on the bit stream.

Advantages of the Invention

According to an image encoder, an image decoder, an image encoding method, an image decoding method, programs for executing these methods, and recording media on which these programs are recorded in accordance with the invention when encoding using not only fixed color spaces, for example, Y, Cb and Cr , but also of different color spaces, it is possible to flexibly select the information of the intra prediction mode and the information of the inter prediction mode used in the corresponding color com onentah, and can perform optimum encoding processing even when the definition of various color spaces.

BRIEF DESCRIPTION OF THE DRAWINGS

1 is a diagram explaining a structure of a video encoder according to a first embodiment.

2 is a diagram for explaining a structure of a video decoder according to a first embodiment.

FIG. 3 is a diagram explaining a method for generating a predicted image in a 4 × 4 intra-prediction mode estimated by the spatial prediction unit 2 shown in FIG. 1.

FIG. 4 is a diagram explaining a method for generating a predicted image in an intra-prediction mode of 16 × 16 estimated by the spatial prediction unit 2 shown in FIG. 1.

FIG. 5 is a flowchart illustrating a decision processing procedure for an intra-prediction mode implemented in the video encoder shown in FIG. 1.

6 is a diagram illustrating an array of data of a bitstream of a video signal output from a video encoder according to a first embodiment.

Fig. 7 is a flowchart for explaining an intra-prediction decoding processing procedure carried out in the video decoder shown in Fig. 2.

Fig. 8 is a diagram for explaining a mode of another data array of a video signal bitstream output from a video encoder according to a first embodiment.

Fig. 9 is a diagram explaining a method for generating a predicted image in an intra-prediction mode adapted to color difference components according to the AVC standard.

Figure 10 is a diagram illustrating traditional and real macroblocks.

11 is a diagram explaining a structure of a video encoder according to a second embodiment.

12 is a diagram for explaining a structure of a video decoder according to a second embodiment.

FIG. 13 is a diagram explaining a method for generating a predicted image in an 8 × 8 intra-prediction mode estimated by the spatial prediction unit 2 shown in FIG. 11.

Fig. 14 is a flowchart illustrating a decision processing procedure for an intra-coding mode implemented in the video encoder shown in Fig. 11.

Fig. 15 is a diagram for explaining an array of data of a bitstream of a video signal output from a video encoder according to a second embodiment.

Fig. 16 is a diagram for explaining another data array of a bitstream of a video signal output from a video encoder according to a second embodiment.

FIG. 17 is a flowchart for explaining an intra-prediction decoding processing procedure carried out in the video decoder shown in FIG.

Fig. 18 is a diagram explaining encoding processing parameters of an intra-prediction mode of a component C0 according to a third embodiment.

FIG. 19 is a diagram explaining encoding processing parameters of an intra prediction mode of component C1 according to a third embodiment.

FIG. 20 is a diagram explaining encoding processing parameters of an intra-prediction mode of component C2 according to the third embodiment.

FIG. 21 is a flowchart showing an encoding processing mode of an intra prediction mode according to the third embodiment.

FIG. 22 is a flowchart showing a different order of intra-prediction mode coding processing according to the third embodiment.

Fig. 23 is a flowchart showing an intra prediction mode decoding processing order according to the third embodiment.

24 is a diagram for explaining another data array of a video signal bitstream output from a video encoder according to a fourth embodiment.

25 is a flowchart showing a different processing order of the intra prediction mode coding according to the fifth embodiment.

Fig. 26 is a diagram for explaining tabulated rules for setting a predicted value according to a fifth embodiment.

Fig. 27 is a flowchart showing an encoding procedure according to a sixth embodiment.

28 is a diagram illustrating a binary sequence structure of CurrIntraPredMode according to the sixth embodiment.

29 is a diagram illustrating another structure of a binary sequence of CurrIntraPredMode according to a sixth embodiment.

30 is a diagram illustrating a structure of a video encoder according to a seventh embodiment.

Fig. 31 is a diagram for explaining a structure of a video decoder according to a seventh embodiment.

Fig. 32 is a diagram illustrating a macroblock unit.

33 is a flowchart showing a decision processing order regarding an inter prediction mode according to a seventh embodiment.

Fig. 34 is a diagram for explaining a data stream of a video stream output from a video encoder according to a seventh embodiment.

Fig. 35 is a flowchart showing a flow of processing carried out by the variable-length decoding unit 25 according to the seventh embodiment.

Fig. 36 is a diagram for explaining another array of video stream data output from a video encoder according to a seventh embodiment.

Fig. 37 is a diagram for explaining yet another data stream of a video stream output from a video encoder according to a seventh embodiment.

Fig. 38 is a flowchart showing a decision processing order regarding an inter prediction mode according to an eighth embodiment.

Fig. 39 is a diagram illustrating a data stream of a bitstream at the macroblock level according to the eighth embodiment.

40 is a flowchart showing an order of processing for generating an inter-predicted image according to an eighth embodiment.

Fig. 41 is a diagram for explaining another data stream of a bitstream at a macroblock level according to an eighth embodiment.

Fig. 42 is a diagram for explaining yet another data stream of a bitstream at the macroblock level according to the eighth embodiment.

Fig. 43 is a flowchart showing a decision processing order regarding an inter prediction mode according to a ninth embodiment.

Fig. 44 is a flowchart showing an order of processing for generating an inter-predicted image according to a ninth embodiment.

Fig. 45 is a diagram for explaining a structure of a motion vector coding unit.

Fig. 46 is a diagram for explaining operations of a motion vector coding unit.

Fig. 47 is a diagram for explaining a structure of a motion vector decoding unit.

Fig. 48 is a diagram for explaining a state of a bitstream syntax.

Fig. 49 is a diagram for explaining a structure of data encoded in units of macroblocks according to an eleventh embodiment.

FIG. 50 is a diagram explaining a detailed structure of encoded data of header information of a component of Cn shown in FIG. 49, according to an eleventh embodiment.

51 is a diagram illustrating another structure of data encoded in units of macroblocks according to an eleventh embodiment.

Fig. 52 is a diagram for explaining a structure of a bitstream according to an eleventh embodiment.

Fig. 53 is a diagram for explaining a slice structure according to an eleventh embodiment.

Fig. 54 is a diagram for explaining an internal structure related to arithmetic encoding processing of the variable-length encoding unit 11 according to the twelfth embodiment.

Fig. 55 is a flowchart showing an arithmetic encoding processing order of a variable-length encoding unit 11 according to a twelfth embodiment.

Fig. 56 is a diagram explaining a detailed processing flow in step S162 shown in Fig. 55 according to a twelfth embodiment.

Fig. 57 is a diagram for explaining the concept of a contextual model (ctx).

Fig. 58 is a diagram for explaining an example of a contextual model relating to a motion vector of a macroblock.

Fig. 59 is a diagram for explaining an internal structure related to arithmetic decoding processing of a variable word length decoding unit 25 according to a twelfth embodiment.

60 is a flowchart showing an arithmetic decoding processing order of a variable-length decoding unit 25 according to a twelfth embodiment.

Fig. 61 is a diagram for explaining a contextual model 11f according to a twelfth embodiment.

Fig. 62 is a diagram for explaining a difference in a mode of a current macroblock according to a twelfth embodiment.

63 is a diagram for explaining structures of an encoder and a decoder according to a thirteenth embodiment.

64 is a diagram explaining a structure of a video encoder according to a thirteenth embodiment.

Fig. 65 is a diagram for explaining a structure of a video decoder according to a thirteenth embodiment.

Fig. 66 is a diagram for explaining general encoding processing according to a fourteenth embodiment.

Fig. 67 is a diagram for explaining independent encoding processing according to a fourteenth embodiment.

Fig. 68 is a diagram for explaining a relationship of a reference motion prediction in a time dimension between images in an encoder and a decoder according to a fourteenth embodiment.

Fig. 69 is a diagram for explaining an example of a structure of a bit stream generated by an encoder according to a fourteenth embodiment and subjected to input / decode processing by a decoder according to a fourteenth embodiment.

70 is a diagram explaining bitstream structures for slice data in cases of general encoding processing and independent encoding processing, respectively.

Fig. 71 is a diagram for explaining a schematic structure of an encoder according to a fourteenth embodiment.

72 is a diagram for explaining a state in which a processing delay on the encoder side is reduced.

73 is a diagram illustrating an internal structure of a first image encoding unit.

Fig. 74 is a diagram for explaining an internal structure of a second image encoding unit.

Fig. 75 is a diagram for explaining a schematic structure of a decoder according to a fourteenth embodiment.

Fig. 76 is a diagram for explaining an internal structure of a first image decoding unit.

Fig. 77 is a diagram for explaining an internal structure of a second image decoding unit.

78 is a diagram illustrating an internal structure of a first image encoding unit subjected to color space conversion processing.

Fig. 79 is a diagram explaining an internal structure of a first image encoding unit subjected to color space conversion processing.

80 is a diagram illustrating an internal structure of a first image encoding unit subjected to color space inverse transform processing.

Fig. 81 is a diagram illustrating an internal structure of a first image encoding unit subjected to color space inverse transform processing.

82 is a diagram illustrating a structure of encoded data of macroblock header information included in a bitstream of a conventional YUV 4: 2: 0 format.

Fig. 83 is a diagram for explaining the internal structure of the prediction block 461 of the first image decoding block, which ensures compatibility of the conventional YUV 4: 2: 0 format with the bitstream.

84 is a diagram explaining a structure of a bitstream of encoded data to be multiplexed according to a fifteenth embodiment.

Fig. 85 is a diagram for explaining information about an encoding type of an image during encoding of image data in an access unit, starting from an AUD NAL unit.

Fig. 86 is a diagram for explaining a structure of a bitstream of encoded data to be multiplexed according to a fifteenth embodiment.

DESCRIPTION OF SYMBOLS

1 video input

2 block spatial prediction

3 subtractor

4 prediction differential signal

5 decision block regarding coding mode

6 encoding mode

7 predicted image

8 conversion unit

9 quantization block

10 quantized conversion coefficient

11 variable-length coding unit

11a contextual model determination unit

11b binary conversion unit

11c occurrence probability generation unit

11d coding unit

11e coded value

11f contextual model

11g memory for storing occurrence probability information

11h state of probability of occurrence

12 block inverse quantization

13 inverse transform block

14 local decoding prediction differential signal

15 locally decoded image (internally decoded image)

16 memory

17 transmission buffer

18 adder

19 coding control unit

20 weight ratio

21 quantization parameters

22 video stream

23 flag identifying the common use of intra-prediction mode

24 release filter control flag

25 variable word length decoding unit

25a decoding unit

25b reconstructed plot value

26 release filter

27 decoded image

28 intra-coding mode

29 basic intra-prediction mode

30 advanced intra prediction mode

31 flag indicating the table of advanced intra prediction modes

32 conversion block size identification flag

33 flag identifying the common use of the mode of intracoding

34 intra-coding mode

35 intra prediction mode

36 flag indicating intra-prediction mode

102 motion compensated prediction unit

106 macroblock type / submacroblock type

123 Interprediction Mode Common Use Identification Flag

123b flag identifying the common use of the motion vector

123c macroblock header common use identification flag

128 basic macroblock type

128b macroblock type

129 basic type of submacroblock

129b sub-macroblock type

130 extended type of macroblock

131 extended type of submacroblock

132 reference reference image identification number

132b reference image identification number

133 basic motion vector information

134 extended reference identification number

135 extended motion vector information

136 profile information

137 motion vector

138, 138a, 138b, 138c skip indication information

139a, 139b, 139c header information

140a, 140b, 140c conversion coefficient data

141 intra prediction mode

142 conversion rate efficiency / inefficiency indication information

143 flag identifying the general use of the parameter state probability of occurrence

144 intra-prediction mode for color difference signal

111 motion vector prediction unit

112 block differential motion vector calculation

113 differential motion vector of a variable-length coding block

250 motion vector decoding unit

251 variable-length differential motion vector decoding unit

252 motion vector prediction unit

253 motion vector calculation unit

301 color space conversion unit

302 converted video signal

303 encoder

304 color space conversion method identification information

305 bit stream

306 decoder

307 decoded image

308 inverse color space conversion unit

310 conversion unit

311 color space conversion method identification information

312 inverse transform block

422a, 422b0, 422b1, 422b2, 422c video stream

423 common coding / independent coding identification signal

427a, 427b decoded image

461 prediction block

462 unlocking filter

463 predicted service information

464 flag indicating the converted block size

465 color space conversion unit

466 color space inverse transform unit

467 alarm information

501, 601 switch

502 color separation unit

503a first image encoding unit

503b0, 503b1, 503b2 second image encoding unit

504 multiplexing unit

602 color component decision block

603a first image decoding unit

603b0, 603b1, 603b2 second image decoding unit

610 top header analysis unit

4611a, 4611b, 4611c change unit

4612 luminance signal prediction unit

4613 block intra-prediction of a color difference signal

4614 luminance signal prediction unit

4615 color difference signal prediction unit

PREFERRED EMBODIMENTS FOR CARRYING OUT THE INVENTION

First Embodiment

According to a first embodiment, an encoder that encodes in a frame in units obtained by equal dividing a video frame inputted in a 4: 4: 4 format into rectangular areas (macroblocks) of 16 x 16 pixels and a decoder corresponding to the encoder will be explained. The characteristics of the invention are communicated to the encoder and decoder based on the encoding system adopted in MPEG-4 AVC (ISO / IEC 14496-10) /ITU-TH.264, which is a non-patent document 1.

The structure of the video encoder according to the first embodiment is shown in FIG. The structure of the video decoder according to the first embodiment is shown in FIG. In FIG. 2, the components indicated by identical positions to the components of the encoder shown in FIG. 1 are identical components.

The operations of the encoder as a whole and the decoder as a whole, decision processing regarding the intra-prediction mode, and intra-prediction decoding processing, which are characteristic operations according to the first embodiment, will be explained based on these figures.

1. Description of the encoder

In the encoder shown in FIG. 1, corresponding video frames are input as input video signal 1 in a 4: 4: 4 format. The input video frames are input to the encoder in units of macroblocks obtained by dividing the three color components into blocks of 16 pixels × 16 pixels of the same size and placing the blocks, as shown in FIG. 10.

First of all, the spatial prediction unit 2 performs intra-prediction processing for each of the color components in units of macroblocks using a locally decoded image 15 stored in the memory 16. Three memory blocks are prepared for the corresponding color components (although three memory blocks are prepared in the explanation of this embodiment, the number of memory blocks can be changed accordingly, depending on the actual implementation). The intra-prediction modes are a 4 × 4 intra-prediction mode for performing spatial prediction, in which, in units of 4 pixel × 4 line blocks shown in FIG. 3, neighboring block pixels are used, and a 16 × 16 intra-prediction mode for spatial prediction, in which , in macroblock units of 16 pixels × 16 lines shown in FIG. 4, neighboring pixels of the macroblock are used.

(a) 4 × 4 intra-prediction mode

A 16 × 16 pixel block of a luminance signal in a macroblock is divided into sixteen blocks formed by 4 × 4 pixel blocks. Any of the nine modes shown in FIG. 3 is selected in units of 4x4 pixel blocks. The pixels of the blocks (upper left, upper, upper right and left) around the block is already encoded, subjected to local decoding processing and stored in memory 16 are used to generate a predicted image.

Intra4 × 4_pred_mode = 0: The adjacent pixel at the top is used as the predicted image as is.

Intra4 × 4_pred_mode = 1: The adjacent pixel on the left is used as the predicted image as it is.

Intra4 × 4_pre_mode = 2: The average of the neighboring eight pixels is used as the predicted image.

Intra4 × 4_pred_mode = 3: A weighted average is calculated for every two to three pixels from neighboring pixels and used as a predicted image (corresponding to the edge 45 degrees to the right).

Intra4 × 4_pred_mode = 4: A weighted average is calculated for every two to three pixels from neighboring pixels and is used as a predicted image (corresponding to the edge 45 degrees to the left).

Intra4 × 4_pred_mode = 5: A weighted average is calculated for every two to three pixels from neighboring pixels and is used as a predicted image (corresponding to the edge 22.5 degrees to the left).

Intra4 × 4_pred_mode = 6: A weighted average is calculated for every two to three pixels from neighboring pixels and is used as a predicted image (corresponding to the edge 67.5 degrees to the left).

Intra4 × 4_pred_mode = 7: A weighted average is calculated for every two to three pixels from neighboring pixels and is used as a predicted image (corresponding to the edge 22.5 degrees to the right).

Intra4 × 4_pred_mode = 8: A weighted average is calculated for every two to three pixels from neighboring pixels and is used as a predicted image (corresponding to the edge 112.5 degrees to the left).

When choosing a 4 × 4 intra-prediction mode, sixteen pieces of mode information are necessary for each macroblock. Thus, in order to reduce the amount of codes of the mode information itself, taking into account the fact that the mode information has a high correlation with the block adjacent to it, the prediction coding is based on the mode information on the neighboring block.

(b) Intra prediction mode 16 × 16

A 16 × 16 intra-prediction coding mode is a mode for predicting blocks of 16 × 16 pixels equivalent in size to a macroblock at a given time. Any of the four modes shown in FIG. 4 is selected in units of macroblocks. In the same way as in 4 × 4 intra-prediction mode, block pixels (upper left, upper, and left) around a block already encoded, subjected to local decoding processing, and stored in memory 16 are used to generate a predicted image.

Intra16 × 16_pred_mode = 0: Sixteen pixels on the underside of the upper macroblock are used as the predicted image.

Intra16 × 16_pred_mode = 1: Sixteen pixels on the right side of the left macroblock are used as the predicted image.

Intra16 × 16_pred_mode = 2: The average value of thirty-two pixels, together including sixteen pixels on the lower side of the upper macroblock (part A in figure 4) and sixteen pixels on the left side of the left macroblock (part B in figure 4), is used as a predicted image.

Intra16 × 16_pred_mode = 3: The predicted image is obtained by predetermined arithmetic processing (weighted summation processing corresponding to the pixel used and the predicted pixel position) using thirty-one pixels, in total including the pixel on the lower right corner of the macroblock, above the upper left fifteen pixels on the lower side of the upper macroblock (part excluding empty pixels) and fifteen pixels on the right side of the left macroblock (part, excluding yuschaya blank pixels).

The video encoder according to the first embodiment is characterized by a change in the intra-prediction processing method for the three color components based on the flag 23 for identifying the common use of the intra-prediction mode. This point will be described in detail below in paragraph 2.

Spatial prediction unit 2 performs prediction processing on all modes or subsets shown in FIGS. 3 and 4 to obtain a prediction difference signal 4 using a subtractor 3. Prediction efficiency of the prediction difference signal 4 is evaluated by decision block 5 regarding the encoding mode. A prediction mode in which the optimal prediction efficiency is obtained for the macroblock specified as the prediction object is output as the encoding mode 6 from the prediction processing performed by the spatial prediction unit 2. Encoding mode 6 includes the corresponding varieties of information of the prediction mode (Intra4 × 4_pred_mode or Intra16 × 16_pred_mode) used for a single forecasting area together with decision information (equivalent to the intra-coding mode, showing 6) indicating whether a 4 × 4 intra-prediction mode or a 16 × 16 intra-prediction mode is used. A single prediction region is equivalent to a 4 × 4 pixel block in the case of intra4 × 4_pred_mode and equivalent to a 16 × 16 pixel block in the case of the intra × 16 × 16 prediction mode. When selecting encoding mode 6, a weighting factor of 20 for each encoding mode established by deciding the encoding control unit 19 can be taken into account. The optimal prediction difference signal 4 obtained using the encoding mode 6 on the decision block 5 regarding the encoding mode is output to the conversion unit 8. The conversion unit 8 converts the input prediction difference signal 4 into a conversion coefficient and outputs the conversion coefficient to quantization unit 9. Quantization unit 9 quantizes the input transform coefficient based on the quantization parameter 21 specified by the encoding control unit 19 and outputs a conversion coefficient assignment to the variable-length coding unit 11 as the quantized transform coefficient 10. The quantized transform coefficient 10 is entropy encoded by, for example, Huffman coding or arithmetic coding on the variable-length coding unit 11. The quantized transform coefficient 10 is restored to the local prediction differential prediction signal 14 by the inverse quantization unit 12 and the inverse transform unit 13. The quantized transform coefficient 10 is added to the predicted image 7, which is generated based on the encoding mode 6, by the adder 18 to generate a locally decoded image 15. The locally decoded image 15 is stored in the memory 16 for later use in intra-prediction processing. An enable filter control flag 24 indicating whether the enable filter is applied to the macroblock is also input to the variable-length encoding unit 11. (In the prediction processing performed by the spatial prediction unit 2, since the pixel data is stored in the memory 16 before being processed by the unlocking filter, the processing of the unlocking filter itself is not necessary for encoding processing. However, the unlocking filter acts according to the direction of the unlocking filter control flag 24 on the side decoder to obtain the final decoded image.)

The flag 23 identifies the common use of the intra-prediction mode, the quantized transform coefficient 10, the coding mode 6, and the quantization parameter 21 entered into the variable-length encoding unit 11 are ordered and formed as a bit stream in accordance with a predetermined rule (syntax) and output in transmission buffer 17. The transmission buffer 17 smooths the bit stream according to the band of the transmission line to which the encoder is connected and the read speed of the recording medium and outputs the bit stream as a video stream a 22. The transmission buffer 17 outputs feedback information to the encoding control unit 19 according to the state of accumulation of the bitstream into the transmission buffer 17 and then controls the amount of generated codes when encoding video frames.

2. Decision processing regarding intra prediction mode in the encoder

The following describes the decision processing regarding the intra-prediction mode, which is a characteristic of the encoder according to the first embodiment. This processing is carried out in units of macroblocks in which three color components are placed. The processing is carried out mainly by the spatial prediction unit 2 and the decision unit 5 regarding the encoding mode in the encoder shown in FIG. 1. A logical flowchart showing the progress of processing is shown in FIG. Image data of the three color components forming the block are indicated below C0, C1 and C2.

First of all, the decision block 5 regarding the coding mode receives the flag 23 for identifying the general use of the intra-prediction mode and makes a decision based on the value of the flag 23 for identifying the general use of the intra-prediction mode whether the intra-prediction mode common to C0, C1 and C2 is used (step S1 in FIG. 5). When using the general intra-prediction mode, the decision block 5 regarding the encoding mode proceeds to step S2 and subsequent steps. If the general intra-prediction mode is not used, the decision block 5 regarding the coding mode proceeds to step S5 and subsequent steps.

When the intra prediction mode is used together for C0, C1, and C2, the decision block 5 regarding the coding mode notifies the spatial prediction block 2 of all 4 × × 4 intra prediction modes that can be selected. The spatial prediction unit 2 evaluates the prediction efficiency for all 4 × 4 prediction modes and selects the optimal 4 × 4 intra prediction mode common to C0, C1, and C2 (step S2). Then, the decision block 5 regarding the coding mode notifies the spatial prediction block 2 of all 16 × 16 intra prediction modes that can be selected. The spatial prediction unit 2 estimates the prediction efficiency of all 16 × 16 intra-prediction modes and selects the optimal 16 × 16 intra-prediction mode common to C0, C1, and C2 (step S3). Finally, the decision block 5 regarding the encoding mode selects the optimal mode with respect to the prediction efficiency in the modes obtained in steps S2 and S3 (step S4), and ends the processing.

When the intra prediction mode is not shared between C0, C1 and C2 and C0, C1 and C2, respectively, the best modes are selected, the decision block 5 regarding the coding mode notifies the spatial prediction block 2 of all 4 × 4 intra prediction modes that can be selected for the components Ci (i <= 0 <3). The spatial prediction unit 2 estimates the prediction efficiency for all 4 × 4 intra-prediction modes and selects the optimal 4 × 4 intra-prediction mode in the Ci components (i <= 0 <3) (step S6). Similarly, the spatial prediction unit 2 selects an optimal 16 × 16 intra-prediction mode (step S7). Finally, in step S8, the spatial prediction unit 2 makes a decision regarding the optimal intra-prediction mode in the components Ci (i <= 0 <3).

As a standard for evaluating the prediction efficiency for the prediction mode carried out in the spatial prediction unit 2, for example, the velocity / distortion value set as Jm = Dm + λRm (λ: positive number) can be used. Dm is the value of coding distortion or prediction error in the case of using the intra prediction mode m. The coding distortion is obtained using the intra-prediction mode m to calculate the prediction error and decode the video signal from the result obtained by converting and quantizing the prediction error to measure the error with respect to the signal before encoding. The magnitude of the prediction error is obtained by calculating the difference between the predicted image and the signal before coding in the case when the intra prediction mode m and quantization of the difference level are applied. For example, Absolute Distance Sum (SAD) is used. Rm is the amount of generated codes when the intra prediction mode m is applied. In other words, Jm is a value specifying a compromise between the amount of codes and the degree of deterioration when the intra-prediction mode m is applied. The intra-prediction mode m, giving the minimum Jm, is the optimal solution.

When the encoder processes in step S2 and subsequent steps, one piece of intra-prediction mode information is allocated to the macroblock including three color components. On the other hand, when the encoder performs the processing in step S5 and subsequent steps, intra prediction mode information is allocated to the color components, respectively. Thus, since the fragments of intra-prediction mode information allocated to the macroblock are different, it is necessary to multiplex the intra-prediction mode common use identification flag 23 into the bitstream and allow the decoder to recognize whether the encoder has completed the processing steps in step S2 and subsequent steps or has completed the processing steps in step S5 and subsequent steps. The data array of such a bitstream is shown in FIG. 6.

The figure shows a data stream of the bitstream at the macroblock level. The intra-coding mode 28 indicates information allowing to distinguish between intra 4 × 4 and intra 16 × 16, and the basic intra-prediction mode 29 indicates information of the general intra-prediction mode in the case where the flag 23 identifying the general use of the intra-prediction mode indicates “common to C0, C1 and C2”. The basic intra-prediction mode 29 indicates the information of the intra-prediction mode for C0, when the flag 23 identifying the general use of the intra-prediction mode indicates "not common for C0, C1 and C2". The advanced intra-prediction mode 30 is multiplexed only when the intra-prediction mode common use identification flag 23 indicates “not common to C0, C1, and C2”. Advanced intra prediction mode 30 indicates intra prediction mode information for C1 and C2. Then, the quantization parameter 21 and the quantized transform coefficient 10 are multiplexed. The encoding mode 6 shown in FIG. 1 is a common member of the intra-coding mode 28 and the intra-prediction modes (basic and advanced) (although the enable filter control flag 24 entered in the variable-length encoding block 11 shown in FIG. 1 is not included in FIG. 6, an enable filter control flag 24 is omitted because the flag is not a component necessary to explain the characteristics of the first embodiment).

In the 4: 2: 0 format, adopted in the traditional video coding standard, the definition of color spaces is fixed to Y, Cb and Cr. In the 4: 4: 4 format, the definition of color spaces is not limited to Y, Cb, and Cr, but various color spaces can be used. By generating intra prediction mode information, as shown in FIG. 6, it is possible to perform optimal coding processing even when the determination of color spaces of the input video signal 1 is diverse. For example, when color spaces are defined as RGB, the structure of the video texture remains the same in the corresponding components R, G, and B. Thus, using information from the general intra-prediction mode, it is possible to reduce the redundancy of the intra-prediction mode information itself and increase the encoding efficiency. On the other hand, when color spaces are defined as Y, Cb, and Cr, the structure of the video texture is integrated into Y. Thus, the general intra-prediction mode does not always give an optimal result. Thus, it is possible to obtain optimal coding efficiency by adaptively using the advanced intra-prediction mode 30.

3. Description of the decoder

The decoder shown in FIG. 2 receives a video stream 22 corresponding to the array shown in FIG. 6, output from the encoder shown in FIG. 1, performs decoding processing in units of macroblocks in which the three color components have the same size (format 4: 4: 4), and restores the corresponding video frames.

First of all, the variable-length decoding unit 25 to which the stream 22 arrives, decodes the stream 22 in accordance with a predetermined rule (syntax) and extracts information including the general prediction mode identification flag 23, the quantized transform coefficient 10, mode encoding 6 and a quantization parameter 21. The quantized transform coefficient 10 is input to the inverse quantization unit 12 together with the quantization parameter 21, and the inverse quantum is processed Niya. Then, the output signal of the inverse quantization unit 12 is input to the inverse transform unit 13 and restored to the local prediction differential prediction signal 14. On the other hand, the encoding mode 6 and the intra-prediction mode common use identification flag 23 are input to the spatial prediction unit 2. The spatial prediction unit 2 receives the predicted image 7 in accordance with these pieces of information. The specific procedure for obtaining the predicted image 7 will be described below. The local decoding prediction difference signal 14 and the prediction image 7 are summed by the adder 18 to obtain an internally decoded image 15 (this is completely the same signal as the locally decoded image 15 in the encoder). The internally decoded image 15 is written back to memory 16 for use in further intra-prediction of the macroblock. Three memory blocks are prepared for the respective color components (although three memory blocks are prepared in the explanation of this embodiment, the number of memory blocks can accordingly be changed according to the design). The unlocking filter 26 is instructed to act on the internally decoded image 15 based on the indication of the unlocking filter control flag 24, decoded by the variable-length decoding unit 25, to obtain the final decoded image 27.

4. Processing of intra prediction decoding in a decoder

The processing of generating an intra-predicted image, which is a characteristic of a decoder according to the first embodiment, is described in detail below. This processing is carried out in units of macroblocks in which three color components are placed. The processing is carried out mainly by the variable-length decoding unit 25 and the spatial prediction unit 2 of the decoder shown in FIG. 2. A logical flowchart showing the progress of processing is shown in FIG.

Steps S10-S14 in the logic flowchart shown in FIG. 7 are performed by the variable-length decoding unit 25. The video stream 22 entering the block 25 decoding with a variable word length is consistent with the data array shown in Fig.6. In step S10, the variable-length decoding unit 25 first decodes the data intra-coding mode 28 shown in FIG. 6. Then, the variable word length decoding unit 25 decodes the intra-prediction mode common use identification flag 23 (step S11). In addition, the variable-length decoding unit 25 decodes the basic intra-prediction mode 29 (step S12). In step S13, the variable-length decoding unit 25 decides whether the intra-prediction mode is used together for C0, C1, and C2 using the result of the flag 23 for identifying the common use of the intra-prediction mode. When using the general intra-prediction mode, the variable-length decoding unit 25 uses the basic intra-prediction mode 29 for all of C0, C1, and C2. If the general intra-prediction mode is not used, the variable-length decoding unit 25 uses the basic intra-prediction mode 29 as the mode for C0 and decodes the advanced intra-prediction mode 30 (step S14) to obtain mode information for C1 and C2. Since the encoding mode 6 for the respective color components is set in the processing steps, the variable-length decoding unit 25 outputs the encoding mode 6 to the spatial prediction unit 2 and obtains intra-predicted images of the corresponding color components in accordance with steps S15-S17. The process for obtaining intra-predicted images is consistent with the procedures shown in FIGS. 3 and 4 and is identical to the processing performed by the encoder shown in FIG. 1.

Variations in the bitstream data array shown in FIG. 6 are shown in FIG. 7, a flag 23 for identifying common usage of an intra-prediction mode is multiplexed as a flag located at a higher data level, for example, at a slice, image, or sequence level, and not as a flag at the macroblock level. An indication flag 31 of the extended intra prediction mode table is provided for selecting a code table defining a code word of the extended intra prediction mode 30 from the plurality of code tables. Therefore, when it is possible to ensure sufficient prediction efficiency according to a change at a higher level equal to or higher than the cutoff level, it is possible to reduce the overhead bit without multiplexing the flag 23 for identifying the general use of the intra-prediction mode at the macroblock level each time the processing is performed. As for the extended intra prediction mode 30, since the display flag of the table of extended intra prediction modes 31 is provided, it is possible to choose the definition of the prediction mode defined for components C1 and C2, instead of identifying the definition of the basic intra prediction mode 29. This makes it possible to carry out encoding processing adapted to the definition of color spaces. For example, when encoding a 4: 2: 0 AVC format, a set of intra-prediction modes other than brightness (Y) is specified for the color difference component (Cb and Cr). In the 4: 2: 0 format, the color difference signal in the macroblock is a signal of 8 pixels × 8 lines. Any of the four modes shown in FIG. 9 is selected in units of macroblocks for decoding processing. Although there are two varieties of Cb and Cr as color difference signals, the same mode is used. With the exception of intra prediction DC intra_chroma_pred_mode = 0, the prediction processing is the same as in the 16 × 16 intra prediction mode shown in FIG. 4. In DC prediction, an 8 × 8 block is divided into four 4 × 4 blocks, and the positions of the pixels, for each of which the average value is calculated, are changed for each of the blocks for processing. In the block indicated in the figure "a + x, a or x", the average value is calculated using eight pixels "a" and "x", when you can use the pixel "a" and the pixel "x", using four pixels "a "when only the pixel" a "can be used, and using only four pixels" x ", when only the pixel" x "can be used. The average value is used as the predicted image 7. The value 128 is used as the predicted image 7 when it is impossible to use the pixels "a" and "x". In the block labeled “b or x,” the average value is calculated using four pixels “b” when the image “b” can be used, and using four pixels “x” when only the pixel “x” can be used.

Thus, when it is necessary to change the set of intra-prediction modes according to the characteristics of the color components, it is possible to obtain a more optimal coding efficiency according to a structure similar to the syntax shown in Fig. 8.

Second Embodiment

According to a second embodiment, another encoder that encodes in a frame in units obtained by equal dividing a video frame inputted in a 4: 4: 4 format into rectangular areas (macroblocks) of 16 x 16 pixels and a decoder corresponding to the encoder will be explained. As in the first embodiment, the characteristics of the invention are communicated to the encoder and decoder based on the encoding system adopted in the MPEG-4 AVC (ISO / IEC 14496-10) /ITU-TH.264 standard, which is Non-Patent Document 1.

The structure of the video encoder according to the second embodiment is shown in FIG. 11. The structure of the video decoder according to the second embodiment is shown in FIG. In FIG. 11, the components indicated by identical positions to the encoder components shown in FIG. 1 are identical components. In FIG. 12, the components indicated by identical positions to the encoder components shown in FIG. 11 are identical components. 11, reference numeral 32 denotes a transform block size identification flag and 33 denotes an intra-encoding mode common use identification flag.

The operations of the encoder as a whole and the decoder as a whole according to the second embodiment, decision processing regarding the intra-coding / prediction mode, and intra-prediction decoding processing, which are characteristic operations of the second embodiment, will be explained based on these figures.

1. Description of the encoder

In the encoder shown in FIG. 11, corresponding video frames are input as input video signal 1 in a 4: 4: 4 format. The input video frames are input to the encoder in units obtained by dividing the three color components into macroblocks of the same size and placing the blocks, as shown in FIG. 10.

The spatial prediction unit 2 performs intra-prediction processing for each of the color components in units of macroblocks using a locally decoded image 15 stored in the memory 16. The intra-prediction modes are a 4 × 4 intra-prediction mode for spatial prediction, in which 4 pixel × 4 lines shown in FIG. 3, using neighboring block pixels, 8 × 8 intra-prediction mode for spatial prediction a method in which, in units of 8 pixel × 8 blocks of lines shown in FIG. 13, neighboring block pixels are used, and a 16 × 16 intra-prediction mode for spatial prediction, in which, in units of macroblocks, 16 pixels × 16 lines shown in 4, neighboring pixels of the macroblock are used. In the encoder according to the second embodiment, the 4 × 4 intra-prediction mode and the 8 × 8 intra-prediction mode are interchanged and are used in accordance with the state of the transform block size identification flag 32. It can be imagined, using the intra-coding mode, as shown in FIG. 6, which of the intra-prediction modes for 4 × 4 prediction, 8 × 8 prediction, and 16 × 16 prediction is used to encode a particular macroblock. In the encoder according to the second embodiment, two types of coding modes are provided as intra-coding modes, namely, an N × N intra-prediction coding mode (N is 4 or 8) for encoding using a 4 × 4 intra-prediction mode or 8 × 8 intra-prediction mode and a mode 16 × 16 intra-prediction coding for performing coding using the 16 × 16 intra-prediction mode. Intracoding modes will be described below, respectively.

(a) N × N intra-prediction coding mode

N × N intra-prediction coding mode is a mode for performing selective coding of a 4 × 4 intra-prediction mode for dividing a block of 16 × 16 pixels of a luminance signal in a macroblock by sixteen blocks formed by 4 × 4 pixel blocks and individually selecting a prediction mode for of each of the 4 × 4 pixel blocks and the 8 × 8 intra-prediction mode for dividing the 16 × 16 pixel block of the brightness signal in the macroblock into four blocks formed by 8 × 8 pixel blocks, and individually selecting the prediction mode for zhdogo blocks of 8 × 8 pixels. The change in the intra-prediction mode 4 × 4 and the intra-prediction mode 8 × 8 is associated with the state of the flag 32 identifying the size of the transform block. This point will be described below. As for the 4 × 4 intra-prediction mode, as explained according to the first embodiment, any of the nine modes shown in FIG. 3 is selected in units of 4 × 4 pixel blocks. The pixels of the blocks (upper left, upper, upper right and left) around the block, already encoded, subjected to local decoding processing and stored in memory 16, are used to generate a predicted image.

On the other hand, in the 8 × 8 intra-prediction mode, any of the nine modes shown in FIG. 13 is selected in units of 8 × 8 pixels. As follows from the comparison with FIG. 3, the 8 × 8 intra-prediction mode is obtained by changing the 4 × 4 intra-prediction mode prediction method to be adapted to the 8 × 8 pixel block.

Intra8 × 8_pred_mode = 0: The adjacent pixel at the top is used as the predicted image as is.

Intra8 × 8_pred_mode = 1: The adjacent pixel on the left is used as the predicted image as it is.

Intra8 × 8_pre_mode = 2: The average of the neighboring eight pixels is used as the predicted image.

Intra8 × 8_pred_mode = 3: A weighted average is calculated for every two to three pixels from neighboring pixels and is used as a predicted image (corresponding to the edge 45 degrees to the right).

Intra8 × 8_pred_mode = 4: A weighted average is calculated for every two to three pixels from neighboring pixels and is used as a predicted image (corresponding to the edge 45 degrees to the left).

Intra8 × 8_pred_mode = 5: A weighted average is calculated for every two to three pixels from neighboring pixels and is used as a predicted image (corresponding to the edge 22.5 degrees to the left).

Intra8 × 8_pred_mode = 6: A weighted average is calculated for every two to three pixels from neighboring pixels and is used as a predicted image (corresponding to the edge 67.5 degrees to the left).

Intra8 × 8_pred_mode = 7: A weighted average is calculated for every two to three pixels from neighboring pixels and used as a predicted image (corresponding to the edge 22.5 degrees to the right).

Intra8 × 8_pred_mode = 8: A weighted average is calculated for every two to three pixels from neighboring pixels and is used as a predicted image (corresponding to the edge 112.5 degrees to the left).

When choosing a 4 × 4 intra-prediction mode, sixteen pieces of mode information are necessary for each macroblock. Thus, in order to reduce the amount of codes of the mode information itself, taking into account the fact that the mode information has a high correlation with the block adjacent to it, the prediction coding is based on the mode information on the neighboring block. Similarly, when choosing an 8 × 8 intra-prediction mode, taking into account the fact that the intra-prediction mode has a high correlation with its neighboring block, the prediction coding is based on the mode information on the neighboring block.

(b) 16 × 16 intra-prediction encoding mode

A 16 × 16 intra-prediction coding mode is a mode for predicting blocks of 16 × 16 pixels equivalent in size to a macroblock at a given time. Any of the four modes shown in FIG. 4 is selected in units of macroblocks. In the same way as in the 4 × 4 intra-prediction mode, block pixels (upper left, upper, and left) around a block already encoded, subjected to local decoding processing and stored in memory 16 are used to generate a predicted image. The types of modes are explained with reference to FIG. 4 according to the first embodiment. In encoding mode with intra-prediction of 16 × 16, the variable block size is always 4 × 4. However, sixteen DCs (DC components, average values) are collected in 4x4 block units. The conversion is applied in two stages, i.e. First, conversion in units of 4 × 4 blocks and conversion of the AC component remaining after removing DC components for each 4 × 4 block are performed.

The video encoder according to the second embodiment is characterized in that the intra-prediction / conversion / coding methods for the three color components are changed based on the common use identification flag of the intra-coding mode 33. This point will be described in detail below in section 2.

The spatial prediction unit 2 evaluates the intra-prediction mode based on the flag 33 identifying the common use of the intra-coding mode for the input signals of the three color components. The flag 33 identifying the general use of the intra-coding mode indicates that the intra-coding mode is entered for each of the three color components, or the same intra-coding mode is highlighted separately for all three components. The reason lies in the basic information described below.

In the 4: 4: 4 format, you can directly use RGB for color spaces other than Y, Cb and Cr, color spaces traditionally used for coding. In the color spaces Y, Cb, and Cr, the components, depending on the texture structure of the video signal, are removed from the Cb and Cr signals. It is highly likely that the optimal intra-coding method varies between the Y component and the two components Cb and Cr. (In fact, in the coding system for encoding a 4: 2: 0 AVC / H.264 format, for example, a high profile 4: 2: 0, the designs of the intra prediction modes used for component Y and components Cb and Cr are different). On the other hand, when coding is performed in RGB color spaces, and not in Y, Cb, and Cr color spaces, texture structure removal among color components is not performed, and the correlation between signal components in an identical space is high. Thus, this makes it possible to increase the coding efficiency due to the possibility of choosing a common intra-coding mode. This point depends on the definition of color spaces and, in addition, depends on the characteristics of the video signal, even if specific color spaces are used. It is desirable that the coding system itself can adaptively cope with such characteristics of video signals. Thus, in this embodiment, an intra-encoding mode common use identification flag 33 is provided in the encoding device so that the video can be flexibly encoded in a 4: 4: 4 format.

The spatial prediction unit 2 performs prediction processing for the respective color components in all intra prediction modes shown in FIGS. 3, 4 and 13, or a predetermined subset according to the state of the flag 33 identifying the common use of the intra-coding mode set as described above, and receives a difference signal prediction 4 using a subtractor 3. The prediction efficiency for the prediction difference signal 4 is estimated by decision block 5 regarding tionary encoding mode. The decision block 5 regarding the encoding mode selects an intra-prediction mode in which the optimal prediction efficiency for the object macroblock is obtained from the prediction processing performed by the spatial prediction unit 2. When choosing N × N intra-prediction, the decision block 5 regarding the encoding mode outputs an encoding mode with N × intra prediction N as the encoding mode 6. When the prediction mode is 4 × 4 intra-prediction, the adoption unit 5 decisions regarding the encoding mode sets the flag 32 identifying the size of the conversion block in the state "conversion to block size 4 × 4". When the prediction mode is 8 × 8 intra-prediction, the decision block 5 regarding the encoding mode sets the conversion block size identification flag 32 to the “conversion to 8 × 8 block size” state. As a method of determining the flag 32 of the identification of the size of the conversion unit, various methods are possible. In the encoding device according to the second embodiment, as the basic method for setting the block size in the conversion of the remainder obtained by N × N prediction, after the optimal N × N intra prediction mode is set by the decision block 5 regarding the encoding mode, the transform block size identification flag 32 is determined according to the value of N mode. For example, the size of the transform block is defined as an 8 × 8 pixel block when the 4 × 4 intra-prediction mode is used. Then, it is very likely that the spatial continuity of the prediction signal is violated in units of 4 × 4 blocks in the prediction difference signal 4 obtained from the prediction. If high frequency components are not generated. Thus, the effect of concentration of signal power due to conversion is reduced. If the size of the transform block is specified as a 4 × 4 pixel block according to the prediction mode, this problem does not occur.

When the decision block 5 regarding the encoding mode selects 16 × 16 intra prediction, the decision block 5 regarding the encoding mode outputs the 16 × 16 intra prediction mode as encoding mode 6. When encoding mode 6 is selected, weights 20 for each encoding mode can be taken into account, established by decision block 19 coding control.

The prediction differential signal 4 obtained by the encoding mode 6 is output to the conversion unit 8. The conversion unit 8 converts the input prediction difference signal into a conversion coefficient and outputs the conversion coefficient to quantization unit 9. Quantization unit 9 quantizes the introduced transformation coefficient based on quantization parameter 21 set coding control unit 19, and outputs a conversion coefficient to coding unit 11 with a variable word length as a quantized conversion factor 10.

When the size of the transform block is expressed in units of 4 × 4 blocks, the prediction difference signal 4 input to the transform block 8 is divided in units of 4 × 4 blocks, converted, and quantized by quantization block 9. When the size of the transform block is expressed in units of 8 × 8 blocks , the prediction difference signal 4 input to the transform block 8 is divided in units of 8 × 8 blocks, is converted, and quantized by the quantization block 9.

The quantized transform coefficient 10 is subjected to entropy coding by, for example, Huffman coding or arithmetic coding on a variable-length coding block 11. The quantized transform coefficient 10 is restored to the local decoding prediction differential signal 14 by the inverse quantization unit 12 and the inverse transform unit 13 in a block size based on the transform block size identification flag 32. The quantized transform coefficient 10 is added to the predicted image 7, which is generated based on the encoding mode 6, by the adder 18 to generate a local decoded image 15. The locally decoded image 15 is stored in the memory 16 for later use in intra-prediction processing. An enable filter control flag 24 indicating whether the enable filter is applied to the macroblock is also input to the variable-length encoding unit 11. (In the prediction processing performed by the spatial prediction unit 2, since the pixel data is stored in the memory 16 before being processed by the unlocking filter, the processing of the unlocking filter itself is not necessary for encoding processing. However, the unlocking filter acts according to the direction of the unlocking filter control flag 24 on the decoder side for obtaining the final decoded image.)

The intra-coding mode common use identification flag 33, the quantized transform coefficient 10, the coding mode 6, and the quantization parameter 21 entered into the variable-length coding unit 11 are ordered and formed as a bit stream in accordance with a predetermined rule (syntax) and are output to transmission buffer 17. The transmission buffer 17 smooths the bit stream according to the band of the transmission line to which the encoder is connected and the read speed of the recording medium and outputs the bit stream as video stream 22. UFER transmission 17 outputs feedback information to the encoding control unit 19 according to the state of accumulation of a bit stream in the transmission buffer 17 and then controls the amount of generated codes in encoding of video frames.

2. Decision processing regarding the intra-coding / prediction mode in the encoder

The intra-coding mode and decision processing regarding the intra-coding / prediction mode, which is a characteristic of the encoder according to the second embodiment, are described in detail below. This processing is carried out in units of macroblocks in which three color components are placed. The processing is carried out mainly by the spatial prediction unit 2 and the decision unit 5 regarding the encoding mode in the encoder shown in FIG. 11. A logical flowchart showing the progress of processing is shown in FIG. Image data of the three color components forming the block are indicated below C0, C1 and C2.

First of all, the encoding mode decision block 5 receives the intra-encoding mode common use identification flag 33 and makes a decision based on the value of the intra-encoding mode common use identification flag 33 whether the intra-coding mode common to C0, C1 and C2 is used (step S20 in FIG. fourteen). When using the general intra-prediction mode, the decision block 5 regarding the encoding mode proceeds to step S21 and subsequent steps. In case of not using the general intra-prediction mode, the decision block 5 regarding the encoding mode proceeds to step S22 and subsequent steps.

When the intra-coding mode is used together for C0, C1, and C2, the decision block 5 regarding the coding mode notifies the spatial prediction block 2 of all intra prediction modes (N × N intra prediction and 16 × 16 intra prediction) that can be selected. The spatial prediction unit 2 evaluates the prediction efficiency for all prediction modes and selects the optimal intra-coding mode and intra-prediction mode for all components (step S21).

On the other hand, when the optimal intra-coding modes are selected for C0, C1, and C2, respectively, the decision block 5 regarding the coding mode notifies the spatial prediction block 2 of all intra-prediction modes (N × N intra-prediction and 16 × 16 intra-prediction) that can be selected for the components Ci (i <= 0 <3). The spatial prediction unit 2 estimates the prediction efficiency for all intra-prediction modes and selects the optimal 4 × 4 intra-prediction mode in the components Ci (i <= 0 <3) (step S23).

When the spatial prediction unit 2 selects the 4 × 4 intra-prediction mode as the mode providing optimal prediction efficiency in the above steps S21 and S23, the transform block size identification flag 32 is set to the “transform to 4 × 4 block size” state. When the spatial prediction unit 2 selects an 8 × 8 intra-prediction mode as the mode providing optimal prediction efficiency, the transform block size identification flag 32 is set in the “transform to 8 × 8 block size” state.

As criteria for evaluating the prediction efficiency for the prediction mode carried out in the spatial prediction unit 2, for example, the velocity / distortion value defined as Jm = Dm + λRm (λ: positive number) can be used. Dm is the value of coding distortion or prediction error in the case of using the intra prediction mode m. The coding distortion is obtained using the intra-prediction mode m to calculate the prediction error and decode the video signal from the result obtained by converting and quantizing the prediction error to measure the error with respect to the signal before encoding. The magnitude of the prediction error is obtained by calculating the difference between the predicted image and the signal before encoding in the case when the intra prediction mode m is applied, and quantizing the difference level. For example, Absolute Distance Sum (SAD) is used. Rm is the amount of generated codes when the intra prediction mode m is applied. In other words, Jm is a value specifying a compromise between the amount of codes and the degree of deterioration when the intra-prediction mode m is applied. The intra-prediction mode m, giving the minimum Jm, is the optimal solution.

When the encoder performs the processing in step S21 and subsequent steps, one piece of intra-coding mode information is allocated to the macroblock including three color components. On the other hand, when the encoder performs the processing in step S22 and subsequent steps, the information of the intra-coding mode is allocated to the color components (three in total), respectively. Thus, since the fragments of intra-prediction mode information allocated to the macroblock are different, it is necessary to multiplex the identification flag of the general use of the intra-encoding mode 23 into the bitstream and allow the decoder to recognize whether the encoder has completed the processing steps in step S21 and subsequent steps or has completed the processing steps in step S23 and subsequent steps. An array of data of such a bitstream is shown in FIG.

15, the intra-coding modes 0 (34a), 1 (34b), and 2 (34c) multiplexed into the bitstream at the macroblock level indicate coding modes 6 for components C0, C1, and C2, respectively. When the intra-coding mode is an N × N intra-prediction coding mode, the transform block size identification flag 32 and the intra-prediction mode information are multiplexed into the bitstream. On the other hand, when the intra-coding mode is a 16 × 16 prediction coding mode, the intra-prediction mode information is encoded as part of the intra-coding mode information. Information about the transform block size identification flag 32 and the intra-prediction mode are not multiplexed into the bitstream. When the intra-coding mode common use flag 33 is set to “common to C0, C1, and C2,” the intra-coding modes 1 (34b) and 2 (34c), the conversion unit size identification flags 1 (32b) and 2 (32c), and the intra-prediction modes 1 ( 35b) and 2 (35c) are not multiplexed into the bitstream (the part circled by the dashed circle in FIG. 15 indicates a branch of the bitstream). In this case, the intra-coding mode 0 (34a), the flag for identifying the size of the transform block 0 (32a), and the intra-prediction mode 0 (35a) play the role of coding information common to all color components. In the example shown in FIG. 15, the intra-encoding mode common use identification flag 33 is multiplexed as bitstream data at a level higher than the macroblock, for example at the slice, image or sequence level. In particular, when the intra-encoding mode common use identification flag 33 is used as an example described according to the second embodiment, since the color spaces often do not change throughout the sequence, the goal can be achieved by multiplexing the intra-encoding mode common use identification flag 33 to the sequence level.

According to a second embodiment, the intra-encoding mode common use identification flag 33 is used to indicate “common to all components”. However, the flag 33 identifying the general use of the intra-coding mode can be used to indicate, according to the definition of the color spaces of the input video signal 1, for example, "common to specific two components, for example, C1 and C2" (in the case of Y, Cb and Cr, etc. it is possible to use the intra-prediction mode together for Cb and Cr). When the total usage range of the common use identification flag 33 of the intra-coding mode is limited only to the intra-coding mode and the N × N intra-prediction mode is used, the transform block size and the N × N prediction mode can be independently selected for each of the color components (Fig. 16). Using the syntax structure shown in FIG. 16, it is possible to change the prediction method for each of the color components and increase the prediction efficiency using information of a joint video encoding mode with a complex pattern, which requires N × N prediction.

If the information on the common mode identification flag 33 of the intra-coding mode is somehow known in advance by the encoder and the decoder, the information on the common mode identification flag 33 of the intra-coding mode does not need to be transmitted in the video bitstream. In this case, for example, in the encoder, an intra-coding mode common use identification flag 33 may be generated to implement stable coding for a certain value or may be transmitted individually from the video bitstream.

3. Description of the decoder

The decoder shown in Fig. 12 receives a video stream 22 corresponding to the array shown in Fig. 15, output from the encoder shown in Fig. 11, performs decoding processing in units of macroblocks in which the three color components have the same size (format 4: 4: 4), and restores the corresponding video frames.

First of all, the variable-length decoding unit 25 to which stream 22 arrives decodes stream 22 in accordance with a predetermined rule (syntax) and retrieves information including an intra-encoding mode common use identification flag 33, quantized transform coefficient 10, mode encoding 6 and a quantization parameter 21. The quantized transform coefficient 10 is input to the inverse quantization unit 12 together with the quantization parameter 21, and inverse quantization processing is performed. Then, the output signal of the inverse quantization unit 12 is input to the inverse transform unit 13 and restored to the local prediction differential prediction signal 14. On the other hand, the encoding mode 6 and the intra-encoding mode common use identification flag 33 are input to the spatial prediction unit 2. The spatial prediction unit 2 receives the predicted image 7 in accordance with these pieces of information. The specific procedure for obtaining the predicted image 7 will be described below. The local decoding prediction difference signal 14 and the prediction image 7 are summed by the adder 18 to obtain an internally decoded image 15 (this is completely the same signal as the locally decoded image 15 in the encoder). The internally decoded image 15 is written back to memory 16 for use in further intra-prediction of the macroblock. Three memory blocks are prepared for the respective color components. The unlocking filter 26 is instructed to act on the internally decoded image 15 based on the indication of the unlocking filter control flag 24, decoded by the variable-length decoding unit 25, to obtain the final decoded image 27.

4. Processing of intra prediction decoding in a decoder

The processing of generating an intra-predicted image, which is a characteristic of a decoder according to a second embodiment, will be described in detail below. This processing is carried out in units of macroblocks in which three color components are placed. The processing is carried out mainly by the variable-length decoding unit 25 and the spatial prediction unit 2 of the decoder shown in FIG. 12. A logical flowchart showing the progress of processing is shown in FIG.

Steps S25-S38 in the logic block diagram shown in FIG. 17 are carried out by the variable-length decoding unit 25. The video stream 22 arriving at the variable-length decoding unit 25 is consistent with the data array shown in FIG. In step S25, the data of the intra-coding mode 0 (34a) (corresponding to the component C0) shown in FIG. 15 are primarily decoded. As a result, when the intra-coding mode 0 (34a) is “N × N intra-prediction”, the variable-length decoding unit 25 decodes the transform block size identification flag 0 (32a) and the intra-prediction mode 0 (35a) (steps S26 and S27). Then, when it is decided that the intra-coding / prediction mode information is common to all color components, based on the state of the intra-coding mode common use identification flag 33, the variable-length decoding unit 25 sets the intra-coding mode 0 (34a), the conversion block size identification flag 0 (32a) and intra prediction mode 0 (35a) as encoding information used for components C1 and C2 (steps S29 and S30). Processing in units of macroblocks is shown in FIG. The intra-coding mode common use identification flag 33 used to make the decision in step S29 is read from bitstream 22 by decoding unit 25 with a variable word length at a level equal to or higher than the slice level, before variable-length decoding unit 25 words will go to the BEGINNING of the process shown in FIG.

When a decision is made in step S29 shown in FIG. 17 that the intra-coding / prediction mode information is encoded for each of the color components, in the next steps S31-S38, the variable-length decoding unit 25 decodes the intra-coding / prediction mode information for the components C1 and C2. The coding modes 6 for the respective color components are set in the processing steps and output to the spatial prediction unit 2 to obtain intra-predicted images for the corresponding color components in accordance with steps S39-S41. The process for obtaining intra-predicted images is consistent with the procedures shown in FIGS. 3, 4 and 13 and is identical to the processing performed by the encoder shown in FIG. 11.

As described above, if the information on the common mode identification flag 33 of the intra-coding mode is previously known to the encoder and the decoder, the decoder can decode, for example, with a pre-fixed value, instead of analyzing the value of the common mode identification flag 33 of the intra-coding mode from the bitstream video signal, or can transmit information separately from the bitstream of the video signal.

In the 4: 2: 0 format, adopted in the traditional video coding standard, the definition of color spaces is fixed to Y, Cb and Cr. In the 4: 4: 4 format, the definition of color spaces is not limited to Y, Cb, and Cr, but various color spaces can be used. By generating coding information on the intramacroblock, as shown in FIGS. 15 and 16, it is possible to perform optimal coding processing according to the determination of the color spaces of the input video signal 1 and the characteristics of the video signal. In addition, it is possible to uniquely interpret the bitstream resulting from such encoding processing to implement decoding and video processing.

Third Embodiment

According to a third embodiment, another example of the structures of the encoder shown in FIG. 11 and the decoder shown in FIG. 12 is described. As in the first embodiment, the characteristics of the invention are communicated to the encoder and decoder based on the encoding system adopted in MPEG-4 AVC (ISO / IEC 14496-10) / ITU-TH.264, which is Non-Patent Document 1. The video encoder according to the third embodiment differs from the encoder according to the second embodiment described with reference to FIG. 11 only with respect to the operations of the variable-length encoding unit 11. The video decoder according to the third embodiment is different from the decoder according to the second embodiment described with reference to FIG. 12 only with respect to the operations of the variable-length decoding unit 25. Otherwise, the video encoder and video decoder perform the same operations as the corresponding devices according to the second embodiment. Only differences will be explained.

1. The procedure for encoding information intra prediction mode in the encoder

In the encoder according to the second embodiment, the variable-length encoding unit 11 indicates an array of data on a bitstream for N × N intra-prediction mode information, but does not specifically indicate an information encoding procedure. In this embodiment, a specific method for implementing an encoding procedure is described. This embodiment is characterized in that, in particular, entropy coding, which uses the correlation of values between color components, is carried out for N × N intra-prediction modes obtained in the corresponding color components, taking into account the case when the values of N × N intra-prediction modes are high correlation between color components.

The following explanation is based on the condition that an array of bit streams in the format shown in FIG. 16 has been received. To simplify the explanation, the value of the identification flag 33 of the common use of the intra-coding mode is set to be shared for C0, C1 and C2, the intra-coding mode is an N × N intra-prediction mode, and the size of the 0-2 transform blocks is 4 × 4 blocks. In this case, all intra-prediction modes 0-2 (35a-35c) are 4 × 4 intra-prediction modes. 18 to 20, the current macroblock to be encoded is designated X. The macroblock to the left of the current macroblock is macroblock A, and the macroblock to the right and up from the current macroblock is macroblock B.

18-20 are used as circuits for explaining the encoding procedure for the respective color components C0, C1, and C2. The logical flowcharts of the procedure are shown in FIGS. 21 and 22.

The state of component C0 of the macroblock X is shown in FIG. The 4 × 4 block to be encoded is designated as block X, and the 4 × 4 blocks to the left and above block X are designated as block A and block B, respectively. There are two cases according to the position of the 4 × 4 block to be encoded. In case 1, the 4 × 4 blocks on the left and above the 4 × 4 block to be encoded are located outside the current macroblock X or belong to macroblock A or macroblock B. In the case of 2 4 × 4 blocks on the left and above the 4 × 4 block to be encoded inside the current macroblock X or belong to macroblock X. In both cases, one 4 × 4 intra-prediction mode is allocated for each X 4 × 4 block in macro-block X. This 4 × 4 intra-prediction mode is indicated by CurrIntraPredMode. The 4 × 4 block A intra prediction mode is indicated by IntraPredModeA, and the 4 × 4 block B intra prediction mode is indicated by IntraPredModeB. IntraPredModeA and IntraPredModeB are information already encoded at the time of block X coding. When coding the 4x4 intra prediction mode of a certain block X, variable-length coding block 11 first extracts these parameters (step S50 in FIG. 21).

The variable-length encoding unit 11 sets the predicted value predCurrIntraPredMode for the CurrIntraPredMode of block X according to the following equation (step S51).

.

.

The variable-length encoding unit 11 encodes the CurrIntraPredMode of component C0. Moreover, if CurrIntraPredMode = predCurrIntraPredMode, then the encoding unit 11 with a word length encodes a 1-bit flag (prev_intra_pred_mode_flag) indicating that CurrIntraPredMode is equal to the predicted value. If CurrIntraPredMode! = PredCurrIntraPredMode, then the variable-length encoding unit 11 compares CurrIntraPredMode and predCurrIntraPredMode. When the CurrIntraPredMode is smaller, the variable-length encoding unit 11 encodes the CurrIntraPredMode as is. When the CurrIntraPredMode is larger, the variable-length encoding unit 11 encodes CurrIntraPredMode-1 (step S52).

The encoding procedure for component C1 will be described with reference to FIG. First, in the same manner as in the encoding procedure for the component C0, the variable-length encoding unit 11 sets the near-encoding parameters, for example IntraPredModeA and IntraPredModeB, according to the position of the X block (step S53).

The variable-length encoding unit 11 sets the predicted candidate value 1 predCurrIntraPredMode1 for the CurrIntraPredMode of block X according to the following equation (step S54).

.

.

If prev_intra_pred_mode_flag = 1 in component C0, then the variable-length encoding unit 11 takes this predCurrIntraPredMode1 as predCurrIntraPredMode in unit X of component C1 as is. The reason is as follows. Taking prev_intra_pred_mode_flag = 1 at the identical position of the block of component C0 means that the correlation between prediction modes is high in the near region of the image in component C0. In this case, when the signal is RGB or the like, from which the correlation of the texture structures between the component C0 and component C1 is not completely removed, it is very likely that in component C1 the correlation is high between regions near the image, as in the component C0. Thus, the variable-length coding unit 11 makes a decision that the predicted value of the component C1 is independent of the intra-prediction mode of the 4 × 4 C0 component.

On the other hand, in component C0, when prev_intra_pred_mode_flag = 0 or rem_intra_pred_mode is encoded (step S55), the variable-length encoding unit 11 sets CurrIntraPredMode of component C0 as the predicted candidate value 2 (step S56). It means that

.

.

This is set as the predicted candidate value due to the following basic information. Encoding rem_intra_pred_mode in component C0 means that the correlation of intra-prediction between regions near the image is low in component C0. In this case, it is assumed that the correlation between regions near the image is also low in component C1. It is likely that intra-prediction modes at the identical block position in different color components give better predicted values.

Finally, the variable-length encoding unit 11 sets the predicted value of CurrIntraPredMode in unit X of component C1 as the value of one of predCurrIntraPredMode1 and predCurrIntraPredMode2 (step S57). Which value is used is additionally encoded with a 1-bit flag (pred_flag). However, pred_flag is only encoded when CurrIntraPredMode matches the predicted value. When CurrIntraPredMode does not match the predicted value (when rem_intra_pred_mode is encoded), predCurrIntraPredMode1 is used as the predicted value.

The above procedure is presented in the form of the following expressions.

// Use predicted candidate value 1

// Use predicted candidate value 2

As a result, prev_intra_pred_mode_flag, pred_flag, and rem_intra_pred_mode are encoded as encoded data (step S58).

The encoding procedure for component C2 will be described with reference to FIG. First of all, in the same manner as in the encoding procedure for the components C0 and C1, the variable-length encoding unit 11 sets the near-encoding parameters, for example IntraPredModeA and IntraPredModeB, according to the position of the X block (step S59).

The variable-length encoding unit 11 sets the predicted candidate value 1 predCurrIntraPredMode1 for the CurrIntraPredMode of block X according to the following equation (step S60).

.

.

If prev_intra_pred_mode_flag = 1 in both components C0 and C1, the variable-length encoding unit 11 takes this predCurrIntraPredMode1 as predCurrIntraPredMode in unit X of component C1 as is. The reason is as follows. The adoption of prev_intra_pred_mode_flag = 1 at the identical position of the block of components C0 and C1 means that the correlation between the prediction modes is high in the near region of the image in components C0 and C1. In this case, when the signal is RGB or the like, from which the correlation of texture structures between component C0, component C1, and component C2 is not completely removed, it is very likely that in component C2 the correlation is also high between regions near the image, as and in components C0 and C1. Thus, the variable-length coding unit 11 decides that the predicted value of the component C2 is independent of the intra-prediction mode of the 4 × 4 components C0 and C1.

On the other hand, in components C0 or C1, when prev_intra_pred_mode_flag = 0 or rem_intra_pred_mode is encoded (step S61), the encoding unit 11 with a word length sets the CurrIntraPredMode of components C0 or C1 as the predicted candidate value 2 (step S62). It means that

This is set as the predicted candidate value due to the following basic information. Encoding rem_intra_pred_mode in components C0 or C1 means that the correlation of intra-prediction between areas near the image in components C0 or C1. In this case, it is assumed that the correlation between regions near the image is also low in the C2 component. It is likely that intra-prediction modes at the identical block position in different color components give better predicted values. According to this idea, when rem_intra_pred_mode is encoded in both components C0 and C1, the current intra-prediction modes C0 and C1 can be candidates for the predicted value. However, the current intra-prediction mode of component C1 is taken as the predicted value. The fact is that when entering YUV color spaces, it is very likely that C0 is processed as a luminance signal and C1 / C2 is processed as a color-difference signal, in which case it is considered that C1 is closer to the prediction mode C2 than C0. When entering RGB color spaces, it is not so important whether C0 or C1 is selected. In general, it is considered beneficial to take the C1 component as the predicted value (the C2 component can be taken as the predicted value depending on the design).

Finally, the variable-length encoding unit 11 sets the predicted value of CurrIntraPredMode in unit X of component C2 as the value of one of predCurrIntraPredMode1 and predCurrIntraPredMode2 (step S63). Which value is used is additionally encoded with a 1-bit flag (pred_flag).

The above procedure is presented in the form of the following expressions.

// Use predicted candidate value 1

// Use predicted candidate value 2

As a result, prev_intra_pred_mode_flag, pred_flag, and rem_intra_pred_mode are encoded as encoded data (step S64).

In the same way, the above-described coding procedure for the 8 × 8 intra-prediction mode can be determined. Coding the N × N intra-prediction mode in such a procedure, one can use the correlation between the N × N intra-prediction mode and the prediction mode selected in other color components, and it is possible to reduce the code size of the prediction mode itself and increase the coding efficiency.

The difference between FIG. 21 and FIG. 22 is whether the encoding processing for the intra-prediction mode is carried out in units of MB individually for each of the color components or together. In the case of FIG. 21, a variable-length encoding unit 11 encodes the corresponding color components in units of 4x4 blocks and arranges sixteen block patterns assembled in the bitstream (step S65). In the case of FIG. 22, the variable-length encoding unit 11 together encodes sixteen 4x4 blocks of the respective color components and arranges the blocks in the bitstream for each of the color components (steps S66, S67, and S68).

In the above procedure, pred_flag is information that is effective only when prev_intra_pred_mode_flag is 1. However, pred_flag can also be effective when prev_intra_pred_mode_flag is 0. In other words, for example, using component C1, encoding can be performed in the procedure described below.

// Use predicted candidate value 1

// Use predicted candidate value 2

According to this method, when rem_intra_pred_mode is encoded in intra-prediction mode in a block at the identical position of component C0, pred_flag is always encoded. However, even when prev_intra_pred_mode_flag = 0, a more accurate predicted value can be used. Thus, an increase in coding efficiency can be expected. In addition, pred_flag can be encoded regardless of whether rem_intra_pred_mode is encoded in intra-prediction mode in a block at the identical position of component C0. In this case, the intra-prediction mode of the component C0 is always used as the predicted candidate value. Below are the expressions corresponding to this case.

// Use predicted candidate value 1

// Use predicted candidate value 2

The pred_flag flag can be set in units of macroblocks or sequences, and not in units of 4x4 blocks. When setting pred_flag in units of macroblocks, the predicted candidate value 1 or the predicted candidate value 2 is shared among all 4x4 blocks in the macroblock. Thus, it is possible to further reduce overhead information transmitted as pred_flag. Since it is set according to the definition of the input color spaces, whether the predicted candidate value 1 or the predicted candidate value 2 is used, pred_flag can be set in sequence units. In this case, there is also no need to transmit pred_flag for each macroblock. In this way, overhead information can be further reduced.

2. The procedure for decoding information intra prediction mode in the decoder

In the decoder according to the second embodiment, the variable-length encoding unit 25 indicates an array of data on the bitstream for information on the N × N intra-prediction mode, but does not specifically indicate an information decoding procedure. According to a third embodiment, a specific method for implementing a decoding procedure is described. The third embodiment is characterized in that, in particular, a bitstream that undergoes entropy encoding that uses the correlation of values between color components is decoded for N × N intra-prediction modes obtained in the respective color components, taking into account the case when the values of intra-prediction modes N × N have a high correlation between color components.

The following explanation is based on the condition that an array of bit streams in the format shown in FIG. 16 has been received. To limit the explanation to the decoding procedure for the intra-prediction mode, the value of the flag 33 identifying the common use of the intra-coding mode in the bitstream is set to be shared for C0, C1 and C2. The N × N intra-prediction mode is referred to as the intra-coding mode. A 4 × 4 block is designated as the dimensions of the 0-2 transform block. In this case, all intra-prediction modes 0-2 (35a-35c) are 4 × 4 intra-prediction modes. As in the encoder, the ratio shown in FIGS. 18-20 is used for the decoder. In the decoder, the current macroblock to be decoded is denoted by X. The macroblock to the left of the current macroblock is macroblock A, and the macroblock to the right and up from the current macroblock is macroblock B. A logical block diagram of the decoding procedure is shown in Fig. 23. In Fig.23, the steps indicated by the same positions as shown in Fig.21 and 22 indicate that the processing is identical to the processing when executing the encoder.

The state of component C0 of the macroblock X is shown in FIG. There are two cases according to the position of the 4 × 4 block to be decoded. In case 1, the 4 × 4 blocks on the left and above the 4 × 4 block to be decoded are located outside the current macroblock X or belong to macroblock A and macroblock B. In the case of 2 4 × 4 blocks on the left and above the 4 × 4 block to be decoded are inside the current macroblock X or belong to the macroblock X. The 4 × 4 block to be decoded is called block X, and the 4 × 4 blocks on the left and above block X are called block A and block B, respectively. In both cases, one 4 × 4 intra-prediction mode is allocated to each X 4 × 4 block in macroblock X. This 4 × 4 intra-prediction mode is CurrIntraPredMode. The 4 × 4 block A intra prediction mode is IntraPredModeA, and the 4 × 4 block B intra prediction mode is IntraPredModeB. Both IntraPredModeA and IntraPredModeB are information already decoded at the time of block X coding. When decoding the 4 × 4 intra prediction mode of a certain block X, first of all, the variable-length decoding block 25 extracts these parameters (step S50).

The variable-length decoding unit 25 sets the predicted value predCurrIntraPredMode for the CurrIntraPredMode of block X according to the following equation (step S51)

.

.

The variable-length decoding unit 25 decodes a 1-bit flag (prev_intra_pred_mode_flag) indicating CurrIntraPredMode = predCurrIntraPredMode. Prev_intra_pred_mode_flag = 1 means that CurrIntraPredMode = predCurrIntraPredMode. Otherwise, the variable-length decoding unit 25 decodes information about (prev_intra_pred_mode_flag = 0) and rem_intra_pred_mode from the bitstream. When, as a result of comparing rem_intra_pred_mode and predCurrIntraPredMode, it turns out that rem_intra_pred_mode is smaller, CurrIntraPredMode = rem_intra_pred_mode is set. When the CurrIntraPredMode is larger, CurrIntraPredMode = rem_intra_pred_mode + 1 is set (step S65).

These procedures in general can be represented as follows.

The decoding procedure for component C1 will be described with reference to FIG. First, in the same manner as in the decoding procedure for the component C0, the variable-length decoding unit 25 sets the near-coding parameters, for example IntraPredModeA and IntraPredModeB, according to the position of the X block (step S53).

The variable word length decoding unit 25 sets the predicted candidate value 1 predCurrIntraPredMode1 for the CurrIntraPredMode of block X according to the following equation (step S54)

.

.

If prev_intra_pred_mode_flag = 1 in component C0, this predCurrIntraPredMode1 is accepted as predCurrIntraPredMode in block X of component C1 as is. This happens for the same reason as in the case of the encoder.

On the other hand, when prev_intra_pred_mode_flag = 0 in component C0 or when rem_intra_pred_mode is decoded (step S55), the variable word length decoding unit 25 sets the CurrIntraPredMode of component C0 as the predicted candidate value 2 (step S56). It means that

.

.

This is set as the predicted candidate value due to the same basic information as in the case of the encoder.

Finally, the variable-length decoding unit 25 sets the predicted value of CurrIntraPredMode in the component block X C1 as the value of one of predCurrIntraPredMode1 and predCurrIntraPredMode2 (step S57). Which value is used is additionally decoded with a 1-bit flag (pred_flag). However, pred_flag is decoded only when CurrIntraPredMode matches the predicted value. When CurrIntraPredMode does not match the predicted value (when rem_intra_pred_mode is decoded), predCurrIntraPredMode1 is used as the predicted value.

After setting the predicted candidate value 1, the predicted candidate value 2, prev_intra_pred_mode_flag, pred_flag and rem_intra_pred_mode by the following procedure, the variable-length word decoding unit 25 decodes CurrIntraPredMode (step S66).

// In this case, pred_flag is not included in the bitstream.

The decoding procedure for component C2 will be described with reference to FIG. First, in the same manner as in the encoding procedure for components C0 and C1, the variable-length decoding unit 25 sets the near-coding parameters, for example IntraPredModeA and IntraPredModeB, according to the position of the X block (step S59).

The variable word length decoding unit 25 sets the predicted candidate value 1 predCurrIntraPredMode1 for the CurrIntraPredMode of block X according to the following equation (step S60).

predCurrIntraPredMode1 = Min (IntraPredModeA, IntraPredModeB).

If prev_intra_pred_mode_flag = 1 in both components C0 and C1, this predCurrIntraPredMode1 is accepted as predCurrIntraPredMode in block X of component C1 as is. This happens for the same reason as in the case of the encoder.

On the other hand, when prev_intra_pred_mode_flag = 0 in components C0 or C1, or when rem_intra_pred_mode is decoded (step S61), the variable word length decoding unit 25 sets the CurrIntraPredMode of components C0 or C1 as the predicted candidate value 2 (step S62).

It means that

This is set as the predicted candidate value due to the same basic information as in the case of the encoder.

Finally, the variable-length decoding unit 25 sets the predicted value of CurrIntraPredMode in block X of component C2 as the value of one of predCurrIntraPredMode1 and predCurrIntraPredMode2 (step S63). Which value is used is additionally decoded with a 1-bit flag (pred_flag). However, pred_flag is only decoded when CurrIntraPredMode matches the predicted value. When CurrIntraPredMode does not match the predicted value (when rem_intra_pred_mode is decoded), predCurrIntraPredMode1 is used as the predicted value.

After setting the predicted candidate value 1, the predicted candidate value 2, prev_intra_pred_mode_flag, pred_flag and rem_intra_pred_mode by the following procedure, the variable-length word decoding unit 25 decodes CurrIntraPredMode (step S71).

// In this case, pred_flag is not included in the bitstream.

In the same way, the above-described 8 × 8 intra-prediction mode decoding procedure can be determined. By decoding the N × N intra-prediction mode in such a procedure, it is possible to reduce the code size of the prediction mode itself and decode the bitstream with increased coding efficiency using the correlation between the N × N intra-prediction mode and the prediction modes selected in other color components.

In the above procedure, pred_flag is information decoded only when prev_intra_pred_mode_flag is 1. However, pred_flag can also be decoded when prev_intra_pred_mode_flag is 0.

In other words, for example, using component C1, encoding can be performed in the procedure described below.

The effect of this method is the same as described in the encoding procedure on the corresponding side of the encoder. In addition, pred_flag can be decoded regardless of whether rem_intra_pred_mode is decoded in intra-prediction mode in a block at the identical position of component C0. In this case, the intra-prediction mode of the component C0 is always used as the predicted candidate value.

Below are the expressions corresponding to this case.

As described in the explanation of the encoder, pred_flag can be included in the bitstream in units of macroblocks or sequences, rather than in units of 4x4 blocks. When setting pred_flag in units of macroblocks, the predicted candidate value 1 or the predicted candidate value 2 is shared among all 4x4 blocks in the macroblock. Thus, the pred_flag overhead to be decoded is reduced. Since it is established, according to the definition of the input color spaces, whether the predicted candidate value 1 or the predicted candidate value 2 is used, pred_flag can be set in sequence units. In this case, there is also no need to transmit pred_flag for each macroblock. Thus, overhead information is further reduced.

Fourth Embodiment

Consider a bitstream in the format shown in FIG. 16 according to a second embodiment. In the explanation of the second embodiment, when the intra-coding mode indicates “N × N intra-prediction”, the intra-prediction modes of the respective color components C0, C1 and C2 are recognized as the 4 × 4 intra-prediction mode or the 8 × 8 intra-prediction mode according to the value of the conversion unit size identification flags 0-2 (32a-32c). According to a fourth embodiment shown in FIG. 24, this bitstream array is changed to transmit, for components C1 and C2, intra-prediction mode indication flags 1 and 2 (36a and 36b) at the sequence level. The intra-prediction mode indication flag is effective when the N × N intra-prediction mode is selected in the intra-coding mode, and the transform block size identification flag indicates a 4 × 4 transform, i.e. in the case of intra prediction mode 4 × 4. The flag indicating the intra-prediction mode allows you to switch between the following two states according to this value.

State 1: for component C1 or C2, the 4 × 4 intra-prediction mode to be used is individually selected from the nine modes shown in FIG. 3 and encoded.

State 2: for component C1 or C2, the 4 × 4 intra-prediction mode is limited to DC prediction, i.e. intra4x4_pred_mode = 2 according to FIG. 3, and intra prediction mode information is not encoded.

For example, when encoding is performed in color spaces, for example, Y, Cb, and Cr, and in the case of a high-resolution video signal, for example HDTV or even higher-resolution video signals, a 4x4 block corresponds to an extremely small image area. In this case, it may be more efficient to record the information of the prediction mode on one piece of information and not transmit information of the prediction mode that forms the overhead than to give room for a component to choose from nine prediction modes, for example, components Cb and Cr that do not directly support the structure texture image. Due to the implementation of such an array of bit streams, it is possible to carry out optimal coding corresponding to the characteristics of the input color spaces and the characteristics of the video signal.

A decoder that receives the bitstream in the format shown in FIG. 24 decodes the intra-prediction mode indication flags (36a and 36b) in the variable-length decoding unit 25 and distinguishes whether the bitstream is encoded in state 1 or state 2, according to flag values Indication of intra-prediction mode. The decoder then makes a decision for component C1 or C2 whether the 4x4 intra prediction mode is decoded from the bitstream or DC prediction, i.e. intra4 × 4_pred_mode = 2 in FIG. 3 is applied continuously.

According to the fourth embodiment, in state 2, for components C1 or C2, the 4x4 intra prediction mode is limited to intra4x4_pred_mode = 2. However, only the information of the prediction mode needs to be fixed on the value of one, or it can be other prediction modes. State 2 can be set to use for component C1 or C2, the same 4 × 4 intra-prediction mode as for C0. In this case, since there is no need to code the 4 × 4 intra-prediction mode for component C1 or C2, overhead bits can be reduced.

Fifth Embodiment

According to a fifth embodiment, another example of the structures of the encoder shown in FIG. 11 and the decoder shown in FIG. 12 is described. As in other embodiments, the characteristics of the invention are communicated to the encoder and decoder according to the fifth embodiment based on the encoding system adopted in MPEG-4 AVC (ISO / IEC 14496-10) /ITU-TH.264, which is Non-Patent Document 1. The video encoder according to the fifth embodiment is different from the encoder shown in FIG. 11, the operation of which is explained according to the second and third embodiments, only with respect to the operations of the variable-length encoding unit 11. The video decoder according to the fifth embodiment is different from the decoder shown in FIG. 12, the operation of which is explained according to the second and third embodiments, only with respect to the operations of the variable-length decoding unit 25. Otherwise, the video encoder and video decoder perform the same operations as described in the second and third embodiments. Only differences will be explained.

1. The procedure for encoding information intra prediction mode in the encoder

In the encoder according to the third embodiment, a specific method for encoding N × N intra-prediction mode information in a bit stream in the format shown in FIG. 16 with a variable-length encoding unit 11 is described. According to a fifth embodiment, another specific method for implementing an encoding procedure is described. The fifth embodiment is characterized in that, in particular, taking into account the fact that the value of the N × N intra-prediction mode reflects the texture structure serving as the image template, a method for implementing adaptive prediction in the near pixel region in the same color component is provided. The following explanation is based on the condition that an array of bit streams in the format shown in FIG. 16 has been received. According to a fifth embodiment, N × N intra-prediction mode information for the respective components C0, C1, and C2 is independently encoded for each of the color components. The encoding method for component C0 also applies to C1 and C2. To simplify the explanation, only the encoding method for the C0 component will be explained. The value of the flag 33 identifying the common use of the intra-coding mode is set to use the intra-coding mode together for C0, C1 and C2. The intra-coding mode is the N × N intra-prediction mode, and the 0-2 (32a-32c) transform block size identification flags are the 4 × 4 block. In this case, all intra-prediction modes 0-2 (35a-35c) are 4 × 4 intra-prediction modes. 18, FIG. 18 is used as a diagram explaining a coding procedure for N × N intra-prediction mode information for a component C0. In Fig. 18, the current block to be encoded is denoted by X. The macroblock to the left of the current block is macroblock A, and the macroblock to the right and above the current macroblock is macroblock B. A logical block diagram of the encoding procedure is shown in Fig. 25.

According to a third embodiment, the smaller of the IntraPredModeA and IntraPredModeB values is uniquely allocated as the predicted predCurrIntraPredMode value for the 4x4 CurrIntraPredMode intra prediction mode allocated to 4x4 X blocks, respectively, shown in FIG. 18. This method is also adopted in this AVC / H.264 standard. As the value of the N × N intra-prediction mode increases, the system for generating the predicted image enters a more complex mode involving pixel interpolation, which takes into account the directivity of the image template. The fact is that a small value is allocated to the mode with high adaptability to the general image template. At a low bit rate, since an increase in the volume of prediction mode codes has a greater influence on the choice of mode than an increase in distortion, this system is useful for the coding efficiency of the encoder as a whole. On the contrary, when the bit rate is relatively high, since an increase in distortion affects the mode selection to a greater extent than an increase in the volume of prediction mode codes, it is not always true that the smaller of the IntraPredModeA and IntraPredModeB values is optimal. Based on this observation, according to the fifth embodiment, the accuracy of the predicted value is improved by adapting the specification of this predicted value according to the states of IntraPredModeA and IntraPredModeB, which is explained below. In this procedure, as a value at which CurrIntraPredMode can be evaluated most efficiently with respect to the image template, the variable-length encoding unit 11 sets predCurrIntraPredMode based on the state of IntraPredModeA and IntraPredModeB (steps S73, S74 and S75).

(1) When both IntraPredModeA and IntraPredModeB are in the range of 0 to 2, MIN (IntraPredModeA, IntraPredModeB) is set to predCurrIntraPredMode.

(2) When IntraPredModeA or IntraPredModeB is 3 or more, and when the prediction directions of IntraPredModeA and IntraPredModeB are completely different (for example, IntraPredModeA is 3 and IntraPredModeB is 4), DC prediction (intra4 × 4_pred_mode = 2) is set to predCurrIntraP.

(3) When IntraPredModeA or IntraPredModeB is 3 or more, and when the prediction directions are the same (e.g., IntraPredModeA is 3, and IntraPredModeB is 7 (predicting from the top right in both IntraPredModeA and IntraPredModeB)), prediction mode, interpolating pixel (in the above mode 7 ), set as predCurrIntraPredMode.

As in the third embodiment, the variable-length encoding unit 11 performs preparatory processing for precoding, for example, IntraPredModeA and IntraPredModeB (steps S50, S53, and S59). As a result, predCurrIntraPredMode is uniquely derived from the values of IntraPredModeA and IntraPredModeB. The tabulated rules for this predicted value task are shown in FIG. 26, shaded portions indicate cases where traditional MIN rules (IntraPredModeA, IntraPredModeB) are not applicable and the best predicted value is derived from the continuity of the image pattern. Class 0 is used in procedure (1). Class 1 is used in procedures (2) and (3).

After predCurrIntraPredMode is established as a result of the procedure, the variable-length encoding unit 11 performs the remaining encoding procedure for the component C0 described according to the third embodiment to complete the encoding (steps S52, S58 and S64).

In this way,

In the same way, the above-described coding procedure for the 8 × 8 intra-prediction mode can be determined. When coding the N × N intra-prediction mode in such a procedure, it is possible to better use the correlation of the prediction mode in the near pixel region in the same color component and it is possible to reduce the amount of codes of the prediction mode itself and increase the coding efficiency.

2. The procedure for decoding information intra prediction mode in the decoder

In the decoder according to the third embodiment, one of the specific procedures for decoding N × N intra-prediction mode information in the variable-length decoding unit 25 is described for the bitstream in the format shown in FIG. 16. According to a fifth embodiment, another specific method for implementing a decoding procedure is described. The fifth embodiment is characterized in that, in particular, taking into account the fact that the value of the N × N intra-prediction mode reflects the texture structure serving as the image template, adaptive prediction is performed in the near pixel region in the same color component to decode the encoded bitstream.

The following explanation is based on the condition that an array of bit streams in the format shown in FIG. 16 has been received. To simplify the explanation, the value of the flag 33 identifying the common use of the intra-coding mode in the bitstream is set to use the intra-coding mode for C0, C1 and C2. The N × N intra-prediction mode is designated as the intra-coding mode, and the 4 × 4 block is indicated as identification flags of the transform block size from 0 to 2 (32a-32c). In this case, all intra-prediction modes 0-2 (35a-35c) are 4 × 4 intra-prediction modes. As in the encoder, only the component C0 will be explained in the decoder using the relation shown in FIG. 18 (C1 and C2 are independently decoded in an equivalent procedure). In the decoder, the current macroblock to be decoded is indicated by X. The macroblock to the left of the current block is macroblock A, and the macroblock to the right and top of the current macroblock is macroblock B.

According to a third embodiment, as described in the encoder explanation, the smaller of the IntraPredModeA and IntraPredModeB values is uniquely allocated as the predicted predCurrIntraPredMode value for the 4x4 CurrIntraPredMode intra prediction mode allocated to the 4x4 X blocks, respectively, shown in FIG. 18. On the other hand, in the decoder according to the fifth embodiment, predCurrIntraPredMode is determined using the table shown in FIG. 26 in a procedure completely identical to the procedure described as the encoding procedure. Since IntraPredModeA and IntraPredModeB are already decoded and known, the processing can be carried out completely identical to the encoding procedure.

The further procedure is equivalent to the decoding procedure for the component C0 described according to the third embodiment. These procedures in general can be represented as follows.

In the same way, the above-described 8 × 8 intra-prediction mode decoding procedure can be determined. By decoding the N × N intra-prediction mode in such a procedure, it is possible to more effectively use the correlation of prediction modes in the near pixel region of the same color component to decode the encoded bitstream with a reduced amount of codes of the prediction mode itself.

In the example described above, predCurrIntraPredMode is fixed using the table shown in FIG. 26 to encode and decode. However, intra-prediction modes, most often appearing for the states IntraPredModeA and IntraPredModeB, can be encoded and decoded, updating them one by one. For example, in the combination “class = 0, IntraPredModeA = 0, IntraPredModeB = 0, predCurrIntraPredMode = 0” shown in FIG. 26, according to the above described embodiment, predCurrIntraPredMode is always 0 when IntraPredModeA = 0 and IntraPredModeB = 0. However, since the video signal itself is a non-stationary signal, there is no guarantee that this combination is the best, depending on the contents of the video signal. In the worst case, it is not entirely unlikely that predCurrIntraPredMode does not match the predicted value in most cases during the video signal. Thus, for example, the frequency with which CurrIntraPredMode appears in the case of IntraPredModeA = 0 and IntraPredModeB = 0 is counted and each time when encoding and decoding CurrIntraPredMode and predCurrIntraPredMode are updated in the prediction mode having the highest frequency of occurrence in relation to the state of IntraPredMode and. In this configuration, the predicted value used for encoding and decoding CurrIntraPredMode can be set equal to the optimal value in light of the contents of the video signal.

Sixth Embodiment

According to a sixth embodiment, another example of the structures of the encoder shown in FIG. 11 and the decoder shown in FIG. 12 is described. As in other embodiments, the characteristics of the invention are communicated to the encoder and decoder according to the sixth embodiment based on the encoding system adopted in MPEG-4 AVC (ISO / IEC 14496-10) /ITU-TH.264, which is non-patent document 1. The video encoder according to the sixth embodiment is different from the encoder shown in FIG. 11, is explained according to the second, third and fifth embodiments only in the operations of the variable-length encoding unit 11. The video decoder according to the sixth embodiment differs from the decoder shown in FIG. 12, is explained according to the second, third and fifth embodiments only in the operations of the variable-length decoding unit 25. Otherwise, the video encoder and video decoder perform the same operations as in the second, third, and fifth embodiments. Only differences will be explained.

1. The procedure for encoding information intra prediction mode in the encoder

In the encoder according to the third and fifth embodiments, a specific method is described for encoding N × N intra-prediction mode information in a bit stream in the format shown in FIG. 16 by a variable-length encoding unit 11. According to a sixth embodiment, another specific method for implementing an encoding procedure is described. The sixth embodiment is characterized in that, in particular, taking into account the fact that the value of the N × N intra-prediction mode reflects the texture structure serving as the image template, a method for performing adaptive arithmetic coding in the near pixel region in an identical color component is provided. The following explanation is based on the condition that an array of bit streams in the format shown in FIG. 16 has been received. According to a sixth embodiment, the N × N intra-prediction mode information for the respective components C0, C1, and C2 is independently encoded for each of the color components. The encoding method for component C0 also applies to C1 and C2. To simplify the explanation, only the encoding method for the C0 component will be explained. The value of the flag 33 identifying the common use of the intra-coding mode is set to use the intra-coding mode together for C0, C1 and C2. The intra-coding mode is the N × N intra-prediction mode, and the 0-2 (32a-32c) transform block size identification flags are the 4 × 4 block. In this case, all intra-prediction modes 0-2 (35a-35c) are 4 × 4 intra-prediction modes. 18, FIG. 18 is used as a diagram explaining a coding procedure for N × N intra-prediction mode information for a component C0. In FIG. 18, the current block to be encoded is indicated by X. The macroblock to the left of the current block is macroblock A, and the macroblock to the right and above the current macroblock is macroblock B. A logical block diagram of the encoding procedure is shown in FIG. 27.

According to the third and fifth embodiments, the smaller of the IntraPredModeA and IntraPredModeB values is uniquely allocated as the predicted predCurrIntraPredMode value for the 4 × 4 CurrIntraPredMode intra prediction mode allocated to the 4 × 4 X blocks, respectively, shown in FIG. 18. When the predicted value is equal to the specified value, prev_intra_pred_mode_flag is set to 1, and coding in the intra-prediction mode 4 × 4 for block X ends. When the predicted value is different from the specified value, the code is passed to rem_intra_pred_mode. In this embodiment, CurrIntraPredMode is directly arithmetically encoded using the IntraPredModeA and IntraPredModeB states. In this case, the encoding procedure is used, consistent with the context adaptive binary arithmetic encoding adopted in the AVC / H.264 standard.

First of all, the variable-length encoding unit 11 represents the CurrIntraPredMode of the encoding object in binary form in accordance with the format shown in FIG. 28 (step S76). The first portion of the binary sequence is a code for classifying CurrIntraPredMode as prediction in the vertical direction or prediction in the horizontal direction (see FIG. 3). In this example, DC prediction (intra4x4_pred_mode = 2) is classified as horizontal prediction. However, DC prediction (intra4x4_pred_mode = 2) can be classified as vertical prediction. The second section gives the completion bit to the values of the prediction mode, which are believed to have the highest frequency of occurrence in the vertical direction and horizontal direction, respectively. The third and subsequent sections are configured with a code followed by completion from one with the highest frequency of occurrence among the other values of the prediction mode (the second and subsequent sections of the binary sequence configuration shown in Fig. 28 are desirable to be set according to the probability of occurrence of characters during the encoding of the actual image data).

The variable-length coding unit 11 performs arithmetic coding, sequentially selecting the occurrence probability tables to be used for the corresponding sections of the binary sequence (0,1). When encoding the first portion, the variable-length encoding unit 11 sets the context used for arithmetic encoding as follows (step S78).

Context A (C _A ): The intra_pred_direction_flag flag representing in binary form whether the intra prediction mode is vertical prediction or horizontal prediction is set for IntraPredModeA and IntraPredModeB. The following four states are set as context values.

For example, when intra4x4_pred_mode takes the values 0, 3, 5, and 7 shown in FIG. 3, intra_pred_direction_flag is classified as vertical prediction (= 0). When intra4x4_pred_mode takes values 1, 2, 4, 6, and 8, intra_pred_direction_flag is classified as horizontal prediction (= 1). The conditional probabilities CurrIntraPredMode based on the states IntraPredModeA and IntraPredModeB are calculated in advance, and the initial appearance probability tables (0,1) defined based on the conditional probabilities are allocated to the four states C _A, respectively. By forming the context in this way, it is possible to more accurately evaluate the conditional probability of the appearance of the first section and increase the efficiency of arithmetic coding. The variable-length encoding unit 11 selects a probability table for the appearance of the first region according to the value of C _A and performs arithmetic encoding. The variable-length encoding unit 11 updates the probability table of occurrence by the encoding value (step S79).

The initial occurrence probability table (0.1), set according to the probability of occurrence of the corresponding value prediction mode, is preliminarily allocated to the second and subsequent sections. Then, the variable-length decoding unit 25 performs binary arithmetic decoding and updating the occurrence probability table in the same manner as for the first portion.

In the same way, the above-described coding procedure for the 8 × 8 intra-prediction mode can be determined. Encoding the N × N intra-prediction mode in such a procedure, adaptive arithmetic coding can be applied to encoding prediction mode information using correlation of prediction modes in the near pixel region of the same color component. This allows you to increase the coding efficiency.

2 . The decoding procedure of intra prediction mode information in the decoder

In the decoder according to the third and fifth embodiments, one of the specific procedures for decoding N × N intra-prediction mode information in the variable-length decoding unit 25 is described for the bitstream in the format shown in FIG. 16. According to a sixth embodiment, another specific method for implementing a decoding procedure is described. The sixth embodiment is characterized in that, in particular, taking into account the fact that the value of the N × N intra-prediction mode reflects the structure of the texture serving as the image template, adaptive arithmetic coding is carried out in the near pixel region in the same color component for decoding the encoded bitstream.

The following explanation is based on the condition that an array of bit streams in the format shown in FIG. 16 has been received. To simplify the explanation, the value of the flag 33 identifying the common use of the intra-coding mode in the bitstream is set to use the intra-coding mode for C0, C1 and C2. The N × N intra-prediction mode is designated as the intra-coding mode, and the 4 × 4 block is indicated as identification flags of the transform block size from 0 to 2 (32a-32c). In this case, all intra-prediction modes 0-2 (35a-35c) are 4 × 4 intra-prediction modes. As in the encoder, only the component C0 will be explained in the decoder using the relation shown in FIG. 18 (C1 and C2 are independently decoded in an equivalent procedure). In the decoder, the current macroblock to be decoded is indicated by X. The macroblock to the left of the current block is macroblock A, and the macroblock to the right and top of the current macroblock is macroblock B.

According to the third and fifth embodiments described in the encoder explanation, the smaller of the IntraPredModeA and IntraPredModeB values is uniquely allocated as the predicted predCurrIntraPredMode value for the 4 × 4 CurrIntraPredMode intra prediction mode allocated to the 4 × 4 X blocks respectively shown in FIG. 18. When prev_intra_pred_mode_flag is decoded and its value is 1, predCurrIntraPredMode is taken as CurrIntraPredMode. When prev_intra_pred_mode_flag is zero, rem_intra_pred_mode is decoded to restore the 4x4 block prediction mode of block X is restored. On the other hand, in this embodiment, CurrIntraPredMode is directly subjected to arithmetic decoding using the states IntraPredModeA and IntraPredModeB. In this case, a decoding procedure is used consistent with the context adaptive binary arithmetic decoding adopted in the AVC / H.264 standard.

The CurrIntraPredMode to be decoded is encoded as a binary sequence in accordance with the format shown in FIG. 28. This sequence is sequentially subjected to binary arithmetic decoding from the left end. As explained in the encoding procedure according to the sixth embodiment, the first portion of the binary sequence is a code for classifying CurrIntraPredMode as prediction in the vertical direction or prediction in the horizontal direction (see FIG. 3). The second and subsequent sections are subjected to configuration of the code, followed by completion from one with the highest frequency of occurrence among the values of the prediction mode. The reason for this code configuration is described in the encoding procedure.

In the decoding process, especially when decoding the first portion, the variable-length decoding unit 25 sets C _A to the same as in the context used in the encoding procedure. The variable-length decoding unit 25 selects an occurrence probability table according to the value of C _A and performs arithmetic decoding to reconstruct the first portion. The variable-length decoding unit 25 updates the probability table of occurrence with a decoding value.

The initial occurrence probability table (0.1), set according to the probability of occurrence of the corresponding value prediction mode, is preliminarily allocated to the second and subsequent sections. Then, the variable-length decoding unit 25 performs binary arithmetic decoding and updating the occurrence probability table in the same manner as for the first portion. Since the binary sequence shown in FIG. 28 is formed so that it is possible to uniquely specify a corresponding value prediction mode, CurrIntraPredMode is decoded when a predetermined number of plots are restored.

In the same way, the above-described 8 × 8 intra-prediction mode decoding procedure can be determined. Due to the decoding of the N × N intra-prediction mode, in such a procedure it is possible to decode a coded bit stream with a reduced amount of codes of the prediction mode itself according to arithmetic coding, which makes it possible to use the prediction mode correlation in the near pixel region of the same color component.

In the above example, other variations of the table shown in FIG. 28 are possible. For example, the binary sequence generation method shown in FIG. 29 can be adopted. Moreover, the context B described below is used for the first section.

Context B (C _B ): The intra_dc_pred_flag flag, representing in binary form whether the intra prediction mode is a vertical DC prediction, is set for IntraPredModeA and IntraPredModeB. The following four states are set as context values.

In Fig. 3, when intra4 × 4_pred_mode is set to 2, intra_dc_pred_flag is set to 1. When intra4 × 4_pred_mode is set to other values, intra_dc_pred_flag is set to 0. Conditional CurrIntraPredMode probabilities based on the states of the IntraPredModeA and IntraPredModeB the probabilities of the occurrence table are calculated in advance 0.1) of the first section, given on the basis of conditional probabilities, are allocated to four states C _B, respectively. In Fig. 29, the first portion is intended to take a value of 0 when CurrIntraPredMode represents DC prediction, and takes a value of 1 when CurrIntraPredMode is different from DC prediction. The above context A (C _A ) is used for the second portion. By forming the context in this way, it is possible to more accurately evaluate the conditional occurrence probabilities for the first section and the second section and increase the efficiency of arithmetic coding.

Seventh Embodiment

According to a seventh embodiment, an encoder that encodes using inter-frame prediction in units obtained by equal dividing a video frame inputted in a 4: 4: 4 format into rectangular areas (macroblocks) of 16 x 16 pixels and a decoder corresponding to the encoder will be explained. The characteristics of the invention are communicated to the encoder and decoder based on the encoding system adopted in MPEG-4 AVC (ISO / IEC 14496-10) /ITU-TH.264.

The structure of the video encoder according to the seventh embodiment is shown in FIG. The structure of the video decoder according to the seventh embodiment is shown in FIG. In Fig. 31, the components indicated by the same positions as the components of the encoder in Fig. 30 are identical components.

The operations of the encoder as a whole and the decoder as a whole and the decision processing regarding the inter prediction mode and the motion compensation prediction decoding processing, which are characteristic operations of the seventh embodiment, will be explained based on these figures.

1. Description of the encoder

In the encoder shown in FIG. 30, corresponding video frames are input as input video signal 1 in a 4: 4: 4 format. The input video frames are input to the encoder in units of blocks obtained by dividing the three color components into macroblocks of the same size and placing the blocks, as shown in FIG. 10.

First of all, the motion compensation prediction unit 102 selects a reference image of one frame from the reference image data to predict motion compensation of one or more frames stored in the memory 16, and performs prediction processing for motion compensation for each of the color components in units of macroblocks. Three memory blocks are prepared for the respective color components (although three memory blocks are prepared in the explanation of this embodiment, the number of memory blocks can accordingly be changed according to the design). Seven types are prepared as block sizes for forecasting with motion compensation. First, in units of macroblocks, as shown in FIGS. 32 (a) to 32 (d), any of the sizes 16 × 16, 16 × 8, 8 × 16 and 8 × 8 can be selected. When selecting 8 × 8, as shown in FIGS. 32 (e) to (h), any of the sizes 8 × 8, 8 × 4, 4 × 8 and 4 × 4 can be selected for each 8 × 8 blocks. Information about the selected size is displayed as a macroblock type, and size information in units of 8 × 8 blocks is displayed as a sub-macroblock type. The identification number and motion vector information for the selected reference image for each block are displayed.

The video encoder according to the seventh embodiment is characterized by a change in the prediction processing method for motion compensation for the three color components based on the common prediction mode identification flag 123. This point will be described in detail below in paragraph 2.

Motion compensation prediction unit 102 performs prediction processing for motion compensation for all block sizes or sub-block sizes shown in FIG. 32 of all motion vectors 137 in a predetermined search range and may select one or more reference images to obtain a prediction differential signal 4 according to the vectors motion 137, one reference image and subtracter 3. Decision block 5 regarding the encoding mode estimates the prediction efficiency of the difference signal prediction 4 and outputs the macroblock type / sub-macroblock type 106, the motion vector 137 and the reference image identification number, at which the optimal prediction efficiency is obtained, to the macroblock to be predicted from the prediction processing performed by the motion compensation prediction unit 102. When choosing the type of macroblock / type of submacroblock 106, one can take into account the weighting factor 20 for each type, which is set by decision on the encoding control unit 19. The motion compensation prediction unit 102 outputs a prediction differential signal 4, obtained by motion compensation prediction based on the type, motion vector 137 and the selected reference image, to the transform unit 8. The transform unit 8 converts the input prediction difference signal 4 into a transform coefficient and outputs a transform coefficient to the quantization unit 9. The quantization unit 9 quantizes the input transform coefficient based on the quantization parameter 21 specified by the unit 19 encoding systematic way, and outputs the transform coefficient coding unit 11 for variable length as a quantized transform coefficient 10. The quantized transform coefficient 10 is subjected to entropy encoding by means such as Huffman coding or arithmetic coding on the coding unit 11 variable length. The quantized transform coefficient 10 is restored to the local prediction differential prediction signal 14 by the inverse quantization unit 12 and the inverse transform unit 13. The quantized transform coefficient 10 is added to the predicted image 7, which is generated based on the macroblock type / submacroblock type 106, the motion vector 137 and the selected reference image, by the adder 18 to generate a locally decoded image 15. The locally decoded image 15 is stored in memory 16 for later use prediction processing for motion compensation. An enable filter control flag 24 indicating whether the enable filter is applied to the macroblock is also input to the variable-length encoding unit 11. (In the prediction processing performed by the motion compensation prediction unit 102, since the pixel data is stored in the memory 16 before being processed by the unlocking filter, the processing of the unlocking filter itself is not necessary for encoding processing. However, the unlocking filter acts according to the direction of the unlocking filter control flag 24 on the decoder side to get the final decoded image.)

Inter-prediction mode common use identification flag 123, quantized transform coefficient 10, macroblock type / sub-macroblock type 106, motion vector 137, reference image identification number and quantization parameter 21 entered into variable-length encoding unit 11 are ordered and formed as a bit stream in accordance with a predetermined rule (syntax) and are output to the transmission buffer 17. The transmission buffer 17 smooths the bit stream according to the band of the transmission line to which the code is connected er, and the read speed of the recording medium, and outputs the bitstream as the video stream 22. The transmission buffer 17 outputs feedback information to the encoding control unit 19 according to the accumulation state of the bitstream into the transmission buffer 17 and then controls the amount of generated codes when encoding video frames.

2. Decision processing regarding the inter prediction mode in the encoder

The decision processing regarding the inter prediction mode, which is a characteristic of the encoder according to the seventh embodiment, is described in detail below. In the following description, the inter prediction mode indicates a block size serving as an input signal for compensating a motion vector, i.e. macroblock type / submacroblock type. Decision processing for the inter-prediction mode means processing for selecting a macroblock type / sub-macroblock type, a motion vector, and a reference image. Processing is carried out in units of macroblocks obtained by placing three color components. The processing is carried out mainly by the motion compensation prediction unit 102 and the decision unit 5 regarding the encoding mode in the encoder shown in FIG. 30. A logical flowchart showing the progress of processing is shown in FIG. Image data of the three color components forming the block are referred to below as C0, C1, and C2.

First of all, the decision block 5 regarding the coding mode receives the common prediction mode identification flag 123 and makes a decision based on the value of the common prediction mode identification flag 123 whether the common inter prediction mode, the common motion vector 137 and the common reference image for C0, C1 are used and C2 (step S100 in FIG. 33). When the inter prediction mode, the motion vector 137 and the reference image are used together, the decision block 5 regarding the encoding mode proceeds to step S101 and subsequent steps. Otherwise, the decision block 5 regarding the encoding mode proceeds to step S102 and subsequent steps.

When the inter prediction mode, motion vector 137 and the reference image are used together for C0, C1, and C2, the coding mode decision block 5 notifies the motion compensation prediction block 102 of all the inter prediction modes, motion vector search ranges, and reference pictures that can be selected. Motion compensation prediction unit 102 estimates prediction performance for all inter prediction modes, motion vector search ranges and reference images, and selects an optimal inter prediction mode, optimal motion vector 137, and optimal reference images common to C0, C1, and C2 (step S101).

When the inter prediction mode, the motion vector 137 and the reference image are not used together for C0, C1 and C2 and the best modes are respectively selected for C0, C1 and C2, the decision block 5 regarding the encoding mode notifies the motion compensation prediction block 102 of all the inter prediction modes, ranges search for the motion vector and reference images that can be selected for the components Ci (i <= 0 <3). The motion compensation prediction unit 102 estimates the prediction efficiency for all inter prediction modes, motion vector search ranges and reference images, and selects an optimal inter prediction mode, an optimal motion vector 137, and an optimal reference image in the components Ci (i <= 0 <3) (steps S102, S103 and S104).

As criteria for evaluating the prediction efficiency for the prediction mode carried out in the motion compensation prediction block 102, for example, the velocity / distortion value defined as Jm, v, r = Dm, v, r + λRm, v, r (λ : positive number). Dm, v, r is the coding distortion or the magnitude of the prediction error in the case when the inter prediction mode m, the motion vectors v in a predetermined range, and the reference image r are used. The coding distortion is obtained using the inter-prediction mode m, motion vectors v and the reference image r to calculate the prediction error and decode the video signal from the result obtained by converting and quantizing the prediction error to measure the error relative to the signal before encoding. The magnitude of the prediction error is obtained by calculating the difference between the predicted image and the signal before coding in the case where the inter prediction mode m, the motion vectors v and the reference image r, and quantizing the difference level are applied. For example, Absolute Distance Sum (SAD) is used. Rm, v, r is the volume of generated codes in the case when the inter prediction mode m, the motion vectors v, and the reference image r are used. In other words, Jm, v, r is a value specifying a compromise between the amount of codes and the degree of deterioration when the inter prediction mode m, motion vectors v, and reference image r are used. Interprediction mode m, which gives the minimum Jm, v, r, motion vectors v and the reference image r give the optimal solution.

When the encoder performs the processing in step S101 and subsequent steps, a pair of pieces of information about the inter prediction mode, motion vectors 137, and the reference image are allocated to the macroblock including three color components. On the other hand, when the encoder processes in step S102 and subsequent steps, the information of the inter prediction mode, the motion vectors 137, and the reference image are allocated to the color components, respectively. Thus, since the pieces of information about the inter prediction modes, motion vectors 137 and the reference image allocated to the macroblock are different, it is necessary to multiplex the identification flag 123 of the general use of the inter prediction mode into the bitstream and allow the decoder to recognize whether the encoder has performed the processing steps in step S101 and subsequent steps or carried out the processing steps in step S102 and subsequent steps. An array of data of such a bitstream is shown in FIG.

An array of bitstream data at the macroblock level is shown in FIG. The type of macroblock indicates intra or inter and includes information serving as a unit of motion compensation during inter. The sub-macroblock type is multiplexed only when the 8 × 8 block size is selected as the macroblock type and includes block size information for each of the 8 × 8 block sizes. The macroblock base type 128 and the sub macroblock base type 129 indicate a common macroblock type and a common sub-macroblock type when the inter-prediction mode common use identification flag 123 indicates “common to C0, C1 and C2”. Otherwise, the base macroblock type 128 and the sub-macroblock base type 129 indicate the macroblock type and sub-macroblock type for C0. The extended macroblock type 130 and the extended sub-macroblock type 131 are multiplexed for C1 and C2, respectively, only when the inter-prediction mode common use identification flag 123 indicates “not common for C0, C1 and C2”. The macroblock type 130 and the extended sub-macroblock type 131 indicate the macroblock type and sub-macroblock type for C1 and C2.

The reference image identification number is information for indicating a reference image selected for each block greater than or equal to the size of an 8 × 8 block serving as a unit of motion compensation. During the inter-frame, since the reference image that can be selected is one frame, one identification number of the reference image is multiplexed for each block. A pair of pieces of motion vector information is multiplexed by motion vector information for each block serving as a unit of motion compensation. The number of reference image identification numbers and pieces of motion vector information that need to be multiplexed is equivalent to the number of blocks serving as motion compensation units included in the macroblock. When the inter-prediction mode common use identification flag 123 indicates “common to C0, C1 and C2”, the basic reference image identification number 132 and the basic motion vector information 133 indicate the common reference image identification number and common motion vector information. Otherwise, the reference reference image identification number 132 and the basic motion vector information 133 indicate the reference image identification number and the motion vector information for C0. The extended reference image identification number 134 and the extended motion vector information 135 are multiplexed for C1 and C2, respectively, only when the inter-prediction mode common use identification flag 123 indicates “not common for C0, C1 and C2”. The reference reference image identification number 134 and the extended motion vector information 135 indicate the reference image identification number and the motion vector information for C1 and C2.

Then, the quantization parameter 21 and the quantized transform coefficient 10 are multiplexed. (Although the enable filter control flag 24 entered in the variable-length encoding unit 11 shown in FIG. 30 is not included in FIG. 34, the enable filter control flag 24 is omitted because the flag is not a component necessary to explain the characteristics of the seventh embodiment implementation.)

In the 4: 2: 0 format, adopted in the traditional video coding standard, the definition of color spaces is fixed to Y, Cb and Cr. In the 4: 4: 4 format, the definition of color spaces is not limited to Y, Cb, and Cr, but various color spaces can be used. By generating the information of the inter prediction mode as shown in FIG. 34, it is possible to carry out optimal coding processing even when the determination of the color spaces of the input video signal 1 is different. For example, when color spaces are defined as RGB, in the area where the structure of the video texture equally remains in the corresponding components R, G, and B, using the information of the general inter prediction mode and the information of the general motion vector, we can reduce the redundancy of the information of the inter prediction mode and the information of the motion vector itself and increase coding efficiency. On the other hand, when color spaces are defined as Y, Cb, and Cr, the structure of the video texture is integrated into Y. Thus, the general inter-prediction mode does not always give an optimal result. Thus, it is possible to obtain optimal coding efficiency by adaptively using the advanced intra-prediction mode 30. On the other hand, for example, in an area (component R is 0) without a hue of red, information of the optimal inter-prediction mode and the optimal vector for the component R and information of the optimal inter-prediction mode and The optimal motion vectors for components G and B should be different. Thus, it is possible to obtain optimum coding efficiency by adaptively using the advanced inter prediction mode, identification information of the extended reference image, and information of the extended motion vector.

3. Description of the decoder

The decoder shown in Fig. 31 receives a video stream 22 corresponding to the array shown in Fig. 34, output from the encoder shown in Fig. 30, performs decoding processing in units of macroblocks in which the three color components have the same size (format 4: 4: 4), and restores the corresponding video frames.

First, the variable-length decoding unit 25 receives a stream 22, decodes the video stream 22 in accordance with a predetermined rule (syntax), and extracts information including an inter prediction mode common use identification flag 123, a quantized transform coefficient 10, a macroblock type / sub-macroblock type 106 , reference picture identification number, motion vector information, and quantization parameter 21. The quantized transform coefficient 10 is input to the inverse quantization block 12 estno with the quantization parameter 21 and inverse quantization processing is performed. Then, the output signal of the inverse quantization unit 12 is input to the inverse transform unit 13 and restored to the local decoding prediction differential signal 14. On the other hand, the macroblock type / sub-macroblock type 106 and the inter-prediction mode common use identification flag 123 are input to the motion compensation prediction unit 102. Block 102 predicting motion compensation receives the predicted image 7 in accordance with these pieces of information. The specific procedure for obtaining the predicted image 7 will be described below. The local decoding prediction difference signal 14 and the prediction image 7 are summed by the adder 18 to obtain an internally decoded image 15 (this is completely the same signal as the locally decoded image 15 in the encoder). The internally decoded image 15 is written back to the memory 16 for later use in motion compensation prediction for the macroblock. Three memory blocks are prepared for the respective color components (although three memory blocks are prepared in the explanation of this embodiment, the number of memory blocks can accordingly be changed according to the design). The unlocking filter 26 is instructed to act on the internally decoded image 15 based on the indication of the unlocking filter control flag 24, decoded by the variable-length decoding unit 25, to obtain the final decoded image 27.

2. Inter prediction decoding processing in a decoder

The decoder shown in Fig. 31 receives a video stream 22 corresponding to the array shown in Fig. 34, output from the encoder shown in Fig. 30, performs decoding processing in units of macroblocks with the same size (4: 4: 4 format) for three color components and restores the corresponding video frames.

Described in detail below is an inter-predicted image generation processing that is a characteristic of a decoder according to a seventh embodiment. This processing is carried out in units of macroblocks in which three color components are placed. The processing is carried out mainly by the variable-length decoding unit 25 and the motion compensation prediction unit 102 in the decoder shown in FIG. The logical flowchart of the processing carried out by the variable-length decoding unit 25 is shown in FIG. 35.

The video stream 22 introduced into the variable-length decoding unit 25 is consistent with the data array shown in FIG. 34. In step S110, the variable-length decoding unit 25 decodes the inter-prediction mode common use identification flag 123 for the data shown in FIG. 34 (step S110). The variable-length decoding unit 25 further decodes the base macroblock type 128 and the sub-macroblock base type 129 (step S111). In step S112, the variable-length decoding unit 25 decides whether the inter prediction mode is used together for C0, C1, and C2 based on the result of the identification flag 123 of the common use of the inter prediction mode. When the inter prediction mode is used together for C0, C1, and C2 (Yes in step S112), the variable word length decoding unit 25 uses the base macroblock type 128 and the submacroblock base type 129 for all C0, C1, and C2. Otherwise (No in step S112), the variable-length decoding unit 25 uses the base macroblock type 128 and the sub-macroblock base type 129 as the mode for C0. The variable-length decoding unit 25 decodes the extended macroblock type 130 and the extended sub-macroblock type 131 for C1 and C2, respectively (step S113) to obtain inter prediction mode information for C1 and C2. The variable word length decoding unit 25 decodes the identification number 132 of the basic reference image and the basic motion vector information 133 (step S114). When the inter-prediction mode common use identification flag 123 indicates “shared between C0, C1, and C2” (Yes in step S115), the variable-length decoding unit 25 uses the basic reference image identification number 132 and the basic motion vector information 133 for all C0, C1 and C2. Otherwise (No in step S115), the variable-length decoding unit 25 uses the base reference image identification number 132 and the basic motion vector information 133 as information for C0. The variable-length decoding unit 25 decodes the extended reference image identification number 134 and the extended motion vector information 135 for C1 and C2, respectively (step S116). Macroblock types 106, reference image identification numbers, and motion vector information for the respective color components are set in the processing steps. Thus, the variable-length decoding unit 25 outputs macroblock types 106, reference image identification numbers and motion vector information to the motion compensation prediction unit 102 to obtain predicted motion-compensated images of the respective color components.

Variations in the bitstream data array shown in FIG. 34 are shown in FIG. In Fig. 36, the common prediction mode identification flag 123 is multiplexed as a flag located at a higher data level, for example, at a slice, image or sequence level, and not as a flag at the macroblock level. Therefore, when it is possible to ensure sufficient prediction efficiency according to a change at a higher level equal to or higher than the cutoff level, it is possible to reduce the overhead bit without multiplexing the flag 123 for identifying the general use of the prediction mode at the macroblock level each time the processing is performed.

34 and 36, an inter prediction mode common use identification flag 123 is multiplexed to each macroblock or higher level of data, such as a slice, image, or sequence. When encoding is carried out in a 4: 4: 4 format without multiplexing the flag 123 for identifying the common use of the inter prediction mode, different inter prediction modes and motion vector information for the respective components can always be used. The data stream of the bitstream in this case is shown in Fig.37. 37, there is no flag 123 for identifying the general use of the inter prediction mode, and profile information 136 indicating that the input image is being processed in a 4: 4: 4 format is multiplexed to higher-level data, for example, a sequence. The extended macroblock type 130, the extended sub-macroblock type 131, the extended reference image identification number 134, and the extended motion vector information 135 are multiplexed according to the result of decoding the profile information.

Eighth Embodiment

According to a seventh embodiment, the macroblock type / submacroblock type, the motion vector and the reference image can be varied for each of the color components. According to an eighth embodiment, a video encoder and a video decoder will be described, characterized by their ability to set the macroblock type / sub-macroblock type common to the respective components, and vary only the motion vector for each of the components. The structures of the video encoder and video decoder according to the eighth embodiment are the same as those in FIGS. 30 and 31 according to the seventh embodiment. However, the structures are different in that instead of the flag 123 identifying the common use of the inter prediction mode, the flag 123b identifying the common use of the motion vector is used.

1. Decision processing regarding an inter prediction mode in an encoder

Decision processing for the inter-prediction mode, which is a characteristic of the encoder according to the eighth embodiment, will be described in detail with reference to processing other than that of the seventh embodiment.

2. Decision processing regarding the inter prediction mode in the encoder

Processing is carried out in units of macroblocks obtained by placing three color components. The processing is carried out mainly by the motion compensation prediction unit 102 and the decision unit 5 regarding the encoding mode in the encoder shown in FIG. 30. A logical flowchart showing the progress of processing is shown in FIG. 38. Image data of the three color components forming the block are referred to below as C0, C1, and C2.

First of all, the coding mode decision block 5 receives the motion vector common use identification flag 123b and makes a decision based on the value of the common motion vector common use identification flag 123b whether the common motion vector 137 is used for C0, C1 and C2 (step S120 in FIG. 37). When using the common motion vector 137, the decision block 5 regarding the encoding mode proceeds to step S121 and subsequent steps. Otherwise, the decision block 5 regarding the encoding mode proceeds to step S122 and subsequent steps.

When the motion vector 137 is shared between C0, C1, and C2, the coding mode decision block 5 notifies the motion compensation prediction block 102 of all the inter prediction modes, motion vector search ranges, and reference pictures that can be selected. Motion compensation prediction unit 102 estimates the prediction efficiency for all inter-prediction modes, motion vector search ranges and reference images, and selects an optimal inter-prediction mode, optimal motion vector 137, and optimal reference images common to C0, C1, and C2 (step S121).

When the motion vector 137 is not shared between C0, C1, and C2 and the best motion vectors are selected for C0, C1, and C2, respectively, the coding mode decision block 5 notifies the motion compensation prediction block 102 of all the inter prediction modes, motion vector search ranges, and reference images that can be selected. The motion compensation prediction unit 102 estimates the prediction efficiency for all inter-prediction modes, motion vector search ranges and reference images and selects the optimal inter-prediction mode and the optimal reference image (step 122) and additionally the optimal motion vector in the components Ci (i <= 0 <3) ( steps S123, S124, and S125).

It is necessary to multiplex the motion vector common use identification flag 123b into the bitstream and enable the motion vector common use identification flag 123b to be recognized on the decoder side. An array of data of such a bitstream is shown in FIG.

The data stream of the bitstream at the macroblock level is shown in Fig. 39. The macroblock type 128b, the sub macroblock type 129b, and the reference picture identification number 132b are “common to C0, C1, and C2”. When the motion vector common use identification flag 123b indicates "common to C0, C1 and C2", the basic motion vector information 133 indicates common motion vector information. Otherwise, the basic motion vector information 133 indicates the motion vector information for C0. Only when the motion vector common use identification flag 123b indicates "not common to C0, C1 and C2", the extended motion vector information 135 is multiplexed for C1 and C2, respectively, and indicates the motion vector information for C1 and C2. The macroblock type / sub-macroblock type 106 shown in FIGS. 30 and 31 are a common member of the macroblock type 128b and the sub-macroblock type 129b shown in FIG. 39.

2. Inter prediction decoding processing in a decoder

The decoder according to the eighth embodiment receives the video stream 22 corresponding to the array shown in Fig. 39, output from the encoder according to the eighth embodiment, performs decoding processing in units of macroblocks with the same size (4: 4: 4 format) for the three color components and restores the corresponding video frames.

Inter prediction image generation processing, which is a characteristic of a decoder according to the eighth embodiment, will be described in detail with reference to processing other than that of the seventh embodiment. This processing is carried out in units of macroblocks in which three color components are placed. The processing is carried out mainly by the variable-length decoding unit 25 and the motion compensation prediction unit 102 in the decoder shown in FIG. A logical flowchart of the processing carried out by the variable-length decoding unit 25 is shown in FIG. 40.

The video stream 22 input to the variable-length decoding unit 25 is consistent with the data array shown in FIG. In step S126, the variable-length decoding unit 25 decodes the macroblock type 128b and the sub-macroblock type 129b common to C0, C1, and C2. The size of the block serving as the motion compensation unit depends on the decoded macroblock type 128b or submacroblock type 129b. Thus, the variable-length decoding unit 25 decodes a reference image identification number 132b common to C0, C1, and C2 for each unit serving as a motion compensation unit (step S127). In step S128, the variable-length decoding unit 25 decodes the motion vector common use identification flag 123b. Then, the variable-length decoding unit 25 decodes the basic motion vector information 133 for each unit serving as a motion compensation unit (step S129). In step S130, the variable-length decoding unit 25 decides whether motion vector 137 is used together for C0, C1, and C2 using the result of the flag 123b for identifying the common use of the motion vector. When the motion vector 137 is shared (Yes in step S130), the variable-length decoding unit 25 uses the basic motion vector information for all C0, C1, and C2. Otherwise (No in step S130), the variable-length decoding unit 25 uses the basic motion vector information 133 as a mode for C0 and decodes the extended motion vector information 135 for C1 and C2, respectively (step S131). Since macroblock types / submacroblock types 106, reference image identification numbers and motion vector information for the respective color components are set in the processing steps, the variable word length decoding unit 25 outputs macroblock types / submacroblock types 106, reference image identification numbers and motion vector information to the block 102 predicting motion compensation to obtain a predicted image with motion compensation for the respective color components.

Variations in the data stream of the bitstream shown in Fig. 39 are shown in Fig. 41. According to FIG. 39, the motion vector common use identification flag 123b is multiplexed as a flag located at a higher data level, for example at a slice, image or sequence level, and not as a macroblock flag. Therefore, when it is possible to provide sufficient prediction efficiency according to a change at a higher level equal to or higher than the slice level, it is possible to reduce the overhead bit without multiplexing the flag 123b identifying the total use of the motion vector at the macroblock level each time the processing is performed.

Referring to FIGS. 39 and 41, the motion vector common use identification flag 123b is multiplexed to each macroblock or higher level of data, such as a slice, image, or sequence. When encoding is carried out in a 4: 4: 4 format without multiplexing, the flag 123b identifying the common use of the motion vector, other information of the motion vector can always be used for the respective components. The data stream of the bitstream in this case is shown in Fig. 42. 42, there is no flag 123b for identifying the common use of the motion vector, and profile information 136 indicating that the input image in 4: 4: 4 format is being processed is multiplexed to higher-level data, for example, a sequence. The extended motion vector information 135 is multiplexed according to the result of decoding the profile information 136.

According to an eighth embodiment, the macroblock type / sub-macroblock type 106 and the reference image are common to the respective color components, and only the motion vector 137 can vary for each of the color components. Therefore, when sufficient prediction efficiency is obtained by adapting only the motion vector 137 to the corresponding color components, overhead bits can be reduced without multiplexing the macroblock type / sub-macroblock type 106 and the reference image identification number for each of the color components.

Ninth Embodiment

According to the seventh embodiment, a decision can be made whether the macroblock type / sub-macroblock type 106, the motion vector 137 and the reference image are used together for the three components or are changed for each of the color components according to the common use identification flag 123 of the inter prediction mode or profile information 136. However, according to the ninth to the embodiment, assuming that the 4: 4: 4 image corresponds to the Y, Cb, Cr format, it can be decided whether different modes are used for the brightness component (Y) and the color difference component (Cb, Cr) (in this case, the common mode is used for two components from the color difference components). A video encoder and a video decoder will be explained, differing in their ability to decide whether the common mode for the three components is used, whether different modes from the corresponding components are used, or different modes are used for the brightness components and color difference components. The structures of the video encoder and video decoder according to the ninth embodiment are identical to those shown in FIGS. 30 and 31 according to the seventh embodiment.

1. Decision processing regarding an inter prediction mode in an encoder

Decision processing for the inter prediction mode, which is a characteristic of the encoder according to the ninth embodiment, will be described in detail with reference to processing other than that of the seventh embodiment.

Processing is carried out in units of macroblocks obtained by placing three color components. The processing is carried out mainly by the motion compensation prediction unit 102 and the decision unit 5 regarding the encoding mode in the encoder shown in FIG. 30. A logical flowchart showing the progress of processing is shown in FIG. 43. Image data of the three color components forming the block are referred to below as C0, C1, and C2.

First of all, the decision block 5 regarding the coding mode receives the general prediction mode identification flag 123 and makes a decision based on the value of the common prediction mode identification flag 123 whether the common inter prediction mode, the common motion vector 137, and the common reference image for C0, C1 are used and C2 (step S132 in FIG. 43). When the inter prediction mode, the motion vector 137 and the reference image are used together, the decision block 5 regarding the encoding mode proceeds to step S133 and the subsequent steps. Otherwise, the decision block 5 regarding the encoding mode proceeds to step S134 and subsequent steps, or to step 137 and subsequent steps.

When the inter prediction mode, motion vector 137 and the reference image are used together for C0, C1, and C2, the coding mode decision block 5 notifies the motion compensation prediction block 102 of all the inter prediction modes, motion vector search ranges, and reference pictures that can be selected. Motion compensation prediction unit 102 estimates the prediction efficiency for all inter-prediction modes, motion vector search ranges and reference images, and selects an optimal inter-prediction mode, optimal motion vector 137, and optimal reference images common to C0, C1, and C2 (step S133).

When the inter prediction mode, the motion vector 137 and the reference image are not used together for C0, C1 and C2 and the best modes are respectively selected for C0, C1 and C2, the decision block 5 regarding the encoding mode notifies the motion compensation prediction block 102 of all the inter prediction modes, ranges search for the motion vector and reference images that can be selected for the components Ci (i <= 0 <3). The motion compensation prediction unit 102 estimates the prediction efficiency for all inter-prediction modes, motion vector search ranges and reference images, and selects an optimal inter-prediction mode, an optimal motion vector 137, and optimal reference images in the components Ci (i <= 0 <3) (steps S134, S135 and S136).

When the inter prediction mode, motion vector 137 and the reference image are used together for C1 and C2, and the best modes are selected for C0 (equivalent to the luminance component) and C1 and C2 (equivalent to the color difference components), the decision block 5 regarding the coding mode notifies the motion compensation prediction block 102 about all inter prediction modes, motion vector search ranges, and reference images that can be selected in component C0. The motion compensation prediction unit 102 estimates prediction efficiencies for all inter prediction modes, motion vector search ranges, and reference images, and selects an optimal inter prediction mode, an optimal motion vector 137, and an optimal reference image in component C0 (step S137). The decision block 5 regarding the encoding mode notifies the motion compensation prediction block 102 of all inter prediction modes, motion vector search ranges, and reference images that can be selected in components C1 and C2. The motion compensation prediction unit 102 estimates prediction efficiencies for all inter-prediction modes, motion vector search ranges and reference images, and selects an optimal inter-prediction mode, an optimal motion vector 137, and an optimal reference image common to C1 and C2 (step S138).

The bitstream data array output by the encoder according to the ninth embodiment is the same as that shown in FIG. 34. When the inter-prediction mode common use identification flag 123 indicates “common to C1 and C2,” the extended macroblock type 130, the extended sub-macroblock type 131, the extended reference identification number 134, and the extended motion vector information 135 are information common to C1 and C2.

2. Inter prediction decoding processing in a decoder

The decoder according to the ninth embodiment receives the video stream 22 corresponding to the array shown in Fig. 34, output from the encoder according to the ninth embodiment, performs decoding processing in units of macroblocks with the same size (4: 4: 4 format) for the three color components and restores the corresponding video frames.

Inter prediction image generation processing, which is a characteristic of a decoder according to the ninth embodiment, will be described in detail with reference to processing other than that of the seventh embodiment. This processing is carried out in units of macroblocks in which three color components are placed. The processing is carried out mainly by the variable-length decoding unit 25 and the motion compensation prediction unit 102 in the decoder shown in FIG. A logical flowchart of the processing carried out by the variable-length decoding unit 25 is shown in FIG.

The video stream 22 introduced into the variable-length decoding unit 25 is consistent with the data array shown in FIG. 34. In step S140, the variable-length decoding unit 25 decodes the inter-prediction mode common use identification flag 123 for the data shown in FIG. 34 (step S140). The variable-length decoding unit 25 further decodes the base macroblock type 128 and the sub-macroblock base type 129 (step S141). In step S142, the variable-length decoding unit 25 decides whether the inter-prediction mode is used together for C0, C1, and C2 based on the result of the identification flag 123 of the common use of the inter-prediction mode. When the inter prediction mode is used together for C0, C1, and C2, the variable-length decoding unit 25 uses the base macroblock type 128 and the sub-macroblock base type 129 for all C0, C1, and C2. Otherwise, the variable-length decoding unit 25 uses the base macroblock type 128 and the sub-macroblock base type 129 as the mode for C0. In addition, when the common mode is used for C1 and C2, the variable-length decoding unit 25 decodes the extended macroblock type 130 and the extended sub-macroblock type 131 common to components C1 and C2 (step S143). When different modes are used for C0, C1, and C2, the variable word length decoding unit 25 decodes the extended macroblock type 130 and the extended sub-macroblock type 131 for C1 and C2, respectively (steps S144, S145, and S146) to obtain mode information for C1 and C2. The variable word length decoding unit 25 decodes the identification number 132 of the basic reference image and the information of the basic motion vector 133 (step S147). When the inter-prediction mode common use identification flag 123 indicates “shared between C0, C1, and C2,” the variable-length decoding unit 25 uses the reference reference picture identification number 132 and the basic motion vector information 133 for all C0, C1, and C2. Otherwise, the variable-length decoding unit 25 uses the reference reference image identification number 132 and the basic motion vector information 133 as information for C0. Furthermore, when the common mode is used for C1 and C2, the variable word length decoding unit 25 decodes the extended reference image identification number 134 and the extended motion vector information 135 common to components C1 and C2 (step 149). When different modes are used for C0, C1, and C2, the variable word length decoding unit 25 decodes the extended reference image identification number 134 and the extended motion vector information 135 for C1 and C2, respectively (steps S150, S151 and S152). The types of macroblock 106, identification numbers of the reference image, and motion vector information for the respective color components are set in the processing steps. Thus, the variable-length decoding unit 25 outputs macroblock types 106, reference image identification numbers and motion vector information to the motion compensation prediction unit 102 to obtain predicted motion-compensated images of the respective color components.

Similarly, in the case of the bitstream data array shown in FIG. 36, when the inter-prediction mode common use identification flag 123 indicates “common to C1 and C2”, extended macroblock type 130, extended sub-macroblock type 131, extended reference identification number 134, and information 135 extended motion vectors are information common to C1 and C2. The operations of the video encoder and video decoder into which the video stream corresponding to the data array shown in FIG. 36 is input and from which the video stream is output are the same as in the case shown in FIG. 34.

According to a ninth embodiment, the macroblock type / sub-macroblock type 106, the motion vector 137, and the reference image may vary for each of the color components. The macroblock type / sub-macroblock type 106 and the reference image can also be common for the respective components, and only the motion vector 137 is common for the three components, change for each component or be common for C1 and C2 and the optimal ones are selected for C0, C1 and C2, respectively. The data stream of the bitstream in this case is consistent with Fig. 39 or Fig. 41. In this case, as in the case described above, when the inter-prediction mode common use identification flag 123 indicates “common to C1 and C2,” the extended motion vector information 135 is information common to C1 and C2.

Tenth Embodiment

According to a tenth embodiment, a method for encoding an input motion vector 137 and multiplexing a motion vector 137 into a bit stream in a variable encoder coding unit 11 described in accordance with a seventh embodiment and a method for decoding a motion vector 137 from a bit stream in decoding unit 25 with variable word length of the decoder corresponding to the encoder.

On Fig shows a structural diagram of a part of the encoding unit 11 with a variable word length of the encoder shown in Fig, which is a coding unit of the motion vector that encodes the motion vector 137.

A method for multiplexing motion vectors 137 of three color components (C0, C1, and C2) into a bit stream in the order of C0, C1, and C2 will be described.

The motion vector 137 for C0 is denoted by mv0. In motion vector prediction unit 111, a predicted vector (mvp0) of motion vector 137 for C0 is determined. As shown in FIG. 46, the motion vectors (mvA0, mvB0 and mvC0) of the block (A, B and C in FIG. 46) adjacent to the block where the motion vector to be encoded (mv0) is located are retrieved from the memory. The motion vectors 137 for A, B, and C are already multiplexed into the bitstream. The median mvA0, mvB0 and mvC0 is calculated as mvp0. The calculated predicted vector mvp0 and the motion vector mv0 to be encoded are input to the differential motion vector calculation unit 112. In block 112 for calculating the difference motion vector, a difference motion vector (mvd0) between mv0 and mvp0 is calculated. The calculated difference motion vector mvd0 is input to the variable-length differential motion vector encoding unit 113 and is subjected to entropy encoding by, for example, Huffman coding or arithmetic coding.

The motion vector (mv1) for C1 is encoded. In the motion vector prediction unit 111, a predicted vector (mvp1) of the motion vector 137 for C1 is determined. As shown in FIG. 46, the motion vectors (mvA1, mvB1 and mvC1) of a block adjacent to the block where the motion vector (mv1) to be encoded and the motion vector (mv0) for C0 are in the same position as in the block, where mv1 is located, they are extracted from memory 16. Motion vectors 137 for A, B, and C are already multiplexed into the bitstream. The median mvA1, mvB1, mvC1 and mv0 is calculated as mvp1. The calculated predicted vector mvp1 and the motion vector mv1 to be encoded are input to the difference motion vector calculation unit 112 for calculating the difference motion vector (mvd1 = mv1-mvp1) between mv1 and mvp1. The calculated difference motion vector mvd1 is input to the variable-length differential-motion vector encoding unit 113 and is subjected to entropy encoding by, for example, Huffman coding or arithmetic coding.

The motion vector (mv2) for C1 is encoded. In motion vector prediction unit 111, a predicted vector (mvp2) of motion vector 137 for C1 is determined. As shown in FIG. 46, the motion vectors (mvA2, mvB2 and mvC2) of the block adjacent to the block where the motion vector (mv2) to be encoded is located, and the motion vectors (mv1 and mv2) for C0 and C1 in the same position, as in the block where mv2 is located, they are retrieved from memory. The median mvA2, mvB2, mvC2, mv0 and mv1 is calculated as mvp2. The calculated predicted vector mvp2 and the motion vector mv2 to be encoded are input to the difference motion vector calculation unit 112 for calculating the difference motion vector (mvd2 = mv2-mvp2) between mv2 and mvp2. The calculated differential motion vector mvd2 is input to the variable-length differential-motion vector encoding unit 113 and is subjected to entropy encoding by, for example, Huffman coding or arithmetic coding.

FIG. 47 is a structural diagram of a portion of a variable-length decoding unit 25 of the encoder shown in FIG. 31, which is a motion vector decoding unit 250 that decodes a motion vector 137.

In motion vector decoding unit 250, motion vectors 137 of the three color components multiplexed onto video stream 22 are decoded in the order of C0, C1, and C2.

In block 251 for decoding a difference motion vector with a variable word length, difference motion vectors (mvd0, mvd1 and mvd2) of the three color components (C0, C1 and C2) multiplexed onto the video stream 22 are extracted and decoded with a variable word length.

In motion vector prediction unit 252, predicted vectors (mvp0, mvp1, and mvp2) of motion vectors 137 for C0, C1, and C2 are calculated. The method for calculating the predicted vectors is the same as that used in the motion vector prediction unit 111 of the encoder.

In block 253 of the calculation of the motion vector, the difference motion vectors and the predicted vectors are summed to calculate the motion vectors (mvi = mvdi + mvpi (i = 0, 1, 2)). The calculated motion vectors 137 are stored in memory 16 for use as predicted candidate vectors.

According to a tenth embodiment, when encoding and decoding motion vectors, the motion vector of the identical block of the color component adjacent to the block where the motion vector to be encoded is located and the motion vectors of other blocks of the color component in the same position as in the block where the motion vector is, to be encoded are used as predicted candidate vectors. Thus, for example, in the absence of continuity of the motion vector of the neighboring block in the same color component in the boundary region of the object, etc. motion vectors of blocks in the same position of different color components are used as predicted candidate vectors. This makes it possible to increase the efficiency of motion vector prediction and reduce the amount of motion vector codes.

Eleventh Embodiment

According to an eleventh embodiment, examples of another encoder and another decoder derived from the encoder and decoder described according to the seventh embodiment will be described. The encoder and decoder according to the eleventh embodiment make a decision, according to a predetermined control signal, whether the components C0, C1 and C2 in the macroblock are encoded in accordance with the individual pieces of header information, and multiplex the information about the control signal to the video stream 22. The encoder and decoder differ in providing means multiplexing the header information necessary for decoding the components C0, C1 and C2 to the video stream 22 according to the control signal and the effective coding of the missed ( whether the uncoded) macroblock at the time of lack of information on a motion vector that should be transmitted according to the control signal, and a conversion coefficient.

In a conventional MPEG video encoding system including AVC, the case where the encoding information to be transmitted is not available for the macroblock to be encoded is specially signalized to implement highly efficient encoding with a minimum amount of macroblock codes. For example, when trying to encode a specific macroblock, image data at a completely identical position in the reference image used for prediction with motion compensation is used as a predicted image (i.e., the motion vector is zero), and the resulting prediction error signal is converted and quantized. As a result, when all transform coefficients after quantization are equal to zero, the amplitude of the obtained prediction error signal is zero on the decoding side, even if inverse quantization is performed. There are no data conversion factors that should be transmitted to the side of the decoder. In addition, when it is assumed that the motion vector is equal to zero, it is possible to determine a special type of macroblock “zero motion vector and the absence of these conversion coefficients”. Such a macroblock is traditionally referred to as a missed macroblock or an uncoded macroblock and is adapted not to transmit unnecessary information by means of a special signaling. In AVC, the assumption of motion vectors is the condition "when the 16 × 16 prediction shown in Fig. 32 (a) is performed and when the predicted values (the predicted vectors mvp0, mvp1 and mvp2) used to encode the motion vector are equal to the actual motion vectors" . In the absence of data conversion coefficients that meet the condition and which should be transmitted, the macroblock is considered to be a missed macroblock. In traditional AVC, when encoding this skipped macroblock, any of the following two methods is selected according to the variable-length encoding system used.

Method 1: The number (word length) of missed macroblocks continuing in the slice is calculated, and the word length is encoded with a variable word length.

Method 2: A flag is encoded to indicate whether each macroblock is a missing macroblock.

The syntax of the bitstream according to the respective methods is shown in Fig. 48. Fig. 48 (a) shows a case where adaptive Huffman coding is used as a variable-length coding system (method 1). Fig. 48 (b) shows a case where adaptive arithmetic coding is used (method 2). In the case of method 1, the signaling for the missed macroblock is carried out by mb_skip_run. In the case of method 2, the signaling for the missed macroblock is carried out using mb_skip_flag. MB (n) indicates the encoded data of the nth macroblock (which is not a missing macroblock). Note that mb_skip_run and mb_skip_flag are allocated in units of macroblocks in which the components C0, C1, and C2 are assembled.

On the other hand, in the encoder and decoder according to the eleventh embodiment, a method for changing header information including a motion vector and the like is provided. for each of the components C0, C1 and C2, according to the state of the control signal, i.e. a signal equivalent to the common use identification flag 123 of the inter prediction mode described according to the seventh embodiment, and signaling the missed macroblock for each of the components C0, C1 and C2. Specific examples of bitstream syntax are shown in FIGS. 49 and 50.

The structure of the data encoded in units of macroblocks output by the encoder according to the eleventh embodiment and input to the decoder according to the eleventh embodiment is shown in FIG. 49. A detailed structure of the encoded data of the Cn component header information in FIG. 49 is shown in FIG. 50. In the following description, in order to explain the advantages of this structure of the bitstream, operations on a decoder that receives the bitstream and restores the video signal will be mainly described. An explanation of the operations of the decoder is given with reference to Fig.

The inter-prediction mode common use identification flag 123 according to the seventh embodiment is expressed as the macro-block header common use identification flag 123c by explaining its definition. The macroblock header common use identification flag 123c is a flag that expresses component C0 header information 139a as basic macroblock header information and indicates whether only component C0 header information 139a is multiplexed as header information shared between components C1 and C2 or header information 139b component C1 and header information 139c of component C2 individually are multiplexed as extended header information, respectively. The macroblock header common use identification flag 123c is extracted from the video stream 22 and is decoded by the variable-length decoding unit 25. When the flag indicates that only header information 139a of component C0 is multiplexed as header information shared between components C1 and C2, decoding that uses header information 139a of component C0 is applied to all components C0, C1 and C2 in the macroblock. When the flag indicates that the header information 139b of the component C1 and the header information 139c of the component C2 are separately multiplexed as extended header information, a decoding in which the header information fragments 139a-139c relating to the respective components C0, C1 and C2 in the macroblock is applied to the component . This point will be explained in more detail below as processing in units of macroblocks.

1.When only header information of component C0 is multiplexed

When the macroblock header common use identification flag 123c indicates that only the C0 component header information 139a is multiplexed as header information shared between components C1 and C2, macroblock decoding is applied to all components C0, C1 and C2 based on various varieties of macroblock header information included into the header information 139a of the component C0. In this case, the skip indication information 138a of the component C0 and the header information 139a of the component C0 are used together for both components C1 and C2, the skip indication information (138b and 138c) and the header information (139b and 139c) for the components C1 and C2 are not multiplexed into the bitstream .

First, the variable-length decoding unit 25 decodes and evaluates the skip indication information 138a of component C0. When the skip indication information 138a of the component C0 indicates “skip,” the variable-length decoding unit 25 considers that the header information 139a of the component C0 is not encoded, and the conversion efficiency information 142 of the conversion information of the header information of the component C0 is zero (no encoded coefficients conversion). Therefore, it is believed that the data of the transform coefficients of the components C0-C2 (140a-140c) are not encoded, and all quantized transform coefficients 10 in the macroblocks are set to zero for output. In addition, the variable-length decoding unit 25 sets the motion vectors 137 of all components C0, C1, and C2 with the same value in accordance with the definition of the missing macroblock and outputs the motion vector 137.

When the skip indication information 138a of the component C0 indicates “not to skip,” the variable-length decoding unit 25 considers that the header information 139a of the component C0 is present and decodes the header information 139a of the component C0. When the macroblock type 128b in the C0 component header information 139a indicates intra-coding, the variable-length decoding unit 25 decodes the intra-prediction mode 141, conversion coefficient efficiency / inefficiency indication information 142 and quantization parameter (if conversion coefficient efficiency / inefficiency indication information 142 is not 0) . If the conversion coefficient efficiency / ineffectiveness indication information 142 is non-zero, the variable word length decoding unit 25 decodes the conversion coefficient data of components C0-C2 (140a-140c) and outputs the conversion coefficient data of component C0 as a quantized conversion coefficient 10. When information 142 the indicator of the efficiency / inefficiency of the conversion coefficients is zero, the variable-length decoding unit 25 considers that all data of the conversion coefficients nents C0-C2 (140a-140c) are zero, and quantized all transform coefficients 10 in the macroblock are set to zero for output. When the macroblock type 128b indicates intercoding, the variable-length decoding unit 25 decodes the necessary sub-macroblock type 129b and further decodes the identification number of the reference image 132b, motion vector information 133b, conversion coefficient efficiency / inefficiency indication information 142, and quantization parameter 21 (if indication information 142 efficiency / inefficiency of conversion factors is not equal to 0). If the conversion coefficient efficiency / ineffectiveness indication information 142 is non-zero, the variable word length decoding unit 25 decodes the conversion coefficient data of components C0-C2 (140a-140c) and outputs the conversion coefficient data of component C0 as a quantized conversion coefficient 10. When information 142 indicators of the efficiency / inefficiency of the conversion coefficients is zero, the variable-length decoding unit 25 considers that all data of the conversion coefficients nents C0-C2 (140a-140c) are zero, and quantized all transform coefficients 10 in the macroblock are set to zero for output. As in the seventh embodiment, the macroblock decoding is carried out in accordance with a predetermined processing procedure using the output signal of the variable-length decoding unit 25 according to the above operation.

2. When header information is multiplexed for components C0, C1, and C2, respectively

When the macroblock header common use identification flag 123c indicates that component C1 header information 139b and component C2 header information 139c are multiplexed as extended header information separately from component C0 header information 139a, macroblock decoding is applied to each of components C0, C1 and C2 based on different varieties of macroblock header information included in the header information of component C0 (139a-139c), respectively. In this case, the skip indication information (138b and 138c) and the header information (139b and 139c) for components C1 and C2 are multiplexed into the bitstream.

First, the variable-length decoding unit 25 decodes and evaluates the skip indication information 138a of component C0. When the skip indication information 138a of the component C0 indicates “skip,” the variable-length decoding unit 25 considers that the header information 139a of the component C0 is not encoded, and the conversion efficiency information 142 of the conversion information of the header information of the component C0 is zero (no encoded coefficients conversion). Therefore, it is considered that the conversion coefficient data 140a of the component C0 is not encoded, and all quantized conversion coefficients in the components C0 are set to zero (i.e., the ratio between the skip indication information 138a of the component C0 and the conversion coefficient efficiency / inefficiency indication information 142 changes according to the flag value Macroblock header common use identification 123c). In addition, the variable-length decoding unit 25 sets the motion vector 137 of the component C0 as defined in the case of missing the component C0 and outputs the motion vector 137.

When the skip indication information 138a of the component C0 indicates “not to skip,” the variable-length decoding unit 25 considers that the header information 139a of the component C0 is present and decodes the header information 139a of the component C0. When the macroblock type 128b in the header information 139a of the component C0 indicates intra-coding, the variable-length decoding unit 25 decodes the intra-prediction mode 141 (the spatial prediction mode in which the pixel closest to the pixel of the prediction object in the frame is used as the predicted value), indication information 142 the efficiency / inefficiency of the conversion coefficients and quantization parameter 21 (if the information 142 indicating the efficiency / inefficiency of the conversion coefficients is not p VNA 0). If the conversion coefficient efficiency / ineffectiveness indication information 142 is non-zero, the variable-length decoding unit 25 decodes the C0 component conversion coefficient data and outputs the C0 component conversion coefficient data as a quantized conversion coefficient 10. When the conversion coefficient efficiency / inefficiency indication information is zero , the variable-length decoding unit 25 considers that all the data of the transform coefficients of the component C0 p Avna to zero. When the macroblock type indicates intercoding, the variable-length decoding unit 25 decodes the necessary sub-macroblock type and further decodes the reference image identification number, motion vector information, conversion coefficient efficiency / inefficiency indication information and quantization parameter (if the conversion coefficient efficiency / inefficiency indication information is not equal 0). If the conversion coefficient efficiency / ineffectiveness indication information is non-zero, the variable word length decoding unit 25 decodes the conversion coefficient data of the C0 component and outputs the conversion coefficient data of the C0 component as a quantized conversion coefficient 10. When the conversion coefficient efficiency / inefficiency information is zero, the block 25, variable-length decoding considers that all data of the transform coefficients of component C0 are equal to zero. The variable word length decoding unit 25 carries out the processing procedure for C1 and C2 in the same manner.

As in the seventh embodiment, the decoding of the respective components C0, C1 and C2 in the macroblock is carried out in accordance with a predetermined processing procedure using the output signal of the variable-length decoding unit 25 according to the above operations.

Operations on the decoder side are mainly described above. Due to this formation of the bitstream, the following effects are obtained. First of all, in the traditional AVC there is only one set of used header information (Fig. 50) per macroblock. It is necessary to jointly implement the intra / inter decision for all C0-C2 components and encode in accordance with this header information. When a signal component equivalent to the luminance signal that transmits the contents of the image signal is equivalently included in three color components, such as in a 4: 4: 4 format, fluctuation of signal characteristics may occur due to the nature of the inclusion of noise, etc. in the input video signals to the respective components. It is not always optimal to code all components of C0-C2 together. Provided that the bitstream structures shown in Figs. 49 and 50 according to the eleventh embodiment are specified, the encoder can select the optimal coding mode (macroblock type including intra / intercoding types), the optimal motion vector, and the macroblock type identification flag 123c; etc., corresponding to the characteristic of the signal, and to encode for each of the components C0-C2 and can increase the encoding efficiency. Traditionally, since the coding is carried out in units of macroblocks in which all the components of C0-C2 are assembled, a decision is made to skip the macroblock, provided that there is no coding information for all components. However, according to the eleventh embodiment, since it is possible to decide on the presence or absence of coding information for each component according to the skip indication information 138 when only a specific component is missing and the other components are not missing, there is no need to decide that none component is not skipped. This allows for more efficient allocation of the amount of codes. In the encoder, the value of the skip indication information 138 is determined by the variable word length encoding unit 11 based on the data 10 of the quantized transform coefficient, the motion vector 137, the reference image identification number 132b and the macroblock type / sub-macroblock type 106 in accordance with the definition of the skipped macroblock uniformly specified in the encoder and decoder.

The structure of the bit stream processed by the encoder and decoder according to the eleventh embodiment may be as shown in FIG. In this example, skip indication information (138), header information (139a-139c), and transform coefficient data (140a-140c) of the respective components C0, C1, and C2 are co-located respectively. In this case, in the skip indication information, the corresponding states C0, C1, and C2 can be placed in 1-bit code symbols or eight states can be co-located in one code symbol. When the correlation with the skip state is high among color components, it is possible to improve the coding efficiency of the skip indication information 138 by collecting code characters to appropriately define contextual arithmetic coding models (described below in the twelfth embodiment).

The macroblock header common use identification flag 123c may be multiplexed into the bitstream in arbitrary level data units, such as a macroblock, slice, image, or sequence. If there is a constant difference in signal characteristics between the color components in the input signal, if the macroblock header common use identification flag 123c is multiplexed in sequence units, efficient coding with less overhead can be performed. If the macroblock header common use identification flag 123c is multiplexed at the image level, an effect can be expected, for example, to improve the balance of coding efficiency and arithmetic operations using a common header in an I image with a small number of macroblock type variations, and using a separate header for each of color components in images P and B with a large number of variations of the macroblock type. Furthermore, it can be said that a change in the image level is also desirable with respect to coding control for a video signal whose characteristic changes for each image, for example, a scene change. When the macroblock header common use identification flag 123c is multiplexed in units of macroblocks, the amount of codes per macroblock increases. On the other hand, it is possible to control whether the header information is shared based on the states of the respective color components of the signal in units of macroblocks. This allows the encoder to be built more satisfactorily, which improves compression efficiency by following local fluctuations in the image signal.

The following method is possible. When the encoding type equivalent to the image type changes at the slice level, as in AVC, the macroblock header common use identification flag 123c is multiplexed for each slice. When the flag indicates “common to C0, C1, and C2,” a bitstream is formed so that the slice includes all pieces of coding information for the three color components. When the flag indicates “not common to C0, C1, and C2,” a bitstream is formed such that one slice includes information about one color component. The state of this method is shown in FIG. 52, the meaning of the slice configuration identification information indicating that “the current slice includes all pieces of encoding information for the three color components” or “the current slice includes encoding information for the specific color component” is given to the header common use identification flag 123c macroblock. It goes without saying that such slicing configuration identification information can be prepared individually from the macroblock header common use identification flag 123c. When a slice is identified as “the current slice includes coding information for a particular color component,” the identification includes an identification indicating “which of the color components is C0, C1, and C2.” When, therefore, a decision is made whether a single macroblock header is used together for components C0, C1 and C2 (mixed slice C0, C1 and C2), or if the macroblock header individually is multiplexed for each of components C0, C1 and C2 (slice C0, slice C1 and slice C2) in units of slices, if these two types of slices are mixed in one image, slice C0, slice C1 and slice C2 are always multiplexed into the bitstream in the set in the form of data obtained by encoding macroblocks at the same position on the screen. In other words, the value of first_mb_in_slice included in the slice header and indicating the position in the image of the front slice macroblock always takes the same value in the same set of slice C0, slice C1 and slice C2. The numbers of macroblocks included in the slice set C0, slice C1 and slice C2 are the same. This state is shown in FIG. Providing such a restriction on the structure of the bitstream, the encoder can encode the bitstream by adaptively choosing the encoding method that has the highest encoding efficiency between the mixed slice C0, C1 and C2 and the set of slice C0, slice C1 and slice C2 according to the characteristic of the local signal in the image. A decoder can receive a bitstream effectively encoded in this way and play a video signal. For example, if bitstream 22, input to the decoder shown in FIG. 31, has this configuration, variable-length decoding unit 25 decodes section identification information of slices from the bitstream each time slice data is input and determines which of the slices shown in Fig. 52, it is necessary to decode. When it is decided based on the identification information of the slicing configuration that the encoded data is in the form of a slice set C0, a slice C1, and a slice C2, the variable-length decoding unit 25 only needs to perform a decoding operation, establishing that the state of the identification flag 123 of the common use of the inter prediction mode (or macroblock header common use identification flag 123c) is “use separate inter prediction modes or (macroblock header) in C0, C1 and C2”. Since it is guaranteed that the first_mb_in_slice value of each slice and the number of macroblocks in the slice are the same, decoding processing can be performed without causing overlap and clearance in the mixed slice C0, C1 and C2 and the image based on the value.

When the signal characteristics of the respective slices C0, C1, and C2 are significantly different, in order to avoid a reduction in coding efficiency by providing such a limitation, identification information can be provided to select at the image level or sequence level whether mixing of a slice having a different configuration identification information value can be allowed slices in the image.

Twelfth Embodiment

According to a twelfth embodiment, examples of another encoder and another decoder derived from the encoder and decoder described according to the eleventh embodiment will be described. The encoder and decoder according to the twelfth embodiment are distinguished by an adaptive setting, when encoding the corresponding components C0, C1 and C2 in a macroblock using an adaptive arithmetic coding system, are the probability of occurrence of a character used for arithmetic coding and the study process used for the probability of occurrence of a character shared by all by components or separately by each of the components according to indication information multiplexed into a bitstream.

According to the twelfth embodiment, in the encoder, only the processing in the variable-length encoding unit 11 shown in FIG. 30 is different from the processing according to the eleventh embodiment. In the decoder, only the processing in the variable-length decoding unit 25 in FIG. 31 is different from the processing according to the eleventh embodiment. Other operations are identical to those of the eleventh embodiment. In the following description, arithmetic encoding and decoding processing, which are the hallmarks of the twelfth embodiment, will be explained in detail.

1. Coding processing

An internal structure related to arithmetic encoding processing in the variable-length encoding unit 11 is shown in FIG. 54. The flow of arithmetic coding processing is shown in FIGS. 55 and 56.

The variable-length encoding unit 11 according to the twelfth embodiment includes a contextual model determination unit 11a that sets contextual models (described below) defined for respective data types, for example, motion vector 137 serving as encoding object data, reference identification number 132b image, macroblock type / submacroblock type 106, intra prediction mode 141 and quantized transform coefficient 10, binary conversion unit 11b, which converts the multi-valued data to binary data in accordance with the binary conversion rules established for the respective data types of the encoding object, an occurrence probability generating unit 11c that gives the probability of occurrence of values (0 or 1) of the respective sections after the binary conversion, the encoding unit 11d, which performs arithmetic coding based on the generated occurrence probabilities, and a memory 11g in which occurrence probability information is stored. The contextual mode determination unit 11a receives various data inputted into the variable-length encoding unit 11 as the data of the encoding object, for example, motion vector 137, reference image identification number 132b, macroblock type / sub-macroblock type 106, intra-prediction mode 141, and a quantized coefficient transformations 10. The output signals of the coding unit 11d are equivalent to the information associated with the macroblock of the video stream 22.

(1) Contextual Model Definition Processing (Step S160 in FIG. 55)

A contextual model is a causal relationship model of the probability of occurrence of an information source symbol with other information that leads to fluctuations in the probability of occurrence. It is possible to carry out coding more adapted to the actual probability of occurrence of a symbol by changing the state of the probability of occurrence in accordance with this causal relationship. The concept of a contextual model (ctx) is shown in Fig. 57. Although, according to FIG. 57, the symbol of the information source is binary, the symbol of the information source may be ambiguous. Options from 0 to 2 for ctx in Fig. 57 are determined on the basis of the assumption that the probability state of the appearance of the symbol of the information source that uses this ctx changes according to the situation. When encoding a video signal according to the twelfth embodiment, the ctx value changes according to a causal relationship between the encoded data in a particular macroblock and the encoded data of the macroblocks around the macroblock. An example of a contextual model related to the macroblock motion vector disclosed in D. Marpe et al. "Video Compression Using Context-Based Adaptive Arithmetic Coding", International Conference on Image Processing 2001, is shown in Fig. 58. According to Fig. 58, the motion vector of block C is an encoding object (more precisely, the predicted difference value is encoded

mvd _k (C) obtained by predicting the motion vector of block C from its surroundings), and ctx_mvd (C, k) indicates the contextual model. mvd _k (A) indicates the predicted value of the motion vector difference in block A, and mvd _k (B) indicates the predicted value of the motion vector difference in block B. The values mvd _k (A) and mvd _k (B) are used to determine the estimated value of e _k (C) changes to the contextual model. The estimated value of e _k (C) indicates the degree of fluctuation of the motion vector in the environment. In general, when this fluctuation is small, mvd _k (C) is small. In contrast, when e _k (C) is large, mvd _k (C) also increases. Thus, it is desirable to adapt the probability of occurrence of a symbol for mvd _k (C) based on e _k (C). A set of variations of this probability of occurrence is a contextual model. In this case, we can say that there are three varieties of variations in the probability of occurrence.

In addition, contextual models are predetermined for coding object data, for example, macroblock type / submacroblock type 106, intra-prediction mode 141, and quantized transform coefficient 10, respectively, and are shared between the encoder and decoder. The contextual model determination unit 11a performs processing for selecting a model predetermined based on the type of such data of the encoding object (a decision as to which variation in the probability of occurrence among the contextual models corresponds to the processing for generating the probability of occurrence in the following paragraph (3)).

(2) Binary Conversion Processing (Step S161 in FIG. 55)

The data of the encoding object is converted into a binary sequence by the binary conversion unit 11b. Contextual models are set according to the corresponding sections (binary positions) of the binary sequence. According to the rule for converting to binary, the data of the encoding object is converted into a binary sequence with a variable word length in accordance with a rough distribution of values that the corresponding binary data can take. Conversion to binary has the advantage of, for example, reducing the number of divisions of the numerical probability scale by encoding the data of the encoding object, which can initially take multiple values, in units of sections instead of directly arithmetic encoding the data of the encoding object and simplifying the arithmetic operation, which makes it possible to reduce contextual models.

(3) Appearance probability generation processing (step S162 in FIG. 55 (details of step S162 are shown in FIG. 56))

In the processes described in paragraphs (1) and (2) above, the binary conversion of the multi-valued data of the encoding object and the setting of contextual models applied to the corresponding sections is completed, and the preparation for encoding is completed. Then, the occurrence probability generating unit 11c carries out generation processing for the occurrence probability state used for arithmetic coding. Since variations in the probability of occurrence for the corresponding values 0/1 are included in the respective contextual models shown in FIG. 54, the probability of occurrence generation unit 11c performs processing with reference to the contextual model 11f determined in step S160. An occurrence probability generating unit 11c sets an estimated value for selecting an occurrence probability indicated

e _k (C) in FIG. 58, and determines, according to the estimated value, which variation in the probability of occurrence is used for a given coding from the options to which contextual models belong (step S162a in FIG. 56). The variable word length encoding unit 11 according to the twelfth embodiment includes a memory for storing occurrence probability information 11g and includes a mechanism for storing the occurrence probability state 11h, which is sequentially updated during the encoding, for each of the color components. The occurrence probability generation unit 11c selects according to the value of the common use identification flag 143 of the occurrence probability state parameter whether the occurrence probability state 11h used for the coding is selected from the occurrence probability states supported for each of the color components C0-C2, or the occurrence probability state for component C0 is shared by components C1 and C2, and determines the occurrence probability state 11h actually used for encoding (steps S162b-S162d on ig.56).

It is necessary to multiplex the flag 143 identifying the common use of the state parameter of the probability of occurrence in the bit stream so that the decoder can make the same choice. In this configuration, the following effects are realized. For example, in the case of FIG. 58, when the macroblock header common use identification flag 123c indicates that the component C0 header information 139a is used for other components, if the macroblock type 128b indicates a 16 × 16 prediction mode, only one e is set for one macroblock _k (C) according to FIG. In this case, the state of the probability of occurrence prepared for the component C0 is always used. On the other hand, when the macroblock header common use identification flag 123c indicates that the header information (139a-139c) corresponding to the respective components is used, if the macroblock type 128b indicates the 16 × 16 prediction mode in all C0, C1 and C2, there may be three variations e _k (C) according to Fig for one macroblock. The coding unit 11d at a later stage may take two options, i.e. jointly, the appearance probability state 11b prepared for the components C0 is used and updated for the corresponding variations, or the appearance probability states 11h prepared for the corresponding color components are individually used and updated. In the first embodiment, when the corresponding components C0, C1, and C2 have essentially the same distribution of motion vectors, the multiplicity of the study increases due to the use and updating of the general occurrence probability state 11h. This makes it possible to more satisfactorily study the probability of the appearance of a motion vector. In the latter case, on the contrary, when the corresponding components C0, C1, and C2 have different distributions of the motion vectors, it is possible to reduce inconsistencies due to the study of the probability of occurrence by using and updating state 11h separately. This makes it possible to more satisfactorily study the probability of the appearance of a motion vector. Since the video signal is non-stationary, when such adaptive control is possible, it is possible to increase the efficiency of arithmetic coding.

(4) Coding processing

Since the probabilities of occurrence of the corresponding values 0/1 on the numerical probability scale necessary for the arithmetic coding process are obtained according to (3), the coding unit 11d performs arithmetic coding in accordance with the process described in the traditional example (step S163 in Fig. 55). The actual encoded value (0 or 1) 11e is fed back to the occurrence probability generation unit 11c. An occurrence probability generating unit 11c counts occurrence frequencies 0/1 to update the occurrence probability state 11h used (step S164). For example, it is assumed that at the moment when coding processing for 100 sections is performed using a specific occurrence probability state 11h, the occurrence probability 0/1 with variation in the occurrence probability is 0.25 and 0.75. When 1 is encoded using the same variation in the probability of occurrence, the frequency of occurrence of 1 is updated, and the probability of occurrence of 0/1 is changed to 0.247 and 0.752. This mechanism allows efficient coding adapted to the actual occurrence probabilities. The encoded value 11e changes to the output of the variable-length encoding unit 11 and is output from the encoder as a video stream 22.

An internal structure associated with arithmetic decoding processing in the variable-length decoding unit 25 is shown in FIG. The sequence of operations for processing arithmetic decoding is shown in Fig. 60.

The variable-length word decoding unit 25 according to the twelfth embodiment includes a contextual model determining unit 11a that indicates types of corresponding decoding object data, for example, motion vector 137, reference picture identification number 132b, macroblock type / sub-macroblock type 106, intra prediction mode 141, and the quantized transform coefficient 10, and sets the contextual models specified jointly for the encoder for the respective types, the block-to-binary conversion unit 11b to a new kind that generates binary conversion rules established based on the data types of the decoding object, an occurrence probability generating unit 11c that gives the probability of occurrence of corresponding sections (0 or 1) in accordance with the binary conversion rules and contextual models, the decoding unit 25a, which performs arithmetic decoding based on the generated probability of occurrence and decodes data, for example, motion vector 137, reference identification number 132b image, macroblock type / submacroblock type 106, intra-prediction mode 141 and quantized transform coefficient 10 according to the binary sequence obtained by arithmetic decoding, and the rules for converting to binary, and a memory 11g in which occurrence probability information is stored. Components 11a-11c and 11g are identical to the internal components of the variable-length encoding unit 11 shown in FIG. 54.

(5) Processing the definition of the contextual model, processing the conversion to binary, and processing the generation of the probability of occurrence

These processes correspond to processes (1) - (3) on the encoder side. Although not shown in the figures, the flag 143 identifying the common use of the state parameter of the probability of occurrence is pre-extracted from the video stream 22.

(6) Arithmetic decoding processing

Since the probability of occurrence of a portion to be decoded is specified in the processes before (6), the decoding unit 25a decodes the value of the portion in accordance with the predetermined arithmetic decoding processing (step S166 in FIG. 60). The reconstructed portion value 25b is fed back to the occurrence probability generating unit 11c. An occurrence probability generating unit 11c counts occurrence frequencies 0/1 to update the occurrence probability state 11h used (step S164). The decoding unit 25a checks each time, when setting the restored value of each section, the matching of the restored value and the binary sequence patterns specified by the binary conversion rules, and outputs the data value indicated by the matched pattern as the value of the decoding data (step S167). Until decoding data is set, decoding unit 25a returns to step S166 and continues decoding processing.

According to an encoder and a decoder including arithmetic encoding processing and arithmetic decoding processing according to the above configurations, more efficient encoding can be performed when the encoded information for each of the color components is adaptively arithmetically encoded according to the macroblock header common use identification flag 123c.

Although not specifically shown in the figures, the unit for multiplexing the flag 143 for identifying the general use of the appearance probability state parameter may be a macroblock unit, a slice unit, an image unit, or a sequence unit. When it is possible to ensure sufficient coding efficiency with a change at a higher level equal to or higher than the slice level, by multiplexing the flag 143 identifying the general use of the probability of occurrence state parameter as a flag located at a higher data level, for example, at the slice level, image or sequence, it is possible to reduce bits of overhead information without multiplexing the flag 143 identifying the common use of the state parameter of the probability of occurrence on exactly macroblocks each time processing is performed.

The occurrence probability state parameter common use identification flag 143 may be information set inside the decoder based on corresponding information included in the bitstream separate from the occurrence probability state parameter common use identification flag 143.

According to a twelfth embodiment, the arithmetic coding of the macroblock header common use identification flag 123c in macroblock units, the model shown in FIG. 61 is used for context model 11f. In FIG. 61, the value of the macroblock header common use identification flag 123c in macroblock X is indicated by IDC _X. When carrying out a coding flag 123c common-use identification header of the macroblock in the macroblock C macro-blocks take the following three states based on the value IDC _A flag 123c common-use identification macroblock header for the macroblock A and a value IDC _B flag 123c common-use identification macroblock header for a macroblock B according to equation to figure.

Value 0: A and B are in the "use a common macroblock header for C0, C1 and C2" mode.

Value 1: one of A and B is in the "use common macroblock header for C0, C1 and C2" mode, and the other is in the "use separate macroblock header for C0, C1 and C2" mode.

Value 2: A and B are in the "use separate macroblock headers for C0, C1 and C2" mode.

The encoding of the macroblock header common use identification flag 123c in this way allows arithmetic coding according to the encoding state of the macroblocks in the environment and improves coding efficiency. From the explanation of the operations of the decoder according to the twelfth embodiment, it is obvious that contextual models are defined by the same procedure on the encoding side and on the decoding side to perform arithmetic decoding.

According to a twelfth embodiment, with respect to the header information shown in FIG. 50 included in the macroblock header (macroblock type, sub-macroblock type, intra-prediction mode, reference image identification number, motion vector, conversion coefficient efficiency / inefficiency indication information and quantization parameter), arithmetic coding is carried out in contextual models defined for the corresponding types of information. As shown in FIG. 62, all contextual models are defined for the current macroblock C with reference to the corresponding information about macroblocks A and B. Here, as shown in FIG. 62 (a), when the macroblock C is in the “use a common macroblock header for C0, C1 and C2 ", and macroblock B is in the" use separate macroblock headers for C0, C1 and C2 "mode, information about a specific color component among C0, C1 and C2 is used as reference information when defining contextual models.

For example, a selection method may be adopted when C0, C1, and C2 correspond to the color components R, G, and B, with component G having the component closest to the luminance signal traditionally used for encoding as a signal that well represents the image structure. The fact is that even in the "use a common macroblock header for C0, C1 and C2" mode, macroblock header information is often set based on component G for encoding.

On the other hand, otherwise, as shown in FIG. 62 (b), when the macroblock C is in the "use separate macroblock headers for C0, C1 and C2" mode, and the macroblock B is in the "use the common macroblock header for C0 , C1 and C2 ", it is necessary to encode and decode the header information about the three color components in macroblock C. In this case, the header information about the corresponding color components is used as reference information when defining contextual models. As for macroblock B, header information common to the three components is used as a value that is the same for the three components. Although this is obvious when the macroblock header common use identification flag 123c indicates the same value for all macroblocks A, B and C, pieces of reference information corresponding to the macroblocks are always present. Thus, fragments of reference information are used.

From the explanation of the operations of the decoder according to the twelfth embodiment, it is obvious that contextual models are defined by the same procedure on the encoding side and on the decoding side to perform arithmetic decoding. After determining which component information the contextual model to be used belongs to, updating the appearance probability state associated with the contextual model is performed based on the state of the identification flag 143 of the general use of the appearance probability state parameter.

According to a twelfth embodiment, arithmetic coding corresponding to probability distributions of occurrence for the corresponding data of the encoding object is also performed for the corresponding data of transform coefficients of components C0, C1, and C2. As this data, encoded data for the three components is always included in the bitstream, regardless of whether the macroblock header is shared. According to a twelfth embodiment, since intra-prediction and inter-prediction are performed on the color spaces of the encoded input signal and a prediction difference signal is obtained, it is believed that the distribution of the transform coefficient data obtained by the integer transform of the prediction difference signal is the same probability distribution of occurrence regardless of the peripheral state, for example, is the macroblock header shown in FIG. 62 used spot. Thus, according to the twelfth embodiment, a common contextual model is defined and used for encoding and decoding, regardless of whether the macroblock header is shared between the respective components C0, C1, and C2.

From the explanation of the operations of the decoder according to the twelfth embodiment, it is obvious that contextual models are defined by the same procedure on the encoding side and on the decoding side to perform arithmetic decoding. After determining which component information the contextual model to be used belongs to, updating the appearance probability state associated with the contextual model is performed based on the state of the identification flag 143 of the general use of the appearance probability state parameter.

Thirteenth Embodiment

According to a thirteenth embodiment, embodiments of another encoder and another decoder derived from the encoder and decoder described according to the seventh through twelfth embodiments will be described. The encoder and decoder according to the thirteenth embodiment are characterized in that the encoder that performs color space conversion processing on the input stage of the encoder described according to the seventh through twelfth embodiments converts the color spaces of the video signal input to the encoder after converting the image to arbitrary color spaces suitable for encoding, and multiplexes into the bitstream information indicating inverse transform processing to return color spaces to color spaces at the time of converting the image on the decoding side, and by means of a configuration for extracting information indicating inverse transform processing from the bitstream, obtains a decoded image using the decoder described in the seventh through twelfth embodiments, and then performs the inverse spatial transform based on information indicating inverse transformation processing.

The structures of the encoder and decoder according to the thirteenth embodiment are shown in FIG. An encoder and a decoder according to a thirteenth embodiment will be explained with reference to FIG.

The encoder according to the thirteenth embodiment includes, in addition to the encoder 303 according to the seventh to twelfth embodiments, a color space conversion unit 301 at a preliminary stage of the encoder 303. The color space conversion unit 301 includes one or a plurality of varieties of color space conversion processing. The color space conversion unit 301 selects a color space conversion processing to be used according to characteristics of an input video signal, system settings, and the like. for performing color space conversion processing on the input video signal, and transmits the converted video signal 302 resulting from the color space conversion processing to the encoder 303. At the same time, the color space conversion unit 301 outputs information for identifying the color space conversion processing used to the encoder 303 as identification information 304 color space conversion method. The encoder 303 multiplexes the color space conversion method identification information 304 into a bitstream 305, in which the converted video signal 302 is compressed encoded according to the method described in the seventh to twelfth embodiments as an encoding object signal, and transmits the color space conversion method identification information 304 by a transmission line or outputs information 304 identifying a method for converting a color space to a recording device that implements Records to the recording medium.

As a prepared method for color space conversion, for example, there are transformations, for example, a conversion from RGB to YUV, traditionally used as a standard

,

,

prediction between color components

C0 = G '= G

C1 = B '= B f (G) (f (G): result of filtering processing for component G)

C2 = R '= R f (G) and

conversion from RGB to YCoGg

C0 = Y = R / 2 + G / 2 + B / 4

C1 = Co = R / 2 B / 2

C2 = Cg = -R / 4 + G / 2 B / 4.

There is no need to limit the input of the color space conversion unit 301 to an RGB signal. Transformation processing is not limited to the above three types of processing.

The decoder according to the thirteenth embodiment includes, in addition to the decoder 306 according to the seventh to twelfth embodiments, a color space inverse transform unit 308 at the last stage of the decoder 306. The decoder 306 receives the bit stream 305 and extracts color space conversion method identification information 304 from the bit stream 305 and outputs identification information 304 of a color space conversion method. In addition, the decoder 306 outputs the decoded image 307 obtained through the operations of the decoder described according to the seventh through twelfth embodiments. The color space inverse transform unit 308 includes an inverse transform processing corresponding to a corresponding color space transform method selected by the color space transform unit 301. The color space inverse transform unit 308 performs processing for indicating the conversion performed by the color space transform unit 301 based on the color space conversion method identification information 304 output by the decoder 306, applying the inverse transform processing to the decoded image 307, and returning the decoded image 307 to the color spaces of the video signal inserted into the encoder according to the thirteenth embodiment.

According to an encoder and a decoder according to a thirteenth embodiment, the optimal conversion processing for color spaces is applied to a video signal to be encoded at a preliminary coding stage and a final decoding processing stage to remove a correlation included in an image signal including three color components before encoding. Thus, it is possible to carry out coding in a state of reduced redundancy and increase compression efficiency. In a conventional standard coding system, for example MPEG, the color spaces of a signal to be encoded are limited only to YUV. However, since the encoder and decoder include a color space conversion unit 301 and a color space inverse conversion unit 308 and color space conversion method identification information 304 is included in the bitstream 305, the color space limitation of the video signal input for encoding can be eliminated. In addition, you can encode a video signal using the optimal conversion selected from a set of varieties of means for removing correlation between color components.

A thirteenth embodiment is described provided that the color space transform unit 301 and the color space inverse transform unit 308 are always activated. However, without activating these processing units, it is also possible to adopt a configuration for encoding at a higher level, for example, a sequence of information indicating compatibility with the traditional standard.

It is also possible to construct a color space transforming unit 301 and a color space inverse transforming unit 308 according to a thirteenth embodiment in an encoder and a decoder according to the seventh to twelfth embodiments for performing color space conversion at a prediction difference signal level. The encoder and decoder thus constructed are shown in FIG. 64 and FIG. 65, respectively. In the encoder shown in FIG. 64, a transform block 310 is provided instead of the transform block 8, and an inverse transform block 312 is provided instead of the inverse transform block 132. In the decoder shown in FIG. 65, an inverse transform block 312 is provided instead of the inverse transform block 13.

First of all, as indicated by the processing of the color space conversion unit 301, the conversion unit 310 selects the optimal conversion processing from the set of color space conversion processing types and performs color space conversion on the prediction difference signal 4 of the components C0, C1 and C2 output from the adoption unit 5 decisions regarding coding mode. After that, the conversion unit 310 performs a conversion equivalent to that of the conversion unit 8 to the result of the color space conversion. The conversion unit 310 transmits identification information of the color space conversion method 311 indicating which conversion is selected to the variable word length encoding unit 11, multiplexes the identification information of the method for converting the color space 311 to a bitstream and outputs the bitstream as a video stream 22. The inverse transformation unit 312 performs an inverse transformation equivalent to that performed by the inverse orthogonal transform unit 13, and then performs inverse processing th color space transform processing using color space transform, indicated by the identification information 311 color space transform method.

At the decoder, the variable-length decoding unit 25 retrieves the color space conversion method identification information 311 from the bitstream and transmits the extraction result to the inverse transform unit 312 to perform the same processing as the processing of the inverse transform unit 312 in the encoder. With this configuration, when it is possible to sufficiently remove, in the area of the predicted difference, the remaining correlation between the color components, you can perform the removal as part of the encoding processing. Thus, an increase in coding efficiency is achieved. However, when separate macroblock headers are used for components C0, C1 and C2, first of all, the prediction method changes for each of the components, for example, intra-prediction for component C0 and inter-prediction for component C1. Thus, correlation may be less easily maintained in the region of the prediction differential signal 4. Thus, when separate macroblock headers are used for components C0, C1, and C2, the transform unit 310 and the inverse transform unit 312 may be instructed not to perform color space conversion. An indication of whether color space conversion is performed in the region of the prediction differential signal 4 may be multiplexed into the bitstream as identification information. Color space conversion method identification information 311 may be changed in units of sequence, image, slice, and macroblock.

In the structures of the encoder and decoder shown in FIGS. 64 and 65, the corresponding data of the transform coefficients of the components C0, C1 and C2 have different signal definition areas for the signal of the encoding object according to the identification information of the color space conversion method identification 311. Thus, it is believed that in the general case, the distribution of the data of the conversion coefficients is another probability distribution of occurrence according to the identification information 311 of the color space conversion method. Thus, when the encoder and decoder are constructed, as shown in FIGS. 64 and 65, the encoder and decoder perform encoding and decoding using contextual models that are associated with a separate occurrence probability state for each of the components C0, C1 and C2 and for each of states of information 311 identifying a color space conversion method.

From the explanation of the operations of the decoder according to the twelfth embodiment, it is obvious that contextual models are defined by the same procedure on the encoding side and on the decoding side to perform arithmetic decoding. After determining which component information the contextual model to be used belongs to, updating the appearance probability state associated with the contextual model is performed based on the state of the identification flag 143 of the general use of the appearance probability state parameter.

Fourteenth Embodiment

According to the fourteenth embodiment, more specific device structures regarding the encoder and decoder described in the embodiments will be described.

According to embodiments, the operations of the encoder and decoder are explained using drawings based on, for example, FIGS. 1, 2, 30, and 31. These drawings explain the operations of jointly inputting an input video signal including three color components into an encoder, implementing in an encoding encoder and at the same time choosing whether the three color components are encoded based on the general prediction mode or macroblock header, or encoded based on individual prediction modes or macroblock headers, bit-wise input the eye obtained by encoding to the decoder, and performing decoding processing in the decoder and simultaneously selecting based on a flag (for example, an intra prediction mode identification flag 23 or an inter prediction mode identification flag 123) decoded or extracted from a bitstream whether the three color components based on the prediction mode or the macroblock header are encoded based on the individual prediction modes or headers macroblock for receiving the reproduced video signal. It has been previously described in detail that a flag can be encoded and decoded in arbitrary level data units, for example, a macroblock, slice, image, or sequence. According to a fourteenth embodiment of the present invention, in particular, the structure and principle of operation of an apparatus for performing encoding and decoding while changing the encoding of three color component signals by means of a common macroblock header and encoding of three color component signals by separate macroblock headers in units of one frame (or one field ) will be explained based on the specific drawings. In the following explanation, unless expressly stated otherwise, the description "one frame" refers to a data block of one frame or one field.

It is assumed that the macroblock header according to the fourteenth embodiment includes: a transform block size identification flag shown in FIG. 15; coding and prediction mode information shown in FIG. 50, for example, macroblock type, submacroblock type, and intra-prediction mode; motion prediction information, for example, a reference image identification number and a motion vector; information indicating the effectiveness / inefficiency of conversion factors; and macroblock overhead information other than the transform coefficient data, for example, a quantization parameter for the transform coefficient.

In the following explanation, coding processing of three color component signals of one frame with a common macroblock header is referred to as “common coding processing”, and coding processing of three color component signals of one frame with separate macroblock headers is referred to as “independent coding processing”. Similarly, decoding processing of image data of a frame from a bit stream in which three color component signals of one frame are encoded by a common macroblock header is referred to as “general decoding processing” and decoding processing of image data of a frame from a bit stream in which three signals of color components of one frame encoded by separate independent macroblock headers, referred to as “independent decoding processing”. In the general encoding processing according to the fourteenth embodiment, as shown in FIG. 66, the input video signal for one frame is divided into macroblocks in a group of three color components. On the other hand, in independent coding processing, as shown in FIG. 67, the input video signal for one frame is divided into three color components, and the three color components are divided into macroblocks consisting of single color components. Thus, the corresponding macroblocks are subject to independent coding processing for the respective component C0, component C1, and component C2. Macroblocks subject to general coding processing include samples of three color components from C0, C1, and C2. Macroblocks subject to independent coding processing include samples of any of the components C0, C1, and C2.

On Fig shows a diagram explaining the relationship of the reference motion prediction in the time dimension between the images in the encoder and the decoder according to the fourteenth embodiment. In this example, a data block indicated by a thick vertical line is defined as an image and the relationship between the image and the access unit is indicated by the surrounding dashed line. In the case of general encoding and decoding processing, one image is data representing a video signal for one frame in which three color components are mixed. In the case of independent processing of encoding and decoding, one image is a video signal for one frame of any of the color components. An access unit is a minimal data unit that provides a time stamp for synchronization with audio / audio information, etc. for video signal. In the case of general encoding and decoding processing, data for one image is included in one access unit (427a in FIG. 68). On the other hand, in the case of independent encoding and decoding processing, three images are included in one access unit (427b in FIG. 68). The fact is that in the case of independent encoding and decoding processing, the playback video signal for one frame does not work until the images are collected at the same display time for all three color components. The numbers indicated above the corresponding images indicate the encoding and decoding processing order in the time dimension of the image (frame_num for AVC). 68, arrows between images indicate a reference direction of motion prediction. In the case of independent encoding and decoding processing, reference motion prediction between images included in an identical access unit and motion reference prediction between different color components is not performed. Images of the respective color components C0, C1, and C2 are encoded and decoded with prediction and motion support only for signals of identical color components. With this structure, in the case of independent encoding and decoding processing according to the fourteenth embodiment, it is possible to encode and decode the corresponding color components, not at all relying on the encoding and decoding processing of other color components. Thus, it is easy to carry out parallel processing.

An AVR defines an IDR image (instant decoder update), which itself performs intra-coding and discards the contents of the reference image memory used for motion-compensated prediction. Since the IDR image is decoded without reliance on any other images, the IDR image is used as a random access point. In the access unit, in the case of general coding processing, one access unit is a single image. However, in the access unit, in the case of independent encoding processing, one access unit is constituted by a plurality of images. Thus, when a specific color component image is an IDR image, assuming that all other color component images are also IDR images, an IDR access unit is set to provide a random access function.

In the following explanation, identification information indicating whether encoding is performed by general encoding processing or by independent encoding processing is called a common encoding / independent encoding identification signal.

69 is a diagram for explaining a structure of a bitstream that is generated by an encoder according to a fourteenth embodiment and is input to a decoder according to a fourteenth embodiment for decoding processing. On Fig shows the structure of the bit stream from the sequence level to the frame level. First of all, the common encoding / independent encoding identification signal 423 is multiplexed with an upper sequence level header (in the case of AVC, a set of sequence parameters, etc.). Corresponding frames are encoded in units of access units. AUD indicates an access block separator NAL unit, which is a unique NAL unit for identifying an access unit gap in the AVC. When the common encoding / independent encoding identification signal 423 indicates “image encoding by general encoding processing”, encoded data for one image is included in the access unit. It is assumed that the image in this case is the above data representing the video signal for one frame in which three color components are mixed. In this case, the encoded data of the i-th access unit form the Slice slice data set (i, j), and “j” is the index of the slice data in one image.

On the other hand, when the common encoding / independent encoding identification signal 423 indicates “image encoding by independent encoding processing”, one image is a video signal for one frame of any of the color components. In this case, the encoded data of the p-th access unit form the slice data set Slice (p, q, r) of the q-th image in the access unit, and “r” is the index of the slice data in one image. In the case of a video signal formed by three color components, for example RGB, the number of values that q can take is three. In the case, for example, when additional data, such as permeability information for alpha mixing, is encoded and decoded as an identical access unit in addition to a video signal that includes three primary colors, or in the case of encoding and decoding a video signal formed by color components (e.g., YMCK, used in color printing), in an amount greater than or equal to four, the number of values that q can take is set to four or more. When selecting independent coding processing, the encoder and decoder according to the fourteenth embodiment encode the corresponding color components forming the video signal, completely independent of each other. Thus, it is possible to freely change the number of fragments of color components, in principle, without changing the encoding and decoding processing. This leads to the fact that even when the signal format for implementing the color representation of the video signal changes in the future, this change can be coordinated with the independent encoding processing according to the fourteenth embodiment.

To implement the structure according to the fourteenth embodiment, the common encoding / independent encoding identification signal 423 is represented as “the number of images included in one access unit and independently encoded without subjecting each other to motion prediction reference”. In this case, the general encoding / independent encoding identification signal 423 can be represented by the number of values that the parameter q can take, and the number of values that the parameter can take is denoted below as num_pictures_in_au. In other words, num_pictures_in_au = 1 indicates “general encoding processing” and num_pictures_in_au = 3 indicates “independent encoding processing” according to the fourteenth embodiment. If there are four or more color components, num_pictures_in_au must have a value greater than 3. Due to this signaling, if the decoder decodes and accesses num_pictures_in_au, the decoder can not only distinguish between data encoded by general encoding processing and data encoded by independent encoding processing, but also simultaneously examine how many images of one color component are present in one access unit. Thus, it is possible to carry out general encoding processing and independent encoding processing smoothly in the bit stream, which allows for compatibility with the expansion of the color representation of the video signal in the future.

70 is a diagram for explaining the structure of a slice data bitstream in the case of general encoding processing and independent encoding processing. In the bitstream encoded by independent coding processing, to achieve the effects described below, the color component identification flag (color_channel_idc) is reported to the header area at the top of the slice data received by the decoder, which allows to determine which image of the color component in the access unit the slice data belongs to. Color_channel_idc groups slices that have the same color_channel_idc value. In other words, between slices with different color_channel_idc values, there is no dependence of encoding and decoding (for example, motion prediction, contextual modeling / study of the probability of occurrence, etc.) on CABAC. With this requirement, the independence of the corresponding images in the access unit in the case of independent encoding processing is ensured. Frame_num (the encoding and decoding processing order of the image to which the slice belongs), multiplexed with the corresponding slice header, is set to the same value in all images of the color component in one access unit.

71 is a diagram for explaining a schematic structure of an encoder according to a fourteenth embodiment. Referring to FIG. 71, general encoding processing is performed in the first image encoding unit 503a and independent encoding processing is performed in the second image encoding units 503b0, 503b1 and 503b2 (prepared for the three color components). The video signal 1 is supplied to the first image encoding unit 503a or the color component separation unit 502 and any of the second image encoding units 503b0-503b2 for each color component via the switch (SW) 501. The switch 501 is activated by the common encoding / independent encoding identification signal 423 and provides an input video signal 1 to the specified path. The following is a description of the case where the common encoding / independent encoding identification signal (num_pictures_in_au) 423 is a signal multiplexed with a set of sequence parameters when the input video signal is a 4: 4: 4 signal and is used to select general encoding processing and independent encoding processing in units of sequences. This case reflects the same concept as the cases of the common use identification flag 123 of the inter prediction mode described according to the seventh embodiment and the common use identification flag 123c of the macroblock header described according to the eleventh embodiment. When general encoding processing is used, it is necessary to perform general decoding processing on the decoder side. When independent encoding processing is used, it is necessary to perform independent decoding processing on the decoder side. Thus, it is necessary to multiplex the common encoding / independent encoding identification signal 423 with the bitstream as information indicating processing. Thus, the common encoding / independent encoding identification signal 423 is input to the multiplexing unit 504. The multiplexing unit of the common encoding / independent encoding identification signal 423 can be any unit, for example, a GOP unit (group of images) consisting of several image groups in a sequence, i.e. e. unit is at a higher level than the level of images.

To perform general encoding processing, the first image encoding unit 503a divides the input video signal 1 into macroblocks in a group of samples of three color components, as shown in FIG. 66, and advances the encoding processing in this unit. The encoding processing in the first image encoding unit 503a will be described below. When you select the independent encoding processing, the input video signal 1 is divided into data for one frame C0, C1, and C2 in the color component separation unit 502 and is supplied to the corresponding second image encoding units 503b0-503b2, respectively. The second image encoding units 503b0-503b2 divide the signal for one frame divided for each color component into macroblocks in the format shown in FIG. 67, and advance encoding processing in this unit. Coding processing in the second image coding units will be described below.

A video signal for one image, consisting of three color components, is input into the first image encoding unit 503a. The encoded data is output as a video stream 422a. A video signal for one image, consisting of one color component, is input to the second image encoding units 503b0-503b2. The encoded data is output as video streams 420b0-422b2. These video streams are multiplexed into the video stream format 422c in the multiplexing unit 504 based on the state of the common encoding / independent encoding identification signal 423 and output.

When multiplexing the video stream 422c in the access unit in the case when independent encoding processing is performed, it is possible to interleave the slice data multiplexed and transmitted in the bit stream between the images (corresponding color components) in the access unit (Fig. 72). In this case, on the decoder side, it is necessary to decide which color component in the access unit the received slice data belongs to. Thus, the color component identification flag is multiplexed with the header area at the top of the slice data, as shown in FIG.

Due to a structure similar to that of the encoder shown in FIG. 71, when the encoder encodes images of three color components according to parallel processing using three sets of each of the image encoding units 503b0-503b2 independent of each other, encoded data can be transmitted without waiting for the completion of the encoded data of other images of the color component, as soon as the slicing data of the own image. In AVC, you can divide a single image into a set of slice data and encode the slice data. You can flexibly change the length of the slice data and the number of macroblocks included in the slice according to the encoding conditions. Between slices adjacent to each other in the image space, since decoding processing independence for slices is ensured, it is impossible to use close contexts, such as intra-prediction and arithmetic coding. Thus, the longer the length of the slice data, the higher the coding efficiency. On the other hand, when an error is mixed into the bitstream during transmission and recording, the return from the error occurs the earlier, the shorter the length of the slice data, and the easier it is to suppress a decrease in quality. When the length and structure of the cut, the order of color components, etc. fixed without multiplexing the identification flag of the color component, the conditions for generating the bit stream are fixed in the encoder. Thus, it is possible to flexibly adapt to various conditions necessary for coding.

If it is possible to form a bitstream, as shown in FIG. 72, in an encoder, it is possible to reduce the size of the transmission buffer necessary for transmission, i.e. processing delay on the encoder side. The processing delay reduction state is shown in FIG. If the multiplexing of slice data between images is not allowed until the encoding of the image of a certain color component is completed, the encoder has to buffer the encoded data of other images. This means that there is a delay at the image level. On the other hand, as shown in the lower part of FIG. 72, if it is possible to perform interleaving at the slice level, the image encoding unit of the specific color component can output encoded data to the multiplexing unit in slice data units and can suppress the delay.

In one image of the color component, the slice data included in the image can be transmitted in the macroblock scan order or can be constructed so as to be able to interleave transmission even in one image.

The operations of the first and second image coding units will be described in detail below.

Description of the operation of the first block coding image

The internal structure of the first image encoding unit 503a is shown in FIG. According to FIG. 73, the input video signal 1 is input in a 4: 4: 4 format and in units of macroblocks in a group of three color components in the format shown in FIG.

First of all, the prediction unit 461 selects a reference image from the reference image data for predicting motion compensation stored in the memory 16a, and performs prediction processing for motion compensation in units of macroblocks. Memory 16a stores a plurality of pieces of reference image data formed by three color components over a plurality of time intervals. Prediction block 461 selects the optimal reference image in units of macroblocks from the reference image data and predicts motion. In order to place the reference image data in the memory 16a, the reference image data can be individually stored for each of the color components in the sequential plane mode, or samples of the corresponding color components can be stored in the sequence of point modes. Seven types are prepared as block sizes for predicting motion compensation. First, any size can be selected from 16 × 16, 16 × 8, 8 × 16 and 8 × 8 in units of macroblocks, as shown in FIG. 32A-FIG. 32D. In addition, when selecting 8 × 8, any size of 8 × 8, 8 × 4, 4 × 8 and 4 × 4 can be selected for each 8 × 8 block, as shown in FIG. 32E-FIG. 32H.

Prediction block 461 performs prediction processing for each macroblock size to compensate for movement on all or part of the block sizes, subblock sizes, motion vectors in a predetermined search range and one or more reference images used. Prediction block 461 receives a differential prediction signal for each block serving as a motion compensation prediction block using motion vectors, and a reference image identification number 463 and a subtractor 3 used for prediction. The prediction efficiency of the differential prediction signal 4 is evaluated in block 5 of the decision regarding the encoding mode. The decision block 5 regarding the coding mode derives the macroblock type / sub-macroblock type 106 and the motion vector / reference image identification information 463, at which the optimal prediction efficiency for the macroblock to be predicted is achieved from the prediction processing performed in the prediction block 461. All pieces of header information macroblocks, for example, macroblock types, submacroblock types, reference image indices and motion vectors, are defined as header information, total For three color components used for coding, and are multiplexed with a bit stream. When evaluating the optimality of prediction efficiency in order to control the volume of an arithmetic operation, it is possible to estimate the magnitude of the prediction error for a predetermined color component (for example, component G of the RGB signal or Y component of the signal YUV). Alternatively, although the volume of the arithmetic operation is increasing, to obtain optimal prediction performance, the magnitude of the prediction error for all color components can be fully estimated. In the final selection of the macroblock type / sub-macroblock type 106, a weighting factor of 20 for each type determined when deciding by the encoding control unit 19 can be taken into account.

Similarly, prediction block 461 also performs intra-prediction. When performing intra-prediction, intra-prediction mode information is output as output signal 463. In the following explanation, when intra-prediction and prediction with motion compensation do not specifically differ, as output signal 463, intra-prediction mode information, motion vector information, reference image identification number are collectively referred to as overhead information forecasting. With regard to intra-prediction, it is possible to estimate the magnitude of the prediction error only for a predetermined color component, or it is possible to fully estimate the magnitude of the prediction error for all color components. Finally, prediction block 461 selects macroblock type intra-prediction or inter-prediction, judging the macroblock type according to the prediction efficiency or coding efficiency in decision block 5 regarding the coding mode.

The prediction unit 461 outputs the selected macroblock type / sub-macroblock type 106 and the differential prediction signal 4 obtained by intra-prediction and motion compensation prediction based on the prediction overhead information 463 to the conversion unit 310. The conversion unit 310 converts the inputted differential prediction signal 4 and outputs the differential signal prediction 4 per quantization block 9 as a transform coefficient. In this case, the size of the block serving as the input signal for the conversion can be selected from 4 × 4 and 8 × 8. When it is possible to select a transform block size, the block size selected during encoding is reflected in the value of the transform block size flag 464 and the flag is multiplexed with the bitstream. The quantization unit 9 quantizes the transform coefficient introduced based on the quantization parameter 21 determined by the encoding control unit 19 and outputs the transform coefficient to the variable-length encoding unit 11 as the quantized transform coefficient 10. The quantized transform coefficient 10 includes information for three color components and is entropy encoded by Huffman coding, arithmetic coding, etc. in block 11 encoding with a variable word length. The quantized transform coefficient 10 is reconstructed to the local prediction differential prediction signal 14 by the inverse quantization unit 12 and the inverse transform unit 312. The quantized transform coefficient 10 is added to the predicted image 7 generated based on the selected macroblock type / sub-macroblock type 106 and prediction overhead information 463, by the adder 18. Thus, a locally decoded image is generated 15. Being subjected to deletion processing block distortion in the de-blocking filter 462, the local decoded image 15 is stored in memory 16a for use in the next prediction processing for motion compensation. An enable filter control flag 24 indicating whether the enable filter is applied to the macroblock is also input to the variable-length encoding unit 11.

The quantized transform coefficient 10, the macroblock type / sub-macroblock type 106, the prediction overhead 463 and the quantization parameter 21 entered into the variable-length encoding unit 11 are ordered and formed as a bit stream in accordance with a predetermined rule (syntax) and are output to transmission buffer 17 as encoded data of a NAL unit in slice data units in one or a group of a plurality of macroblocks in the format shown in FIG. The transmission buffer 17 smooths the bit stream according to the band of the transmission line to which the encoder is connected and the read speed of the recording medium, and outputs the bit stream as the video stream 422a. The transmission buffer 17 applies feedback to the encoding control unit 19 according to the state of accumulation of bit streams in the transmission buffer 17 and adjusts the amount of generated codes at the next encoding of the video frames.

The output of the first image encoding unit 503a is a slice of a unit of three components and is equivalent to the amount of codes in units of a group of access units. Thus, the transmission buffer 17 can be placed in the multiplexing unit 504 as is.

In the first image encoding unit 503a according to the fourteenth embodiment, it can be decided that all the slice data in the sequence is a slice in which C0, C1 and C2 are mixed (i.e., a slice in which pieces of information of the three color components are mixed) according to the signal 423 identification of common coding / independent coding. Thus, the color component identification flag is not multiplexed with the slice header.

Description of the operation of the second block coding image

The internal structure of the second image encoding unit 503b0 (503b1, 503b2) is shown in FIG. According to Fig. 74, it is assumed that the input video signal 1a is input in units of macroblocks consisting of a sample of one color component in the format shown in Fig. 67.

First of all, the prediction unit 461 selects a reference image from the reference image data for predicting motion compensation stored in the memory 16b, and performs prediction processing for motion compensation in units of macroblocks. In memory 16, a plurality of pieces of reference image data formed by one color component can be stored for a plurality of time intervals. Prediction block 461 selects the optimal reference image in units of macroblocks from the reference image data and predicts motion. The memory 16b in units of a group of three color components can be used in conjunction with the memory 16a. Seven types are prepared as block sizes for predicting motion compensation. First, any size can be selected from 16 × 16, 16 × 8, 8 × 16 and 8 × 8 in units of macroblocks, as shown in FIG. 32A-FIG. 32D. In addition, when selecting 8 × 8, any size of 8 × 8, 8 × 4, 4 × 8 and 4 × 4 can be selected for each 8 × 8 block, as shown in FIG. 32E-FIG. 32H.

Prediction block 461 performs prediction processing for each macroblock size to compensate for movement on all or part of the block sizes, subblock sizes, motion vectors in a predetermined search range and one or more reference images used. Prediction block 461 receives a differential prediction signal 4 for each block serving as a motion compensation prediction block using motion vectors, and a reference image identification number 463 and a subtractor 3 used for prediction. The prediction efficiency of the differential prediction signal 4 is evaluated in block 5 of the decision regarding the encoding mode. The decision block 5 regarding the coding mode outputs the macroblock type / submacroblock type 106 and the motion vector information / reference image identification number 463, at which the optimal prediction efficiency for the macroblock to be predicted is achieved from the prediction processing performed in the prediction block 461. All pieces of information macroblock header, for example, macroblock types, submacroblock types, reference image indices and motion vectors, are defined as header information in relation to one color component of the input video signal 1 used for encoding, and are multiplexed with a bit stream. When evaluating the optimality of forecasting efficiency, only the magnitude of the prediction error is estimated for one color component to be encoded. In the final selection of the macroblock type / sub-macroblock type 106, a weighting factor of 20 for each type determined when deciding by the encoding control unit 19 can be taken into account.

Similarly, prediction block 461 also performs intra-prediction. During intra-prediction, the intra-prediction mode information is output as an output signal 463. In the following explanation, when intra-prediction and motion-compensated prediction are not specifically distinguished, the output signal 463 is called prediction overhead information including intra-prediction mode information, motion vectors and a reference identification number Images. In addition, with respect to intra-prediction, only the magnitude of the prediction error is estimated for one color component to be encoded. Finally, prediction block 461 selects macroblock type intra-prediction or inter-prediction, judging the macroblock type according to prediction efficiency or coding efficiency.

The prediction unit 461 outputs the selected macroblock type / sub-macroblock type 106 and the differential prediction signal 4 obtained by the prediction service information 463 to the conversion unit 310. The conversion unit 310 converts the input differential prediction signal 4 of one color component and outputs the differential prediction signal 4 to the quantization unit 9 as a conversion factor. In this case, the size of the block serving as the input signal for the conversion can be selected from 4 × 4 and 8 × 8. When a choice is possible, the block size selected during encoding is reflected in the value of the conversion block size flag 464, and the flag is multiplexed with the bitstream. The quantization unit 9 quantizes the transform coefficient introduced based on the quantization parameter 21 determined by the encoding control unit 19 and outputs the transform coefficient to the variable-length encoding unit 11 as the quantized transform coefficient 10. The quantized transform coefficient 10 includes information for one color component and is entropy encoded by Huffman coding, arithmetic coding, etc. in block 11 encoding with a variable word length. The quantized transform coefficient 10 is reconstructed to the local prediction differential prediction signal 14 by the inverse quantization unit 12 and the inverse transform unit 312. The quantized transform coefficient 10 is added to the predicted image 7 generated based on the selected macroblock type / sub-macroblock type 106 and prediction overhead information 463, by the adder 18. Thus, a locally decoded image is generated 15. Being subjected to deletion processing block distortion in the de-blocking filter 462, the local decoded image 15 is stored in the memory 16b to be used in the following processing for motion compensation prediction. An enable filter control flag 24 indicating whether the enable filter is applied to the macroblock is also input to the variable-length encoding unit 11.

The quantized transform coefficient 10, the macroblock type / sub-macroblock type 106, the prediction overhead 463 and the quantization parameter 21 entered into the variable-length encoding unit 11 are ordered and formed as a bit stream in accordance with a predetermined rule (syntax) and are output to transmission buffer 17 as encoded data of a NAL unit in slice data units in one or a group of a plurality of macroblocks in the format shown in FIG. The transmission buffer 17 smooths the bit stream according to the band of the transmission line to which the encoder is connected, and the read speed of the recording medium, and outputs the bit stream as a video stream 422b0 (422b1, 422b2). The transmission buffer 17 applies feedback to the encoding control unit 19 according to the state of accumulation of bit streams in the transmission buffer 17 and adjusts the amount of generated codes at the next encoding of the video frames.

The output of each of the second image encoding units 503b0-503b2 is a slice consisting of only one color component data. When it is necessary to adjust the amount of codes in units of the group of access units, a common transmission buffer in units of multiplexed slices of all color components can be provided in the multiplexing unit 504 to provide feedback to the encoding control unit 19 of the corresponding color components based on the occupied buffer volume. In this case, coding control can be carried out using only the amount of information about the generation of all color components, and can also be carried out taking into account the status of the transmission buffer 17 of each of the color components. When encoding is controlled using only the amount of information about the generation of all color components, it is also possible to implement a function equivalent to the transmission buffer 17 with a common transmission buffer in the multiplexing unit 504 and to delete the transmission buffer 17.

In the second image encoding units 503b0-503b2 according to the fourteenth embodiment, it can be decided that all the slice data in the sequence is a slice of one color component (i.e., slice C0, slice C1 or slice C2) according to the common encoding / independent identification signal 423 coding. Thus, the color component identification flag is always multiplexed with the slice header so that it can be decided on the decoder side which slice corresponds to which image data in the access unit. Thus, the respective second image encoding units 503b0-503b2 can transmit output signals from the respective transmission buffers 17 at a time when data for one slice is accumulated without accumulating output signals for one image.

The common encoding / independent encoding identification signal (num_pictures_in_au) may simultaneously provide information for distinguishing data encoded by general encoding processing from data encoded by independent encoding processing (general encoding identification information), and information indicating how many images of one color component are present in one access unit (number of color components). However, two kinds of information can be encoded as independent pieces of information.

The first image encoding unit 503a and the second image encoding units 503b0-503b2 differ only in whether macro heading information is processed as information common to the three components, or as information of one color component, and the structure of the slice data bitstream. You can implement most of the basic processing blocks, for example, prediction blocks, transform blocks and inverse transform blocks, quantization blocks and inverse quantization blocks, and enable filters shown in FIGS. 73 and 74, can be implemented in function blocks common to the first image encoding block 503a and second image encoding units 503b0-503b2, with the only difference being whether the information of the three color components is processed together or only the information of one color component is being processed . Thus, it is possible to implement not only the completely independent encoding processing unit shown in FIG. 71, but also various encoders by properly combining the main components shown in FIGS. 73 and 74. If the configuration of the memory 16a in the first image encoding unit 503a is provided in in the serial plane mode, it is possible to share a memory structure for storing a reference image between the first image encoding unit 503a and the second image encoding units 503b0-503b2.

Although not shown in the figure, in the encoder according to the fourteenth embodiment, assuming there is an imaginary stream buffer (an image buffer for encoding) that buffers the video stream 422c, consistent with the arrays shown in Figs. 69 and 70, and imaginary frame memory (image buffer for decoding), which buffers the decoded images 427a and 427b, the video stream 422c is generated to avoid overflow or underfill of the image buffer for encoding and failure of the image buffer for decoding. This control is mainly carried out by the encoding control unit 19. Therefore, when the video stream 422c is decoded in accordance with the operations (imaginary buffer models) of the image buffer for encoding and the image buffer for decoding in the decoder, it is guaranteed that the decoder does not fail. Imaginary buffer models are defined below.

The operations of the image buffer for encoding are carried out in units of access units. As described above, when performing general decoding processing, the encoded data of one image is included in one access unit. When performing independent decoding processing, encoded image data for the number of color components (for three images in the case of three components) are included in one access unit. The operations specified for the image buffer for encoding are the time when the first bit and the last bit of the access unit are entered into the image buffer for encoding, and the time when the bit of the access unit is read from the image buffer for encoding. By definition, reading from the image buffer for encoding is instantaneous. It is assumed that all bits of the access unit are read from the image buffer for encoding at the same time. When a bit of the access block is read from the image buffer for encoding, the bit is entered into the analysis block of the top header. As described above, the bit undergoes decoding processing in a first image decoding unit or a second image decoding unit and is output as a color video frame coupled in units of an access unit. Processing from reading bits from the image buffer for encoding and outputting the image as a color video frame in units of access units is carried out instantly according to the definition of the imaginary buffer model. A color video frame constructed in units of access units is entered into the image buffer for decoding, and the time of output of the color video frame from the image buffer for decoding is calculated. The output time from the image buffer for decoding is the value calculated by adding the predetermined delay time to the reading time from the image buffer for encoding. This delay time can be multiplexed with a bit stream to control the decoder. When the delay time is 0, i.e. when the output time from the image buffer for decoding is equal to the reading time from the image buffer for encoding, the color video frame is entered into the image buffer for decoding and simultaneously output from the image buffer for decoding. In other cases, i.e. when the output time from the image buffer for decoding is longer than the reading time from the image buffer for encoding, the color video frame is stored in the image buffer for decoding until the time comes to output from the image buffer for decoding. As described above, operations from the image buffer for decoding are specified in units of access units.

75 is a diagram for explaining a schematic structure of a decoder according to a fourteenth embodiment. Referring to FIG. 75, general decoding processing is performed in the first image decoding unit 603a. Independent decoding processing is performed in decision block 602 regarding the color component and second image decoding blocks 603b0 (prepared for the three color components).

The video stream 422c is divided into units of NAL units in block 610 analysis of the upper header. Top header information, such as a sequence parameter set and an image parameter set, is decoded as is and stored in a predetermined memory area to which the first image decoding unit 603a, the color component decision block 602, and the second image decoding units 603b0-603b2 are able to access top header information. The common coding / independent coding identification signal (num_pictures_in_au) 423, multiplexed in sequence units, is decoded and maintained as part of the header information.

The decoded num_pictures_in_au is supplied to the switch (SW) 601. If num_pictures_in_au = 1, the switch 601 supplies the slice NAL unit for each image to the first image decoding unit 603a. If num_pictures_in_au = 3, the switch 601 supplies the slice NAL block to the decision block 602 regarding the color component. In other words, if num_pictures_in_au = 1, the first image decoding unit 603a performs general decoding processing. If num_pictures_in_au = 3, the three second image decoding blocks 603b0-603b2 carry out independent decoding processing. A detailed description of the operations of the first and second image decoding blocks is given below.

The decision block 602 regarding the color component decides which image of the color component in the current access unit corresponds to the slice NAL block according to the value of the color component identification flag shown in FIG. 70, and distributes and supplies the slice NAL block to the corresponding second blocks 603b0-603b2 image decoding. With such a structure of the decoder, it turns out that even if a bit stream is received obtained by interleaving and coding a slice in an access unit, as shown in Fig. 72, it is easy to decide which slice belongs to which image of the color component and correctly decode the bit stream.

Description of the operation of the first block image decoding

The internal structure of the first image decoding unit 603a is shown in FIG. The first image decoding unit 603a receives the video stream 442c consistent with the arrays shown in FIGS. 69 and 70, which is output from the encoder shown in FIG. 71, in mixed slice units C0, C1 and C2, after dividing the video stream in units of the NAL unit. The first image decoding unit 603a performs decoding processing when the macroblock consists of samples of the three color components shown in FIG. 66 and restores the output video frame.

The video stream 442c is input to a variable word length decoding unit 25. The variable-length decoding unit 25 interprets the video stream 442c in accordance with a predetermined rule (syntax) and extracts the quantized transform coefficient 10 for the three components and macroblock header information (macroblock type / sub-macroblock type 106, prediction overhead information 463, block size indication flag 464 transforms and quantization parameter 21), shared between the three components. The quantized transform coefficient 10 is input to the inverse quantization unit 12, which performs the same processing as the first image encoding unit 503a performs, together with the quantization parameter 21, and undergoes inverse quantization processing. Then, the output signal of the inverse quantization unit 12 is input to the inverse transform unit 312, which performs the same processing as the first image encoding unit 503a performs, and is restored to the local prediction differential prediction signal 14 (if the transform block size flag 464 is present in the video stream 422c, which is addressed by the flag 464 indicating the size of the transform block in the inverse quantization step and in the inverse transform processing step). On the other hand, only the processing of accessing the prediction overhead information 463 to generate the prediction image 7 in the prediction block 461 in the first image encoding unit 503a is included in the prediction block 461. The macroblock type / sub-macroblock type 106 and the prediction overhead information 463 are input to the prediction block 461 for obtaining the predicted image 7 for the three components. When the macroblock type indicates intra-prediction, the predicted image 7 for the three components is obtained from the prediction overhead 463 in accordance with the intra-prediction mode information. When the macroblock type indicates inter-prediction, the predicted image 7 for the three components is obtained from the prediction overhead 463 in accordance with the motion vector and the reference image index. The local prediction differential prediction signal 14 and the prediction image 7 are added by an adder 18 to obtain an internally decoded image 15 for the three components. Since the internally decoded image (locally decoded image) 15 is used to predict the motion compensation of the following macroblocks, after applying block distortion removal processing to samples of the internally decoded image for the three components in the enable filter 462, which performs the same processing as the first encoding unit 503a performs image, the internally decoded image 15 is displayed as the decoded image 427a and stored in the memory 16a. In this case, the unlock filter processing is applied to the internally decoded image 15 based on the instruction of the unlock filter control flag 24, interpreted by the variable-length decoding unit 25. A plurality of pieces of reference image data formed by three color components over a plurality of time intervals is stored in the memory 16a. The prediction unit 461 selects a reference image indicated by the reference image index extracted from the bitstream in units of macroblocks from the reference image data and generates a predicted image. In order to place the reference image data in the memory 16a, the reference image data can be individually stored for each of the color components in the sequential plane mode, or samples of the corresponding color components can be stored in the sequence of point modes. The decoded image 427a includes three color components and is directly converted to a color video frame forming an access unit 427a0 in the general decoding processing.

Description of the operation of the second block image decoding

The internal structure of each of the second image decoding blocks 603b0-603b2 is shown in FIG. Each of the second image decoding units 603b0-603b2 receives a video stream 442c consistent with the arrays shown in FIGS. 69 and 70 output from the decoder shown in FIG. 71, in units of the slice NAL units C0, C1 or C2 allocated by the acceptance unit 602 decisions regarding the color component, after the video stream is divided in units of NAL units in the header block 610, performs decoding processing with a macroblock consisting of a sample of one color component shown in Fig. 67 as a unit and restores to output video frame.

The video stream 422c is input to a variable word length decoding unit 25. The variable-length decoding unit 25 interprets the bitstream 422c in accordance with a predetermined rule (syntax) and extracts the quantized transform coefficient 10 for one color component and macroblock header information (macroblock type / sub-macroblock type 106, prediction overhead information 463, indication flag 464 transform block size and quantization parameter 21), shared for one color component. The quantized transform coefficient 10 is input to the inverse quantization unit 12, which performs the same processing as the second image encoding unit 503b0 (503b1, 503b2) performs, together with the quantization parameter 21, and undergoes inverse quantization processing. Then, the output signal of the inverse quantization unit 12 is input to the inverse transform unit 312, which performs the same processing as the second image encoding unit 503b0 (503b1, 503b2), and is restored to the differential local decoding prediction signal 14 (if the conversion unit size flag 464 indicates is present in the video stream 422c, which is addressed by the flag 464 indicating the size of the transform block in the inverse quantization step and in the inverse transform processing step). On the other hand, only the processing of accessing the prediction overhead information 463 to generate the prediction image 7 in the prediction block 461 in the second image encoding unit 503b0 (503b1, 503b2) is included in the prediction block 461. The macroblock type / sub-macroblock type 106 and the prediction overhead information 463 are input to the prediction block 461 to obtain a predicted image 7 for one color component. When the macroblock type indicates intra-prediction, the predicted image 7 for one color component is obtained from prediction overhead information 463 in accordance with the intra-prediction mode information. When the macroblock type indicates inter-prediction, the predicted image 7 for one color component is obtained from the prediction overhead 463 in accordance with the motion vector and the reference image index. The local prediction differential prediction signal 14 and the prediction image 7 are added by an adder 18 to obtain an internally decoded image 15 for a macroblock of one color component. Since the internally decoded image 15 is used for motion-compensated prediction of the following macroblocks, after applying block distortion removal processing to the samples of the internally decoded image for one color component in the enable filter 26, which performs the same processing as the second block 503b0 (503b1, 503b2) encoding the image, the internally decoded image 15 is output as the decoded image 427b and stored in the memory 16b. In this case, the unlock filter processing is applied to the internally decoded image 15 based on the instruction of the unlock filter control flag 24, interpreted by the variable-length decoding unit 25. The decoded image 427b includes only a sample of one color component and is constructed as a color video frame by linking, in units of access units 427b0, a decoded image 427b as outputs of other corresponding second image decoding units 603b0-603b2 to be processed in parallel, shown in FIG. 75.

As follows from the above, the first image decoding unit 603a and the second image decoding units 603b0-603b2 differ only in whether the macroblock header information is processed as information common to the three components or as information of one color component and the structure of the slice data stream. You can implement most of the basic decoding processing units, for example, motion compensation, inverse transform, and inverse quantization prediction processing shown in FIGS. 73 and 74, in function blocks common to the first image encoding unit 503a and the second image encoding units 503b0-503b2. Thus, it is possible to implement not only the completely independent decoding processing unit shown in FIG. 75, but also the various decoders, appropriately combining the main components shown in FIGS. 76 and 77. Furthermore, if the memory configuration is 16a in the first encoding unit 503a the image is provided in serial plane mode, the structures of the memory units 16a and 16b between the first image decoding unit 603a and the second image decoding units 603b0-603b2 can be shared.

It is obvious that the decoder shown in FIG. 75 is capable of receiving and decoding a bitstream output from the encoder, is capable of always fixing the common encoding / independent encoding identification signal 423 to “independent encoding processing” and independently encoding all frames without using the first block 503a encoding an image as another variation of the encoder shown in FIG. As another variation of the decoder shown in FIG. 75, in the form of use, provided that the common encoding / independent encoding identification signal 423 is always fixed to “independent encoding processing”, the decoder can be constructed as a decoder that does not include a switch 601 and a first image decoding unit 603a and performs only independent decoding processing.

The common encoding / independent encoding identification signal (num_pictures_in_au) includes information for distinguishing data encoded by general encoding processing from data encoded by independent encoding processing (common encoding identification information), and information indicating how many images of one color component are present in one access unit (number of color components). However, two kinds of information can be encoded as independent pieces of information.

If the first image decoding unit 603a includes a function for decoding a bitstream according to the high AVC profile, in which the three components are coded in the traditional YUV 4: 2: 0 format as an object, and the upper header analysis unit 610 decides in which format the bitstream is encoded, with reference to the profile identifier decoded from the bitstream 422c, and transmits the decision result to the switch 601 and the first image decoding unit 603a as part of the signal line information for a common encoding / independent encoding identification signal 423, it is also possible to construct a decoder that provides compatibility of the conventional YUV 4: 2: 0 format with the bitstream.

In the first image encoding unit 503a according to the fourteenth embodiment, pieces of information of the three color components are mixed in the slice data, and completely identical intra / inter prediction processing is applied to the three color components. Accordingly, the signal correlation between the color components can remain in the space of the prediction error signal. As a means of removing correlation of a signal to a prediction error signal, for example, the color space conversion processing described in accordance with the thirteenth embodiment can be used. Examples of the first image encoding unit 503a having such a structure are shown in FIGS. 78 and 79. FIG. 78 shows an example in which color space conversion processing is performed at the pixel level before conversion processing is performed. A color space converting unit 465 is placed before the transform unit 310, and a color space inverse transforming unit 466 is placed after the inverse transforming unit 312. FIG. 79 shows an example in which color space conversion processing is performed while the frequency component to be processed, appropriately selected in relation to the coefficient data obtained after the conversion processing has been performed. A color space conversion unit 465 is located after the conversion unit 310, and a color space inverse conversion unit 466 is located before the inverse conversion unit 312. This allows you to adjust the high-frequency noise component included in the specific color component so that it does not extend to other color components, hardly including noise in itself. When the frequency component to be processed by the color space conversion is made adaptively selectable, pieces of signaling information 467 to decide on the choice of encoding time are multiplexed with the bitstream on the decoding side.

In the color space transform processing, the set of system transforms described according to the thirteenth embodiment may be switched in units of macroblocks and used according to the characteristic of the image signal to be encoded, or a decision regarding the presence or absence of conversion may be carried out in units of macroblocks. You can also specify in advance the types of selectable conversion systems at the sequence level and indicate the conversion system to be selected in units of image, slice, macroblock, etc. You can choose whether color space conversion processing is performed before conversion or after conversion. When implementing these varieties of adaptive coding processing, it is possible to evaluate the coding efficiency for all choices using decision block 5 regarding the coding mode to select the option with the highest coding efficiency. When performing these varieties of adaptive coding processing, fragments of signaling information 467 for deciding on the timing of coding are multiplexed with the bitstream on the decoding side. Signaling can be indicated at a level other than the macroblock level, for example, at the slice, image, GOP or sequence level.

The decoders corresponding to the encoders shown in Figs. 78 and 79 are shown in Figs. 80 and 81. Fig. 80 shows a decoder that decodes a bitstream encoded by the encoder shown in Fig. 78 by performing color space conversion before conversion processing . The variable-length decoding unit 25 decodes signaling information 467 from the bitstream as information about the presence or absence of conversion for selecting whether the conversion is performed in block 466 of the inverse color space conversion and information for selecting a conversion system performed in block 466 of the inverse color conversion space, and provides information to block 466 inverse color space conversion. The decoder shown in FIG. 80 executes, in a color space inverse transform unit 466, color space transform processing for the prediction error signal after the inverse transform based on these varieties of information. On Fig shows a decoder that decodes the bitstream encoded by the encoder shown in Fig, selecting the frequency component to be processed, after processing the conversion and converting the color space. The variable-length decoding unit decodes signaling information 467 from the bitstream as identification information including information about the presence or absence of a conversion, to select whether the conversion is performed in the inverse color space conversion unit 466, and information to select a conversion system performed in a color space inverse transform unit, information for selecting a frequency component in which color space is converted and, etc. and provides information to block 466 inverse color space conversion. The decoder shown in FIG. 81 executes, in an inverse color space transforming unit 466, color space transform processing for these transform coefficients after inverse quantization based on these varieties of information.

In the decoders shown in FIGS. 80 and 81, as in the decoder shown in FIG. 75, if the first image decoding unit 603a includes a function for decoding a bitstream according to a high AVC profile in which the three components are jointly encoded in the traditional format YUV 4: 2: 0 as an object, and the upper header analysis unit 610 decides in which format the bitstream is encoded, with reference to the profile identifier decoded from bitstream 422c, and passes the decision result to switch 601 and the first image decoding lok 603a as part of the information signal to signal line 423 identifying the common encoding / independent encoding, also possible to construct a decoder that secures compatibility of the traditional format YUV 4: 2: 0 format with the bit stream.

The structure of the encoded data of the macroblock header information included in the bitstream in the traditional YUV 4: 2: 0 format is shown in FIG. 82. The data differs from the Cn component header information shown in FIG. 50 in that when the macroblock type is intra prediction, the encoded data of the color difference intra prediction mode 144 is turned on. When the macroblock type is inter prediction, although the structure of the encoded macroblock header information data is the same as that of the Cn component header information shown in FIG. 50, the motion vector of the color difference component is generated by a method other than the luminance component generation method using the reference image identification number and motion vector information included in macroblock header information.

The operations of the decoder are explained below to ensure compatibility of the traditional YUV 4: 2: 0 format with the bitstream. As described above, the first image decoding unit 603a has a function for decoding a bitstream in the traditional YUV 4: 2: 0 format. The internal structure of the first image decoding unit is the same as shown in FIG.

The operations of the first image decoding unit and the variable word length decoding unit 25 having a function for decoding a bit stream in the traditional YUV 4: 2: 0 format are explained below. When the video stream 422c enters the variable-length decoding unit, the variable-word decoding unit decodes the color difference format indication flag. The color-difference signal format indication flag is a flag included in the header of the sequence parameter of the video stream 422c and indicating whether the input video signal format is 4: 4: 4, 4: 2: 2, 4: 2: 0 or 4: 0: 0. The decoding processing for the macroblock header information of the video stream 422c is switched according to the value of the color difference format indication flag. When the macroblock type indicates intra-prediction and the color difference indication flag indicates 4: 2: 0 or 4: 2: 2, the intra-prediction mode of the color difference signal 144 is decoded from the bitstream. When the color difference format format indication flag indicates 4: 4: 4, decoding of the color difference intra prediction mode 144 is skipped. When the color format format indication flag indicates 4: 0: 0, since the input video signal has a format (4: 0: 0 format) formed only by the luminance signal, decoding of the color-difference intra prediction mode 144 is skipped. The decoding processing for macroblock header information other than the color difference intra prediction mode 144 is the same as in the variable word length decoding unit of the first image decoding unit 603a, and does not include a function for decoding a bitstream in the traditional YUV 4: 2 format: 0. Therefore, when the video stream 422c is input to the variable word length decoding unit 25, the variable word length decoding unit 603a extracts a color difference format indication flag (not shown), a quantized transform coefficient for the three components 10, and macroblock header information (macroblock type / submacroblock type 106, prediction overhead 463, transform block size flag 464 and quantization parameter 21). The color-difference signal format indication flag (not shown) and prediction overhead information 463 are input to the prediction block 461 to obtain a predicted image 7 for three components.

The internal structure of the prediction block 461 of the first image decoding block, which ensures compatibility of the conventional YUV 4: 2: 0 format with the bitstream, is shown in FIG. The operations of the prediction unit are explained below.

The switching unit 4611a decides on the type of macroblock. When the macroblock type indicates intra-prediction, the switching unit 4611b decides on the value of the color difference format indication flag. When the value of the color format format indication flag indicates 4: 2: 0 or 4: 2: 2, the prediction unit obtains the predicted image 7 for the three components from the prediction overhead information in accordance with the intra prediction mode information and the intra difference prediction mode information. A predicted image of the luminance signal among the three components is generated in the luminance signal intra prediction block 4612 in accordance with the intra prediction mode information. A predicted image of the color difference signal of the two components is generated in block 4613 for predicting the color difference signal, which performs processing other than processing the luminance component in accordance with the information of the intra prediction mode of the color difference signal. When the value of the color-format signal format indication flag indicates 4: 4: 4, predicted images of all three components are generated in the luminance signal intra-prediction unit 4612 in accordance with the intra-prediction mode information. When the value of the color format indicator indication flag indicates 4: 0: 0, since the 4: 0: 0 format is formed only by the luminance signal (one component), only the predicted image of the luminance signal is generated in the luminance signal intra prediction block 4612 in accordance with the intra prediction mode information.

When the macroblock type indicates inter prediction in the switching unit 4611a, the switching unit 4611c decides on the value of the color difference format indication flag. When the value of the color-format signal format indication flag indicates 4: 2: 0 or 4: 2: 2, with respect to the luminance signal, the predicted image is generated from the prediction overhead information 463 in the luminance signal prediction unit 4614 in accordance with the motion vector and the reference image index and in accordance with the method for generating a predicted image for a luminance signal that meets the AVC standard. As for the predicted image of the color difference signal of the two components, in the color difference signal inter prediction unit 4615, the motion vector obtained from the prediction overhead information 463 is scaled based on the color difference signal format to generate the color difference motion vector. The predicted image is generated from the reference image indicated by the reference image index, which is obtained from the prediction overhead information 463, based on the color-difference motion vector in accordance with the method that meets the AVC standard. When the value of the color-format signal format indication flag indicates 4: 0: 0, since the 4: 0: 0 format is constituted only by the luminance signal (one component), a predicted image of the luminance signal is generated in the luminance signal prediction unit 4614 in accordance with the motion vector and the reference image index .

As described above, means for generating a predicted image of a color difference signal in a conventional YUV 4: 2: 0 format are provided, and means for generating predicted images of the three components is switched according to a flag value of a color difference format indication flag decoded from the bitstream. Thus, it is possible to build a decoder that provides compatibility of the traditional YUV 4: 2: 0 format with the bitstream.

If information indicating a bitstream that can be decoded even in a decoder that does not support color space conversion processing, for example the decoder shown in FIG. 75, is provided in the video stream 422c supplied to the decoders shown in FIGS. 80 and 81, in units of parameters sequences and the like, in all decoders shown in FIGS. 80, 81 and 75, it is possible to decode the bitstream corresponding to the decoding performance of each of the decoders. Accordingly, bitstream compatibility can be easily ensured.

Fifteenth Embodiment

According to a fifteenth embodiment of the present invention, another embodiment different from the encoder and decoder according to the fourteenth embodiment shown in FIGS. 71, 75 and the like, only by the structure of the bit stream to be input and output will be described. An encoder according to a fifteenth embodiment performs multiplexing of encoded data with the bitstream structure shown in FIG. 84.

In the bitstream whose structure is shown in FIG. 69, the NAL AUD block includes primary_pic_type information as its element. The table in FIG. 85 provides information on the type of image encoding during encoding of image data in the access unit, starting from the NAL AUD unit.

For example, when primary_pic_type = 0, this indicates that the image is fully intra-encoded. When primary_pic_type = 1, this indicates that the slice to be intra-encoded and the slice for which motion-compensated prediction can be performed using only one list of reference images can be mixed in the image. Since primary_pic_type is information that specifies the encoding mode in which one image can be encoded, on the encoder side, encoding suitable for various conditions, for example, the characteristics of the input video signal and the random access function, can be performed using this information. According to the fourteenth embodiment, since there is only one primary_pic_type for one access unit, when performing independent encoding processing, primary_pic_type is common to three color component images in the access unit. According to the fifteenth embodiment, when independently encoding each of the images of the color component, primary_pic_type for the remaining two images of the color component is additionally inserted into the NAL AUD block shown in FIG. 69 according to the value num_pictures_in_au. Alternatively, as in the bitstream structure shown in FIG. 84, the encoded data of each image of the color component begins with a NAL unit (color channel separator) indicating the start of the image of the color component, and corresponding primary_pic_type information is included in this NAL CCD unit. In this structure, since the encoded data of the respective color component images for a single image is jointly multiplexed, the color component identification flag (color_channel_idc) described according to the fourteenth embodiment is included in the NAL CCD block, and not in the slice header. Therefore, it is possible to consolidate the color component identification flag information to be multiplexed with corresponding slices into data in image units. Thus, overhead information can be reduced. Since the NAL CCD block, constructed only as a string of bytes, needs to be detected only once for each color component image to confirm color_channel_idc, you can quickly find the top of the color component image without performing variable-length decoding processing. Thus, on the decoder side, the color_channel_idc in the slice header does not need to be confirmed each time to separate the NAL block to be decoded for each color component. It is possible to smoothly feed data to a second image decoding unit.

On the other hand, with this structure, the effect of reducing the buffer size and processing delay of the encoder described with reference to FIG. 72 according to the fourteenth embodiment is weakened. Thus, a color component identification flag can be constructed to indicate at a higher level (sequence or GOP) whether the encoded data is multiplexed in slice units or in image units of the color component. By adopting such a structure of the bitstream, it is possible to implement a flexible implementation of the encoder according to the form of using the encoder.

In addition, according to another embodiment, the multiplexing of the encoded data can be accomplished using the bitstream structure shown in FIG. 86, the color_channel_idc and primary_pic_type included in the CCD NAL unit shown in FIG. 84 are included in the corresponding AUD. In the structure of the bitstream according to the fifteenth embodiment of the present invention, also in the case of independent encoding processing, one image (color component) is included in one access unit. With such a structure, as in the structures described above, there is an effect of reducing overhead information due to the possibility of consolidating the information of the identification flag of the color component into data in image units. In addition, since the NAL AUD block, constructed only as a string of bytes, needs to be detected to confirm color_channel_idc only once for one image, it is possible to quickly find the upper part of the image of the color component without performing variable-length decoding processing. Thus, on the decoder side, the color_channel_idc in the slice header does not need to be confirmed each time to separate the NAL block to be decoded for each color component. Accordingly, it is possible to smoothly supply data to a second image decoding unit. On the other hand, since the image of one frame or one field is formed by three access units, it is necessary to simultaneously indicate three access units as image data. Thus, in the bitstream structure shown in FIG. 86, AUD can be assigned sequence numbers (encoding and decoding orders in the time dimension, etc.) of the respective images. With this structure, on the decoder side, it is possible to check the decoding and display orders of the corresponding images, the attributes of the color components, the IDR affiliation, and the like, without decoding the slice data at all. It is possible to effectively carry out editing and special reproduction at the bitstream level.

In the bitstream structure shown in FIGS. 69, 84 or 86, information indicating the number of slice NAL units included in a single color component image may be stored in the AUD or CCD regions.

For all embodiments, the transform processing and the inverse transform processing may be a transform that guarantees orthogonality, such as DCT, or may be a transform, such as AVC, combined with quantization and inverse quantization processing to approximate orthogonality, rather than a strict transform. e.g. DCT. In addition, the prediction error signal can be encoded as information at the pixel level without conversion.

Industrial applicability

The present invention is applicable to a digital image signal encoder and a digital image signal decoder used for encoding a compressed image, transmitting compressed image data, and the like.

Claims (63)

1. An image encoder containing
a predicted image generating unit that generates a predicted image in accordance with a set of prediction modes indicating methods for generating a predicted image,
a decision unit for the prediction mode that evaluates the prediction efficiency of the predicted image output from the predicted image generation unit for deciding on a predetermined prediction mode, and
a coding unit that subjects the output of the decision unit regarding the prediction mode to coding with a variable word length in which
the decision unit regarding the prediction mode makes a decision, based on a predetermined control signal, which of the general prediction mode and the individual prediction mode are used for the respective color components forming the input image signal, and multiplexes the information about the control signal into a bit stream, multiplexes when used general prediction mode, information of the general prediction mode into the bitstream and multiplexes when the general mode is pro nozirovaniya not used, prediction mode information for each of the color components on the bit stream. 2. The image encoder according to claim 1, wherein the decision block regarding the prediction mode makes a decision regarding the prediction mode in units of macroblocks and processes the prediction mode based on a control signal that changes in units of macroblocks. 3. The image encoder according to claim 1, wherein the decision block regarding the prediction mode makes a decision regarding the prediction mode in units of macroblocks and processes the prediction mode based on a control signal that changes in units of a sequence including a plurality of frames. 4. The image encoder according to claim 1, wherein the decision block regarding the prediction mode multiplexes when the general prediction mode is used, information of the general prediction mode into a bit stream as identification information of the general prediction mode, and multiplexes when the general prediction mode is not used , prediction mode information for each of the color components in the bitstream as general prediction mode identification information used and I. 5. An image decoder containing
a decoding unit that decodes the identification information of the common prediction mode indicating which of the general prediction mode and the individual prediction mode is used for the respective color components constituting the input image signal, and decodes the prediction modes of the corresponding color components based on the value of the common use prediction mode identification information , and
a predicted image generating unit that generates a predicted image based on the prediction modes decoded by the decoding unit, wherein
the decoding unit performs decoding of the image data based on the predicted image generated by the predicted image generation unit. 6. The image decoder according to claim 5, in which the decoding unit performs decoding in units of macroblocks and decodes and uses the identification information of the prediction mode of common use in units of macroblocks. 7. The image decoder according to claim 5, in which the decoding unit performs decoding in units of macroblocks and decodes and uses the identification information of the prediction mode of general use in units of a sequence including a plurality of frames. 8. The image decoder according to claim 5, in which the decoding unit performs decoding in units of macroblocks and decodes and uses the identification information of the prediction mode of common use in units of macroblocks. 9. A method of encoding an image comprising the steps of:
generating a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image,
evaluating the prediction efficiency of the generated predicted image to make a decision regarding a predetermined prediction mode, decide, based on a predetermined signal, which of the general prediction mode and the individual prediction mode are used for the respective color components forming the input image signal,
and multiplexing the control signal information into the bitstream, multiplexing when the common prediction mode is used, general prediction mode information into the bitstream and multiplexing when not using the common prediction mode, prediction mode information for each of the color components into the bitstream, and
subjected to the output signal obtained at the stage of decision making and multiplexing, encoding with a variable word length. 10. A computer-readable recording medium on which an image encoding program is recorded instructing a computer to perform steps in which
generating a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image,
evaluate the forecasting efficiency of the generated predicted image to make a decision regarding a predetermined prediction mode, make a decision, based on a predetermined signal, which of the general prediction mode and the individual prediction mode are used for the respective color components that form the input image signal, and information about the control signal is multiplexed into the bitstream, multiplex when a common prediction mode is used, general prediction mode information into a bitstream and multiplexed when the common prediction mode is not used, prediction mode information for each of the color components in the bitstream, and
subjected to the output signal obtained at the stage of decision making and multiplexing, encoding with a variable word length. 11. A method of decoding an image comprising the steps of:
decoding general-use prediction mode identification information indicating which of the general prediction mode and the individual prediction mode are used for the respective color components constituting the input image signal, and predicting prediction modes of the respective color components based on the value of the general-use prediction mode identification information,
generating a predicted image based on the decoded prediction modes and
decode image data based on the generated predicted image. 12. A computer-readable recording medium on which an image decoding program is recorded instructing the computer to perform the steps in which
decoding general-use prediction mode identification information indicating which of the general prediction mode and the individual prediction mode are used for the respective color components constituting the input image signal, and predicting prediction modes of the respective color components based on the value of the general-use prediction mode identification information,
generating a predicted image based on the decoded prediction modes and decoding the image data based on the generated predicted image. 13. An image encoder containing
a predicted image generating unit that generates a predicted image in accordance with a set of prediction modes indicating methods for generating a predicted image,
a decision block regarding the prediction mode, which evaluates the prediction efficiency of the predicted image output from the predicted image generation block, for deciding on a predetermined prediction mode, and a prediction mode encoding block that subjects the output of the decision block to the prediction mode to variable-length coding words, moreover
the decision unit regarding the prediction mode individually makes a decision regarding the prediction mode for each of the color components in units of image areas of the prediction object, and
the prediction mode encoding unit selects the prediction mode information near the image area for the same color component or the prediction mode information at a position on the screen that is identical to the image area in other color components to specify the predicted values of the prediction modes and encode the prediction mode information. 14. The image encoder of claim 13, wherein the prediction mode encoding unit multiplexes, into a bit stream, identification information indicating which of the prediction mode information is near the image area for the same color component and the prediction mode information at a position on the screen that is identical to the image area in other color components, used as a predicted value. 15. An image decoder containing
a prediction mode decoding unit that decodes prediction modes individually encoded for respective color components constituting an input image signal,
a predicted image generating unit that generates a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image, and
a decoding unit that generates, based on the prediction modes of the respective color components set by the prediction mode decoding unit, a predicted image using the predicted image generation unit for decoding image data, wherein
the prediction mode decoding unit selects, when decoding the prediction mode allocated to the specific prediction unit, the prediction mode information near the same color component or the prediction mode information allocated to the prediction units in the same position on the screen of different color components and sets the predicted value of the prediction mode for decoding. 16. The image decoder according to clause 15, in which the prediction mode decoding unit decodes, from the bitstream, identification information indicating which of the prediction mode information near the same color component and the prediction mode information allocated to the prediction units in the same position on the screen are different color components are used as the predicted value, and sets the predicted value based on the value of the identification information. 17. An image encoding method, comprising the steps of:
generating a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image,
evaluating the forecasting efficiency of the generated predicted image for making a decision regarding a predetermined prediction mode and individually making a decision about the prediction mode for each of the color components in units of image areas of the prediction object, and
the output signal obtained at the decision-making stage regarding the prediction mode is subjected to coding with a variable word length, the prediction mode information is selected near the image area for the same color component or the prediction mode information at a position on the screen that is identical to the image area in another color component for specifying the predicted values of the prediction modes and the encoding of the information of the prediction mode. 18. A computer-readable recording medium on which an image encoding program is recorded instructing a computer to perform steps in which
generating a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image,
evaluating the forecasting efficiency of the generated predicted image for making a decision regarding a predetermined prediction mode and individually making a decision about the prediction mode for each of the color components in units of image areas of the prediction object, and
the output signal obtained at the decision-making stage regarding the prediction mode is subjected to coding with a variable word length, the prediction mode information is selected near the image area for the same color component or the prediction mode information at a position on the screen that is identical to the image area in another color component for specifying the predicted values of the prediction modes and the encoding of the information of the prediction mode. 19. An image decoding method comprising the steps of:
prediction modes are decoded, individually encoded for the respective color components forming the input image signal, and when decoding the prediction mode allocated to a specific prediction block, prediction mode information is selected near the identical color component or prediction mode information allocated to the prediction blocks in the same position on the screen , different color components for specifying the predicted values of the prediction modes and the implementation decoding,
generating a predicted image in accordance with a set of prediction modes indicating methods for generating a predicted image, and
generating a predicted image according to the step of generating the predicted image based on the prediction modes of the respective color components set in the prediction mode decoding step. 20. A computer-readable recording medium on which an image decoding program is recorded instructing a computer to perform steps in which
prediction modes are decoded, individually encoded for the respective color components forming the input image signal, and when decoding the prediction mode allocated to a specific prediction block, prediction mode information is selected near the identical color component or prediction mode information allocated to the prediction blocks in the same position on the screen , different color components for specifying the predicted values of the prediction modes and the implementation decoding,
generating a predicted image in accordance with a set of prediction modes indicating methods for generating a predicted image, and
generating a predicted image according to the step of generating the predicted image based on the prediction modes of the respective color components set in the prediction mode decoding step. 21. An image encoder containing
a predicted image generation unit that generates a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image,
a decision unit for the prediction mode that evaluates the prediction efficiency of the predicted image output from the predicted image generation unit for deciding on a predetermined prediction mode, and
a prediction mode encoding unit that subjects the output of the decision unit regarding the prediction mode to encoding with a variable word length in which
the decision unit regarding the prediction mode individually makes a decision regarding the prediction mode for each of the color components in units of image areas of the prediction object, and
the prediction mode coding unit refers to a lookup table obtained by tabulating combinations of states of a plurality of pieces of information of a prediction mode near an image area for an identical color component and predicted values for prediction modes of a coding object to uniquely specify a predicted value of prediction mode information from states of a plurality of pieces of information of a prediction mode near areas of image and coding. 22. The image encoder according to item 21, in which the prediction mode encoding unit dynamically updates the predicted values of the prediction mode information in the lookup table based on the states of the plurality of pieces of prediction mode information near the image area for the same color component and the values of the prediction modes of the encoding object. 23. An image decoder containing
a prediction mode decoding unit that decodes prediction modes individually encoded for respective color components constituting an input image signal,
a predicted image generating unit that generates a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image, and
a decoding unit that generates a predicted image using the predicted image generation unit based on the prediction modes of the respective color components set by the prediction mode decoding unit for decoding the image data,
when decoding the prediction mode allocated to a specific prediction block, the search table obtained by tabulating combinations of states of a plurality of pieces of information of the prediction mode near the image area for the same color component and the predicted values for the prediction modes of the encoding object to uniquely specify the predicted value prediction mode information from aggregate fragment states in the information of the prediction mode near the image area and the implementation of coding. 24. The image decoder of claim 23, wherein the prediction mode decoding unit dynamically updates the predicted values of the prediction mode information in the lookup table based on the states of the plurality of pieces of prediction mode information near the image area for the same color component and the values of the decoded prediction modes. 25. An image encoder containing
a predicted image generation unit that generates a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image,
a decision unit for the prediction mode that evaluates the prediction efficiency of the predicted image output from the predicted image generation unit for deciding on a predetermined prediction mode, and
a prediction mode encoding unit that subjects the output of the decision unit regarding the prediction mode to encoding with a variable word length, wherein
the decision unit regarding the prediction mode individually makes a decision regarding the prediction mode for each of the color components in units of image areas of the prediction object, and
the prediction mode encoding unit changes the value of the prediction mode of the encoding object into a binary sequence and, when encoding the corresponding sections of the binary sequence, performs arithmetic coding, dynamically changing the probability of occurrence according to the context established according to the states of the set of pieces of information of the prediction mode near the image area for an identical color component. 26. The image encoder of claim 25, wherein the prediction mode encoding unit uses, as a context established according to the states of the plurality of pieces of prediction information near the image area for the same color component, a state indicating which of the predictions in the vertical direction of the image surface and prediction in the horizontal direction of the image surface are classified prediction modes. 27. The image encoder of claim 25, wherein the prediction mode encoding unit uses, as a context established according to the states of the plurality of pieces of prediction information near the image region for the same color component, a state indicating which of the DC prediction and prediction is different from DC prediction, used as a prediction mode. 28. An image decoder containing
a prediction mode decoding unit that decodes prediction modes individually encoded for respective color components constituting an input image signal,
a predicted image generating unit that generates a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image, and
a decoding unit that generates a predicted image using the predicted image generation unit based on the prediction modes of the respective color components set by the prediction mode decoding unit for decoding the image data,
the prediction mode decoding unit performs, when decoding the prediction mode allocated to a specific prediction unit, arithmetic decoding, dynamically changing the probability of occurrence according to the context set according to the states of the set of fragments, the prediction mode information near the image area for the same color space, when decoding the value of the prediction mode of the decoding object represented by a binary sequence for each th section of the binary sequence. 29. The image decoder of claim 28, wherein the prediction mode decoding unit uses, as a context established according to the states of the plurality of pieces of prediction information near the image area for the same color component, a state indicating which of the predictions in the vertical direction of the image surface and prediction in the horizontal direction of the image surface are classified prediction modes. 30. The image decoder of claim 28, wherein the prediction mode decoding unit uses, as a context established according to the states of the plurality of pieces of prediction information near the image area for the same color component, a state indicating which of the DC prediction and prediction is different from DC prediction, is a prediction mode. 31. A method of encoding an image comprising the steps of:
generating a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image,
evaluating the forecasting efficiency of the generated predicted image for making a decision regarding a predetermined prediction mode and individually making a decision about the prediction mode for each of the color components in units of image areas of the prediction object, and
the output signal obtained at the decision-making stage regarding the prediction mode is subjected to coding with a variable word length and the lookup table obtained by tabulating combinations of states of a plurality of pieces of information of the prediction mode near the image area for the same color component and predicted values for the prediction modes of the encoding object is consulted, for a unique task of the predicted value of the information of the prediction mode from states with a plurality of pieces of information of the prediction mode near the image area and the implementation of coding. 32. A computer-readable recording medium on which an image encoding program is recorded instructing a computer to perform steps in which
generating a predicted image in accordance with a set of prediction modes indicating methods for generating a predicted image,
evaluating the forecasting efficiency of the generated predicted image for making a decision regarding a predetermined prediction mode and individually making a decision about the prediction mode for each of the color components in units of image areas of the prediction object, and
the output signal obtained at the decision-making stage regarding the prediction mode is subjected to coding with a variable word length and the lookup table obtained by tabulating combinations of states of a plurality of pieces of information of the prediction mode near the image area for the same color component and predicted values for the prediction modes of the encoding object is consulted, for a unique task of the predicted value of the information of the prediction mode from states with a plurality of pieces of information of the prediction mode near the image area and the implementation of coding. 33. A method of decoding an image, instructing the computer to perform the steps in which
prediction modes are decoded, individually encoded for the corresponding color components that form the input image signal, and when decoding the prediction mode allocated to a specific prediction block, they look up the lookup table obtained by tabulating combinations of states of a plurality of pieces of information of the prediction mode near the image region for an identical color component and predicted values for prediction modes of the encoding object, for unique about setting the predicted value of the information of the prediction mode from the states of the set of pieces of information of the prediction mode near the image area and decoding,
generating a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image, and
generating a predicted image according to the step of generating the predicted image based on the prediction modes of the respective color components set in the prediction mode decoding step. 34. A computer-readable recording medium on which an image decoding program is recorded instructing a computer to perform steps in which
prediction modes are decoded, individually encoded for the corresponding color components that form the input image signal, and when decoding the prediction mode allocated to a specific prediction block, they look up the lookup table obtained by tabulating combinations of states of a plurality of pieces of information of the prediction mode near the image region for an identical color component and predicted values for prediction modes of the decoding object, for unique oh setting the predicted value of the information of the prediction mode from the states of the set of pieces of information of the prediction mode near the image area and decoding,
generating a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image, and
generating a predicted image according to the step of generating the predicted image based on the prediction modes of the respective color components set in the prediction mode decoding step. 35. A method of encoding an image comprising the steps of:
generating a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image,
evaluating the forecasting efficiency of the generated predicted image for making a decision regarding a predetermined prediction mode and individually making a decision about the prediction mode for each of the color components in units of image areas of the prediction object, and
subjected to the output signal obtained at the decision-making stage regarding the prediction mode, change the value of the prediction mode of the encoding object in a binary sequence and, when encoding the corresponding sections of the binary sequence, perform arithmetic coding, dynamically changing the probability of occurrence according to the context established according to the states of the set of fragments of the mode information prediction near image area for identical color component. 36. A computer-readable recording medium onto which an image encoding program is recorded instructing a computer to perform steps in which
generating a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image,
evaluate the forecasting efficiency of the generated predicted image to make a decision regarding a predetermined forecasting mode and individually decide on a forecast mode for each of the color components in units of image areas of the prediction object,
subjected to the output signal obtained at the decision-making stage regarding the prediction mode, change the value of the prediction mode of the encoding object in a binary sequence and, when encoding the corresponding sections of the binary sequence, perform arithmetic coding, dynamically changing the probability of occurrence according to the context established according to the states of the set of fragments of the mode information prediction near image area for identical color component. 37. A method of decoding an image comprising the steps of:
the prediction modes are decoded, individually encoded for the corresponding color components forming the input image signal, and, when decoding the prediction mode allocated to the specific prediction block, arithmetic decoding is performed, dynamically changing the probability of occurrence according to the context established according to the states of the set of pieces of information of the prediction mode near the region images for identical color space, when decoding is a prediction mode of a decoding entity represented by a binary sequence for each portion of the binary sequence,
generating a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image, and
generating a predicted image according to the step of generating the predicted image based on the prediction modes of the respective color components set in the decoding step of the prediction mode for decoding the image data. 38. A computer-readable recording medium on which an image decoding program is recorded instructing a computer to perform steps in which
the prediction modes are decoded, individually encoded for the corresponding color components forming the input image signal, and, when decoding the prediction mode allocated to the specific prediction block, arithmetic decoding is performed, dynamically changing the probability of occurrence according to the context established according to the states of the set of pieces of information of the prediction mode near the region images for identical color space, when decoding is a prediction mode of a decoding entity represented by a binary sequence for each portion of the binary sequence,
generating a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image, and
generating a predicted image according to the step of generating the predicted image based on the prediction modes of the respective color components set in the decoding step of the prediction mode for decoding the image data. 39. An image encoder containing
a predicted image generation unit that generates a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image, a set of reference picture identification numbers and a set of motion vectors,
a decision unit for the prediction mode that evaluates the prediction efficiency of the predicted image output from the predicted image generation unit for deciding on a predetermined prediction mode, a predetermined reference picture identification number and a predetermined motion vector, and
a coding unit that subjects the output of the decision unit regarding the prediction mode to coding with a variable word length, wherein
the decision unit regarding the prediction mode makes a decision based on a predetermined control signal whether the common prediction mode, the identification number of the common reference image and the common motion vector are used for the corresponding color components that form the input image signal, or whether a separate prediction mode, the identification number of the individual reference image and a separate motion vector for each of the color components, for the corresponding colors x components, multiplexes the information about the control signal into the bitstream, multiplexes when using the common prediction mode, the identification number of the common reference image and the common motion vector, the information of the general prediction mode, the identification number of the common reference image and the common motion vector into the bit stream, and multiplexes, when the common prediction mode, the identification number of the common reference image and the common motion vector are not used, the forecast mode information ation, the reference image identification number and motion vector for each of the color components on the bit stream. 40. The image encoder according to § 39, in which the decision block regarding the prediction mode makes a decision regarding the prediction mode in units of macroblocks and processes the prediction mode based on a control signal that changes in units of macroblocks. 41. The image encoder of claim 39, wherein the decision block regarding the prediction mode makes a decision regarding the prediction mode in units of macroblocks and processes the prediction mode based on a control signal that changes in units of a sequence including a plurality of frames. 42. The image encoder of claim 39, wherein the decision block regarding the prediction mode multiplexes when the common prediction mode, the identification number of the common reference image and the common motion vector, the information of the general prediction mode, the identification number of the common reference image and the common motion vector in the bitstream as identification information of the general prediction mode, and multiplexes when the general prediction mode, the identification number The common reference image and the common motion vector are not used, the prediction mode information for each of the color components, the identification number of the common reference image and the common motion vector in the bitstream are used as identification information for the common use prediction mode. 43. An image decoder containing
a decoding unit that decodes the identification information of the common use prediction mode indicating whether the common prediction mode, the identification number of the common reference image and the common motion vector are used for the respective components forming the input image signal, or the separate prediction mode, separate identification number, and separate vector are used motion for each of the color components, and decodes prediction modes, reference numbers are reference th images and motion vectors of the respective color components based on the value of the identification information of the general prediction mode, and
a predicted image generating unit that generates a predicted image based on the predicted modes, reference picture identification numbers and motion vectors decoded by the decoding unit,
the decoding unit performs decoding of the image data based on the predicted image generated by the predicted image generation unit. 44. The image decoder according to item 43, in which the decoding unit performs decoding in units of macroblocks and decodes and uses the identification information of the prediction mode of General use in units of a sequence including a set of frames. 45. A method of encoding an image comprising the steps of:
generating a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image, a set of identification numbers of reference images, and a set of motion vectors,
evaluate the forecasting efficiency of the generated predicted image for deciding on a predetermined prediction mode, a predetermined reference image identification number and a predetermined motion vector, make a decision based on a predetermined control signal whether the common prediction mode, the identification number of the common reference image and the common motion vector for the corresponding color components that form the input image signal, or a separate prediction mode, the identification number of a separate reference image and a separate motion vector for each of the color components are used, and information about the control signal is multiplexed into a bit stream, multiplexed when the common prediction mode, the identification number of the common reference image and the common motion vector, information of the general prediction mode, identification number of the common reference image and the common motion vector in bits the output stream and multiplex when the common prediction mode, the identification number of the common reference image and the common motion vector are not used, the information of the prediction mode, the identification number of the reference image and the motion vector for each of the color components in the bit stream, and
subjected to the output signal obtained at the stage of decision making, encoding with a variable word length. 46. A computer-readable recording medium on which an image encoding program is recorded instructing a computer to perform steps in which
generating a predicted image in accordance with a set of prediction modes indicating a method for generating a predicted image, a set of identification numbers of reference images, and a set of motion vectors,
evaluate the forecasting efficiency of the generated predicted image for deciding on a predetermined prediction mode, a predetermined reference image identification number and a predetermined motion vector, make a decision based on a predetermined control signal whether the common prediction mode, the identification number of the common reference image and the common motion vector for the corresponding color components that form the input image signal, or a separate prediction mode, the identification number of a separate reference image and a separate motion vector for each of the color components are used, and information about the control signal is multiplexed into a bit stream, multiplexed when the common prediction mode, the identification number of the common reference image and the common motion vector, information of the general prediction mode, identification number of the common reference image and the common motion vector in bits the output stream and multiplex when the common prediction mode, the identification number of the common reference image and the common motion vector are not used, the information of the prediction mode, the identification number of the reference image and the motion vector for each of the color components in the bit stream, and
subjected to the output signal obtained at the stage of decision making, encoding with a variable word length. 47. A method of decoding an image comprising the steps of:
decode the general prediction mode identification information indicating whether the common prediction mode, the identification number of the common reference image and the common motion vector are used for the respective color components forming the input image signal, or whether a separate prediction mode, a separate identification number, and a separate motion vector are used for each from color components, and predicting prediction modes, reference picture identification numbers and motion ctori for the respective color components based on the value of the identification information of the general prediction mode,
generating a predicted image based on the decoded prediction modes, reference picture identification numbers and motion vectors, and
decode image data based on the generated predicted image. 48. A computer-readable recording medium on which an image decoding program is recorded instructing a computer to perform steps in which
decode the general prediction mode identification information indicating whether the common prediction mode, the identification number of the common reference image and the common motion vector are used for the respective color components forming the input image signal, or whether a separate prediction mode, a separate identification number, and a separate motion vector are used for each from color components, and predicting prediction modes, reference picture identification numbers and motion vectors of the respective color components based on the value of the identification information of the common use prediction mode,
generating a predicted image based on the decoded prediction modes, reference picture identification numbers and motion vectors, and
decode image data based on the generated predicted image. 49. An image encoder that subjects an input image signal including a plurality of color components to image compression using motion compensation prediction for each of a predetermined partial image region, the image encoder comprises
a decision block regarding a coding mode that determines, based on a predetermined estimation standard, coding modes including indicating motion compensated prediction modes for indicating whether the image signal included in the partial image region is intra-encoded or encoded according to inter-frame prediction with motion compensation, and instructions for generating the predicted image when encoding the image signal according to the inside Adrov motion-compensated prediction,
a predicted image generation unit that generates a motion-compensated predicted image based on motion-compensated prediction modes, reference image identification numbers and motion vectors, and
an encoding unit that exposes encoding modes, reference image identification numbers and motion vectors to variable-length encoding for each of the partial image areas, and multiplexes encoding modes, reference image identification numbers and motion vectors into a bit stream, wherein
the decision block regarding the encoding mode makes a decision, according to a predetermined control signal, whether the common encoding mode is applied to the respective color components forming the partial image area, or whether a separate encoding mode is applied to each of the color components,
the coding unit multiplexes information about the control signal into a bitstream, multiplexes when the common coding mode is applied to the corresponding color components forming the partial image area, the common coding mode, the identification number of the common reference image and the common motion vector into the bitstream, and multiplexes when the coding modes individually apply to the corresponding color components forming the region of the partial image, a separate encoding mode, identically tele- communications separate reference image number and a separate motion vector on the bit stream for each of the color components, and
the predicted image generation unit generates a predicted image for the respective color components based on the coding mode, the reference image identification number and the motion vector set according to the control signal. 50. The image encoder according to § 49, further comprising a decision block regarding skipping of the partial image area, which estimates the result of motion-compensated intra-frame prediction and decides whether encoding information to be transmitted is present for the partial image area,
the decision block regarding skipping the partial image area makes a decision when the control signal indicates that the common coding mode is applied to the corresponding color components forming the partial image area regarding the presence or absence of coding information in the partial image area based on the presence of coding information for at least at least one of the color components in the area of a partial image or lack of coding information for all color com onentov and decides, when the control signal indicates that the separate encoding modes are applied to the respective color components forming the partial image region, with respect to the presence or absence of encoding information for each of the color components, and
the coding unit multiplexes, in accordance with the result of the decision block regarding skipping the partial image area, skipping information of the image area indicating the presence or absence of coding information in the partial image area into the bit stream in units of the partial image area in which the plurality of color components are assembled, or in units of the respective color components. 51. An image decoder that decodes image data with a bit stream, which is obtained when an input image signal including a plurality of color components is compressed encoded using motion-compensated prediction in units of predetermined areas of a partial image, the image decoder contains
a decoding unit that extracts information about the control signal from the bitstream indicating whether the image signal included in the partial image area is intra-encoded or encoded according to inter-frame prediction with motion compensation, and indicating whether encoding modes including mode indication are decoded motion-compensated prediction for indicating methods for generating a predicted image when encoding an image signal according to inter-frame motion compensated prediction, as information common to all color components in the partial image areas, or as information for each of the color components in the partial image area, and decodes coding modes, reference image identification numbers, and motion vectors for each of the partial image areas in accordance with the information about the control signal and extracts encoding modes, identification numbers of the reference image and motion vectors from the bitwise current, and
a predicted image generating unit that generates a motion compensated predicted image based on motion compensated prediction modes, reference image identification numbers and motion vectors, wherein
the decoding unit extracts when the information about the control signal indicates that the common encoding mode applies to the corresponding color components forming the partial image area, the common encoding mode, the identification number of the common reference image and the common motion vector from the bitstream, and extracts when the encoding modes separately apply to the corresponding color components forming the partial image area, a separate encoding mode, the reference number of the reference images and motion vector from the bitstream for each of the color components, and
the predicted image generation unit generates a motion compensated predicted image for the respective color components based on the encoding mode, the reference image identification number and the motion vectors extracted by the decoding unit, in accordance with the information of the control signal. 52. The image decoder according to § 51, in which
the decoding unit includes a skip information decoding unit that decodes skipping information of the image area indicating whether encoding information to be transmitted to the partial image area is present as a result of inter-frame prediction with motion compensation in units of the partial image area, wherein
the skip information decoding unit decodes when the control signal indicates that the common coding mode is applied to the respective color components forming the partial image area, skip information of the image area common to all color components to identify the presence of coding information in the partial image area and decodes when a control signal indicates that individual coding modes are applied to the respective color components forming the partial region The images, image region skip information for each of the color components to identify presence of encoding information in the partial image area, and
the decoding unit decodes when the encoding information to be transmitted is present, the encoding mode, the reference image identification number and the motion vector based on the skip information of the image area, and extracts the encoding mode, the reference image identification number and the motion vector from the bitstream. 53. An image encoding method according to which an input image signal including a plurality of color components is subjected to image compression using motion compensation prediction for each of a predetermined region of a partial image, the image encoding method comprising
a decision-making step regarding the encoding mode, in which, based on a predetermined estimation standard, encoding modes including indicating motion compensation prediction modes for indicating whether the image signal included in the partial image region is subjected to intra-frame encoding or is encoded according to inter-frame motion-compensated prediction, and guidance on methods for generating a predicted image when encoding an image signal according to internal motion compensation frame prediction,
a predicted image generation step of generating a motion-compensated predicted image based on motion-compensated prediction modes, reference image identification numbers and motion vectors, and
a coding step in which coding modes, reference picture identification numbers and motion vectors are subjected to variable-length encoding for each of the partial image areas, and coding modes, reference picture identification numbers and motion vectors are multiplexed into a bit stream, wherein
at the stage of deciding on the encoding mode, a decision is made according to a predetermined control signal whether the common encoding mode is applied to the respective color components forming the partial image area, or whether a separate encoding mode is applied to each of the color components,
in the encoding step, information about the control signal is multiplexed into the bitstream, multiplexed when the common encoding mode is applied to the corresponding color components forming the partial image area, the common encoding mode, the identification number of the common reference image and the common motion vector into the bit stream, and multiplexed when the modes encodings are individually applied to the corresponding color components forming the region of the partial image, a separate encoding mode, ide the identification number of the individual reference image and a separate motion vector in the bit stream for each of the color components, and
in the predicted image generation step, a motion compensated predicted image is generated for the respective color components based on the coding mode, the reference image identification number and the motion vector set according to the control signal. 54. A computer-readable recording medium on which an image encoding program is recorded instructing a computer to execute
a decision step regarding the encoding mode, which indicates whether the image signal included in the predetermined region of the partial image is intraframe encoded or encoded according to the intra-frame prediction with motion compensation, and a decision is made according to the predetermined control signal whether the common mode is applied encoding to a plurality of color components forming a partial image region, or a separate encoding mode is applied to each from color components
a predicted image generation step of generating a motion compensated predicted image for the respective color components based on coding modes, reference image identification numbers and motion vectors set according to the control signal, and
a coding step in which information about a control signal is multiplexed into a bit stream, multiplexed when a common coding mode is applied to corresponding color components forming a partial image area, a common coding mode, an identification number of a common reference image and a common motion vector into a bit stream, and multiplexed, when the encoding modes are individually applied to the respective color components forming the partial image region, a separate encoding mode identification, the identification number of a single reference image and a separate motion vector into the bitstream for each of the color components. 55. An image decoding method, according to which image data is decoded with a bit stream, which is obtained when an input image signal including a plurality of color components is subjected to compression coding using prediction with motion compensation in units of predetermined areas of a partial image, wherein the image decoding method comprises
a decoding step of extracting information about the control signal from the bitstream indicating whether the image signal included in the partial image area is intra-encoded or encoded according to inter-frame prediction with motion compensation, and indicating whether encoding modes including the indication are decoded motion compensation prediction modes for indicating methods for generating a predicted image when encoding an image signal according to inter-frames motion prediction, as information common to all color components in the partial image areas, or as information for each color component in the partial image area, and coding modes, reference image identification numbers, and motion vectors for each of the areas are decoded partial image in accordance with the information about the control signal and extract the encoding modes, identification numbers of the reference image and motion vectors from the bitmap stream, and
a step for generating a predicted image in which a predicted image with motion compensation is generated based on the modes
prediction with motion compensation, reference image identification numbers and motion vectors, moreover
in the decoding step, it is retrieved when the information about the control signal indicates that the common coding mode is applied to the corresponding color components forming the partial image area, the common coding mode, the identification number of the common reference image and the common motion vector from the bitstream, and is extracted when the coding modes by separately apply to the corresponding color components forming the partial image area, a separate encoding mode, reference identification number o images and motion vector from the bitstream for each of the color components, and
in the predicted image generation step, a motion compensated predicted image is generated for the respective color components based on the encoding mode, the reference image identification number and motion vectors extracted in the decoding step in accordance with the control signal information. 56. A computer-readable recording medium on which an image decoding program is recorded instructing a computer to execute
a decoding step of extracting information about the control signal from the bitstream indicating whether the encoding mode is decoded, which includes indicating a motion-compensated prediction mode for indicating whether the image signal included in the predetermined partial image region is intra-frame encoded or encoded according to intraframe prediction with motion compensation, and indicating a method for generating a predicted image when encoding an image signal According to the motion compensation intra-frame prediction, as information common to all color components in the partial image area, or as information for each of the color components in the partial image area, is extracted when the control signal information indicates that the common coding mode is applied to the respective color components forming the partial image region, the common coding mode, the identification number of the common reference image and the common vector motion from the bitstream and retrieved when the information about the control signal indicates that the coding modes are individually applied to the respective color components forming the partial image area, a separate coding mode, an identification number of a separate reference image, and a separate motion vector for each of the color components from the bit flow, and
a motion compensation predicted image generating step of generating a motion compensation predicted image for the respective color components based on the encoding mode, the reference image identification number and the motion vector extracted in the decoding step in accordance with the control signal information. 57. An image encoder containing
a predicted image generation unit that generates a predicted image in accordance with a plurality of pieces of overhead information indicating a method for generating a predicted image,
a decision unit that evaluates the prediction efficiency of the predicted image output from the predicted image generation unit and makes a decision regarding a predetermined overhead information, and
a coding unit that subjects the output of the decision unit to coding with a variable word length, wherein
the decision unit makes a decision, based on the common use identification information, whether common overhead information is used for the respective color components constituting the input image signal, or if separate overhead information is used for each color component, multiplexes the common use identification information into a bit stream, multiplexes, when general overhead information is used, general overhead information in a bitstream and multiplexes when used Separate overhead information, overhead information for each of the color components into the bitstream is provided. 58. An image decoder containing
a decoding unit that decodes common use identification information indicating whether common overhead information is used for the respective color components forming the input image signal, or separate overhead information is used for each color component, and decodes overhead information for the respective color components based on the information value identification of common use, and
a predicted image generating unit that generates a predicted image based on overhead information decoded by a decoding unit, wherein
the decoding unit performs decoding of the image data based on the predicted image generated by the predicted image generation unit. 59. An image encoder containing
a predicted image generation unit that generates a predicted image in accordance with a plurality of pieces of overhead information indicating a method for generating a predicted image,
a decision unit that evaluates the prediction efficiency of the predicted image output from the predicted image generation unit for deciding on a predetermined overhead information, and
a coding unit that subjects the output of the decision unit to coding with a variable word length, wherein
the decision unit multiplexes information about a predetermined control signal, including the number of color components, into the bitstream and multiplexes overhead information for the corresponding color components into the bitstream based on the control signal. 60. An image decoder containing
a decoding unit that decodes a predetermined control signal including information on the number of color components constituting the input image signal, and decodes overhead information for the corresponding color components based on the value of the control signal, and
a predicted image generating unit that generates a predicted image based on overhead information decoded by the decoding unit and a control signal including information on the number of color components, wherein
the decoding unit performs decoding of the image data based on the predicted image generated by the predicted image generation unit. 61. An image encoder containing
a predicted image generation unit that generates a predicted image in accordance with a plurality of pieces of overhead information indicating a method for generating a predicted image,
a decision unit that evaluates the prediction efficiency of the predicted image output from the predicted image generation unit for deciding on a predetermined overhead information, and
a coding unit that subjects the output of the decision unit to coding with a variable word length, wherein
a decision unit multiplexes a color component identification flag to determine an image to which
the slice data belongs to the bitstream and multiplexes the overhead information of the respective color components into the bitstream based on the identification flag of the color component to determine the image to which the slice data belongs. 62. An image decoder containing
a decoding unit that decodes the color component identification flag to determine the image to which the slice data belongs, and decodes overhead information of the corresponding color components based on the color component identification flag to determine the image to which the slice data belongs, and
a predicted image generating unit that generates a predicted image based on overhead information decoded by a decoding unit, wherein
the decoding unit performs decoding of the image data based on the predicted image generated by the predicted image generation unit. 63. An image decoder containing
a decoding unit that decodes, in units of slices, a color component identification flag for determining an image to which the slice data belongs, and generates a decoded image of respective color components based on a color component identification flag for determining a image to which the slice data belongs,
moreover, the decoding unit performs decoding of the image from the slice encoding data set having the identical value of the identification flag of the color component to determine the image to which the slice data belongs.

Images