JP2003319394A - Encoding apparatus and method, decoding apparatus and method, recording medium, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image encoding apparatus for receiving image information with an interlaced scanning format, applying field/frame adapted encoding processing to the information at a macro block level and employing the ABT (Adaptive Block Transform) for orthogonal transformation. <P>SOLUTION: A field/frame discrimination section 53 discriminates a field- based encoding of a micro block or a frame-based encoding of the macro block which has a higher encoding efficiency. An ABT block size decision section 55 decides a block size providing the highest encoding efficiency. A field/frame conversion section 56 converts a received image into an image with a field structure before the orthogonal transform processing when the macro block is subjected to field-based encoding. An ABT conversion section 57 applies orthogonal transform processing to the received image according to the decided block size. An inverse encoding section 59 applies inverse encoding to a Frame/ Field Flag or the like. This invention is applicable to an image information encoding apparatus and an image information decoding apparatus. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an encoding device and method, a decoding device and method, a recording medium, and a program, for example, when an image signal is encoded at a higher compression rate than before and is transmitted or stored. The present invention relates to an encoding device and method suitable for use, a decoding device and method, a recording medium, and a program.

[0002]

BACKGROUND ART Recently, for the purpose of handling an image as a digital signal and efficiently transmitting and accumulating the digital signal, orthogonality transform such as discrete cosine transform and motion compensation are performed by utilizing redundancy peculiar to image information. MP to compress
A device conforming to a method such as EG (Moving Picture Expert Group) is becoming popular both for information distribution at broadcasting stations and for information reception at general homes.

In particular, the MPEG2 (ISO / IEC 13818-2) compression method is a standard defined as a versatile image compression method, and includes both interlaced scanning images and progressive scanning images, as well as standard resolution images and high definition images. It is a standard that covers images and is currently widely used in a wide range of applications for professional use and consumer use, as represented by the DVD (Digital Versatile Disk) standard.

By using the MPEG2 compression method, for example, 4 to 8 Mbps for a standard resolution interlaced scan image having 720 × 480 pixels and 18 for a high resolution interlaced scan image having 1920 × 1088 pixels. By assigning a code amount (bit rate) of 22 Mbps to 22 Mbps, a high compression rate and good image quality can be realized.

By the way, MPEG2 was mainly intended for high-quality coding suitable for broadcasting, but since it did not support a coding method with a higher compression rate, it was used as a coding method with a higher compression rate. , MPEG4 encoding system was standardized. Regarding the image encoding method, 1998 1
In February, the standard was approved as an international standard as ISO / IEC 14496-2.

Further, in recent years, ITU-T (International Telecommunication Unio), which is a telecommunications standardization division of the International Telecommunications Union, has been initially designed for the purpose of image coding for video conferences.
Standardization called H.26L (ITU-T Q6 / 16 VCEG) by the n-Telecommunication Standardization Sector) is under way.

H. 26L requires a larger amount of calculation in the encoding process and the decoding process as compared with the conventional encoding methods such as MPEG2 and MPEG4, but is known to achieve higher encoding efficiency. There is.

[0008] Furthermore, as a part of the activities of MPEG4, the H.264 standard is currently being collaborated. H.26L based on H.26. 26
Joint Mode is a standardization of coding technology that achieves higher coding efficiency by incorporating functions not supported by L.
l of Enhanced-Compression Video Coding.

Here, a conventional image information coding apparatus using orthogonal transformation such as discrete cosine transformation or Karhunen-Loeve transformation and motion compensation will be described with reference to FIG. FIG. 1 shows an example of the configuration of a conventional image information encoding device.

In the image information coding apparatus, an input image signal which is an analog signal is converted into a digital signal by the A / D converter 1 and then supplied to the screen rearrangement buffer 2. The screen rearrangement buffer 2 rearranges the image information from the A / D converter 1 according to the GOP (Group of Pictures) structure of the image compression information output by the image information encoding apparatus.

First, an image in which intra (intra-image) coding is performed will be described. Regarding the image to be intra-coded in the screen rearrangement buffer 2, the image information is supplied to the orthogonal transformation unit 4 via the adder 3. The orthogonal transform unit 4 performs orthogonal transform (discrete cosine transform, Karhunen-Loeve transform, etc.) on the image information, and the obtained transform coefficient is supplied to the quantizer 5.
In the quantization unit 5, according to the control from the rate control unit 8 based on the data amount of the conversion coefficient accumulated in the accumulation buffer 7,
Quantization processing is performed on the transform coefficient supplied from the orthogonal transform unit 4.

In the lossless encoding unit 6, the encoding mode is determined from the quantized transform coefficient and the quantization scale supplied from the quantization unit 5, and the lossless encoding for the determined encoding mode ( Variable length coding, arithmetic coding or the like) is performed to form information to be inserted in the header part of the image coding unit. Further, the encoded coding mode is supplied to and accumulated in the accumulation buffer 7. Accumulation buffer 7
The coded coding mode accumulated in is output to the subsequent stage as image compression information.

In addition, the lossless encoding unit 6 performs lossless encoding on the quantized transform coefficient, and stores the encoded transform coefficient in the accumulation buffer 7. Accumulation buffer 7
The coded transform coefficients accumulated in the above are also output to the subsequent stage as image compression information.

The inverse quantization unit 9 inversely quantizes the transform coefficient quantized by the quantization unit 5. Inverse orthogonal transform unit 1
In 0, the inverse quantized transform coefficient is applied to the inversely quantized transform coefficient to generate decoded image information. The loop filter 11 removes the block distortion of the generated decoded image information. The decoded image information from which the block distortion has been removed is stored in the frame memory 12.

Next, an image to be inter-coded will be described. Regarding the image to be inter-encoded in the screen rearrangement buffer 2, the image information is supplied to the adder 3 and the motion prediction / compensation unit 13.

In the motion prediction / compensation unit 13, image information for reference corresponding to the image to be inter-encoded from the screen rearrangement buffer 2 is read from the frame memory 12 and the motion prediction / compensation process is performed. Then, the reference image information is generated and supplied to the adder 3. Further, the motion vector information obtained during the motion prediction / compensation processing by the motion prediction / compensation unit 13 is supplied to the lossless coding unit 6.

In the adder 3, the reference image information from the motion prediction / compensation unit 13 is converted into a difference signal from the image information of the image to be inter-encoded from the screen rearrangement buffer 2.

When processing an image to be inter-coded, the orthogonal transform unit 4 performs an orthogonal transform on the difference signal, and the obtained transform coefficient is supplied to the quantizer 5.
In the quantizer 5, the transform coefficient supplied from the orthogonal transformer 4 is quantized under the control of the rate controller 8.

The lossless encoding unit 6 determines the encoding mode based on the transform coefficient and the quantization scale quantized by the quantization unit 5, the motion vector information supplied from the motion prediction / compensation unit 13, and the like. Then, lossless encoding is performed on the determined encoding mode, and information to be inserted into the header portion of the image encoding unit is generated. The encoded coding mode is stored in the storage buffer 7. The encoded encoding mode accumulated in the accumulation buffer 7 is output as image compression information.

Further, the lossless coding unit 6 performs lossless coding processing on the motion vector information from the motion prediction / compensation unit 13 to generate information to be inserted in the header part of the image coding unit. .

The processing after the inverse quantizer 9 in the case of processing an image to be inter-coded is similar to that in the case of processing an image to be intra-coded, and therefore its explanation is omitted. To do.

Next, a conventional image information decoding apparatus for inputting the image compression information output from the conventional image information encoding apparatus shown in FIG. 1 to restore an image signal will be described with reference to FIG. FIG. 2 shows an example of the configuration of a conventional image information decoding device.

In the image information decoding apparatus, the input image compression information is temporarily stored in the accumulation buffer 21 and then transferred to the lossless decoding unit 22. The lossless decoding unit 22 performs lossless decoding (variable length decoding, arithmetic decoding, or the like) on the image compression information based on a predetermined format of the image compression information, and the encoding mode stored in the header section. The information is acquired and supplied to the inverse quantization unit 23. Similarly, the lossless decoding unit 22 acquires the quantized transform coefficient and supplies it to the inverse quantization unit 23. Further, when the frame to be decoded is inter-coded, the lossless decoding unit 22 also decodes the motion vector information stored in the header portion of the image compression information,
The information is supplied to the motion prediction / compensation unit 30.

The dequantization unit 23 dequantizes the quantized transform coefficient supplied from the lossless decoding unit 22, and supplies the obtained transform coefficient to the inverse orthogonal transform unit 24. The inverse orthogonal transform unit 24 performs an inverse orthogonal transform (inverse discrete cosine transform, inverse Karhunen-Loeve transform, etc.) on the transform coefficient based on a predetermined image compression information format.
Give.

Here, when the target frame is intra-coded, the image information subjected to the inverse orthogonal transform is stored in the screen rearrangement buffer 26 via the adder 25, and D The signal is converted into an analog signal by the / A converter 27 and output to the subsequent stage. The image information subjected to the inverse orthogonal transformation is stored in the frame memory 29 after the block distortion is removed by the loop filter 28.

When the target frame is inter-coded, the motion vector information from the lossless decoding unit 22 and the image information stored in the frame memory 29 in the motion prediction / compensation unit 30. A reference image is generated on the basis of and and is supplied to the adder 25. In the adder 25, the reference image from the motion prediction / compensation unit 28 and the output from the inverse orthogonal transform unit 25 are combined to generate image information. Note that the other processing is the same as that of the intra-coded frame, and therefore the description thereof is omitted.

By the way, in the MPEG2, when the input image signal is in the interlaced scanning format, the field / frame adaptive encoding processing can be performed at the macroblock level.

Currently, H.264. No such specification is defined in 26L, but the document "" Interlace Coding Tools fo
r H.26L Video Coding (L.Wang et al., VCEG-O37, Dec.2
001) "" (hereinafter referred to as Reference 1) is described in H. 26L
It has been proposed to extend the specification of (1) to enable field / frame adaptive coding processing at the macroblock level.

The field / frame adaptive encoding process at the macroblock level, which is proposed in Document 1, will be described.

The current H.264. In 26L, seven types of modes (modes 1 to 7) as shown in FIG. 15 are defined as units of motion prediction / compensation in a macroblock.

Document 1 proposes that a syntax corresponding to a macroblock of image compression information has a Frame / Field Flag between Run and MB_type as shown in FIG. When the value of Frame / Field Flag is 0, it indicates that the macroblock is subjected to frame-based coding, and when the value of Frame / Field Flag is 1, it is subjected to field-based coding. It is shown that.

When the value of Frame / Field Flag is 1, that is, when field-based coding is performed,
The pixels in the macroblock are rearranged in units of rows as shown in FIG.

When the value of Frame / Field Flag is 1, five types of modes (modes 1a to 5a) shown in FIG. 6 corresponding to modes 3 to 7 in FIG. 3 are defined as units of motion prediction / compensation in the macroblock. Has been done.

For example, in mode 2a of FIG. 6, among 8 × 8 blocks 0 to 3 obtained by dividing a macroblock into four,
Blocks 0 and 1 belong to the same field parity, and blocks 2 and 3 belong to the same field parity. Further, for example, in mode 3a of FIG. 6, out of 4 × 8 blocks 0 to 8 obtained by dividing a macro block into eight, blocks 0 to 3 belong to the same field parity, and blocks 4 to 7 belong to the same field parity. .

The intra prediction mode when the value of Frame / Field Flag is 1 will be described. For example, pixels a to p located in the 4 × 4 block shown in FIG.
Even if the value of Field Flag is 1, it is adjacent 4
Intra prediction is performed using the pixels A to I located in the × 4 block, but the pixels a to p and the pixels A to I are used.
Are all belonging to the same field parity.

A case where the pixels A to I belong to the same macroblock as the pixels a to p will be described with reference to FIG. The pixels a to p existing in the 4 × 4 block 7 obtained by dividing the macroblock into 16 are subjected to intra prediction using the pixels A to I existing at the ends of the adjacent blocks 2, 3, and 6.

A case where the pixels A to I belong to a macro block different from the pixels a to p will be described with reference to FIG.

FIG. 9A shows a case where the values of the Frame / Field Flags for the macroblock on the left side and the macroblock on the upper side of the macroblock to be processed are 1 respectively. In this case, the intra prediction of the pixels existing in the 4 × 4 block C obtained by dividing the macro block to be processed into 16 is performed by the pixels existing in the 4 × 4 block A obtained by dividing the left macro block into 16 and the upper macro. It is performed using the pixels existing in a 4 × 4 block B obtained by dividing a block into 16 parts. Intra prediction of the pixels existing in the 4 × 4 block C ′ is performed using the pixels existing in the 4 × 4 block A ′ and the pixels existing in the 4 × 4 block B ′.

In FIG. 9B, the value of the Frame / Field Flag for the macroblock to be processed is 1, and the Frame / Field Fl for the macroblocks on the left and upper sides of the Frame / Field Flag.
The case where the value of ag is 0 is shown. In this case, the intra prediction of the pixels existing in the 4 × 4 block C obtained by dividing the macro block to be processed into 16 is performed by the pixels existing in the 4 × 4 block A obtained by dividing the left macro block into 16 and the upper macro block. Is performed by using the pixels existing in the 4 × 4 block B obtained by dividing 16 into 16 blocks. Four
The intra prediction of the pixels existing in the × 4 block C ′ is 4
Pixels existing in × 4 block A ′ and 4 × 4 block B
Is performed using the pixels existing in.

Next, regarding intra prediction of color difference signals,
This will be described with reference to FIG. Frame / Field Flag value is 1
If, then only one type of intra-prediction mode for color difference signals is defined.

In FIG. 10, A to D respectively represent 4 × 4 blocks of color difference signals. Blocks A and B belong to the first field, and blocks C and D belong to the second field. s _{0 to} s ₂ are the total values of the color difference signals existing in the blocks belonging to the first field parity among the blocks adjacent to the blocks A to D. s ₃ to s
₅ is a total value of the color difference signals existing in the blocks belonging to the second field parity among the blocks adjacent to the blocks A to D.

The prediction values A to D respectively corresponding to the blocks A to D are predicted according to the following equation (1) when s _{0 to} s ₅ are all in the image frame. A = (s ₀ + s ₂ +4) / 8 B = (s ₁ +2) / 4 C = (s ₃ + s ₅ +4) / 8 D = (s ₄ +2) / 4 (1)

[0043] However, one of the s ₀ to _{s 5, s 0, s 1} , s
_If only ₃ and s ₄ exist in the image frame, blocks A to D
Predicted values A to D respectively corresponding to are predicted according to the following equation (2). A = (s ₀ +2) / 4 B = (s ₁ +2) / 4 C = (s ₃ +2) / 4 D = (s ₄ +2) / 4 (2)

[0044] Furthermore, among the s ₀ to s _5, if only s ₂ s ₅ exist in the picture frame, the predicted values corresponding to the blocks A to D, are predicted according to Expressions (3). A = (s ₂ +2) / 4 B = (s ₂ +2) / 4 C = (s ₅ +2) / 4 D = (s ₅ +2) / 4 (3)

FIG. 11 shows a method of coding the residual component of the color difference signal after intra prediction as described above. That is, after performing orthogonal transform processing on each 4 × 4 block, a 2 × 2 block as shown in the figure is generated using the DC components of the first field and the second field, and the orthogonal transform processing is performed again. To be done.

Next, the motion prediction / compensation process when the value of Frame / Field Flag is 1 will be described. Fram
If the value of e / Field Flag is 1, the motion prediction compensation mode is inter 16 × 16 mode, inter 8 × 1
There are 6 types of modes: 6 mode, inter 8 × 8 mode, inter 4 × 8 mode, and inter 4 × 4 mode.

For example, the inter 16 × 16 mode is a mode in which the motion vector information for the first field, the motion vector information for the second field, and the reference frame in the inter 8 × 16 mode are equivalent.

Code_Numbers 0 to 5 are assigned to the six types of motion prediction / compensation modes, respectively.

The current H.264. In 26L, multiple frame prediction in which a plurality of reference frames can be provided as shown in FIG. 12 is defined. Current frame-based H.264. In the 26L standard, the information about the reference frame is defined at the macroblock level, and Code_Number is defined for the frame encoded immediately before.
0 is assigned, and Code_Number 1 to 5 are assigned to the frames coded 1 to 5 times before.

On the other hand, when field-based coding is performed, Code_Number0 is assigned to the first field of the frame coded immediately before, and Code_Number1 is assigned to the second field. Part 1
For the first field of the frame encoded the previous time
Code_Number2 is assigned and Code_Number3 is assigned to the second field. Code_Num for the first field of the frame encoded one more time before
ber4 is assigned and Code_Nu for the second field
mber5 is assigned.

Further, with respect to a macroblock in which field-based coding is performed, a reference field for the first field and a reference field for the second field are separately defined.

Next, the motion vector information prediction method when the value of Frame / Field Flag is 1 will be described. The median prediction defined in 26L will be described with reference to FIG. 16 × 1 corresponding to the 16 × 16 macroblock E shown in FIG.
6, 8 × 8, or 4 × 4 motion vector information is predicted using the median of the motion vector information of adjacent macroblocks A to C.

However, among the macro blocks A to C,
For those that do not exist in the image frame, the median is calculated assuming that the value of the corresponding motion vector information is 0.
For example, when the macro blocks D, B, and C do not exist in the image frame, the motion vector information corresponding to the macro block A is used as the prediction value. When the macroblock C does not exist in the image frame, the median is calculated using the motion vector information of the macroblock D instead.

The reference frames of the macro blocks A to D do not necessarily have to be the same.

Next, the case where the block size of the macroblock is 8 × 16, 16 × 8, 8 × 4, or 4 × 8 will be described with reference to FIG. Note that the macroblock E of interest and the macroblocks A to D adjacent to the macroblock E are arranged as shown in FIG.

FIG. 14A shows a case where the block size of the macro blocks E1 and E2 is 8 × 16. Regarding the left macroblock E1, if the left-neighboring macroblock A refers to the same frame as the macroblock E1, the motion vector information of the macroblock A is used as a prediction value. When the macroblock A adjacent to the left refers to a frame different from the macroblock E1, the median prediction described above is applied.

Regarding the right macroblock E2, when the macroblock C adjacent to the upper right refers to the same frame as the macroblock E2, the motion vector information of the macroblock C is used as a prediction value. When the macroblock C adjacent to the upper right corner refers to a frame different from the macroblock E2, the median prediction described above is applied.

FIG. 14B shows the case where the block size of the macro blocks E1 and E2 is 16 × 8. Regarding the upper macroblock E1, when the macroblock B adjacent to the upper macroblock E1 refers to the same frame as the macroblock E1, the motion vector information of the macroblock B is used as a prediction value. If the macroblock B adjacent to the top refers to a frame different from the macroblock E1, the above-described median prediction is applied.

Regarding the lower macroblock E2, when the left-neighboring macroblock A refers to the same frame as the macroblock E2, the motion vector information of the macroblock A is used as a prediction value. When the macroblock A adjacent to the left refers to a frame different from the macroblock E2, the median prediction described above is applied.

FIG. 14C shows the macroblocks E1 to E8.
The block size is 8 × 4. The above median prediction is applied to the left macroblocks E1 to E4, and the right macroblock E5 is applied.
To E8, left macroblocks E1 to E
The motion vector information of 4 is used as the predicted value.

FIG. 14D shows the macroblocks E1 to E8.
The block size is 4 × 8. The above-described median prediction is applied to the upper macroblocks E1 to E4, and the lower macroblock E5 is applied.
To E8, upper macroblocks E1 to E8
The motion vector information of 4 is used as the predicted value.

Even when the value of Frame / Field Flag is 1, the prediction of the horizontal direction component of the motion vector information is based on the method described above with reference to FIGS. 13 and 14. However, with respect to the vertical component, field-based blocks and frame-based blocks coexist, so the following processing is performed. Note that the macroblock E of interest and the macroblocks A to D adjacent thereto are arranged as shown in FIG.

When the macroblock E is frame-based coded and any of the adjacent macroblocks A to D is field-based coded, the vertical component of the motion vector information for the first field, Two times the average value of the vertical component of the motion vector information for the second field is calculated, and the prediction process is performed by assuming that this is equivalent to the frame-based motion vector information.

When the macroblock E is field-based coded and any of the adjacent blocks A to D is frame-based coded, the value of the vertical component of the motion vector information is divided by two. Prediction processing is performed with the quotient corresponding to a field-based motion vector.

By the way, the present H.264. At 26L,
Quantization processing is performed after discrete cosine transform processing in units of 4 × 4 blocks.
BT Coding for Higher Resolution Video "(M. Wien et a
l. Joint Video Team of ISO / IEC MPEG & ITU-T VCEG, J
VT-B053) ”(hereinafter referred to as Reference 2), the block size is 4 as a unit of discrete cosine transform processing.
It is disclosed that four types of blocks of x4, 4x8, 8x4, and 8x8 are used.

Hereinafter, use of four types of blocks having block sizes of 4 × 4, 4 × 8, 8 × 4, and 8 × 8 disclosed in Document 2 as a unit of discrete cosine transform processing (hereinafter, (Also referred to as orthogonal transformation using ABT), but before that, the current H.264 standard is described. The discrete cosine transform processing and the quantization processing defined by the standard 26L will be described.

For example, if the quantized pixel values are a, b,
If c and d are used and the conversion coefficients are A, B, C, and D, the process corresponding to the discrete cosine transform is expressed by the following equation (4). A = 13a + 13b + 13c + 13d B = 17a + 7b-7c + 17d C = 13a-13b-13c + 13d D = 7a-17b + 17c-7d (4)

Further, the coefficients after conversion are a ′, b ′,
Assuming c ′ and d ′, the process corresponding to the inverse discrete cosine transform is expressed by the following equation (5). a '= 13A + 17B + 13C + 17D b' = 13A + 7B-13C-17D c '= 13A-7B-13C + 17D d' = 13A-17B + 13C-7D ... (5)

Here, the quantized pixel value a and the coefficient a'after conversion have the relationship shown in the following expression (6). a '= 676a (6)

The relationship between the quantized pixel value a and the coefficient a'after conversion is due to the fact that the normalization process is not included in the equations (4) and (5). Note that the normalization process is executed as will be described later by the final shift operation after dequantization.

H. In 26L, a quantization parameter QP that can take values from 0 to 31 is defined as a parameter for performing quantization / inverse quantization. In the quantization step, each time the value of the quantization parameter QP increases by 1,
Designed to increase by 12%. That is,
If the value of the quantization parameter QP is increased by 6, the quantization step size will be doubled.

The quantization parameter QP embedded in the image compression information is for the luminance signal. Therefore, hereinafter, in order to distinguish from the quantization parameter QP for the luminance signal, the quantization parameter for the color difference signal is set to Q.
_Described as P _chroma . Quantization parameter QP for the luminance signal and QP for the color difference signal
The correspondence with _chroma is as follows. That is,
For example, the value 0 of the quantization parameter QP for the luminance signal
To 17 correspond to the QP _chroma values 0 to 17 as the quantization parameters for the color difference signals, respectively. Further, for example, the value of the quantization parameter QP for the luminance signal is 20
Respectively set the quantization parameter for the color difference signal to Q.
Corresponds to a value of 19 for P _chroma . QP 0,1,2, ..., 17,18,19,20,21,22,23,24,25,26,27,28,29,30,31 QP _chroma 0,1,2, ...・, 17,17,18,19,20,20,21,22,22,23,23,24,24,25,25

H. In 26L, arrays A (QP) and B (QP) are defined for quantization / dequantization. A (QP = 0,1, ..., 31) 620,553,492,439,391,348,3
10,276,246,219,195,174,15
5,138,123,110,98,87,78,6
9,62,55,49,44,39,35,31,2
7, 24, 22, 19, 17 B (QP = 0, 1, ..., 31) 3881, 4351, 4890, 5481, 6154,
6914, 7761, 8718, 10987, 1233
9,13828,15523,17435,1956
1,1873,24552,27665,3084
7,34870,38807,43747,4910
3,54683,61694,68745,7761
5,89113,100253,109366,126
635,141533

The relationship shown in the following equation (7) is established between the array A (QP) and the array B (QP). A (QP) × B (QP) × 676 ² = 2 ⁴⁰ (7)

For example, the process of quantizing the coefficient K is executed according to the following equation (8). LEVEL = (K × A (QP) + f × 2 ²⁰ ) / 2 ²⁰ (8) However, with respect to f in the formula (8), | f | is 0 to 0.5.
And the sign of f coincides with the coefficient K.

Further, for example, the inverse quantization process is executed according to the following equation (9). K '= LEVEL x B (QP) ... (9)

Next, the ABT (Adapti
An encoding method including orthogonal transformation using ve block transform) will be described.

FIG. 15 shows the seven types of modes defined for inter-coded macroblocks. That is, in mode 1, the block size for motion compensation / prediction is 16 × 16, and the block size for discrete cosine transform is 8 × 8. In mode 2, the block size for motion compensation / prediction is 8 × 16, and the block size for discrete cosine transform is 8 × 8. In mode 3, the block size for motion compensation / prediction is 16 × 8, and the block size for discrete cosine transform is 8 × 8. In mode 4, both the block size for motion compensation / prediction and the block size for discrete cosine transform are 8 × 8. In mode 5, both the block size for motion compensation / prediction and the block size for discrete cosine transform are 8 × 4. In mode 6, both the block size for motion compensation / prediction and the block size for discrete cosine transform are 4 × 8. Mode 7 is motion compensation
Both the block size for prediction and the block size for discrete cosine transform are 4 × 4.

Similarly, for macro blocks to be intra-coded, seven types of modes shown in FIG. 15 are defined. However, in modes 4 to 7, H.264. The intra prediction specified in 26L is performed, but in modes 0 to 3, only intra DC prediction is performed.

When the block size is 4 × 4 or more,
Directing by filtering the pixel values located at the boundaries of blocks before performing Directional prediction.
The efficiency of ional forecast can be increased. This filtering is applied to block boundaries that are 8 pixels in length. Here, the filtering process for the pixels located at the upper boundary of the block will be described. The same applies to the processing for the pixel value located at the left boundary of the block.

For example, an element existing in an N × M block (N, M = 4 or 8) is defined as P (i, j) (i = 0 to N-).
1; j = 0 to M−1), and the already-decoded pixels located at the upper boundary of the block N × M block are P (n, −1) (n = 0 to N−1). If pixel P
In (n, -1), the 3-tap FI shown in the following equation (10) is used.
Directional prediction is performed using the pixel P ′ (n, −1) obtained by applying the R filter. [7, 18, 7] // 32 (10)

Next, the orthogonal transformation using the ABT disclosed in Document 2 will be described. The orthogonal transformation using the ABT is performed by using the 4 × 4 matrix T ₄ shown in the following equation (11) and the following equation (1
This is performed according to the following equation (13) using the 8 × 8 matrix T ₈ shown in 2). T ₄ = 13 13 13 13 17 7 -7 -17 13 -13 -13 13 7 -17 17 -7 ・ ・ ・ (11)

T ₈ = 17 17 17 17 17 17 17 17 24 20 12 6 -6 -12 -20 -24 23 7 -7 -23 -23 -7 7 23 20 -6 -24 -12 12 24 6 -20 17 -17 -17 17 17 -17 -17 17 12 -24 6 20 -20 -6 24 -12 7 -23 23 -7 -7 23 -23 7 6 -12 20 -24 24 -20 12 -6 ...・ (12)

[C _{n × m} ] = [T _m ] × [B _{n × m} ] × [T _n ] ^T (13) However, in the equation (13), [B _{n × m} ] is n Pixels of × m blocks (n, m = 4 or 8), and [T _n ].
Is an orthogonal transformation matrix in the horizontal direction, and [T _m ]
Is an orthogonal transformation matrix in the vertical direction. [C _{n × m} ]
Is an orthogonal transform coefficient, but the normalization is performed at the same time as the quantization processing as described later.

In the coding method including orthogonal transformation using ABT disclosed in Document 2, the current H.264 standard is used. Corresponding to 26L, a variable A _{n × m} (QP) and a variable B _{n × m} (QP) having the quantization parameter QP as a parameter are defined as follows.

A _{4 × 4} (QP = 0 to 31) = 620,553,49
2,439,391,348,310,276,246,219,195,174,155,138,123,
110,98,87,78,69,62,55,49,44,39,35,31,27,24,22,19,1
7 B _{4 × 4} (QP = 0 to 31) = 3881,4351,4890,5481,61
54,6914,7761,8718,9781,10987,12339,13828,15523,174
35,19561,21873,24552,27656,30847,34870,38807,4374
7,49103,54683,61694,68745,77615,89113,100253,10936
6,126635, 141533 A _{8 × 4} (QP = 0 to 31) = 335,299,266,237,211,18
8,168,149,133,118,105,94,84,75,67,59,53,47,42,37,3
4,30,26,24,21,19,17,15,13,12,10,9 B _{8 × 4} (QP = 0 to 31) = 2100,2353,2645,2968,33
34,3742,4188,4721,5289,5962,6700,7484,8375,9380,10
500,11924,13274,14968,16750,19014,20691,23450,2705
8,29313,33500,37026,41382,46900,54116,58625,70350,
78167 A _{8 × 8} (QP = 0 to 31) = 181,162,144,128,114,10
2,91,81,72,64,57,51,45,40,36,32,29,25,23,20,18,16,
14,13,11,10,9,8,7,6,6,5 B _{8 × 8} (QP = 0 to 31) = 1136,1270,1428,1607,18
04,2017,2260,2539,2857,3214,3609,4033,4571,5142,57
14,6428,7093,8228,8943,10285,11428,12856,14693,158
23,18700,20570,22855,25712,29385,34283,34283,41139

The variable A _{n × m} (QP) and the variable B
_{For n × m} (QP), the relationship shown in the following equation (14) is established. A _{n × m} (QP) × B _{n × m} (QP) × N ² _{n × m} = 2 ⁴⁰ (14) However, in the formula (14), N ² _{n × m} is as shown in FIG. Is.

The quantization processing and the inverse quantization processing are
Current H. Similar to 26L, it is executed according to equations (8) and (9).

The two-dimensional array [C _{n × m} ] that has been orthogonally transformed according to the equation (13) corresponds to the block size shown in FIG.
It is rearranged into a one-dimensional array by any of the four scanning methods shown in FIG.

The loop filter used in the coding method including orthogonal transformation using ABT is the current H.264 standard. 26L is the same as the method defined in 26L, but when the block size of orthogonal transformation is 4 × 8, 8 × 4, or 8 × 8,
The filtering process is not performed at the 4 × 4 sub-block boundary in the block.

Regarding lossless encoding, the current H.264 standard is used. 26L
In the case of UVLC (Universal
Variable Length Code) and CA, which is a type of arithmetic coding
BAC (Context-based adaptive binary arithmetic codin
Two types of g) are defined, and the user can select and apply either UVLC or CABAC to the lossless encoding method. In the ABT encoding method of Document 2, CABAC for ABT is specified.

In the coding method including orthogonal transformation using ABT, it is necessary to code the intra ABT mode to which Code_Number is assigned as shown in FIG. 18, and this intra ABT mode is the current H ． MB_type in a frame to be inter-coded at 26L
It is encoded using the context model defined for.

For 4x4, 4x8, 8x4, and 8x8 blocks, the intra-prediction mode needs to be coded, but the current H.264 standard. The intra prediction mode defined in 26L is applied as it is.

Regarding CBPY, the current H.264 standard is used. The same processing as that of 26L is also applied to the ABT encoding method.

Regarding LEVEL, in the modes 5 to 7 among the seven types of modes shown in FIG. A method similar to that of 26L is applied. Modes 1 to 4
, CBP = 1 means that at least one non-zero coefficient exists in the 8 × 8 block, so that for the first LEVEL, the first EOB (End Of Bl)
ock) is encoded without being present.

Regarding RUN, in the 4 × 4 block, the current H.264 is set. Although the method defined by 26L is applied, binarization is performed on 4 × 8, 8 × 4, and 8 × 8 blocks based on a table shown in FIG. FIG. 20 shows a context model for encoding these RUN information using CABAC.

Regarding motion estimation, the current H.264 standard is used. In 26L, Hadamard transform is applied to a plurality of 4 × 4 blocks. Also, the absolute value sum SATD (the sum) of the coefficients obtained by orthogonally transforming the difference value generated by motion estimation
of the absolute values of the transformed block di
(fferences) is used to perform motion prediction with an integer pixel precision or less.

The Hadamard transform is also used in the encoding process including the orthogonal transform using the ABT. However, the normalization process requires an operation of dividing by √2, but in Reference 2, it is 18 as an approximate value of 1 / √8.
1/512 is used. FIG. 21 shows the normalization factor after Hadamard transform.

The selection of the intra mode will be described.
In Low Complexity Mode, the above-mentioned absolute value sum SAT
The intra mode that minimizes D is selected. Hig
In h Complexity Mode, mode determination using the cost function J shown in the following equation (15) is performed. J = D + λ _mode R (15) However, in Expression (15), D is distortion and R is bit cost. Also, λ _mode is the current H.264 standard. It is a Lagrange multiplier used in 26L.

Specifically, first, 4 × 4, 4 × 8, 8 ×
The optimal intra prediction mode is selected for each of the 4 and 8 × 8 ABT blocks. Only DC prediction is applied to ABT blocks other than 4 × 4. For each of these, the value of the cost function J is calculated using equation (15), and the optimum ABT block is selected.

[0101]

By the way, in the method proposed in Document 2, when the input image information is the interlaced scanning format, only one of the image information of the field structure and the frame structure is encoded. However, it is expected that higher coding efficiency can be realized by combining the field / frame adaptive coding processing at the macroblock level proposed in Reference 1.

The present invention has been made in view of such a situation. The image information of the interlaced scanning format is input, the field / frame adaptive encoding processing is performed at the macroblock level, and the ABT is used for orthogonal transformation. It is an object to realize an image encoding device to be used.

[0103]

The encoding device of the present invention is
Orthogonal transform means for dividing a macroblock into any of a plurality of preset orthogonal transform block sizes and performing orthogonal transform, and block size for determining the orthogonal transform block size so as to maximize the coding efficiency. It is characterized by including a determining means, a quantizing means for quantizing the processing result of the orthogonal transforming means, and a lossless encoding means for losslessly encoding the processing result of the quantizing means.

The orthogonal transform block size is 4 ×
There can be four types, 4,4 × 8, 8 × 4, and 8 × 8.

The block size determining means determines the orthogonal transform block size in correspondence with the motion prediction / compensation block size when the macro block is an inter macro block and is subjected to field-based coding processing. You can

When the macroblock is an inter macroblock and is subjected to field-based coding processing, and the block size for motion prediction / compensation is 16 × 8, the block size determining means determines the block for orthogonal transform. The size can be determined to be 8 × 8.

When the macroblock is an inter macroblock and is subjected to field-based coding processing, and the block size for motion prediction / compensation is 8 × 8, the block size determining means determines the block for orthogonal transformation. The size can also be set to 8 × 8.

When the macroblock is an inter macroblock and is subjected to field-based coding processing, and the block size for motion prediction / compensation is 8 × 4, the block size determining means determines the block for orthogonal transformation. The size can also be set to 8 × 4.

When the macroblock is an inter macroblock and is subjected to field-based coding processing, and the block size for motion prediction / compensation is 4 × 8, the block size determining means determines the block for orthogonal transformation. The size can also be set to 4 × 8.

When the macroblock is an inter-macroblock and the field-based coding process is performed, and the block size for motion prediction / compensation is 4 × 4, the block size determining means determines the orthogonal transform block. The size can also be set to 4 × 4.

The coding apparatus of the present invention can further include field / frame judging means for judging whether the coding process at the macroblock level is field-based or frame-based.

The determination by the field / frame determining means and the determination by the block size determining means can be performed simultaneously.

The determination by the field / frame determination means and the determination by the block size determination means are such that the absolute value sum SATD of the coefficients obtained by orthogonally transforming the difference value generated by the motion prediction processing is minimized. , Can be done at the same time.

The determination by the field / frame determining means and the determination by the block size determining means can be performed simultaneously based on the value of the cost function J.

The field / frame determination means sums up the squared sum V of the differences between vertically adjacent pixels in the frame.
and _ar1, in response to a result of comparison between the sum of the squares of the differences between adjacent pixels in the vertical direction in the field V _ar2, the encoding processing at the macroblock level, or to any frame-based and field-based determination You can

The block size determining means determines the block size for orthogonal transformation so that the sum SATD of the absolute values of the coefficients obtained by orthogonally transforming the difference value generated by the motion prediction processing becomes the minimum. be able to.

The block size determining means may determine the orthogonal transform block size based on the value of the cost function J.

The lossless encoding means is a field / frame_flag indicating the determination result by the field / frame determining means,
And MB showing the result of determination by the block size determination means
_type can also be losslessly encoded.

The lossless encoding means may use UVLC.

The lossless encoding means may use CABAC.

The encoding method of the present invention comprises an orthogonal transformation step of dividing a macroblock into any of a plurality of preset orthogonal transformation block sizes and performing orthogonal transformation.
The block size determination step for determining the block size for orthogonal transformation, the quantization step for quantizing the processing result of the orthogonal transformation step, and the lossless encoding for the processing result of the quantization step so that the coding efficiency is the highest. And a lossless encoding step.

The program of the first recording medium of the present invention is
An orthogonal transformation step of dividing a macroblock into one of a plurality of preset orthogonal transformation block sizes and performing orthogonal transformation, and a block size that determines the orthogonal transformation block size so that the coding efficiency is highest. It is characterized by including a determining step, a quantization step for quantizing the processing result of the orthogonal transformation step, and a lossless coding step for losslessly coding the processing result of the quantization step.

The first program of the present invention comprises an orthogonal transformation step of dividing a macroblock into any of a plurality of preset orthogonal transformation block sizes and performing orthogonal transformation, and the highest coding efficiency. In the computer, a block size determination step for determining a block size for orthogonal transformation, a quantization step for quantizing the processing result of the orthogonal transformation step, and a lossless encoding step for losslessly encoding the processing result of the quantization step are stored in a computer. It is characterized by being executed.

The decoding apparatus of the present invention decodes the compressed image information and indicates whether the encoding process at the macroblock level is field-based or frame-based.
frame_flag, MB_type indicating the block size in the orthogonal transform process included in the encoding process, a quantized orthogonal transform coefficient, a decoding unit that generates prediction mode information and motion vector information, and a quantized orthogonal transform Inverse quantization means for inverse quantizing the coefficient, based on MB_type, inverse orthogonal transformation means for inverse orthogonal transforming the processing result of the inverse quantization means, according to the prediction mode information and motion vector information, based on the decoded image information And a reference image generating means for generating a reference image.

The decoding apparatus of the present invention can further include frame structure conversion means for converting the processing result of the inverse orthogonal means into a frame structure based on field / frame_flag.

The decoding apparatus of the present invention may further include field structure conversion means for converting the decoding information before being supplied to the reference image generation means into a field structure based on field / frame_flag.

The decoding method of the decoding apparatus of the present invention decodes compressed image information, and field / frame_flag indicating whether the encoding process at the macroblock level is field-based or frame-based MB_type indicating the block size in the included orthogonal transform processing, a decoding step for generating quantized orthogonal transform coefficients, prediction mode information and motion vector information, and dequantizing the quantized orthogonal transform coefficients Inverse quantization step and MB
_type, an inverse orthogonal transform step of performing an inverse orthogonal transform of the processing result of the inverse quantization means, and a reference image generation step of generating a reference image based on the decoded image information according to prediction mode information and motion vector information. It is characterized by

The program of the second recording medium of the present invention is
Decodes compressed image information, field / frame_flag indicating whether the encoding process at the macroblock level is field-based or frame-based, MB_type indicating the block size in the orthogonal transform process included in the encoding process, Decoding step for generating the quantized orthogonal transform coefficient, prediction mode information and motion vector information, dequantizing step for dequantizing the quantized orthogonal transform coefficient, and dequantizing based on MB_type An inverse orthogonal transform step of performing an inverse orthogonal transform on the processing result of the means, and a reference image generation step of generating a reference image based on the decoded image information according to the prediction mode information and the motion vector information.

The second program of the present invention decodes the compressed image information and includes field / frame_flag which indicates whether the encoding process at the macroblock level is field-based or frame-based, and is included in the encoding process. MB_type indicating the block size in the orthogonal transform processing, the decoding step of generating the quantized orthogonal transform coefficient, the prediction mode information and the motion vector information, and the inverse step of dequantizing the quantized orthogonal transform coefficient. Quantization step and MB_t
A computer includes an inverse orthogonal transform step for performing an inverse orthogonal transform on the processing result of the inverse quantization means based on ype, and a reference image generation step for generating a reference image based on the decoded image information according to prediction mode information and motion vector information. It is characterized by making it execute.

In the coding apparatus and method and the first program of the present invention, the orthogonal transform block size is determined so that the coding efficiency is maximized, and the macroblock is set to the determined orthogonal transform block size. Is divided and orthogonally transformed, the result of the orthogonal transformation is quantized, and the result of the quantization is losslessly encoded.

Decoding apparatus and method of the present invention, and second
In the program, compressed image information is decoded,
Field / frame_indicating whether the encoding process at the macroblock level is field-based or frame-based
flag, MB_type indicating the block size in the orthogonal transformation process included in the encoding process, quantized orthogonal transformation coefficient, prediction mode information and motion vector information are generated, and the quantized orthogonal transformation coefficient is reversed. After being quantized, the inverse quantization processing result is subjected to inverse orthogonal transformation based on MB_type. Further, a reference image is generated based on the decoded image information according to the prediction mode information and the motion vector information.

The encoding device and the decoding device may be devices independent of each other, or may be blocks for performing the encoding process and the decoding process of the signal processing device.

[0133]

BEST MODE FOR CARRYING OUT THE INVENTION An image information coding apparatus to which the present invention is applied will be described below with reference to FIG.

In the image information coding apparatus, the A / D
The converter 51 converts the input image information, which is an analog signal, into a digital signal. The screen rearrangement buffer 52 is
GOP of image compression information output by the image information encoding device
The input images are rearranged according to the structure.

The field / frame determination section 53 determines which of the coding efficiency is higher when the macroblock is field-based and when it is frame-based, and generates a corresponding Frame / Field Flag. And outputs it to the field / frame converters 56, 64, 67 and the lossless encoder 59.

When the macroblock is inter-coded, the adder 54 inputs the input image from the screen rearrangement buffer 52 via the field / frame determination unit 53 and the reference image from the motion prediction / compensation unit 68. A difference image with respect to is generated and output to the ABT block size determination unit 55. Further, when the macroblock is intra-coded, the adder 54 outputs the input image input from the screen rearrangement buffer 52 via the field / frame determination unit 53 to the ABT block size determination unit 55 as it is.

The ABT block size determining unit 55 is 4 ×
The block size with the highest coding efficiency is determined from 4, 4 × 8, 8 × 4, and 8 × 8, and is notified to the subsequent stage.
When the macroblock is field-based coded, the field / frame conversion unit 56 converts the input image before being supplied to the ABT conversion unit 57 from a frame structure to a field structure, as shown in FIG.

The ABT converter 57 orthogonally transforms the input image according to the block size determined by the ABT block size determiner 55, and outputs the obtained ABT transform coefficient to the quantizer 58. In the ABT converter 57, the same modes as the seven types of modes shown in FIG. 15 are defined corresponding to the case where the macroblock is encoded on a frame basis. Further, five types of modes shown in FIG. 24 are defined corresponding to the case where the macroblock is encoded on the field basis. That is, in mode 1,
The block size for motion compensation / prediction is 16 × 8, and the block size for discrete cosine transform is 8 × 8. In mode 2, both the block size for motion compensation / prediction and the block size for discrete cosine transform are 8 × 8. In mode 3, both the block size for motion compensation / prediction and the block size for discrete cosine transform are 8 × 4. Mode 4
Has a block size for motion compensation / prediction and a block size for discrete cosine transform of 4 × 8. Mode 5
Has a block size for motion compensation / prediction and a block size for discrete cosine transform of 4 × 4.

The quantizer 58 uses the ABT from the ABT converter 57.
Quantize the transform coefficients. The lossless encoding unit 59 losslessly encodes each syntax element input from the quantization unit 58 and the motion prediction / compensation unit 68, and Frame / Field Flag input from the field / frame determination unit 53 by UVLC or CABAC. To do. The accumulation buffer 60 accumulates each encoded syntax element and outputs it to the subsequent stage as image compression information. The rate control unit 61 includes a storage buffer 60.
The quantization unit 58 is feedback-controlled so that the bit rate of the image compression information output from the image encoding apparatus becomes the target value based on the storage amount of the.

The inverse quantizer 62 uses the quantized ABT.
Dequantize the transform coefficients. The inverse ABT conversion unit 63 performs inverse ABT conversion on the ABT conversion coefficient with the determined block size,
Decoded image information is generated. When the macroblock is field-based encoded, the field / frame conversion unit 64 converts the decoded image information before being stored in the frame memory 66 into a frame structure. That is, FIG.
The operation opposite to the operation of the field / frame conversion unit 56 shown in FIG. The loop filter 65 removes block distortion of the decoded image information before being stored in the frame memory 66. The frame memory 66 stores the decoded image information that is the source of the reference image.

The field / frame conversion unit 67 converts the decoded image information in the frame memory 66 from the frame structure to the field structure before the motion prediction / compensation unit 68 searches for the motion vector corresponding to the field mode. The motion prediction / compensation unit 68 generates the optimum prediction mode information and the motion vector information and also generates the reference image by the motion prediction process.

The control section 69 controls each section of the image information coding apparatus based on the control program recorded in the recording medium 70.

Here, the field / frame determination unit 53
The determination of whether the macroblock is encoded based on the field or the frame will be described.

In Low Complexity Mode, the following equation (16)
As shown in,
Sum of squared difference V_ar1And as shown in the following equation (17),
Sum of squares of differences between vertically adjacent pixels in the field V
_ar2And compare V_ar1Than V _ar2Is small, fee
It is judged that the coding efficiency is higher in the field-based coding.
And then V_ar1Than V_ar2If the frame is not small,
So that the encoding efficiency of the one encoded with
To do.   V_ar1= Σ_uΣ_v{(X [2v] [u] -X [2v + 1] [u])²+ (X [2v + 1] [u] -X [2v + 2] [u])²}                                                           ... (16)   V_ar2= Σ_uΣ_v{(X [2v] [u] -X [2v + 2] [u])²+ (X [2v + 1] [u] -X [2v + 3] [u])²}                                                           ... (17)

However, in equations (16) and (17),
The operator Σ _u represents the sum of u = 0 to 15, and the operator Σ _v
Indicates the sum total of v = 0 to 6. Also, X [v] [u]
Indicates the (v, u) element of the macroblock luminance signal.

In the High Complexity Mode, not only the field / frame determination unit 53 but also the ABT block size determination unit 28 operates to determine whether the macroblock coding is field-based or frame-based.
That is, for a total of 12 types of modes, seven types defined for frame mode encoding equivalent to those shown in FIG. 15 and five types defined for field mode encoding shown in FIG. Then, the cost function J defined by the equation (15) is used to select the mode that maximizes the coding efficiency of the macroblock.

Next, the process of determining the block size with the highest coding efficiency by the ABT block size determining unit 55 will be described.

In the Low Complexity Mode, when the field-based coding is performed and when the frame-based coding is performed, the mode that minimizes the sum of absolute values SATD described in the background art is selected. Has been done.

Even in Low Complexity Mode,
The field / frame determination device 26 and the ABT block size 28 may operate together. That is, a total of 12 types of 7 types defined for the frame mode coding equivalent to that shown in FIG. 15 and 5 types defined for the field mode coding shown in FIG.
The sum of absolute values SATD may be calculated for each type of mode, and the mode corresponding to the minimum value may be selected.

Next, field / frame_generated by the field / frame determination unit 53 by the lossless encoding unit 59.
The process of losslessly encoding the flag will be described. Note that fi
When the value of eld / frame_flag is 1 (when the macroblock is field-based coded), P picture / B
The MB_type for the picture is also encoded.

When CABAC is used in the lossless encoding unit 59, the following context model is defined for MB / type for field / frame_flag and P picture / B picture.

Of the macro blocks A, B and C arranged as shown in FIG. 25, the macro block C is
The context model regarding the frame / field flag will be described. The context model ctx_fifr_flag (C) regarding the frame / field flag of the macroblock C is defined by the following expression (21). ctx_fifr_flag (C) = a + 2b (18) However, in Expression (18), a and b are the values of the frame / field flags of the macroblocks A and B, respectively.

Next, the context model regarding the MB_type for the I picture will be described. frame / field
When the flag is 1, as shown in FIG. 26, Code_Number is assigned to each of the five MB_types of the macroblock C included in the I picture. Context model ctx_mb_ corresponding to these Code_Number
type_intra_field (C) is defined by the following equation (19). ctx_mb_type_intra_field (C) = A + B (19)

However, in the equation (19), A is 0 when the macroblock A is Intra4 × 4, and I
It is 1 when the size is ntra16 × 16. Similarly, B
Is 0 when the macroblock B is Intra4 × 4
And 1 if Intra 16 × 16. Therefore, the context model ctx_mb_type_intra_fiel
d (C) has a value of 0, 1, or 2. The adjacent macroblocks A and B may be field-based coded or frame-based coded.

Next, the context model relating to MB_type for P / B pictures will be described. When the macroblock C is included in the P picture, the context model ctx_mb_type corresponding to the MB_type of the macroblock C
_inter_field (C) is defined by the following expression (23). When included in the B picture, it is defined by the following equation (24). ctx_mb_type_inter_field (C) ＝ ((A == skip)? 0: 1) ＋2 ((B == skip)? 0: 1) ・ ・ ・ (20) ctx_mb_type_inter_field (C) ＝ ((A == Direct)? 0 : 1) +2 ((B == Direct)? 0: 1) ・ ・ ・ (21)

However, in the equation (20), the operator ((A =
= Skip)? 0: 1) indicates 0 when the macroblock A is in the Skip mode and 1 when the macroblock A is not in the Skip mode. Similarly, the operator ((B =
= Skip)? 0: 1) indicates 0 when the macroblock B is in the Skip mode and 1 when the macroblock B is not in the Skip mode.

In addition, in the equation (21), the operator ((A == D
irect)? 0: 1) indicates 0 when the macroblock A is in the Direct mode, and 1 when the macroblock A is not in the Direct mode. Operator ((B == Dir
ect)? 0: 1) indicates 0 when the macroblock B is in the Direct mode, and 1 when the macroblock B is not in the Direct mode.

Therefore, the context model ctx_mb_type_inter_field (C) corresponding to the MB_type of the macroblock C in the inter frame has three kinds of values for the P picture and the B picture, respectively.

The adjacent macroblocks A and B are
It may be field-based coded or frame-based coded.

MB in non-binarized P picture
_type is binarized by the table shown in FIG. 27A. In addition, MB_t in a B-picture that has not been binarized
ype is binarized by the table shown in FIG. 27B.

For a macroblock to be frame-based coded, if it belongs to a P picture, 10 kinds of MB
_type is defined. On the other hand, for a macroblock to be field-based coded, when belonging to a P picture, 16 × 16 mode and 8 × 16 mode are not defined among the 16 types. That is, for field-based encoded macroblocks,
Eight types of MB_type are defined for P pictures.

For macroblocks to be frame-based coded, there are 18 MB_types for B pictures.
Is defined. On the other hand, for a macroblock to be field-based coded, when belonging to a B picture, among the 18 types, a forward 16 × 16 mode, a backward 16 × 16 mode, a forward 8 × 16 mode, and a backward 16 × 16 mode Direction 8x16 mode is not defined. That is,
For field-based encoded macroblocks, 14 types of MB_type are defined for B pictures.

As described above, according to the image information coding apparatus of the present invention, the image information in the interlaced scanning format is input, the field / frame adaptive coding processing is performed at the macroblock level, and the orthogonal transformation is performed. Is ABT
Is used, it is possible to achieve higher coding efficiency than in the past.

Next, FIG. 28 shows a configuration example of an image information decoding device corresponding to the image information encoding device of FIG.

In the image information decoding apparatus, the accumulation buffer 81 accumulates the input image compression information and outputs it to the lossless decoding unit 82 as appropriate. Lossless decoding unit 82
Decodes the image compression information encoded using UVLC or CABAC, outputs the obtained quantized ABT transform coefficient to the dequantization unit 83, and sets field / frame_flag to the field / frame conversion unit 85. Output to P picture / B
The MB_type for the picture is output to the inverse ABT converter 84,
The prediction mode information and the motion vector information are output to the motion prediction / compensation unit 92.

The inverse quantizing unit 83 uses the quantized ABT.
Dequantize the transform coefficients. The inverse ABT transform unit 84 performs inverse orthogonal transform on the ABT transform coefficient with a block size defined by MB_type among 4 × 4, 4 × 8, 8 × 4, and 8 × 8.

The field / frame conversion unit 85, when the macroblock is field-based coded,
The orthogonal transform coefficient that has been subjected to the inverse ABT transform process before being input to the adder 86 is transformed from the field structure to the frame structure. When the macroblock is inter-coded, the adder 86 combines the difference image subjected to the inverse ABT conversion and the reference image from the motion prediction / compensation unit 92 to generate an output image, Output to the latter stage. Also, the adder 86
When the macroblock is inter-coded, outputs the image subjected to the inverse ABT conversion to the subsequent stage as an output image as it is.

The screen rearrangement buffer 87 rearranges the images output from the adder 86 according to the GOP structure of the input image compression information and outputs the rearranged images to the D / A converter 88. D / A
The conversion unit 88 converts the output image, which is a digital signal, into an analog signal.

The loop filter 89 removes block distortion of the output image from the adder 86. Frame memory 90
Stores the image information that is the source of the reference image from which the block distortion has been removed. The field / frame converter 91
When the macro block is field-based coded, the image information stored in the frame memory 90 is converted into a field structure before being supplied to the motion prediction / compensation unit 92. The motion prediction / compensation unit 92 includes a lossless decoding unit 82.
Based on the prediction mode information and the motion vector information from the image information stored in the frame memory 90 or the image information converted into the field structure by the field / frame conversion unit 91 to generate a reference image and adder Output to 86.

According to the image information decoding device configured as described above, the original image information can be obtained by decoding the image compression information output by the image information encoding device in FIG.

The series of processes described above can be executed by hardware, but can also be executed by software. When a series of processes is executed by software, a program that constitutes the software can execute various functions by installing a computer in which dedicated hardware is installed or various programs. It is installed in a possible general-purpose personal computer or the like from the recording medium 70 of FIG. 22, for example.

This recording medium 70 is distributed separately from the computer to provide the program to the user.
Magnetic disks (including flexible disks) and optical disks (CD-ROM (Compact
c-Read Only Memory), DVD (including Digital Versatile Disc)), magneto-optical disc (including MD (Mini Disc)),
Alternatively, it is not only configured by a package medium such as a semiconductor memory, but also configured by a ROM or a hard disk in which a program is recorded and which is provided to a user in a state where the program is installed in advance in a computer.

In the present specification, the steps for writing the program recorded in the recording medium are not limited to the processing performed in time series according to the order described, but are not necessarily performed in time series, but may be performed in parallel. Alternatively, it also includes processes that are individually executed.

[0174]

As described above, according to the first aspect of the present invention,
It is possible to realize an image coding apparatus that receives image information in an interlaced scanning format, performs field / frame adaptive coding processing at the macroblock level, and uses ABT for orthogonal transformation.

Further, according to the second aspect of the present invention, the image information of the interlaced scanning is restored based on the compressed image information which is adaptively coded at the macroblock level on the field basis or the frame basis. It becomes possible to realize an image decoding device.

[Brief description of drawings]

FIG. 1 is a block diagram showing an example of the configuration of a conventional image information encoding device using orthogonal transformation and motion compensation.

FIG. 2 is a block diagram showing a configuration example of an image information decoding device corresponding to the image information decoding device in FIG.

FIG. It is a figure for demonstrating seven types of modes which show the unit of the motion prediction / compensation with respect to a macroblock defined in 26L.

FIG. 4 is a block diagram of H.264 to enable field / frame adaptive coding at the macroblock level. It is a figure showing the syntax of image compression information in the case of expanding 26L.

[Fig. 5] Fig. 5 is a diagram for explaining a process of rearranging pixels of a macroblock in units of rows when performing field-based coding on the macroblock.

[Fig. 6] Fig. 6 is a diagram for describing seven types of modes defined as units of motion prediction / compensation when field-encoding a macroblock.

FIG. 7 is a diagram for explaining an intra prediction mode disclosed in Document 1.

[Fig. 8] Fig. 8 is a diagram for describing an operation of performing intra prediction in a macroblock when the macroblock is field-based encoded.

[Fig. 9] Fig. 9 is a diagram for explaining an operation of performing intra prediction across a plurality of macroblocks when the macroblock is field-based encoded.

[Fig. 10] Fig. 10 is a diagram for explaining an operation of performing intra prediction on a color difference signal when field-based coding a macroblock.

[Fig. 11] Fig. 11 is a diagram for describing an operation of encoding a residual component of a color difference signal when field-based encoding a macroblock.

FIG. 12: H. It is a figure for demonstrating the concept of multiple frame prediction prescribed | regulated in 26L.

[Fig. 13] Fig. 13 is a diagram for describing a prediction method of motion vector information when field-encoding a macroblock.

FIG. 14: H. The prediction mode of the block defined by 26L is 8 × 16, 16 × 8, 8 × 4, or 4 × 8.
FIG. 6 is a diagram for explaining a method of generating a predicted value of motion vector information in the case of being.

FIG. 15 is a diagram for explaining seven types of modes for inter macroblocks defined in Reference 2.

FIG. 16 is a diagram showing specific values of N ² _{n × m} in expression (14).

FIG. 17 is a diagram for explaining a scanning method that rearranges orthogonal transformation coefficients of a two-dimensional array into one-dimensional data, which is defined in Reference 2.

FIG. 18 is a diagram showing an intra ABT block mode defined in Reference 2.

FIG. 19 is 4 × 8, 8 × 4, which is defined in Reference 2.
It is a figure which shows the table for binarizing RUN with respect to and 8x8 block.

FIG. 20 is a diagram showing a context model for encoding RUN in each block size of an ABT block, which is defined in Document 2, using CABAC.

FIG. 21 is a diagram showing a normalization factor after Hadamard transform for each block size of an ABT block, which is disclosed in Document 2;

[Fig. 22] Fig. 22 is a block diagram illustrating a configuration example of an image information encoding device which is an embodiment of the present invention.

FIG. 23 is a diagram for explaining a process of converting pixels of a macro block from a frame structure to a field structure.

FIG. 24 is a diagram for explaining five types of modes used when a macroblock is field-based encoded in the ABT conversion unit 57 of FIG. 22.

FIG. 25 is a diagram for explaining a context model defined for Frame / Field Flag of a macroblock when the lossless encoding unit 59 of FIG. 22 performs CABAC encoding.

FIG. 26 is a diagram showing an intra ABT block mode used when a macroblock is field-based encoded.

[Fig. 27] Fig. 27 is a diagram illustrating a table for binarizing MB_types of macroblocks belonging to a P picture and a B picture when the macroblock is field-based encoded.

28 is a block diagram illustrating a configuration example of an image information decoding device which is an embodiment of the present invention and corresponds to the image information encoding device in FIG. 22.

[Explanation of symbols]

53 field / frame determination unit, 55 ABT block size determination unit, 56 field / frame conversion unit, 57 ABT conversion unit, 59 lossless encoding unit, 6
3 inverse ABT converter, 64 field / frame converter, 65 loop filter, 67 field / frame converter, 69 controller, 70 recording medium, 8
2 lossless decoding unit, 84 inverse ABT conversion unit, 85 field / frame conversion unit, 91 field / frame conversion unit

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Peter Kuhn             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation (72) Inventor Yoichi Yagasaki             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation F-term (reference) 5C059 LC03 MA00 MA02 MA03 MA04                       MA05 MA14 MA21 MA22 MA23                       MC11 MC32 MC34 MC38 ME01                       ME11 NN01 NN21 NN28 PP05                       PP06 PP07 RC12 RC40 SS07                       SS20 TA12 TA13 TA18 TA23                       TA24 TA46 TB07 TC02 TC04                       TC08 TD02 TD06 UA02 UA32                       UA33 UA39                 5J064 AA02 BA04 BA09 BA16 BB03                       BB12 BC01 BC06 BC08 BC11                       BC14 BC16 BC26 BD02 BD03

Claims (34)

[Claims] 1. Inputting image information of interlaced scanning,
In an encoding device that adaptively performs field-based or frame-based encoding processing at the macroblock level, an orthogonal transformation that divides a macroblock into any of a plurality of preset orthogonal transformation block sizes and performs orthogonal transformation Means, a block size determining means for determining the block size for orthogonal transformation so that the coding efficiency is highest, a quantizing means for quantizing the processing result of the orthogonal transforming means, and a processing for the quantizing means An encoding device, comprising: a lossless encoding unit that losslessly encodes a result. 2. The block size for orthogonal transformation is 4 ×
4. The encoding device according to claim 1, wherein there are four types, 4, 4 × 8, 8 × 4, and 8 × 8. 3. The block size determining means, when the macroblock is an inter macroblock and is subjected to the field-based coding processing, the block size determining means for the orthogonal transform corresponding to a block size for motion prediction / compensation. The encoding device according to claim 1, wherein the block size is determined. 4. The block size determination means, when the macroblock is an inter macroblock and the field-based coding process is performed, and when the motion prediction / compensation block size is 16 × 8. The coding apparatus according to claim 3, wherein the block size for orthogonal transformation is determined to be 8x8. 5. The block size determining means, when the macroblock is an inter macroblock and is subjected to the field-based coding process, and when the motion prediction / compensation block size is 8 × 8. The coding apparatus according to claim 3, wherein the block size for orthogonal transformation is also determined to be 8x8. 6. The block size determination means, when the macro block is an inter macro block and the field-based coding process is performed, and when the motion prediction / compensation block size is 8 × 4. The coding apparatus according to claim 3, wherein the block size for orthogonal transformation is also determined to be 8x4. 7. The block size determining means, when the macroblock is an inter macroblock and the field-based coding process is performed, and when the motion prediction / compensation block size is 4 × 8. The coding apparatus according to claim 3, wherein the orthogonal transform block size is also determined to be 4x8. 8. The block size determining means, when the macroblock is an inter macroblock and is subjected to the field-based coding process, and the motion prediction / compensation block size is 4 × 4. The coding apparatus according to claim 3, wherein the block size for orthogonal transformation is also determined to be 4x4. 9. The field / frame determining unit according to claim 1, further comprising a field / frame determining unit that determines whether the encoding process at the macroblock level is the field base or the frame base. Encoding device. 10. The encoding apparatus according to claim 9, wherein the determination by the field / frame determination unit and the determination by the block size determination unit are performed at the same time. 11. The determination by the field / frame determining means and the determination by the block size determining means minimize the sum of absolute values SATD of coefficients obtained by orthogonally transforming the difference values generated by the motion prediction processing. like,
The encoding device according to claim 10, wherein the encoding is performed at the same time. 12. The encoding apparatus according to claim 10, wherein the determination by the field / frame determining unit and the determination by the block size determining unit are performed simultaneously based on the value of the cost function J. . 13. The field / frame determination means compares the square sum V _ar1 of the difference between pixels vertically adjacent in the _frame with the square sum V _ar2 of the difference between pixels vertically adjacent in the field. 10. The encoding apparatus according to claim 9, wherein it is determined whether the encoding process at the macroblock level is based on the field or the frame based on a result. 14. The block size determination means determines the block size for orthogonal transformation so that a sum SATD of absolute values of coefficients obtained by orthogonally transforming a difference value generated by a motion prediction process is minimized. The encoding device according to claim 1, wherein 15. The encoding apparatus according to claim 1, wherein the block size determining means determines the orthogonal transform block size based on a value of a cost function J. 16. The lossless encoding means is a field / fram indicating a determination result by the field / frame determining means.
The encoding device according to claim 1, wherein e_flag and MB_type indicating a determination result by the block size determining unit are also losslessly encoded. 17. The encoding device according to claim 1, wherein the lossless encoding means uses UVLC. 18. The encoding apparatus according to claim 1, wherein the lossless encoding means uses CABAC. 19. The reversible encoding means includes field / fr indicating a determination result by the field / frame determining means.
The encoding device according to claim 18, wherein a context model is defined for each of ame_flag and MB_type indicating a determination result by the block size determining unit. 20. Frame / field fla of macroblock C
The context model ctx_fifr_flag (C) regarding g is defined by the following equation using the values a and b of the frame / field flags of the macroblocks A and B adjacent to the macroblock C. ctx_fifr_flag (C) = The encoding device according to claim 19, wherein a + 2b. 21. A context model ctx_mb_type corresponding to MB_type of a macroblock C included in an I picture.
_intra_field (C) is 0 when each of the macroblocks A and B adjacent to the macroblock C is Intra4 × 4, and is 1 when it is Intra16 × 16.
20. The encoding apparatus according to claim 19, wherein ctx_mb_type_intra_field (C) = A + B defined by the following equation using a value of A, B. 22. A first context model ctx_m corresponding to MB_type of a macroblock C included in a P picture.
b_type_inter_field (C) is an operator (A == Skip) which indicates 0 or 1 depending on whether or not each of the macro blocks A and B adjacent to the macro block C is in the Skip mode?
0: 1), ((B == Skip)? 0: 1) is used to define ctx_mb_type_inter_field (C) ＝ ((A == skip)? 0: 1) ＋2 ((B
== skip)? 0: 1), The encoding device according to claim 19. 23. When the macroblock C included in a P picture is encoded on the field basis, Code_Numbers 0 to 7 corresponding to the MB_type of the macroblock C.
Are 0, 100, 101, 11000 and 11 respectively.
2 in 001, 11010, 11011 or 11100
The encoding device according to claim 22, wherein the encoding device is digitized. 24. A first context model ctx_m corresponding to MB_type of a macroblock C included in a B picture.
b_type_inter_field (C) is an operator (A == Dire) indicating 0 or 1 corresponding to whether or not each of the macro blocks A and B adjacent to the macro block C is in the Direct mode.
ctx_mb_type_inter_field (C) ＝ ((A == Direct)? 0: 1) ＋2 which is defined by the following formula using (ct)? 0: 1) and ((B == Direct)? 0: 1)
The encoding device according to claim 19, wherein ((B == Direct)? 0: 1). 25. When the macroblock C included in a B picture is encoded on the field basis, Code_Numbers 0 to 1 corresponding to MB_type of the macroblock C.
3 is 0, 100, 101, 11000, 1 respectively
1001, 11010, 11011, 11100, 11
1000, 1110001, 1110010, 1110
011, 1110100, 1110101 or 111
The encoding device according to claim 24, wherein the encoding device is binarized to 0110. 26. An encoding method of an encoding device which adaptively performs field-based or frame-based encoding processing at a macroblock level by inputting interlaced scanning image information, wherein a plurality of preset orthogonal transforms are set. Transformation step of dividing a macroblock into any of the block sizes for orthogonal transformation, a block size determination step of determining the orthogonal transformation block size so that the coding efficiency is maximized, and the orthogonal transformation step And a lossless coding step for losslessly coding the processing result of the quantization step. 27. A program for adaptively performing field-based or frame-based encoding processing at a macroblock level by inputting interlaced-scan image information, the method comprising a plurality of preset orthogonal transformation block sizes. An orthogonal transformation step of dividing a macroblock into either one and orthogonal transformation, a block size determination step of determining the orthogonal transformation block size so that the coding efficiency is highest, and a processing result of the orthogonal transformation step. A recording medium having a computer-readable program recorded thereon, comprising: a quantization step of quantizing; and a lossless encoding step of losslessly encoding a processing result of the quantization step. 28. One of a plurality of orthogonal transform block sizes preset in a computer which adaptively performs field-based or frame-based coding processing at a macroblock level using image information of interlaced scanning as input. An orthogonal transformation step of dividing a macroblock into orthogonal transforms, a block size determination step of determining the orthogonal transform block size so that the coding efficiency is maximized, and a quantization result of the orthogonal transformation step. And a reversible encoding step for reversibly encoding the processing result of the quantization step. 29. A decoding device for decompressing the image information based on the compressed image information in which the interlaced scanning image information is adaptively encoded in the field base or frame base at the macroblock level, Decoding, field / frame_flag indicating whether the encoding process at the macroblock level is the field base or the frame base, MB_type indicating the block size in the orthogonal transform process included in the encoding process, Decoding means for generating quantized orthogonal transform coefficient, and prediction mode information and motion vector information, dequantizing means for dequantizing the quantized orthogonal transform coefficient, based on the MB_type, the Inverse orthogonal transform means for inverse orthogonal transforming the processing result of the inverse quantizer means, the prediction mode information and the motion vector According broadcasting, decoding apparatus characterized by comprising a reference image generating means for generating a reference image based on the decoded image information. 30. The decoding apparatus according to claim 28, further comprising frame structure conversion means for converting a processing result of the inverse orthogonal means into a frame structure based on the field / frame_flag. 31. The field structure conversion means according to claim 28, further comprising field structure conversion means for converting the decoding information before being supplied to the reference image generation means into a field structure based on the field / frame_flag. Decoding device. 32. A decoding method of a decoding device, which restores image information based on compressed image information in which interlaced-scan image information is adaptively encoded on a macroblock level on a field-based or frame-based basis. Decoding image information, field / frame_flag indicating whether the encoding process at the macroblock level is the field-based or the frame-based, the block size in the orthogonal transform process included in the encoding process, MB_type shown, a quantized orthogonal transform coefficient, and a decoding step of generating prediction mode information and motion vector information, an inverse quantization step of dequantizing the quantized orthogonal transform coefficient, and to the MB_type An inverse orthogonal transform step of performing an inverse orthogonal transform on the processing result of the inverse quantization means, based on the prediction mode information According spare the motion vector information, decoding method characterized by comprising a reference image generating step of generating a reference image based on the decoded image information. 33. A program for restoring the image information based on the compressed image information in which the interlaced scanning image information is adaptively encoded in the field base or the frame base at the macroblock level, the compressed image information Field / frame_flag indicating whether the encoding process at the macroblock level is the field base or the frame base, MB_type indicating the block size in the orthogonal transform process included in the encoding process. , A quantized orthogonal transform coefficient, and a decoding step of generating prediction mode information and motion vector information, an inverse quantization step of dequantizing the quantized orthogonal transform coefficient, based on the MB_type, An inverse orthogonal transform step of performing an inverse orthogonal transform on the processing result of the inverse quantization means, the prediction mode information and the previous According motion vector information, a recording medium readable program computer, which comprises a reference image generating step of generating a reference image based on the decoded image information is recorded. 34. A computer for restoring the image information based on the compressed image information in which the image information of the interlaced scanning is adaptively encoded in the field base or the frame base at the macroblock level, and decodes the compressed image information. Then, field / frame_flag indicating whether the encoding process at the macroblock level is the field-based or the frame-based, MB_type indicating the block size in the orthogonal transform process included in the encoding process, and a quantum. Decoding step of generating the orthogonal transform coefficient that has been quantized, and prediction mode information and motion vector information, an inverse quantization step of dequantizing the quantized orthogonal transform coefficient, and the inverse step based on the MB_type. An inverse orthogonal transform step of performing an inverse orthogonal transform on the processing result of the quantization means, the prediction mode information and the motion According vector information, a program for executing the reference image generation step of generating a reference image based on the decoded image information.