JP2003264836A - Image processing apparatus and image processing method, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To decode an MPEG stream into an image with high image quality. <P>SOLUTION: A tap coefficient corresponding to a class of a target pixel which is a targeted pixel is acquired out of tap coefficients for each class obtained by a prescribed learning, and a predicted tap used for a prescribed prediction arithmetic operation with the acquired tap coefficient is produced a two-dimensional DCT coefficient and information obtained from the two-dimensional DCT coefficient. Then using the tap coefficient of the class of the target pixel and the predicted tap to perform the prescribed prediction arithmetic operation obtains a pixel value of the target pixel. In this case, the predicted tap comprises the two-dimensional DCT coefficient of a target block including the target pixel and a pixel value of a pixel obtained by applying MPEG decoding or the like to the two-dimensional DCT coefficient of other block adjacent to the target block. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus, an image processing method, a program, and a recording medium, and in particular, for example, decodes encoded data obtained by MPEG-encoding image data into high-quality image data. The present invention relates to an image processing apparatus, an image processing method, a program, and a recording medium that enable the above.

[0002]

2. Description of the Related Art MPEG (Moving Picture Experts Grou)
p) In the coding method such as 1 or 2, the image data is a block unit of 8 × 8 pixels, and DCT (Discrete Cosine Transfo
rm) conversion and further quantization to obtain encoded data. Therefore, in an MPEG decoder conforming to the MPEG standard, encoded data is inversely quantized and then inversely DCT-transformed to be decoded.

As described above, in the MPEG encoding system,
Since the image data is DCT-transformed in block units and the resulting DCT coefficients are quantized, the decoded image obtained by the MPEG decoder conforming to the MPEG standard is affected by the quantization at the time of encoding. Various distortions such as block distortion and mosquito noise occur.

As a method of reducing the distortion generated in the decoded image, for example, as described in JP-A-11-187400 and JP-A-11-205792,
There is a method of performing post-processing on a decoded image to detect and correct distortion occurring in the decoded image.

[0005]

However, as described above, in the method of detecting and correcting the distortion, it is necessary to detect the position of the block boundary. Furthermore, when the encoded data is image data encoded by the MPEG2 system, it is necessary to determine whether the DCT type that can be set in macroblock units is a frame DCT or a field DCT. is there. Therefore, if the block boundary position is detected or the DCT type is erroneously determined, it may be difficult to sufficiently remove the distortion generated in the decoded image.

Further, the above-described method performs the filter processing for correction regardless of the phase of the cosine waveform which forms the basis of the distortion waveform generated in the decoded image, and therefore, the optimum distortion is not always required. Not a removal method.

The present invention has been made in view of such a situation, and it is possible to obtain a high-quality decoded image in which various distortions are sufficiently reduced.

[0008]

A first image processing apparatus of the present invention corresponds to a class of a pixel of interest, which is a pixel of interest, among tap coefficients for each class obtained by predetermined learning. And a prediction tap generation unit that generates a prediction tap used in a predetermined prediction calculation of the tap coefficient acquired by the acquisition unit from the two-dimensional DCT coefficient and information obtained from the two-dimensional DCT coefficient, The present invention is characterized by including a prediction calculation unit that obtains the pixel value of the target pixel by performing a predetermined prediction calculation using the tap coefficient of the class of the target pixel and the prediction tap.

According to the first image processing method of the present invention, of the tap coefficients for each class obtained by predetermined learning,
A prediction tap used in a predetermined prediction calculation of an acquisition step of acquiring a pixel corresponding to a class of a pixel of interest, which is a pixel of interest, and a tap coefficient acquired in the acquisition step is defined as a two-dimensional DCT coefficient and a two-dimensional A prediction tap generation step of generating a pixel value of the pixel of interest by performing a predetermined prediction calculation using the prediction tap generation step of generating from the information obtained from the DCT coefficient, the tap coefficient of the class of the pixel of interest, and the prediction tap. And is provided.

A first program of the present invention is an acquisition step of acquiring a tap coefficient corresponding to a class of a pixel of interest, which is a pixel of interest, out of tap coefficients for each class obtained by predetermined learning. A prediction tap to be used in a predetermined prediction calculation with the tap coefficient obtained in the step, a prediction tap generation step of generating from the information obtained from the two-dimensional DCT coefficient, a tap coefficient of the class of the pixel of interest, , Using prediction taps and
A prediction calculation step of obtaining a pixel value of a target pixel by performing a predetermined prediction calculation.

A first recording medium of the present invention comprises an acquisition step of acquiring a tap coefficient corresponding to a class of a pixel of interest, which is a pixel of interest, out of tap coefficients for each class obtained by predetermined learning. A prediction tap used in a predetermined prediction calculation with the tap coefficient acquired in the acquisition step is generated from the two-dimensional DCT coefficient and information obtained from the two-dimensional DCT coefficient, and a tap coefficient of the class of the pixel of interest. And a prediction calculation step of obtaining a pixel value of a target pixel by performing a predetermined prediction calculation by using the prediction tap and the prediction tap.

The second image processing apparatus of the present invention is provided with at least two sets of teacher data, which is image data to be a teacher for learning.
Two-dimensional DCT encoded by three-dimensional DCT conversion
Student data generation means for outputting coded data including coefficients as student data to be a student for learning, and tap for predicting target teacher data, which is a pixel of interest among pixels forming teacher data The prediction tap used with the coefficient is the two-dimensional DCT coefficient that constitutes the student data,
Prediction tap generation means for generating from the information obtained from the two-dimensional DCT coefficient, and the prediction error of the prediction value of the teacher data of interest obtained by performing a predetermined prediction calculation using the prediction tap and the tap coefficient are statistically minimized. And a learning unit that performs learning for obtaining the tap coefficient for each class.

According to a second image processing method of the present invention, at least 2 sets of teacher data, which is image data to be a teacher for learning, are set.
Two-dimensional DCT encoded by three-dimensional DCT conversion
A student data generation step of outputting encoded data including coefficients as student data to be a student of learning, and tapping to predict attention teacher data which is a pixel of interest among pixels constituting teacher data A prediction tap used together with the coefficient, a prediction tap generation step of generating a two-dimensional DCT coefficient that constitutes student data and information obtained from the two-dimensional DCT coefficient, and a predetermined prediction calculation using the prediction tap and the tap coefficient. And a learning step for learning for each class a tap coefficient that statistically minimizes the prediction error of the prediction value of the teacher data of interest obtained by.

A second program of the present invention encodes teacher data, which is image data to be a teacher for learning, by performing at least two-dimensional DCT conversion, and encodes encoded data including two-dimensional DCT coefficients to a learning student. The student data generation step of outputting the student data as the student data, and the prediction tap used with the tap coefficient to predict the attention teacher data which is the pixel of interest among the pixels configuring the teacher data are configured as the student data. A two-dimensional DCT coefficient, a prediction tap generation step for generating information obtained from the two-dimensional DCT coefficient, and prediction of a predicted value of teacher data of interest obtained by performing a predetermined prediction calculation using the prediction tap and the tap coefficient. And a learning step for learning for each class a tap coefficient that statistically minimizes the error. And it features.

A second recording medium of the present invention encodes teacher data, which is image data to be a teacher of learning, by performing at least two-dimensional DCT conversion to encode encoded data including two-dimensional DCT coefficients. A student data generation step of outputting as student data to be a student, and a prediction tap used with a tap coefficient to predict attention teacher data which is a pixel of interest among pixels forming teacher data The two-dimensional DCT coefficients that make up the
The prediction tap generation step of generating from information obtained from the two-dimensional DCT coefficient and the prediction error of the prediction value of the teacher data of interest obtained by performing a predetermined prediction calculation using the prediction tap and the tap coefficient are statistically minimized. A program having a learning step of performing learning for obtaining a tap coefficient for each class is recorded.

In the first image processing apparatus and image processing method of the present invention, and the program and recording medium,
Of the tap coefficients for each class obtained by the predetermined learning, the one corresponding to the class of the pixel of interest, which is the pixel of interest, is acquired, and the prediction tap used for the predetermined prediction calculation with the acquired tap coefficient Are generated from the two-dimensional DCT coefficient and the information obtained from the two-dimensional DCT coefficient. Then, the pixel value of the pixel of interest is obtained by performing a predetermined prediction calculation using the tap coefficient of the class of the pixel of interest and the prediction tap.

In the second image processing apparatus and image processing method, the program and the recording medium of the present invention,
Teacher data, which is image data to be a teacher for learning, is encoded by performing at least two-dimensional DCT conversion,
Encoded data including the two-dimensional DCT coefficient is output as student data to be a student for learning. Then, of the pixels forming the teacher data, the prediction tap used with the tap coefficient to predict the attention teacher data that is the pixel of interest is a two-dimensional DCT coefficient and two-dimensional DCT coefficient that form the student data. Generated from the information obtained from
A tap coefficient that statistically minimizes the prediction error of the predicted value of the teacher data of interest obtained by performing a predetermined prediction calculation using the prediction tap and the tap coefficient is obtained for each class.

[0018]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below, but before that, a decoding system based on MPEG will be briefly described.

FIG. 1 shows, for example, M which decodes coded data coded by the MPEG2 system in accordance with MPEG.
The structural example of a PEG decoder is shown.

Encoded data (video stream) obtained by encoding image data in the MPEG2 system
Are supplied to the separating unit 1. The separation unit 1 uses the coded block pattern (Coded Block Pattern
rn) (hereinafter, appropriately referred to as CBP), DCT type, VLC (variable length coding) code of quantized DCT coefficient, quantization scale, motion vector, motion compensation type (frame motion type, field motion type), etc. Is output separately.

The DCT coefficient extraction / inverse quantization unit 2 outputs the quantized DCT coefficient output from the separation unit 1 (hereinafter referred to as appropriate).
VLC code (referred to as quantized DCT coefficient), quantization scale, and DCT type, and decodes the DCT coefficient. That is, the DCT coefficient extraction / inverse quantization unit 2 performs variable length decoding on the VLC code of the quantized DCT coefficient output by the separation unit 1, and obtains the quantized DCT coefficient for each block of 8 × 8 pixels. Furthermore, the DCT coefficient extraction / inverse quantization unit 2
The quantized DCT coefficient for each block is inversely quantized by the quantization scale output by the separating unit 1 to obtain the DCT coefficient for each block. The DCT coefficient for each block obtained by the DCT coefficient extraction / inverse quantization section 2 is supplied to the inverse DCT conversion section 3.

The inverse DCT transform unit 3 uses the inverse DCT of the DCT coefficient for each block from the DCT coefficient extraction / inverse quantization unit 2.
It is converted and supplied to the motion compensation addition unit 6.

The motion compensation adder 6 includes an inverse DCT converter 3
In addition to the result of the inverse DCT conversion output by, the CBP and DCT types output by the separation unit 1 are supplied. The motion compensation addition unit 6 adds the predicted image stored in the image memory 5 to the inverse DCT result from the inverse DCT conversion unit 3 as necessary, based on the CBP or DCT type.
A block of pixel values of × 8 is decoded and output.

That is, in MPEG coding, blocks of I pictures are intra-coded, blocks of P pictures are intra-coded or forward predictive coded, and blocks of B-pictures are intra-coded. , Forward predictive coding, backward predictive coding, or bidirectional predictive coding.

Here, in the forward predictive coding, an image of a frame (or field) temporally preceding the frame (or field) of the block to be coded is used as a reference image, and the reference image is motion-compensated. The difference between the obtained predicted image of the block to be encoded and the block to be encoded is calculated, and the difference value (hereinafter,
The residual image is appropriately DCT-transformed.

In the backward predictive coding, the block to be coded is obtained by using the image of the frame temporally subsequent to the frame of the block to be coded as a reference image and performing motion compensation on the reference image. Prediction image of
The difference from the block to be encoded is obtained, and the difference value (residual image) is DCT-transformed.

Further, in the bidirectional predictive coding, two frames (or fields), which are a frame temporally preceding and a frame subsequent to the frame of the block to be coded, are used.
Is used as a reference image, the difference between the prediction image of the block to be encoded and the block to be encoded, which is obtained by performing motion compensation on the reference image, is obtained, and the difference value (residual image) is calculated. DCT conversion is performed.

Therefore, the block is non-intra (non-i
ntra) encoding (forward predictive encoding, backward predictive encoding, or bidirectional predictive encoding), the inverse DCT transform result output by the inverse DCT transform unit 3 is the residual image (the original image and its The difference value from the predicted image), and the motion compensation addition unit 6 adds the residual image and the predicted image stored in the image memory 5 to decode the non-intra coded block.

On the other hand, when the block output from the inverse DCT converter 4 is intra-coded, the motion compensation adder 6 uses the block as a decoding result as it is.

When the motion compensation adder 6 obtains the decoding result of the block for one frame (or field), that is, the decoded image of one frame (or field), the decoded image is stored in the image memory (I, P picture). Image memory) 7
Is supplied to the picture selection unit 8.

When the decoded image supplied from the motion compensation adder 6 is an I-picture or P-picture image, the image memory 7 temporarily uses the decoded image as a reference image of encoded data to be subsequently decoded. Remember. In addition, MP
In EG2, since the B picture is not used as a reference image, if the decoded image supplied from the motion compensation addition unit 6 is a B picture image, the decoded image is not stored in the image memory 7.

The picture selection unit 8 selects and outputs the decoded image output by the motion compensation addition unit 6 or the frame (or field) of the decoded image stored in the image memory 7 in the display order. That is, in the MPEG2 method, the display order of the image frames (or fields) and the decoding order (encoding order) do not match, so the picture selection unit 8 causes the frame (or field) of the decoded images arranged in the decoding order. ) Are rearranged in the display order and output.

The decoded images arranged in the display order in this manner are supplied to, for example, a display (not shown) and displayed.

On the other hand, the motion compensating section 4 receives the motion vector and the motion compensating type output from the separating section 1, and based on the motion compensating type, the frame (or field) to be the reference picture is read from the picture memory 7. read out. further,
The motion compensation unit 4 performs motion compensation on the reference image read from the image memory 7 according to the motion vector output by the separation unit 1, and supplies the predicted image obtained as a result to the image memory 5 for storage. Let

The predictive image thus stored in the image memory 5 is added to the residual image output from the inverse DCT transform unit 3 in the motion compensation addition unit 6 as described above.

The MPEG decoder of FIG. 1 requires a memory for timing adjustment and a synchronizing signal for absorbing the delay time in each block, but the illustration is omitted. The same applies to an image processing device and a learning device described later.

Next, FIG. 2 shows a configuration example of an embodiment of an image processing apparatus to which the present invention is applied. In the figure, parts corresponding to those in the MPEG decoder of FIG. 1 are designated by the same reference numerals, and description thereof will be omitted below as appropriate.

The image processing apparatus shown in FIG. 2 comprises a preprocessing unit 11, a buffer memory 12, a class classification unit 13, a tap coefficient storage unit 14, and an image reconstruction unit 15,
For example, encoded data (bit stream) obtained by encoding image data by the MPEG2 system is decoded.

That is, the encoded data is supplied to the preprocessing unit 11. In addition, the preprocessing unit 11 includes
In addition to the encoded data, the image that has already been decoded is supplied from the image reconstruction unit 15 as a reference image.

The preprocessing unit 11 includes a separation unit 1, a DCT coefficient extraction / inverse quantization unit 2, a motion compensation unit 4, an image memory 5, and a DC.
The T conversion unit 21 and the frequency domain motion compensation addition unit 22 are configured to perform preprocessing on encoded data.

That is, the DCT conversion unit 21 is supplied with the DCT type from the separation unit 1 and the image memory 5
Therefore, the motion compensation unit 4 supplies the predicted image obtained by performing the motion compensation process on the reference image.

Here, the block which is the object of decoding now is referred to as a block of interest as appropriate.

In the following, a block composed of pixel values will be referred to as a pixel block, and a block composed of DCT coefficients will be referred to as a DCT block.

Further, hereinafter, the pixel block or DCT block which is the target block is referred to as a target pixel block or target DCT block, respectively, as appropriate.

The DCT transform unit 21 recognizes the DCT type of the target block output by the DCT coefficient extraction / inverse quantization unit 2 based on the DCT type supplied from the separation unit 1. Further, the DCT conversion unit 21 uses the DC of the target block.
Based on the T type, 8 × 8 pixels having the same size as the block are selected from the predicted image stored in the image memory 5, and are converted into DCT coefficients. The DCT coefficient obtained from this predicted image (hereinafter, appropriately referred to as predicted DCT coefficient) is
It is supplied from the DCT transform unit 21 to the frequency domain motion compensation addition unit 22.

The frequency domain motion compensation adder 22 has a DC
The T transform unit 21 supplies the predicted DCT coefficient of 8 × 8, and the separation unit 1 also supplies the CBP of the macroblock including the target block (hereinafter, appropriately referred to as the target macroblock).
Is supplied to the DCT coefficient extraction / inverse quantization unit 2
From, the block of interest is supplied.

The frequency domain motion compensation addition unit 22 adds each DCT coefficient of the target block and the corresponding predicted DCT coefficient, if necessary, based on the CBP of the target macro block to obtain the pixel value of the target block. A DCT block of interest obtained by DCT conversion is obtained.

That is, when the block of interest is intra-coded, the DCT coefficient of the block of interest is the DCT transform of the block of pixel values (pixel block), so that frequency domain motion compensation is performed. Adder 2
In No. 2, the block of interest is directly used as the DCT block of interest.

When the target block is non-intra coded, the target block is DCT-transformed between the pixel value block (pixel block) and the difference value (residual image) from the predicted image. Because it is a thing,
The frequency domain motion compensation addition unit 22 calculates each DCT coefficient of the block of interest and the 8 × 8 prediction DC obtained by DCT transforming the prediction image of 8 × 8 pixels in the DCT conversion unit 21.
By adding the corresponding one of the T coefficients,
Find the DCT block of interest.

The DCT block of interest obtained by the frequency domain motion compensation addition unit 22 is output by the separation unit 1.
D of the DCT block of interest (including the macro block)
It is associated with the CT type and supplied from the preprocessing unit 11 to the buffer memory 12.

The buffer memory 12 temporarily stores the set of the DCT block of interest and its DCT type supplied from the preprocessing unit 11. As will be described later, the class classification unit 13 is configured to perform processing using not only the information of the block of interest but also the information of four blocks adjacent to the block of interest in the vertical and horizontal directions. Therefore, the buffer memory 12 has a two-dimensional D for one macroblock added to the macroblocks for two lines on the screen.
It has at least a storage capacity capable of storing the CT coefficient and the DCT type.

Since the DCT coefficient of the block stored in the buffer memory 12 is output from the frequency domain motion compensation addition unit 22 of the preprocessing unit 11, it does not depend on the picture type of the block, and Whether the pixel value of the original image (the original image, not the residual image) of the original image is DC, regardless of whether the image is intra-coded or non-intra-coded.
It has been T-converted.

The original image mentioned here means an image obtained by adding the residual image and the predicted image, and
It is not the original image itself that was the target of encoding.

The class classification unit 13 includes a one-dimensional inverse DCT transformation unit 31, an adjacent one-dimensional DCT coefficient selection / transformation unit 32, an AC.
The buffer memory 1 includes a power calculator 33, an AC inner product calculator 34, and class code generators 36 and 37.
Based on the DCT coefficient and DCT type of the block stored in No. 2, each pixel of the block of interest is classified into one of a plurality of classes.

That is, the one-dimensional inverse DCT transform unit 31 calculates the DCT coefficient of the block stored in the buffer memory 12 as
By performing the one-dimensional inverse DCT transform, a horizontal one-dimensional DCT coefficient representing a horizontal spatial frequency component and a vertical one-dimensional DCT coefficient representing a vertical spatial frequency component are obtained.

Here, the vertical one-dimensional DCT coefficient and the horizontal one-dimensional DCT coefficient will be collectively referred to as a one-dimensional DCT.
It is called a coefficient.

The DCT coefficient included in the MPEG encoded data represents spatial frequency components in two directions, the horizontal direction and the vertical direction. In order to distinguish it from the one-dimensional DCT coefficient, It is called a two-dimensional DCT coefficient as appropriate.

Further, hereinafter, two-dimensional D is calculated from pixel values as appropriate.
Two-dimensional DCT transform and two-dimensional DC transform to CT coefficient
The conversion from the T coefficient to the pixel value is a two-dimensional inverse DCT conversion,
Say each. DC performed by the DCT converter 21 of FIG.
The T transform is a two-dimensional DCT transform, and the inverse DCT transform performed by the inverse DCT transform unit 3 in FIG. 1 is a two-dimensional inverse DCT transform.

The one-dimensional DCT coefficient obtained by the one-dimensional inverse DCT transform unit 31 is the adjacent one-dimensional DCT coefficient selection / transform unit 3.
2, the AC power calculator 33, the AC inner product calculator 34, and the class code generator 36.

Adjacent one-dimensional DCT coefficient selecting / converting unit 32
Receives the DCT type of the block of interest from the buffer memory 12, and based on the DCT type, the one-dimensional DCT coefficients of pixels (columns) adjacent to the block of interest (hereinafter, appropriately referred to as adjacent one-dimensional DCT coefficient) One-dimensional inverse DCT
It is acquired from the one-dimensional DCT coefficient supplied from the conversion unit 31, and is supplied to the AC power calculation unit 33, the AC inner product calculation unit 34, and the class code generation unit 36.

The AC power calculation unit 33 uses the one-dimensional inverse DCT.
The AC component power of the AC component of the one-dimensional DCT coefficient supplied from the coefficient conversion unit 31 (hereinafter appropriately referred to as AC power) is obtained, and the AC of the one-dimensional DCT coefficient supplied from the adjacent one-dimensional DCT coefficient selection / conversion unit 32 is calculated. The power is calculated and supplied to the class code generators 36 and 37.

The AC inner product calculating unit 34 is supplied from the AC component of the one-dimensional DCT coefficient of the boundary portion of the target block supplied from the one-dimensional inverse DCT coefficient converting unit 31 and the adjacent one-dimensional DCT coefficient selecting / converting unit 32. AC components of adjacent one-dimensional DCT coefficients are regarded as vector components, and the inner product of the two vectors (hereinafter, appropriately referred to as A
C inner product). The AC inner product calculated by the AC inner product calculation unit 34 is supplied to the class code generation unit 36.

The class code generation unit 36 classifies the pixels constituting the block of the luminance signal Y, and the class code generation unit 37 classifies the pixels constituting the block of the color difference signal (color signal) C. .

Here, the coded data is, for example, MPEG coded of color image data.

Then, the one-dimensional inverse DCT transform unit 31 determines the one-dimensional DCT coefficient of the target block when the target block is a block of the luminance signal by the adjacent one-dimensional DCT coefficient selection / transform unit 32. Is a block of luminance signals, adjacent 1 to the block of interest
The AC power calculation unit 33 calculates the dimensional DCT coefficient and the AC power obtained for the target block when the target block is a block of the luminance signal, by the AC inner product calculation unit 3
4 is adapted to supply the AC inner products obtained for the target block when the target block is the block of the luminance signal to the class code generating unit 36, respectively. Each pixel of the block of interest is classified into classes based on the information supplied to.

Further, the class code generator 36 generates a class code representing a class obtained by classifying the pixels of the luminance signal of the target block, and supplies the class code to the tap coefficient memory 14. Here, the class code for the pixels of the block of the luminance signal, which is obtained by the class code generation unit 36, is hereinafter referred to as a luminance class code, as appropriate.

The brightness class code output from the class code generating unit 36 for each pixel of the target block is also supplied to the class code generating unit 37. Further, when the target block is a block of color difference signals, the AC power obtained from the target block is also supplied from the AC power calculating unit 33 to the class code generating unit 37.

The class code generator 37 receives the AC power of the target block supplied from the AC power calculator 33,
Each pixel of the target block is classified into a class based on the brightness class code of the pixel of the block of the brightness signal corresponding to the target block of the color difference signal supplied from the class code generation unit 36.

Then, the class code generation unit 37 generates a class code representing a class obtained by classifying the pixels of the color difference signal of the target block, and supplies it to the tap coefficient storage unit 14. Here, the class code of the pixel of the block of the color difference signal, which is obtained by the class code generation unit 37, is hereinafter referred to as a color difference class code, as appropriate.

The tap coefficient storage unit 14 is supplied from the tap coefficient selection units 41 and 42 and the coefficient memories 43 and 44, and the class code generation unit 3 of the class classification unit 13 is provided.
The tap coefficient corresponding to the class code supplied from 6 or 37 is acquired and supplied to the image reconstruction unit 15.

That is, the tap coefficient selecting unit 41 is supplied with the luminance class code output from the class code generating unit 36, and the tap coefficient selecting unit 42 is output from the class code generating unit 37. A color difference class code is supplied.

Further, the coefficient memory 43 stores tap coefficients for each class, which are obtained by learning to be described later, with respect to pixels of the luminance signal, and the coefficient memory 44 obtains pixels of the color difference signal by learning. The tap coefficient for each class is stored.

Then, the tap coefficient selecting unit 41 sets the brightness class code supplied from the class code generating unit 36 to
The coefficient is given to the coefficient memory 43 as an address, whereby the tap coefficient of the class corresponding to the class code is read from the coefficient memory 43 and supplied to the adaptive processing unit 51 of the image reconstructing unit 15.

Further, the tap coefficient selecting section 42 gives the color difference class code supplied from the class code generating section 37 to the coefficient memory 44 as an address, whereby the tap coefficient of the class corresponding to the class code is calculated as a coefficient. It is read from the memory 44 and supplied to the adaptive processing unit 51 of the image reconstructing unit 15.

The image reconstructing section 15 is composed of the image memory 7, the picture selecting section 8 and the adaptive processing section 51, and the two-dimensional DCT coefficient stored in the buffer memory 12 and the coefficient of the tap coefficient storing section 14. Memories 43 and 4
The image is decoded (reconstructed) using the tap coefficient supplied from No. 4.

That is, the adaptive processing unit 51 calculates the two-dimensional DCT coefficient of the target block stored in the buffer memory 12 as
By performing adaptive processing using the tap coefficients supplied from the coefficient memories 43 and 44 of the tap coefficient storage unit 14, the pixel values of the target block are converted. Furthermore, the adaptive processing unit 5
When 1 decodes the pixel value of one frame (or field), that is, the image data of one frame (or field), the decoded image data is supplied to the image memory 7 and the picture selection unit 8.

The image memory 7 stores, as a reference image, the decoded image data of I picture and P picture of the decoded image data supplied from the adaptive processing unit 51. The picture selection unit 8 selects and outputs the decoded image data supplied from the adaptive processing unit 51 or the decoded image data stored in the image memory 7 in the display order.

Here, in the adaptive processing section 51, as described above, the adaptive processing for converting the DCT coefficient into the pixel value using the tap coefficient is performed.

That is, in the adaptive process, the DCT coefficient is decoded into the original pixel value by, for example, linearly combining the DCT coefficient and a predetermined tap coefficient to obtain the predicted value of the original pixel.

More specifically, for example, the pixel value of an image is used as teacher data, the image is DCT-transformed in block units, and the DCT coefficient obtained by quantization and dequantization is calculated by the student. As data, a predicted value E [y] of a pixel value y of a pixel, which is teacher data, a set of several DCT coefficients x ₁ , x ₂ , ... And a predetermined tap coefficient w ₁ ,
Consider that the linear linear combination model defined by the linear combination of w ₂ , ... Is used. In this case, the predicted value E
[Y] can be expressed by the following equation.

E [y] = w ₁ x ₁ + w ₂ x ₂ + ... (1)

In order to generalize the equation (1), a matrix W made up of a set of tap coefficients w _j , a matrix X made up of a set of student data x _ij , and a matrix Y made up of a set of prediction values E [y _j ]. '
To

[Equation 1] , The following observation equation holds.

[0083]       XW = Y '                                                       ... (2)

Here, the component x _ij of the matrix X is the j-th element in the i-th student data set (student data set used for prediction of the _i-th teacher data y _i ) (prediction tap). This means the student data, and the component w _{j of the} matrix W represents the tap coefficient with which the product of the j-th student data in the student data set is calculated. Further, y _i represents the i-th teacher data, and thus E [y _i ] represents the predicted value of the i-th teacher data. Note that y on the left side of the equation (1) is the matrix Y
Is intended to omit the suffix i of the component y _i of, also, x _1, x ₂ on the right-hand side of Equation (1), ... also it is obtained by omitting the suffix i of the component x _ij of the matrix X.

It is considered that the prediction value E [y] close to the original pixel value y is obtained by applying, for example, the least square method to the observation equation of the equation (2). In this case, a matrix Y made up of a set of true pixel values y serving as teacher data and a matrix E made up of a set of residuals e of the prediction value E [y] for the pixel values y are

[Equation 2] , The following residual equation is established from the equation (2).

[0086]       XW = Y + E                                                       ... (3)

In this case, the predicted value E close to the original pixel value y
The tap coefficient w _j for obtaining [y] is the squared error

[Equation 3] Can be obtained by minimizing.

Therefore, the above-mentioned squared error is converted into the tap coefficient w _j.
When the differentiated value becomes 0, that is, the tap coefficient w _j satisfying the following equation is the optimum value for obtaining the predicted value E [y] close to the original pixel value y.

[0089]

[Equation 4] ... (4)

Therefore, first, the equation (3) is changed to the tap coefficient w
By differentiating with _j , the following equation holds.

[0091]

[Equation 5] ... (5)

From equations (4) and (5), equation (6) is obtained.

[0093]

[Equation 6] ... (6)

Further, considering the relationship among the student data x _ij , the tap coefficient w _j , the teacher data y _i , and the residual e _{i in} the residual equation of the equation (3), the following equation is obtained from the equation (6). You can get the normal equation.

[0095]

[Equation 7] ... (7)

In the normal equation shown in the equation (7), the matrix (covariance matrix) A and the vector v are

[Equation 8] When the vector W is defined as shown in Formula 1, it can be expressed by the formula AW = v ... (8).

For each normal equation in the equation (7), by preparing a certain number of sets of student data x _ij and teacher data y _i , the same number as the number J of tap coefficients w _{j to} be obtained is obtained. Can be, therefore, equation (8)
By solving for the vector W (however, in order to solve the equation (8), the matrix A in the equation (8) needs to be regular), and the optimum tap coefficient (here, the square error is minimized. The tap coefficient) w _j can be obtained. In addition,
In solving equation (8), for example, the sweep method (Ga
Uss-Jordan elimination method) can be used.

As described above, the optimum tap coefficient, that is, the tap coefficient w _j that minimizes the statistical error of the predicted value of the pixel value is obtained, and the tap coefficient w _j is further used to calculate Predicted value E close to the original pixel value y according to equation (1)
The adaptive processing is to obtain [y].

As described above, the pixel value of the image is used as the teacher data, and the image is divided into D units in block units.
D obtained by CT conversion, quantization and dequantization
When the CT coefficient is used as the student data, the DCT coefficient (the DCT coefficient obtained by DCT transforming the pixel value, quantizing, and dequantizing the pixel value) is converted into the original pixel by the linear prediction calculation of Expression (1). It is possible to obtain the optimum tap coefficient for conversion into (a predicted value of) the value. And the DCT coefficient is
Since the data is in the frequency domain and the pixel value is a signal in the time domain, the adaptive processing for converting the DCT coefficient into the pixel value by using the tap coefficient as described above is performed in the frequency (Frequency)
And the abbreviation of Time can be called FT conversion.

In the above case, the image is two-dimensional D
DC obtained by CT conversion, quantization and dequantization
While the T coefficient is used as the student data and the original image is used as the teacher data to obtain the tap coefficient, the tap coefficient may be, for example, a two-dimensional DCT transform of the original image,
Further, it is possible to use DCT coefficients obtained by quantization and dequantization as student data, and obtain DCT coefficients obtained by performing two-dimensional DCT conversion of the original image as teacher data. In this case, DC with quantization error
It is possible to obtain the optimum tap coefficient for converting the T coefficient into (a predicted value of) the DCT coefficient having no quantization error.

The tap coefficient is the two-dimensional DC of the original image.
T-transformation, further quantization, inverse quantization, two-dimensional inverse DCT
It is also possible to use the decoded image obtained by conversion as the student data and obtain the original image as the teacher data. In this case, inverse quantization and two-dimensional inverse DCT
It is possible to obtain the optimum tap coefficient for converting the image decoded by the conversion into (the predicted value of) the original image.

That is, according to the adaptive processing, arbitrary data conversion is performed depending on what kind of data is adopted as the teacher data and the student data (the data corresponding to the student data is changed to the data corresponding to the teacher data). It is possible to obtain the optimum tap coefficient for (converting).

Next, FIG. 3 shows a configuration example of the DCT transform unit 21 and the frequency domain motion compensation addition unit 22 of FIG.

The DCT transform unit 21 is composed of a (8 × 8 pixel) sampling unit 61 and a DCT unit 62, and is adapted to generate a predicted DCT coefficient by subjecting the predicted image stored in the image memory 5 to two-dimensional DCT conversion. ing.

That is, the sampling unit 61 is supplied with the predicted image stored in the image memory 5 and the DCT type of the target block (including the macroblock) output from the separating unit 1 (FIG. 2). ing.

Here, in the image memory 5, the motion compensation unit 4
From FIG. 2, it is 1 that is the same size as the macroblock.
A predicted image of 6 × 16 pixels is supplied, and the image memory 5 stores 16 × 1 pixels corresponding to the target block.
The predicted image of 6 pixels is stored. Therefore, the image memory 5
Has a storage capacity capable of storing an image of at least 16 × 16 pixels.

The sampling section 61 samples the 16 × 16 pixel prediction image stored in the image memory 5 according to the DCT type of the block of interest, and generates an 8 × 8 pixel prediction image of the same size as the block. To do.

That is, 16 × 1 stored in the image memory 5
The 6-pixel predicted image has a frame structure and may or may not match the structure of the target block.

Specifically, when the block of interest has a frame structure, the block of interest and the predicted image of 16 × 16 pixels stored in the image memory 5 have the same structure.

Therefore, as shown in FIG. 4A, the target block is the upper left corner of the macroblock (target macroblock),
In the case of the lower left, upper right, or lower right block, the sampling unit 61 selects the upper left, lower left, upper right, or right of the 16 × 16 pixel prediction image stored in the image memory 5 as illustrated in FIG. 4B. Each of the lower 8 × 8 pixels is sampled, thereby obtaining an 8 × 8 pixel prediction image at a position spatially corresponding to each pixel of the block of interest, and D
It is supplied to the CT unit 62.

Here, in FIG. 4 (similarly in FIG. 5 described later), the shaded lines represent odd lines (top fields), and the unshaded lines are even lines (bottom lines). Field).

On the other hand, if the block of interest has a field structure, the block of interest and the prediction image of 16 × 16 pixels stored in the image memory 5 have different structures.

That is, in this case, in the macroblock including the target block (target macroblock), the upper 8 lines are odd lines (top field) and the lower 8 lines are even, as shown in FIG. 5A. It is composed of lines (bottom field).

Therefore, when the target block is the upper left block of the target macro block, the 8 × 8 pixels of the target block are predicted to be 16 × 16 pixels stored in the image memory 5 as shown in FIG. 5B. It corresponds to the eight pixels on the left side of eight odd lines (shaded lines) in the image. If the block of interest is the block at the lower left of the macro block of interest, the block of interest 8 × 8
As shown in FIG. 5B, the pixels correspond to the eight pixels on the left side of the eight even lines (lines not shaded) of the 16 × 16 pixel prediction image stored in the image memory 5. Furthermore, if the block of interest is the block on the upper right of the macro block of interest, the 8 ×
As shown in FIG. 5B, 8 pixels correspond to 8 pixels on the right side of eight odd lines (shaded lines) in the 16 × 16 pixel prediction image stored in the image memory 5. If the target block is the block at the lower right of the target macroblock, 8 × 8 of the target block
As shown in FIG. 5B, the pixel corresponds to the eight pixels on the right side of the eight even lines (lines not shaded) of the 16 × 16 pixel prediction image stored in the image memory 5.

Therefore, when the target block has a field structure, the sampling section 61 determines that the image memory 5
Corresponding to the position of the target block in the target macroblock of the 16 × 16 pixel predicted image stored in
The 8 × 8 pixels as described above are sampled, and thereby, a predicted image of 8 × 8 pixels at a position spatially corresponding to each pixel of the target block is obtained and supplied to the DCT unit 62.

The DCT section 62 performs a two-dimensional DCT conversion on the 8 × 8 pixel prediction image supplied from the sampling section 61 and located spatially corresponding to each pixel of the block of interest. , And supplies it to the frequency domain motion compensation addition unit 22.

In the sampling unit 61, the structure of the block of interest is recognized based on the DCT type of the block of interest (including the macro block of interest) output by the separation unit 1 (FIG. 2).

In FIG. 3, the frequency domain motion compensation addition unit 22 is composed of a DCT coefficient selection unit 71, an addition unit 72, and a selection unit 73, and the DCT coefficient extraction / inverse quantization unit 2
The two-dimensional DCT coefficient of the original image of the target block is added by adding the two-dimensional DCT coefficient of the target block supplied from (FIG. 2) and the predicted DCT coefficient supplied from the DCT transform unit 21 as necessary. Find the coefficient.

That is, the DCT coefficient selecting section 71
The two-dimensional DCT coefficient of the target block output by the coefficient extracting / inverse quantizing unit 2 (FIG. 2) and the CBP of the target block (including the target macroblock) output by the separating unit 1 (FIG. 2) are supplied. It has become.

When the target block is intra-coded, the two-dimensional DCT coefficient of the target block is
Since the original image of the block of interest is a two-dimensional DCT transform, the DCT coefficient selecting unit 71
Output as is. The output of the DCT coefficient selection unit 71 is supplied to the addition unit 72 and the selection unit 73.

If the block of interest is intra-coded, the adder 72 does not perform any processing, and
The selection unit 73 selects the output of the DCT coefficient selection unit 71,
It is supplied to the buffer memory 12 (FIG. 2) in the subsequent stage.

Therefore, when the target block is intra-coded, that is, the two-dimensional DCT coefficient of the target block output by the DCT coefficient extraction / inverse quantization unit 2 (FIG. 2) is the original block of the target block. When the image is a two-dimensional DCT-transformed image, the DCT coefficient extraction / inverse quantization unit 2
The block of interest output by (FIG. 2) is directly supplied to the buffer memory 12 (FIG. 2).

On the other hand, when the block of interest is non-intra coded, the DCT coefficient selecting unit 71 refers to the CBP of the block of interest and refers to the two-dimensional DC of the residual image.
The presence or absence of the T coefficient is recognized. That is, when the target block is non-intra coded, the two-dimensional DCT coefficient of the residual image is arranged in the target block in principle (in the video stream). Of 2
When the dimensional DCT coefficients are all 0, CBP is 0
Therefore, the two-dimensional DCT coefficient is not arranged. Then, in this case, the image of the target block matches the predicted image.

Therefore, the DCT coefficient selecting unit 71 is the one in which the block of interest is non-intra coded, and its C
When BP is 0, 0 is output as the two-dimensional DCT coefficient of the residual image.

Further, the DCT coefficient selecting unit 71 is a block in which the block of interest is non-intra coded, and its CB
When P is 1, the block of interest output by the DCT coefficient extraction / inverse quantization unit 2 (FIG. 2) is the two-dimensional D of the residual image.
Since the CT coefficient is arranged, the two-dimensional DCT coefficient is output.

The output of the DCT coefficient selecting unit 71 is supplied to the adding unit 72 and the selecting unit 73, as described above.

If the target block is non-intra coded, the adding section 72 selects the DCT coefficient selecting section 71.
And the two-dimensional DCT coefficient of the predicted image output by (the DCT unit 62 of) the DCT conversion unit 21 are added, whereby the two-dimensional DCT coefficient of the original image is obtained for the target block, and the selection unit Supply to 73.

When the target block is non-intra coded, the selecting section 73 selects the output of the calculating section 72 and supplies it to the buffer memory 12 (FIG. 2) in the subsequent stage.

Therefore, when the target block is non-intra coded, the CB of the target block is
When P is 0, the image of the block of interest matches the predicted image. Therefore, in the addition unit 72, 0 output from the DCT coefficient selection unit 71 and the DCT coefficient (predicted DCT) of the predicted image output from the DCT unit 62. (Coefficient) is added to obtain the two-dimensional DCT coefficient of the original image of the block of interest.

When the CBP of the target block is 1, the DCT coefficient of the residual image of the target block output from the DCT coefficient selecting section 71 in the adding section 72 and the DCT
The DCT coefficient (predicted DCT) of the predicted image output by the T unit 62
(Coefficient) is added to the two-dimensional DCT coefficient of the original image of the target block.

Then, the selecting unit 73 selects and outputs the two-dimensional DCT coefficient of the original image of the target block, which is obtained by the adding unit 72 as described above.

In the pre-processing unit 11 (FIG. 2), the residual image and the predicted image are added in the frequency domain to obtain the two-dimensional DCT coefficient of the original image.
Two-dimensional DCT coefficient of residual image and two-dimensional DC of predicted image
The two-dimensional DCT coefficient of the original image is obtained by adding the T coefficient and this. However, since this is performed in the class classification unit 13 in the subsequent stage using the DCT coefficient in the frequency domain, the preprocessing unit This is because it is considered convenient to unify the processing of 11 and the class classification unit 13 so as to be performed in the frequency domain.

Therefore, the two-dimensional DCT coefficient of the original image is
Alternatively, the residual image and the predicted image may be added in the time domain, and the addition result may be obtained by two-dimensional DCT transformation.

Next, the processing of the one-dimensional inverse DCT transform section 31 shown in FIG. 2 will be described with reference to FIGS. 6 and 7.

MPEG and JPEG (Joint Photographic
In the image coding method using DCT transform such as Experts Group), the image data is divided into two in the horizontal direction and the vertical direction.
Dimensional DCT transformation (two-dimensional DCT transformation) / inverse DCT transformation (two-dimensional inverse DCT transformation) is performed.

Pixel values in a block of 8 × 8 pixels as shown in FIG. 6A are represented by a matrix X of 8 rows × 8 columns and DC in a block of 8 × 8 as shown in FIG. 6B.
If the T coefficient is represented by a matrix F of 8 rows × 8 columns, 2
The dimensional DCT transform / two-dimensional inverse DCT transform can be expressed by the following equation.

CXC ^T = F (9) C ^T FC = X (10)

Here, the superscript T represents transposition. Also,
C is a DCT transformation matrix of 8 rows × 8 columns, and the component c _ij of the i + 1-th row and the j + 1-th column is represented by the following equation.

C _ij = A _i × cos ((2j + 1) × i × π / 16) (11)

However, in the equation (11), when i = 0, A _i = 1 / (2√2), and when i ≠ 0, A _i
= 1/2. Further, i and j are integer values in the range of 0 to 7.

Expression (9) represents a two-dimensional DCT conversion for converting the pixel value X into a two-dimensional DCT coefficient F, and the expression (10)
Represents a two-dimensional inverse DCT transform that transforms a two-dimensional DCT coefficient F into a pixel value X.

Therefore, according to the equation (10), the two-dimensional DC
The T coefficient F is multiplied by the matrix C ^T from its left side,
The pixel value X is converted by multiplying the matrix C from the right side, but the one-dimensional inverse DCT conversion unit 31 uses the two-dimensional DC.
The one-dimensional DCT coefficient is obtained by multiplying the T coefficient F by the matrix C ^T from the left side or by only the matrix C from the right side.

That is, the one-dimensional inverse DCT transform unit 31 multiplies the two-dimensional DCT coefficient F by only the matrix C ^T from the left side. In this case, as shown in FIG. 6C, the two-dimensional DCT
A vertical one-dimensional inverse DCT transform in which the vertical direction of the coefficient F is transformed into the spatial domain and the horizontal direction is kept in the frequency domain is performed, and as a result, the horizontal one-dimensional DCT coefficient representing the spatial frequency component in the horizontal direction is obtained. vXhF can be obtained.

The one-dimensional inverse DCT transform unit 31 multiplies the two-dimensional DCT coefficient F by only the matrix C from the right side. In this case, as shown in FIG. 6D, the horizontal one-dimensional inverse DCT transform is performed in which the horizontal direction of the two-dimensional DCT coefficient F is transformed into the spatial domain, and the vertical direction remains the frequency domain, and as a result, It is possible to obtain the vertical one-dimensional DCT coefficient hXvF representing the spatial frequency component in the vertical direction.

When the vertical one-dimensional inverse DCT transformation is performed on the two-dimensional DCT coefficient F of horizontal × vertical 8 × 8, 8 sets (8 rows) of 8 × 1 horizontal one-dimensional DCT coefficient are obtained. (Fig. 6C). In addition, the two-dimensional DCT coefficient F is
When the horizontal one-dimensional inverse DCT transform is performed, 1 × 8 vertical 1
Eight sets (8 columns) of dimensional DCT coefficients will be obtained (FIG. 6D).

For the 8 × 1 horizontal one-dimensional DCT coefficient in a certain row, the DCT coefficient at the left end is the DC component (DC component) of the pixel values of the eight pixels in that row (the average of the pixel values of the eight pixels). Value), and the other seven DCT coefficients are
It represents the horizontal AC component of the row. Also, regarding the 1 × 8 vertical one-dimensional DCT coefficient in a certain column, the DCT coefficient in the uppermost row represents the DC component of the pixel value of the 8 pixels in that column, and the other seven DCT coefficients are It represents the AC component in the vertical direction.

Here, according to the equation (9), the horizontal one-dimensional D
The CT coefficient is a horizontal one-dimensional DC obtained by multiplying the pixel value X corresponding to the two-dimensional DCT coefficient F by the matrix C ^T from the right side.
It can also be obtained by performing T conversion. The vertical one-dimensional DCT coefficient can also be obtained by performing a vertical one-dimensional DCT transformation by multiplying the pixel value X corresponding to the two-dimensional DCT coefficient F by the matrix C from the left side of the pixel value X.

FIG. 7 shows an actual image and the two-dimensional DCT coefficient, horizontal one-dimensional DCT coefficient, and vertical one-dimensional DCT coefficient for the image.

FIG. 7 shows an image of 8 × 8 blocks,
2D DCT coefficient for the image, 1D horizontal DC
The T coefficient and the vertical one-dimensional DCT coefficient are shown. Further, FIG. 7A shows an actual image, and FIG. 7B shows a two-dimensional DCT.
FIG. 7C shows the coefficients, and FIG. 7D shows the horizontal one-dimensional DCT coefficients.
, Respectively, show the vertical one-dimensional DCT coefficients.

Here, the image of FIG. 7A has an 8-bit pixel value, and the DCT coefficient obtained from such a pixel value can take a negative value. However, in the embodiments of FIGS. 7B to 7D, 128 (= 2 ⁷ ) is added to the obtained DCT coefficient, and if the added value is less than 0, it is clipped to 0 and the added value is Is 256
The above becomes 0 by clipping to 255.
DCT coefficients in the range of to 255 are shown.

Since the information of the entire block of 8 × 8 pixels is reflected in the two-dimensional DCT coefficient, the two-dimensional DCT coefficient grasps local information such as information of a specific pixel in the block. Is difficult. On the other hand, the horizontal one-dimensional DCT coefficient or the vertical one-dimensional DCT coefficient reflects the information of only one row or one column in which the block is present, so that it is compared with the two-dimensional DCT coefficient in the block. The local information of can be easily grasped.

That is, the feature of a certain row of a block can be grasped from the 8 × 1 horizontal 1-dimensional DCT coefficient of the row, and the feature of a certain column is 1 × 8 vertical 1-dimensional DCT of the column.
It can be understood from the T coefficient. Further, the feature of a pixel in a block is that the horizontal 1-dimensional DCT coefficient of 8 × 1 in the row in which the pixel is located and the 1 × 8 in the column in which the pixel is located.
Can be understood from the vertical one-dimensional DCT coefficient of

The state of the boundary between adjacent blocks on the left and right is a two-dimensional D in which the information of the entire block is reflected.
Using a vertical one-dimensional DCT coefficient that represents a spatial frequency component in the vertical direction at the boundary of blocks is more preferable than using a CT coefficient.
It can be grasped more accurately. Furthermore, the state of the boundary between vertically adjacent blocks is also a horizontal one-dimensional D that represents the spatial frequency component in the horizontal direction at the block boundary, rather than the two-dimensional DCT coefficient that reflects the information of the entire block.
It is possible to grasp more accurately by using the CT coefficient.

Next, the adjacent-dimension DCT coefficient selecting / transforming unit 32 of FIG.
From among the one-dimensional DCT coefficients supplied from 1, the one-dimensional DCT coefficient (adjacent 1
Dimensional DCT coefficient) is obtained.
The coefficient is a block of interest and a block spatially adjacent to the block of interest (hereinafter, appropriately referred to as an adjacent block).
Since it is used to analyze the state of the block boundary between and, it is necessary that the pixel column adjacent to the boundary of the block of interest in the spatial domain is one-dimensional DCT-transformed.

However, in MPEG2, since it is possible to select the frame structure and the field structure in units of macroblocks, the structure of the target macroblock including the target block and the macroblocks adjacent to the target macroblock (hereinafter referred to as appropriate). , Adjacent macroblocks), the one-dimensional DCT coefficient adjacent to the target block in the adjacent block adjacent to the target block is one-dimensional DCT-transformed in the spatial domain to the pixel column adjacent to the boundary of the target block. It may not be a thing.

Therefore, the adjacent one-dimensional DCT coefficient selecting / transforming unit 32 recognizes the structure of the target block and the adjacent block according to the DCT type supplied from the buffer memory 12, and in the spatial domain with the structure of the target block as a reference. The one-dimensional DCT coefficient (adjacent one-dimensional DCT coefficient) obtained by one-dimensional DCT transforming the pixel row of the adjacent block adjacent to the boundary of the block of interest is acquired.

Here, referring to FIGS. 8 to 23, when the structure of the block of interest is used as a reference, the pixel row of the adjacent block adjacent to the boundary of the block of interest is subjected to one-dimensional DCT conversion in the spatial domain. One-dimensional DCT
The coefficient will be described.

Note that in FIGS. 8 to 23, the macroblock of interest is MB _N, and the macroblocks that are adjacent to the macroblock of interest MB _N vertically and horizontally are MB _U , MB _D , and
Expressed as MB _L and MB _R.

Further, the upper left, lower left, upper right and lower right blocks of the macro block MB _N of interest are respectively
B _NUL , B _NDL , B _NUR , and B _NDR , which are upper adjacent macroblocks (macroblocks adjacent to the target block) M
The upper left, lower left, upper right, and lower right blocks of B _U are represented as B _UUL , B _UDL , B _UUR , and B _UDR , respectively. Also, the upper left, lower left, upper right, and lower right blocks of the lower adjacent macroblock (macroblock adjacent below the block of interest) MB _D are _denoted as B _DUL , B _DDL , B _DUR , and B _DDR , respectively, and left adjacent The upper left, lower left, upper right, and lower right blocks of a macroblock (macroblock adjacent to the left of the block of interest) MB _L are _denoted by B _LUL , B _LDL , B _LUR , and B _LDR , respectively.
Further, the blocks on the upper left, lower left, upper right, and lower right of the right adjacent macroblock (the macroblock adjacent to the right of the block of interest) MB _R are respectively B _RUL , B _RDL , B _{R UR} , B
_{Expressed as RDR} .

FIG. 8 shows that the block B _{NUL at} the upper left of the macro block MB _N of interest is the block of interest.
In the spatial domain, the upper boundary of the block of interest B _NUL ,
That is, the pixel row of the adjacent block adjacent to the uppermost 8 pixels (hereinafter, appropriately referred to as upper adjacent pixel row) is shown.

In FIG. 8, the shaded line represents the uppermost pixel column of the _target block B _NUL , and the shaded line represents the upper adjacent pixel column.

When the block B _{NUL on} the upper left of the macro block of interest MB _N is the block of interest, the upper adjacent block is not the block of the macro block of interest MB _N and the block B on the lower left of the upper adjacent macro block MB _{U. UDL
Therefore, which} pixel column is the upper adjacent pixel column that is adjacent to the uppermost pixel column of the target block B _{NU L} is
Macro block MB _{N of} interest and upper adjacent macro block M
It is necessary to consider the structure of both B _U.

That is, when the DCT types of the target macroblock MB _N and the upper adjacent macroblock MB _U are both frame DCT, the _target block B _NUL is the upper left 8 × 8 pixel of the _target macroblock MB _N. As shown in FIG. 8A, the uppermost 8 pixels (shaded portion) are the lowermost 8 pixels of the lower left block B _UDL of the upper adjacent macroblock MB _U of the frame structure in the spatial area. Adjacent to (the shaded area). Therefore, with respect to the horizontal one-dimensional DCT coefficient on the top row of the block of interest B _NUL , the horizontal one-dimensional DCT coefficient of the pixel column adjacent to the upper side in the spatial domain (hereinafter, appropriately referred to as the upper adjacent one-dimensional DCT coefficient) is the block B It becomes a horizontal one-dimensional DCT coefficient obtained by horizontal one-dimensional DCT conversion of the eight pixels in the bottom row of the _UDL .

The DCT type of the macro block MB _N of interest is the frame DCT, and the upper adjacent macro block MB _N
When the DCT type of _U is the field DCT, the block of interest B _NUL is 8 at the upper left of the macroblock of interest MB _N.
8 pixels in the top row, as shown in FIG. 8B.
The pixel (the shaded portion) is adjacent to the 8 pixels (the shaded portion) in the bottom row of the lower left block B _UDL of the upper adjacent macroblock MB _U of the field structure in the spatial area. Therefore, the upper adjacent one-dimensional DCT coefficients for the horizontal one-dimensional DCT coefficients of the top row of the target block B _NUL is block B _UD horizontal 1-D D 8 pixels in the bottom row of _L
It becomes a horizontal one-dimensional DCT coefficient obtained by CT conversion.

The DCT type of the macroblock MB _N of interest is the field DCT, and the upper adjacent macroblock M
DCT type B _U is the case of frame DCT, the block of interest B _NUL is eight 8 pixel to the left of the odd lines of the target macroblock MB _N, as shown in FIG. 8C,
The 8 pixels (the shaded portion) in the top row are the upper adjacent macroblocks MB of the frame structure in the space area.
_Lower left block of _U B 8 of 7th line (7th line from the top) of _UDL
Adjacent to the pixel (hatched portion). Therefore, the upper adjacent one-dimensional DCT coefficient for the horizontal one-dimensional DCT coefficient in the top row of the block of interest B _NUL is the 7th block of the block B _UDL .
Horizontal one-dimensional DC obtained by horizontal one-dimensional DCT conversion of 8 pixels in a row
It becomes the T coefficient.

The DCT type of the macroblock MB _N of interest is the field DCT, and the upper adjacent macroblock M
DCT type of B _U also, in the case of field DCT, the block of interest B _NUL is eight 8 pixel to the left of the odd lines of the target macroblock MB _N, as shown in FIG. 8D, 8 pixels in the uppermost row The shaded portion is adjacent to the 8 pixels (the shaded portion) in the bottom row of the upper left block B _UUL of the upper adjacent macroblock MB _U of the field structure in the spatial area. Therefore, the upper adjacent one-dimensional DCT coefficient for the horizontal one-dimensional DCT coefficient in the top row of the block of interest B _NUL becomes a horizontal one-dimensional DCT coefficient obtained by _performing horizontal one-dimensional DCT conversion of the eight pixels in the bottom row of the block B _UUL .

Next, FIG. 9 shows the macroblock MB _{N of} interest.
If the upper left block B _NUL is the block of interest, in the spatial region, the lower boundary of the block of interest B _NUL , that is, the pixel row of the adjacent block adjacent to the 8 pixels in the bottom row (hereinafter, appropriately, The lower adjacent pixel column) is shown.

In FIG. 9, the shaded line represents the pixel row in the bottom row of the _target block B _NUL , and the shaded line represents the lower adjacent pixel row.

[0169] block which is adjacent to the bottom of the case block B _NUL of the upper left corner of the attention macro block MB _N is the target block, from the lower left corner of the block B _NDL in the attention macro block MB _N, of the block of interest B _NUL Only the structure of the macroblock MB _N of interest needs to be taken into consideration as to which pixel column the lower adjacent pixel column adjacent below the pixel column in the bottom row becomes.

That is, the DCT of the macroblock MB _N of interest
If the type is frame DCT, block B of interest
_NUL is the upper left 8 × 8 pixels of the macro block MB _N of interest, and as shown in FIG. 9A, the 8 pixels (the shaded portion) of the bottom row of the macro block MB _N are the focus of the frame structure in the spatial region. It is adjacent to the uppermost 8 pixels (hatched portion) of the block B _{NDL at} the lower left of the macro block MB _N. Therefore, the horizontal 1D DCT of the bottom row of the block of interest B _{NU L}
The horizontal 1-dimensional DCT coefficient of a pixel row adjacent to the lower side in the spatial domain with respect to the coefficient (hereinafter, as appropriate, the lower adjacent 1-dimensional DCT coefficient
(T coefficient) is a horizontal one-dimensional DCT coefficient obtained by performing horizontal one-dimensional DCT conversion on the eight pixels in the uppermost row of the block B _NDL .

When the DCT type of the macro block MB _N of interest is the field DCT, the block of interest B
_NUL is the 8 pixels on the left side of the 8 odd lines of the macroblock MB _N of interest, and as shown in FIG. 9B, the 8 pixels in the bottom row (the shaded portion) are the fields in the spatial domain. It is adjacent to the uppermost 8 pixels (hatched portion) of the upper left block B _DUL of the lower adjacent macroblock MB _D adjacent below the target macroblock MB _N of the structure. Therefore, the horizontal 1 at the bottom of the block of interest B _NUL
The lower adjacent one-dimensional DCT coefficient to the _three- dimensional DCT coefficient becomes a horizontal one-dimensional DCT coefficient obtained by _performing horizontal one-dimensional DCT conversion on the eight pixels in the uppermost row of the block B _DUL .

Next, FIG. 10 shows the macroblock MB of interest.
_If the block B _{NUL at} the upper left of _N is the block of interest, in the spatial region, the left boundary of the block of interest B _NUL , that is, the pixel row of the adjacent block adjacent to the eight pixels in the leftmost column (hereinafter, appropriately , The left adjacent pixel column).

In FIG. 10, the shaded line represents the leftmost pixel row of the _target block B _NUL ,
The shaded line represents the left adjacent pixel column.

When the block B _{NUL at} the upper left of the macro block MB _N of interest is the block of interest, the block adjacent to the left is not the block of the macro block MB _N of interest and the block B at the upper right of the macro block MB _L of the left neighbor. _{LUR
Therefore} , it is determined which pixel row is the left adjacent pixel row adjacent to the left of the leftmost pixel row of the target block B _{NU L.}
Macro block MB _{N of} interest and left adjacent macro block M
Both structures of _BL need to be considered.

That is, when the DCT types of the macro block MB _{N of} interest and the left adjacent macro block MB _U are both frame DCT, the block of interest B _NUL is the upper left 8 × 8 pixel of the macro block of interest MB _N. , FIG. 10A
As shown in, the 8 pixels in the leftmost column (the shaded portion) are the 8 pixels in the rightmost column of the block _{BLUR on} the upper right of the left adjacent macroblock MB _L of the frame structure in the spatial area ( Adjacent to the shaded part). Therefore, with respect to the leftmost vertical 1D DCT coefficient of the block B _NUL of interest, the vertical 1D DCT coefficient (hereinafter, appropriately referred to as left adjacent 1D DCT coefficient) of the pixel row adjacent on the left side in the spatial domain is a block. It becomes a vertical one-dimensional DCT coefficient obtained by _performing vertical one-dimensional DCT conversion on the eight pixels in the rightmost column of _BLUR .

Macroblock MB of interest_NDCT type
Is the frame DCT and the left adjacent macroblock MB
_LIf the DCT type of is a field DCT, pay attention
Block B_NULIs the macroblock MB of interest_N8 on the upper left
× 8 pixels, and as shown in FIG. 10B, the leftmost column
8 pixels (shaded part) are
Left adjacent macroblock MB of field structure_LOn the upper right
Block B_LURThe top 4 pixels in the rightmost column in and the bottom right
Block B_LDR8 pixels in total in the top 4 pixels in the rightmost column
Adjacent to the element (the shaded area). Therefore, attention
Block B_NULFor the vertical one-dimensional DCT coefficient in the leftmost column of
The left adjacent one-dimensional DCT coefficient is _LURIn
Top 4 pixels on rightmost column and block B_LDROn the rightmost column in
Vertical 1 obtained by vertical 1-dimensional DCT conversion for a total of 8 pixels of 4 pixels
It becomes the dimensional DCT coefficient.

The DCT type of the macroblock MB _N of interest is the field DCT, and the left adjacent macroblock M
When the DCT type of _BL is a frame DCT, the block of interest B _NUL is the 8 pixels on the left side of the eight odd lines of the macro block of interest MB _N , and as shown in FIG. pixels (the portion that is shaded) is in the spatial domain, and 4 pixels in the odd rows of the rightmost column in the top right corner of the block B _LUR left neighboring macroblock MB _L of the frame structure, block B _LDR the lower right It is adjacent to a total of 8 pixels (the hatched portion) of the 4 pixels in the odd-numbered rows. Therefore, the left-adjacent one-dimensional DCT coefficient for the vertical one-dimensional DCT coefficient in the leftmost column of the block of interest B _NUL is 4 pixels in the odd row in the rightmost column in the block B _LUR and the block B _LUR .
A vertical one-dimensional DCT coefficient is obtained by vertical one-dimensional DCT conversion of a total of eight pixels of the four pixels in the odd-numbered rows in the _LDR .

The DCT type of the target macroblock MB _N is the field DCT, and the left adjacent macroblock M
When the DCT type of _BL is also the field DCT, the block of interest B _NUL is the 8 pixels on the left side of the eight odd lines of the macro block of interest MB _N , and as shown in FIG. 10D, 8 in the leftmost column. pixels (the portion that is shaded) is in the spatial domain, adjacent to the 8 pixels of the rightmost column in the upper right of the block B _LUR left neighboring macroblock MB _L field structure (portions are hatched) . Therefore, the left adjacent one-dimensional DCT coefficient for the vertical one-dimensional DCT coefficient in the leftmost column of the block of interest B _NUL becomes a vertical one-dimensional DCT coefficient obtained by _performing vertical one-dimensional DCT conversion on the eight pixels in the rightmost column of the block B _LUR .

Next, FIG. 11 shows the macroblock MB of interest.
_If the block B _{NUL at} the upper left of _N is the block of interest, in the spatial region, the right boundary of the block of interest B _NUL , that is, the pixel row of the adjacent block adjacent to the eight pixels in the rightmost row (hereinafter, appropriately , Right adjacent pixel column).

In FIG. 11, the shaded line represents the rightmost pixel row of the _target block B _NUL ,
The shaded line represents the right adjacent pixel column.

When the block B _{NUL at} the upper left of the macro block MB _N of interest is the block of interest, the block adjacent to the right becomes the block B _NUR at the upper right of the macro block MB _N of interest, so the block B _NUL of the block of interest B _NUL The right adjacent pixel row that is adjacent to the right of the rightmost pixel row is always the block _NUR.
8 pixels in the leftmost column of the target macroblock MB
_It is not necessary to consider _N or the structure of macroblocks adjacent to the target macroblock MB _N.

That is, when the block B _{NUL at} the upper left of the macro block MB _N of interest is the block of interest,
As shown in, the 8 pixels in the rightmost column (the shaded portion) are the 8 pixels in the leftmost column of the block B _{NUR to} the right of the block of interest B _NUL in the spatial area (shaded). Adjacent to). Therefore, the vertical one-dimensional DCT coefficient of the rightmost column of the block of interest B _NUL is the block of the vertical one-dimensional DCT coefficient of the pixel column adjacent on the right side in the spatial domain (hereinafter, appropriately referred to as the right adjacent one-dimensional DCT coefficient). B
It becomes a vertical one-dimensional DCT coefficient obtained by _performing vertical one-dimensional DCT conversion on the eight pixels in the leftmost column of the _NUR .

Next, FIG. 12 shows the macroblock MB of interest.
_If the block B _{NDL at} the lower left of _N is the block of interest, in the spatial region, the pixel row of the adjacent block adjacent to the upper boundary of the block of attention B _NDL , that is, the uppermost 8 pixels (upper adjacent pixel row) ) Is shown.

Note that, in FIG. 12, the shaded line represents the uppermost pixel column of the _target block B _NDL ,
The hatched line represents the upper adjacent pixel column.

[0185] block adjacent to the upper side of the case block B _NDL in the lower left corner of the attention macro block MB _N is the target block, from the upper left corner of the block B _NUL in the attention macro block MB _N, of the block of interest B _NDL Only the structure of the macroblock MB _N of interest needs to be taken into consideration as to which pixel column the upper adjacent pixel column adjacent to the uppermost pixel column becomes.

That is, the DCT of the macroblock MB _N of interest
If the type is frame DCT, block B of interest
_{The NDL} is the lower left 8 × 8 pixel of the macro block MB _N of interest, and as shown in FIG. 12A, the 8 pixels (the shaded portion) of the top row of the macro block MB _N are the focus of the frame structure in the spatial region. It is adjacent to the bottom row 8 pixels (hatched portion) of the upper left block B _NUL of the macro block MB _N. Therefore, the horizontal one-dimensional DCT in the top row of the block of interest B _{ND L}
The upper adjacent one-dimensional DCT coefficient for the coefficient is the block B
It becomes a horizontal one-dimensional DCT coefficient obtained by performing horizontal one-dimensional DCT conversion on the eight pixels in the bottom row of _NUL .

When the DCT type of the macro block MB _N of interest is the field DCT, the block of interest B
_NDL is the 8 pixels on the left side of the 8 even lines of the macro block MB _N of interest, and as shown in FIG. 12B, the 8 pixels in the top row (the shaded portion) are the frames in the spatial domain. The block B at the lower left of the upper adjacent macroblock MB _U adjacent to the macroblock MB _N of interest in the structure
It is adjacent to the bottom 8 pixels of _UDL (the shaded area). Therefore, the upper adjacent one-dimensional DCT coefficient for the horizontal one-dimensional DCT coefficient in the uppermost row of the block of interest B _NDL becomes a horizontal one-dimensional DCT coefficient obtained by _performing horizontal one-dimensional DCT conversion of the eight pixels in the lowermost row of the block B _UDL .

Next, FIG. 13 shows the macroblock MB of interest.
_If the block B _{NDL at} the lower left of _N is the block of interest, in the spatial domain, the pixel column of the adjacent block adjacent to the lower boundary of the block of interest B _NDL , that is, the 8 pixels in the bottom row (lower adjacent pixel Columns).

In FIG. 13, the shaded line represents the pixel row in the bottom row of the target block B _NDL ,
The hatched line represents the lower adjacent pixel column.

[0190] target macroblock lower left block B _NDL block MB _N is adjacent underneath the case of a block of interest, the upper left block B of the lower adjacent macroblocks MB _D is not a block in the target macroblock MB _{N DUL}
Therefore, which of the pixel columns is the lower adjacent pixel column that is adjacent to the lowermost pixel column of the target block B _{ND L} is
Macroblock MB _{N of} interest and lower adjacent macroblock M
Both structures of _BD need to be considered.

That is, when the DCT types of the target macroblock MB _N and the lower adjacent macroblock MB _D are both frame DCT, the _target block B _NDL is the lower left 8 × 8 pixel of the target macroblock MB _N. , FIG. 13A
As shown in FIG. 8, the 8 pixels in the bottom row (the shaded portion) are the 8 pixels in the top row of the upper left block B _DUL of the lower adjacent macroblock MB _D of the frame structure in the spatial area (the shaded _areas are shaded). Adjacent part). Therefore, the lower adjacent one-dimensional DCT coefficient to the horizontal one-dimensional DCT coefficient in the bottom row of the block of interest B _NDL is the horizontal one-dimensional DCT coefficient obtained by _performing the horizontal one-dimensional DCT conversion of the eight pixels in the top row of the block B _DUL.
It becomes a coefficient.

Macroblock MB of interest_NDCT type
Is the frame DCT and the lower adjacent macroblock MB
_DIf the DCT type of is a field DCT, pay attention
Block B_NDLIs the macroblock MB of interest_N8 at the bottom left of
X8 pixels, and as shown in FIG. 13B,
8 pixels (shaded part) are
Field structure lower adjacent macroblock MB_DOn the upper left
Block B_DUL8 pixels in the top row of (the shaded area
Min) adjacent to. Therefore, attention block B_NDLBottom line
Lower adjacent one-dimensional DCT coefficient for the horizontal one-dimensional DCT coefficient of
Number is block B _DUL8 pixels in the top row of the horizontal one-dimensional D
It becomes a horizontal one-dimensional DCT coefficient obtained by CT conversion.

The DCT type of the macroblock MB _N of interest is the field DCT, and the lower adjacent macroblock M
When the DCT type of B _D is the frame DCT, the block of interest B _NDL is the 8 pixels on the left side of the eight even lines of the macro block of interest MB _N , and as shown in FIG. The (shaded portion) is adjacent to the 8 pixels (hatched portion) in the second row of the upper left block B _DUL of the lower adjacent macroblock MB _D of the frame structure in the spatial area. . Therefore, the lower adjacent one-dimensional DCT coefficient with respect to the horizontal one-dimensional DCT coefficient in the bottom row of the _target block B _NDL is a horizontal one-dimensional DCT coefficient _{obtained by performing} horizontal one-dimensional DCT conversion on the 8 pixels in the second row of the block B _DUL .

The DCT type of the macroblock MB _N of interest is the field DCT, and the lower adjacent macroblock M
In the case of the DCT type of B _D as well, in the case of the field DCT, the block of interest B _NDL is the eight pixels on the left side of the eight even lines of the macroblock of interest MB _N , and as shown in FIG. In the spatial area, the shaded portion is adjacent to the uppermost 8 pixels (shaded portion) of the lower left block B _DDL of the lower adjacent macroblock MB _D of the field structure. Therefore, the lower adjacent one-dimensional DCT coefficient with respect to the horizontal one-dimensional DCT coefficient in the bottom row of the target block B _NDL is a horizontal one-dimensional DCT coefficient obtained by performing horizontal one-dimensional DCT conversion of the eight pixels in the top row of the block B _DDL .

Next, FIG. 14 shows the macroblock MB of interest.
_When the block B _{NDL at} the lower left of _N is the block of interest, in the spatial domain, the pixel block of the adjacent block adjacent to the left boundary of the block of interest B _NDL , that is, the eight pixels in the leftmost column (left adjacent pixel Columns).

In FIG. 14, the shaded line represents the leftmost pixel row of the _target block B _NDL ,
The shaded line represents the left adjacent pixel column.

When the block B _{NDL at} the lower left of the macro block of interest MB _N is the block of interest, the block adjacent to the left is not the block in the macro block of interest MB _N and the block at the lower right of the macro block MB _L of the left adjacent block. _BLDR
Therefore, it is determined which pixel row is the left adjacent pixel row adjacent to the left of the leftmost pixel row of the target block B _{ND L.}
Macro block MB _{N of} interest and left adjacent macro block M
Both structures of _BL need to be considered.

That is, when the DCT types of the target macroblock MB _N and the left adjacent macroblock MB _L are both frame DCT, the _target block B _NDL is the lower left 8 × 8 pixels of the target macroblock MB _N. , Figure 14A
As shown in, the 8 pixels in the leftmost column (the shaded portion) are 8 pixels in the rightmost column of the lower right block B _LDR of the left adjacent macroblock MB _L of the frame structure in the spatial domain. Adjacent to (the shaded area). Therefore, the left adjacent one-dimensional DCT coefficient to the vertical one-dimensional DCT coefficient in the leftmost column of the block of interest B _NDL is a vertical one-dimensional DCT obtained by performing vertical one-dimensional DCT conversion on the eight pixels in the rightmost column of the block B _LDR.
It becomes a coefficient.

Macroblock MB of interest_NDCT type
Is the frame DCT and the left adjacent macroblock MB
_LIf the DCT type of is a field DCT, pay attention
Block B_NDLIs the macroblock MB of interest_N8 at the bottom left of
X8 pixels, and as shown in FIG. 14B, the leftmost column
8 pixels (shaded part) are
Left adjacent macroblock MB of field structure_LOn the upper right
Block B_LURLower 4 pixels of the rightmost column in and the lower right
Block B_LDR8 pixels in total in the lower 4 pixels of the rightmost column in
Adjacent to the element (the shaded area). Therefore, attention
Block B_NDLFor the vertical one-dimensional DCT coefficient in the leftmost column of
The left adjacent one-dimensional DCT coefficient is _LURIn
The bottom 4 pixels of the rightmost column and block B_LDRUnder the rightmost column in
Vertical 1 obtained by vertical 1-dimensional DCT conversion for a total of 8 pixels of 4 pixels
It becomes the dimensional DCT coefficient.

The DCT type of the macroblock MB _N of interest is the field DCT, and the left adjacent macroblock M
When the DCT type of B _L is the frame DCT, the block of interest B _NDL is the 8 pixels on the left side of the eight even lines of the macro block of interest MB _N , and as shown in FIG. pixels (the portion that is shaded) is in the spatial domain, and 4 pixels in the even rows of the rightmost column in the top right corner of the block B _LUR left neighboring macroblock MB _L of the frame structure, block B _LDR the lower right It is adjacent to a total of 8 pixels (the shaded portion) of the 4 pixels in the even row. Therefore, the left-adjacent one-dimensional DCT coefficient for the vertical one-dimensional DCT coefficient in the leftmost column of the block of interest B _NDL is 4 pixels in the even row of the rightmost column in the block B _LUR and the block B _LUR .
A vertical one-dimensional DCT coefficient is obtained by vertical one-dimensional DCT conversion of a total of eight pixels of four pixels in the even row in the _LDR .

The DCT type of the target macroblock MB _N is the field DCT, and the left adjacent macroblock M
In the case of the DCT type of B _L as well, in the case of the field DCT, the block of interest B _NDL is the 8 pixels on the left side of the 8 even lines of the macro block of interest MB _N , and as shown in FIG. pixels (the portion that is shaded) is in the spatial domain, adjacent to the 8 pixels of the rightmost column of the block B _LDR lower right of the left neighboring macroblock MB _L field structure (portions are hatched) To do. Therefore, the left adjacent one-dimensional DCT coefficient for the vertical one-dimensional DCT coefficient in the leftmost column of the block of interest B _NDL is a vertical one-dimensional DCT coefficient obtained by performing vertical one-dimensional DCT conversion of the eight pixels in the rightmost column of the block B _LDR .

Next, FIG. 15 shows the macroblock MB of interest.
_When the block B _{NDL at} the lower left of _N is the block of interest, in the spatial domain, the pixel block of the adjacent block adjacent to the right boundary of the block of interest B _NDL , that is, the 8 pixels in the rightmost column (right adjacent pixel Columns).

In FIG. 15, the shaded line represents the rightmost pixel row of the _target block B _NDL ,
The shaded line represents the right adjacent pixel column.

When the block B _{NDL at} the lower left of the macro block MB _N of interest is the block of interest, the block adjacent to the right becomes the block B _NDR at the bottom right of the macro block of interest MB _N , so the block of interest B _NDL right adjacent pixel columns adjacent to the right of the pixel row of the right-most column, always, block _NDR of
8 pixels in the leftmost column of the target macroblock MB
_It is not necessary to consider _N or the structure of macroblocks adjacent to the target macroblock MB _N.

That is, when the lower left block B _NDL of the macro block MB _N of interest is the block of interest, FIG.
As shown in, the 8 pixels in the rightmost column (the shaded portion) are the 8 pixels in the leftmost column of the block B _NDR next to the block B _{NDL on} the right of the spatial region (hatched). Adjacent to). Therefore, the right adjacent one-dimensional DCT for the vertical one-dimensional DCT coefficient in the rightmost column of the _target block B _NDL
The coefficient becomes a vertical one-dimensional DCT coefficient obtained by performing vertical one-dimensional DCT conversion on the eight pixels in the leftmost column of the block B _NDR .

FIG. 16 shows that when the block B _{NUR at} the upper right of the macro block MB _N of interest is the block of interest, it is adjacent to the upper boundary of the block of interest B _NUR in the spatial area, that is, adjacent to the uppermost 8 pixels. The pixel row (upper adjacent pixel row) of the adjacent block is shown.

In FIG. 16, the shaded line represents the uppermost pixel column of the _target block B _NUR ,
The hatched line represents the upper adjacent pixel column.

When the block B _{NUR at} the upper right of the macro block MB _N of interest is the block of interest, the block adjacent to the top is not the block of the macro block MB _N of interest and the block at the lower right of the macro block MB _U of the upper adjacent block. B _{UDR
Therefore} , it is determined which pixel row is the upper adjacent pixel row adjacent to the uppermost pixel row of the target block B _{NU R.}
Macro block MB _{N of} interest and upper adjacent macro block M
It is necessary to consider the structure of both B _U.

That is, when the DCT types of the target macroblock MB _N and the upper adjacent macroblock MB _U are both frame DCT, the _target block B _NUR is the upper right 8 × 8 pixel of the _target macroblock MB _N. 16A
As shown in FIG. 5, the 8 pixels in the uppermost row (the shaded portion) are the 8 pixels in the lowermost row of the block B _{UDR at} the lower right of the upper adjacent macroblock MB _U of the frame structure in the spatial area (the diagonal line). Adjacent to). Therefore, the upper-adjacent one-dimensional DCT coefficient for the horizontal one-dimensional DCT coefficient of the top row of the block of interest B _NUR is the horizontal one-dimensional DCT obtained by _performing the horizontal one-dimensional DCT conversion of the eight pixels of the bottom row of the block B _UDR.
It becomes a coefficient.

The DCT type of the macroblock MB _N of interest is the frame DCT, and the upper adjacent macroblock MB _N
When the DCT type of _U is the field DCT, the block B _NUR of interest is 8 at the upper right of the macro block MB _N of interest.
16 × 8 pixels, and as shown in FIG. 16B, the uppermost 8 pixels (shaded portion) are
It is adjacent to the lowermost 8 pixels (hatched portion) of the lower right block B _UDR of the upper adjacent macroblock MB _U of the field structure. Therefore, the block of interest B over the adjacent one-dimensional DCT coefficients for the horizontal one-dimensional DCT coefficients of the top row of _NUR is block B _UD horizontal 1-D D 8 pixels in the bottom row of _R
It becomes a horizontal one-dimensional DCT coefficient obtained by CT conversion.

The DCT type of the macroblock MB _N of interest is the field DCT, and the upper adjacent macroblock M
DCT type B _U is the case of frame DCT, the block of interest B _NUR are eight right 8 pixels in the odd lines of the target macroblock MB _N, as shown in FIG. 16C, 8 pixels in the uppermost row The shaded portion is adjacent to the 8 pixels (the shaded portion) in the seventh row of the lower right block B _UDR of the upper adjacent macroblock MB _U of the frame structure in the spatial area. To do. Therefore, the upper adjacent one-dimensional DCT coefficient for the horizontal one-dimensional DCT coefficient in the top row of the block of interest B _NUR becomes a horizontal one-dimensional DCT coefficient obtained by performing horizontal one-dimensional DCT conversion of the eight pixels in the seventh row of the block B _UDR .

The DCT type of the macroblock MB _N of interest is the field DCT, and the upper adjacent macroblock M
DCT type of B _U also, in the case of field DCT, the block of interest B _NUR are eight right 8 pixels in the odd lines of the target macroblock MB _N, as shown in FIG. 16D, 8 pixels in the uppermost row The shaded portion is adjacent to the 8 pixels (the shaded portion) in the bottom row of the upper right block B _UUR of the upper adjacent macroblock MB _U of the field structure in the spatial area. Therefore, the upper adjacent one-dimensional DCT coefficient for the horizontal one-dimensional DCT coefficient in the uppermost row of the block of interest B _NUR becomes the horizontal one-dimensional DCT coefficient obtained by _performing the horizontal one-dimensional DCT conversion of the eight pixels in the lowermost row of the block B _UUR .

Next, FIG. 17 shows the macroblock MB of interest.
_If the block B _{NUR at} the upper right of _N is the block of interest, in the spatial region, the pixel column of the adjacent block adjacent to the lower boundary of the block of interest B _NUR , that is, the 8 pixels in the bottom row (lower adjacent pixel Columns).

In FIG. 17, the shaded line represents the pixel row in the bottom row of the _target block B _NUR ,
The hatched line represents the lower adjacent pixel column.

When the block B _{NUR at} the upper right of the macro block MB _N of interest is the block of interest, the block adjacent to the bottom is the block B _{NDR at the} lower right of the macro block of interest MB _N , and _{therefore the} block of interest B _NUR It is sufficient to consider only the structure of the macroblock MB _N of interest as to which pixel column the lower adjacent pixel column adjacent below the pixel column in the bottom row of is.

That is, the target macroblock MB_NDCT
If the type is frame DCT, block B of interest
_NURIs the macroblock MB of interest_NIn the upper right of 8 × 8 pixels
Yes, as shown in FIG. 17A, 8 pixels (shadow
Indicates the frame structure in the space area.
Building attention macro block MB_NBlock B at the bottom right of_NDRof
It is adjacent to the uppermost 8 pixels (hatched portion).
Therefore, attention block B _NURHorizontal 1-dimensional DC in the bottom row of
The lower adjacent one-dimensional DCT coefficient for the T coefficient is the block B
_NDRHorizontal 1-dimensional DCT conversion of 8 pixels in the top row of
It becomes a one-dimensional DCT coefficient.

When the DCT type of the macro block MB _N of interest is the field DCT, the block B of interest
_NUR is the 8 pixels on the right side of the 8 odd lines of the macro block of interest MB _N , and as shown in FIG. 17B, the 8 pixels in the bottom row (the shaded portion) are the frames in the spatial domain. Block B on the upper right of the lower adjacent macroblock MB _D that is adjacent below the macroblock MB _N of interest in the structure
It is adjacent to the top 8 pixels of the _DUR (the shaded area). Therefore, the lower adjacent one-dimensional DCT coefficient for the horizontal one-dimensional DCT coefficient in the bottom row of the _target block B _NUR becomes a horizontal one-dimensional DCT coefficient obtained by _performing horizontal one-dimensional DCT conversion of the eight pixels in the top row of the block B _DUR .

Next, FIG. 18 shows the macroblock MB of interest.
_If the block B _{NUR at} the upper right of _N is the block of interest, in the spatial region, the pixel block of the adjacent block adjacent to the left boundary of the block of interest B _NUR , that is, the 8 pixels in the leftmost column (left adjacent pixel Columns).

In FIG. 18, the shaded line represents the leftmost pixel row of the _target block B _NUR ,
The shaded line represents the left adjacent pixel column.

[0220] attention macro block to the upper right corner of the block B _NUR is adjacent to the left of the case is a target block of block MB _N, since become block B _NUL in the upper left within the attention macro block MB _N, of the block of interest B _NUR The left adjacent pixel row that is adjacent to the left of the leftmost pixel row is always the block B.
Since the 8 pixels of the rightmost column of _NUL, and target macroblock MB _N, there is no need to consider the structure of a macroblock adjacent to the target macroblock MB _N.

That is, when the block B _{NUR at} the upper right of the macro block MB _N of interest is the block of interest, FIG.
As shown in, the 8 pixels in the leftmost column (the shaded portion) are the 8 pixels in the rightmost column of the block B _NUL adjacent to the left of the block of interest B _NUR in the spatial area (shaded). Adjacent to). Therefore, the left adjacent one-dimensional DCT for the vertical one-dimensional DCT coefficient in the leftmost column of the block of interest B _NUR
The coefficient becomes a vertical one-dimensional DCT coefficient obtained by performing a vertical one-dimensional DCT conversion on the eight pixels in the rightmost column of the block B _NUL .

Next, FIG. 19 shows the macroblock MB of interest.
_If the block B _{NUR at} the upper right of _N is the block of interest, in the spatial area, the pixel block of the adjacent block adjacent to the right boundary of the block of interest B _NUR , that is, the eight pixels in the rightmost column (right adjacent pixel Columns).

In FIG. 19, the shaded line represents the rightmost pixel row of the _target block B _NUR ,
The shaded line represents the right adjacent pixel column.

When the block B _{NUR on} the upper right of the macro block MB _N of interest is the block of interest, the block adjacent to the right is not the block in the macro block MB _N of interest and the block B on the upper left of the macro block MB _R of the right neighbor. _{RUL
Therefore} , it is determined which pixel row is the right adjacent pixel row adjacent to the right of the rightmost pixel row of the block of interest B _{NU R.}
Macro block MB _{N of} interest and right adjacent macro block M
Both structures of B _R need to be considered.

That is, when the DCT types of the target macroblock MB _N and the right adjacent macroblock MB _R are both frame DCT, the _target block B _NUR is the upper right 8 × 8 pixel of the _target macroblock MB _N. , FIG. 19A
As shown in, the 8 pixels in the rightmost column (the shaded portion) are 8 pixels in the leftmost column of the upper left block B _RUL of the right adjacent macroblock MB _R of the frame structure in the spatial domain ( Adjacent to the shaded part). Therefore, the right adjacent one-dimensional DCT coefficient to the vertical one-dimensional DCT coefficient in the rightmost column of the block of interest B _NUR is a vertical one-dimensional DCT _{obtained by subjecting} the eight pixels in the leftmost column of the block B _RUL to vertical one-dimensional DCT.
It becomes a coefficient.

Macroblock MB of interest_NDCT type
Is the frame DCT and the right adjacent macroblock MB
_RIf the DCT type of is a field DCT, pay attention
Block B_NURIs the macroblock MB of interest_N8 on the upper right
X8 pixels, and as shown in FIG. 19B, in the leftmost column
8 pixels (shaded part) are
Field structure right adjacent macroblock MB_LOn the upper left
Block B_RULUpper 4 pixels of the leftmost column in and the lower left
Block B_RDL8 pixels in total in the upper 4 pixels of the leftmost column in
Adjacent to the element (the shaded area). Therefore, attention
Block B_NURFor the vertical one-dimensional DCT coefficient in the rightmost column of
The right adjacent one-dimensional DCT coefficient is _RULIn
Upper 4 pixels on the leftmost column and block B_RDLAbove the leftmost column
Vertical 1 obtained by vertical 1-dimensional DCT conversion for a total of 8 pixels of 4 pixels
It becomes the dimensional DCT coefficient.

The DCT type of the target macroblock MB _N is the field DCT, and the right adjacent macroblock M
When the DCT type of B _R is the frame DCT, the block of interest B _NUR is the eight pixels on the right side of the eight odd lines of the macro block of interest MB _N , and as shown in FIG. Pixels (shaded portions) are four pixels in the leftmost column odd row in the upper left block B _RUL of the right adjacent macroblock MB _R of the frame structure and the lower left block B _RDL in the spatial domain. It is adjacent to a total of 8 pixels of 4 pixels in odd-numbered rows (hatched portions). Therefore, the right-adjacent one-dimensional DCT coefficient to the vertical one-dimensional DCT coefficient in the rightmost column of the block of interest B _NUR is 4 pixels in the odd row of the rightmost column in the block B _RUL and the block B _RUL .
A vertical one-dimensional DCT coefficient is obtained by vertical one-dimensional DCT conversion of a total of eight pixels of four pixels in the odd row in _RDL .

The DCT type of the macroblock MB _N of interest is the field DCT, and the right adjacent macroblock M
When the DCT type of B _R is also the field DCT, the block of interest B _NUR is the 8 pixels on the right side of the eight odd lines of the macroblock of interest MB _N , and as shown in FIG. 19D, 8 in the rightmost column. The pixel (the shaded portion) is adjacent to the eight pixels (the shaded portion) in the leftmost column of the upper left block B _RUL of the right adjacent macroblock MB _R in the field structure in the spatial area. . Therefore, the right adjacent one-dimensional DCT coefficient with respect to the vertical one-dimensional DCT coefficient in the rightmost column of the block B _NUR of interest is a vertical one-dimensional DCT coefficient _{obtained by performing} vertical one-dimensional DCT conversion of the eight pixels in the leftmost column of the block B _RUL .

Next, FIG. 20 shows the macroblock MB of interest.
_If the block B _{NDR at the} lower right of _N is the block of interest, in the spatial region, the pixel block of the adjacent block adjacent to the upper boundary of the block of interest B _NDR , that is, the uppermost 8 pixels (upper adjacent pixel Columns).

In FIG. 20, the shaded line represents the pixel row in the top row of the _target block B _NDR ,
The hatched line represents the upper adjacent pixel column.

When the block B _{NDR at the} lower right of the macro block of interest MB _N is the block of interest, the upper adjacent block is the block B _NUR at the upper right of the macro block of interest MB _N , so the block of interest B _NDR It is sufficient to consider only the structure of the macro block MB _N of interest as to which pixel column the upper adjacent pixel column adjacent to the uppermost pixel column of is.

That is, the target macroblock MB_NDCT
If the type is frame DCT, block B of interest
_NDRIs the macroblock MB of interest_NAt the lower right of 8 × 8 pixels
As shown in FIG. 20A, there are 8 pixels (shadow
Indicates the frame structure in the space area.
Building attention macro block MB_NBlock B at the bottom right of_NDRof
It is adjacent to the uppermost 8 pixels (hatched portion).
Therefore, attention block B _NDRHorizontal one-dimensional DC in the top row of
The upper adjacent one-dimensional DCT coefficient for the T coefficient is the block B
_NDRHorizontal 1-dimensional DCT conversion of 8 pixels in the top row of
It becomes a one-dimensional DCT coefficient.

If the DCT type of the macro block MB _N of interest is the field DCT, the block B of interest
_{The NDR} is the 8 pixels on the right side of the 8 even lines of the macro block MB _N of interest, and as shown in FIG. 20B, the 8 pixels in the top row (the shaded portion) are the frames in the spatial domain. The block B on the lower right of the upper adjacent macroblock MB _U adjacent to the macroblock MB _N of interest in the structure
It is adjacent to the bottom 8 pixels of _UDR (the shaded area). Therefore, the upper adjacent one-dimensional DCT coefficient for the horizontal one-dimensional DCT coefficient in the uppermost row of the _target block B _NDR becomes a horizontal one-dimensional DCT coefficient obtained by performing horizontal one-dimensional DCT conversion of the eight pixels in the lowermost row of the block B _UDR .

Next, FIG. 21 shows the macroblock MB of interest.
_If the block B _{NDR at the} lower right of _N is the block of interest, in the spatial region, the pixel column of the adjacent block adjacent to the lower boundary of the block of interest B _NDR , that is, the 8 pixels in the bottom row (lower adjacent Pixel column) is shown.

In FIG. 21, the shaded line represents the pixel row in the bottom row of the target block B _NDR ,
The hatched line represents the lower adjacent pixel column.

[0236] the target macroblock which block B _NDR lower right block MB _N is adjacent underneath the case of a block of interest, the top right of the block of lower adjacent macroblocks MB _D is not a block in the target macroblock MB _N B _DUR
Therefore, which pixel column is the lower adjacent pixel column adjacent below the pixel column in the bottom row of the block of interest B _{ND R} is
Macroblock MB _{N of} interest and lower adjacent macroblock M
Both structures of _BD need to be considered.

That is, when the DCT types of the target macroblock MB _N and the lower adjacent macroblock MB _D are both frame DCT, the _target block B _NDR is the lower right 8 × 8 pixel of the _target macroblock MB _N. Yes, FIG. 21A
As shown in, the 8 pixels in the bottom row (the shaded portion) are 8 pixels in the top row of the block B _{DUR in} the upper right of the lower adjacent macroblock MB _D of the frame structure in the spatial area Adjacent part). Therefore, the lower adjacent 1D DCT coefficient to the horizontal 1D DCT coefficient in the bottom row of the block of interest B _NDR is the horizontal 1D DCT obtained by horizontally 1D DCT transforming the 8 pixels in the top row of the block B _DUR.
It becomes a coefficient.

Macro block MB of interest_NDCT type
Is the frame DCT and the lower adjacent macroblock MB
_DIf the DCT type of is a field DCT, pay attention
Block B_NDRIs the macroblock MB of interest_N8 at the bottom right of
X8 pixels, and as shown in FIG. 21B,
8 pixels (shaded part) are
Field structure lower adjacent macroblock MB_DOn the upper right
Block B_DUR8 pixels in the top row of (the shaded area
Min) adjacent to. Therefore, attention block B_NDRBottom line
Lower adjacent one-dimensional DCT coefficient for the horizontal one-dimensional DCT coefficient of
Number is block B _DUR8 pixels in the top row of the horizontal one-dimensional D
It becomes a horizontal one-dimensional DCT coefficient obtained by CT conversion.

The DCT type of the macroblock MB _N of interest is the field DCT, and the lower adjacent macroblock M
When the DCT type of B _D is the frame DCT, the block of interest B _NDR is the 8 pixels on the right side of the eight even lines of the macro block of interest MB _N , and as shown in FIG. The (shaded portion) is adjacent to the 8 pixels (hatched portion) in the second row of the block B _DUR at the upper right of the lower adjacent macroblock MB _D of the frame structure in the spatial area. . Therefore, the lower adjacent one-dimensional DCT coefficient with respect to the horizontal one-dimensional DCT coefficient in the bottom row of the block of interest B _NDR is a horizontal one-dimensional DCT coefficient obtained by _performing horizontal one-dimensional DCT conversion of 8 pixels in the second row of the block B _DUR .

The DCT type of the target macroblock MB _N is the field DCT, and the lower adjacent macroblock M
In the case of the DCT type of B _D as well, in the case of the field DCT, the _target block B _NDR is the 8 pixels on the right side of the eight even lines of the target macroblock MB _N , and as shown in FIG. The shaded portion is adjacent to the uppermost 8 pixels (shaded portion) of the lower right block B _DDR of the lower adjacent macroblock MB _D of the field structure in the spatial area. Therefore, the lower-adjacent one-dimensional DCT coefficient with respect to the horizontal one-dimensional DCT coefficient in the bottom row of the block B _NDR of interest is a horizontal one-dimensional DCT coefficient obtained by performing horizontal one-dimensional DCT conversion of the eight pixels in the top row of the block _BDDR .

Next, FIG. 22 shows the macroblock MB of interest.
_If the block B _{NDR at the} lower right of _N is the block of interest, in the spatial domain, the pixel block of the adjacent block adjacent to the left boundary of the block of interest B _NDR , that is, the leftmost column of eight pixels (left adjacent Pixel column) is shown.

In FIG. 22, the shaded line represents the leftmost pixel row of the _target block B _NDR ,
The shaded line represents the left adjacent pixel column.

When the block B _{NDR at the} lower right of the macro block MB _N of interest is the block of interest, the block adjacent to the left is the block B _NDL at the bottom left of the macro block of interest MB _N , and therefore the block of interest B _NDR left neighboring pixel columns adjacent to the left of the pixel columns of the leftmost column, always block _NDL of
8 pixels in the rightmost column of the target macroblock MB
_It is not necessary to consider _N or the structure of macroblocks adjacent to the target macroblock MB _N.

That is, when the lower right block B _{NDR of the} macro block MB _N of interest is the block of interest, FIG.
As shown in, the 8 pixels in the leftmost column (the shaded portion) are the 8 pixels in the rightmost column of the block B _NDL adjacent to the left of the block of interest B _NDR (hatched). Adjacent to). Therefore, the left adjacent one-dimensional DCT for the vertical one-dimensional DCT coefficient in the leftmost column of the block of interest B _NDR
The coefficient becomes a vertical one-dimensional DCT coefficient obtained by subjecting the eight pixels in the rightmost column of the block B _NDL to vertical one-dimensional DCT.

Next, FIG. 23 shows the macroblock MB of interest.
_If the block B _{NDR at the} lower right of _N is the block of interest, in the spatial domain, the pixel block of the adjacent block adjacent to the right boundary of the block of interest B _NDR , that is, the rightmost eight pixels (right adjacent Pixel column) is shown.

In FIG. 23, the shaded line represents the rightmost pixel row of the _target block B _NDR ,
The shaded line represents the right adjacent pixel column.

If the block B _{NDR at the} lower right of the macro block MB _N of interest is the block of interest, the block adjacent to the right is not the block in the macro block of interest MB _{N but} the block to the right of the right adjacent macro block MB _R. Block B _RDL
Therefore, which pixel row is the right adjacent pixel row adjacent to the right of the rightmost pixel row of the target block B _{ND R} is
Macro block MB _{N of} interest and right adjacent macro block M
Both structures of B _R need to be considered.

That is, when the DCT types of the target macroblock MB _N and the right adjacent macroblock MB _R are both frame DCT, the _target block B _NDR is the lower right 8 × 8 pixel of the _target macroblock MB _N. Yes, FIG. 23A
As shown in, the 8 pixels in the rightmost column (the shaded portion) are 8 pixels in the leftmost column of the lower left block B _RDL of the right adjacent macroblock MB _R of the frame structure in the spatial domain ( Adjacent to the shaded part). Therefore, the right adjacent one-dimensional DCT coefficient to the vertical one-dimensional DCT coefficient in the rightmost column of the block of interest B _NDR is a vertical one-dimensional DCT obtained by _subjecting the eight pixels in the leftmost column of the block B _RDL to vertical one-dimensional DCT conversion.
It becomes a coefficient.

Macroblock MB of interest_NDCT type
Is the frame DCT and the right adjacent macroblock MB
_RIf the DCT type of is a field DCT, pay attention
Block B_NDRIs the macroblock MB of interest_N8 at the bottom right of
23 pixels, and as shown in FIG. 23B, the rightmost column
8 pixels (shaded part) are
Field structure right adjacent macroblock MB_ROn the upper left
Block B_RULBottom 4 pixels of the leftmost column in and the bottom left
Block B_RDL8 pixels in total in the lower 4 pixels of the leftmost column in
Adjacent to the element (the shaded area). Therefore, attention
Block B_NDRFor the vertical one-dimensional DCT coefficient in the rightmost column of
The right adjacent one-dimensional DCT coefficient is _RULIn
Lower 4 pixels on the leftmost column and block B_RDLUnder the leftmost column in
Vertical 1 obtained by vertical 1-dimensional DCT conversion for a total of 8 pixels of 4 pixels
It becomes the dimensional DCT coefficient.

The DCT type of the macro block MB _N of interest is the field DCT, and the right adjacent macro block M
When the DCT type of B _R is the frame DCT, the block of interest B _NDR is the eight pixels on the left side of the eight even lines of the macro block of interest MB _N , and as shown in FIG. Pixels (shaded portions) are 4 pixels in the leftmost even-numbered row in the upper left block B _RUL of the right adjacent macroblock MB _R of the frame structure and the lower left block B _RDL in the spatial domain. 8 pixels in total, 4 pixels in the even-numbered row in the leftmost column (hatched portion)
Adjacent to. Therefore, the right adjacent one-dimensional DCT coefficient to the vertical one-dimensional DCT coefficient in the leftmost column of the block of interest B _NDR is the sum of the four pixels in the even row in the leftmost column in the block B _RUL and the four pixels in the even row in the block B _RDL . It becomes a vertical one-dimensional DCT coefficient obtained by performing vertical one-dimensional DCT conversion on 8 pixels.

The DCT type of the macro block MB _N of interest is the field DCT, and the right adjacent macro block M
When the DCT type of BR is also the field DCT, the block of interest B _NDR is the 8 pixels on the right side of the eight even lines of the macro block of interest MB _N , and as shown in FIG. 23D, the 8 pixels in the rightmost column. The (shaded portion) is adjacent to 8 pixels (hatched portion) in the leftmost column of the lower left block B _RDL of the right adjacent macroblock MB _R in the field structure in the spatial area. Therefore, the right adjacent one-dimensional DCT coefficient with respect to the vertical one-dimensional DCT coefficient in the rightmost column of the block B _NDR of interest is a vertical one-dimensional DCT coefficient obtained by performing vertical one-dimensional DCT conversion of the eight pixels in the leftmost column of the block B _RDL .

Next, FIG. 24 shows adjacent one-dimensional DCT coefficients (upper adjacent one-dimensional DCT coefficient, lower adjacent one-dimensional DCT coefficient,
3 shows a configuration example of the adjacent one-dimensional DCT coefficient selecting / converting unit 32 in FIG. 2 that acquires a left adjacent one-dimensional DCT coefficient and a right adjacent one-dimensional DCT coefficient).

In the adjacent one-dimensional DCT coefficient selection / transformation unit 32, the one-dimensional DCT coefficients (horizontal one-dimensional DCT coefficient and vertical one-dimensional DCT coefficient output by the one-dimensional inverse DCT transformation unit 31 (FIG. 2) are output.
The three-dimensional DCT coefficient) is supplied to the memory 81, and the macroblock of interest and the adjacent macroblock (upper adjacent macroblock, lower adjacent macroblock, left adjacent macro) stored in the buffer memory 12 (FIG. 12). The DCT type of the block and the right adjacent macroblock) is supplied to the sampling unit 83 and the selection unit 85.

The control unit 80 controls each block constituting the adjacent one-dimensional DCT coefficient selection / transformation unit 32.

The memory 81 has a one-dimensional inverse DCT transform unit 31.
The one-dimensional DCT coefficient output by (FIG. 2) is temporarily stored.

The vertical one-dimensional inverse DCT transform section 82 reads the vertical one-dimensional DCT coefficient stored in the memory 81,
Vertical one-dimensional inverse DCT transform, so that horizontal × vertical is 1 ×
8 pixel columns are obtained and output. The 1 × 8 pixel row output by the vertical one-dimensional inverse DCT conversion unit 82 is a sampling unit 83.
Is supplied to.

The sampling section 83 has the buffer memory 1
2 based on the DCT type of the macroblock of interest and the adjacent macroblock stored in FIG.
Sampling is performed on the pixel array supplied from the T conversion unit 83, a 1 × 8 pixel array is reconstructed from the pixels obtained as a result of the sampling, and the pixel array is supplied to the vertical one-dimensional DCT conversion unit 84.

The vertical one-dimensional DCT conversion unit 84 performs a vertical one-dimensional DCT conversion on the 1 × 8 pixel row supplied from the sampling unit 83, whereby the horizontal 1 × 8 vertical 1
The dimensional DCT coefficient is obtained and supplied to the selection unit 85.

The selecting section 85 selects the horizontal one-dimensional DCT coefficient or the vertical one-dimensional DCT coefficient stored in the memory 81, or the vertical one-dimensional DCT coefficient output from the vertical one-dimensional DCT converting section 84, and selects the adjacent one-dimensional DCT coefficient. Output as a coefficient.

Next, the processing of the adjacent one-dimensional DCT coefficient selection / transformation unit 32 (adjacent one-dimensional DCT coefficient selection / transformation processing) of FIG. 24 will be described with reference to the flowchart of FIG.

The memory 81 stores the one-dimensional D of the macro block of interest, the upper adjacent macro block, the lower adjacent macro block, the left adjacent macro block, and the right adjacent macro block.
The CT coefficient is supplied and stored. The sampling unit 83 and the selection unit 85 are supplied with the DCT types of the macroblock of interest, the upper adjacent macroblock, the lower adjacent macroblock, the left adjacent macroblock, and the right adjacent macroblock.

Then, in step S1, the control unit 8
0 determines the position of the target block in the target macroblock. In step S1, the block of interest is
When it is determined that the block is the upper left, lower left, upper right, or lower right of the block of interest, steps S2, S3, S
4 or S5, the upper left block process, the lower left block process, the upper right block process, or the lower right block process is performed, and the process ends.

The processing shown in the flowchart of FIG. 25 is performed, for example, every time the target block is changed.

Next, the upper left block processing in step S2 of FIG. 25 will be described with reference to the flowchart of FIG.

In the upper left block processing, the upper adjacent one-dimensional DC is applied to the block B _NUL on the upper left of the macro block of interest.
T coefficient, lower adjacent one-dimensional DCT coefficient, left adjacent one-dimensional DCT
Coefficients, right-adjacent one-dimensional DCT coefficients are obtained.

That is, in the upper left block processing, first, in step S11, the selection unit 85 determines the DCT type of the target macroblock and the upper adjacent macroblock.

If it is determined in step S11 that the DCT types of the macroblock of interest and the upper adjacent macroblock are both frame DCT, step S11.
12, the selection unit 85, as described in FIG. 8A,
Horizontal 1 obtained by horizontal 1-dimensional DCT transforming the 8 pixels in the bottom row in the lower left block B _UDL of the upper adjacent macroblock MB _U
The dimensional DCT coefficient is acquired as the upper adjacent one-dimensional DCT coefficient.

That is, in the case where the _target block is the block B _NUL at the upper left of the _target macroblock, and when the _target macroblock and the upper adjacent macroblock are both frame structures, the upper adjacent one-dimensional DCT coefficient is
As described in FIG. 8A, the upper adjacent macroblock MB _U
Is a horizontal one-dimensional DCT coefficient obtained by horizontally one-dimensionally DCT-transforming the eight pixels in the bottom row in the lower left block B _UDL of the block B _UDL . This horizontal one-dimensional DCT coefficient is a horizontal one-dimensional DCT coefficient of the eighth row of the block B _UDL . . The horizontal one-dimensional DCT coefficient of the eighth row of the block B _UDL is stored in the memory 81, and the selection unit 85 determines the memory 8 in the memory 8 in step S12.
From 1, the horizontal one-dimensional DCT coefficient of the eighth row of the block B _UDL is read and selected, and is output as the upper adjacent one-dimensional DCT coefficient.

In step S11, the DCT type of the target macroblock is the frame DCT, and the DCT type of the upper adjacent macroblock is the field DCT.
8B, the selection unit 85 performs horizontal one-dimensional DCT conversion on the 8 pixels in the bottom row of the lower left block B _UDL of the upper adjacent macroblock MB _U , as described with reference to FIG. 8B. The obtained horizontal one-dimensional DCT coefficient is acquired as the upper adjacent one-dimensional DCT coefficient.

That is, when the target block is the block B _NUL at the upper left of the _target macro block, and the target macro block has the frame structure and the upper adjacent macro block has the field structure, the upper adjacent one-dimensional D
As described in FIG. 8B, the CT coefficient is 8 in the bottom row in the lower left block B _UDL of the upper adjacent macroblock MB _U.
It is a horizontal one-dimensional DCT coefficient obtained by performing horizontal one-dimensional DCT conversion on a pixel, and this horizontal one-dimensional DCT coefficient is a block B _UDL.
Is the horizontal one-dimensional DCT coefficient of the eighth row of. Block B
The horizontal one-dimensional DCT coefficient of the eighth row of the _UDL is stored in the memory 81, and the selection unit 85 reads the horizontal one-dimensional DCT coefficient of the eighth row of the block B _UDL from the memory 81 in step S13.
Dimensional DCT coefficient is read out and selected, and upper one-dimensional DC
Output as T coefficient.

Further, in step S11, the DCT type of the macroblock of interest is the field DCT,
The DCT type of the upper adjacent macroblock is the frame DCT
8C, the selecting unit 85 selects the horizontal 1-dimensional DCT of the 8 pixels in the seventh row of the lower left block B _UDL of the upper adjacent macroblock MB _U as described in FIG. 8C. The converted horizontal one-dimensional DCT coefficient is acquired as an upper adjacent one-dimensional DCT coefficient.

That is, when the target block is the block B _NUL at the upper left of the _target macro block, and the target macro block has the field structure and the upper adjacent macro block has the frame structure, the upper adjacent one-dimensional D
As described with reference to FIG. 8C, the CT coefficient is a horizontal one-dimensional DCT coefficient obtained by horizontally one-dimensionally DCT-converting the eight pixels in the seventh row of the lower left block B _UDL of the upper adjacent macroblock MB _U. The DCT coefficient is the horizontal one-dimensional DCT coefficient in the seventh row of block B _UDL . Horizontal one-dimensional DCT coefficients of the seventh row of the block B _{U DL} is stored in the memory 81, selection unit 85, in step S14, from the memory 81, the seventh row of the horizontal 1-D D block B _UDL
The CT coefficient is read and selected, and is output as the upper adjacent one-dimensional DCT coefficient.

Further, in step S11, the DCT types of the macroblock of interest and the upper adjacent macroblock are
If both are determined to be the field DCT, the process proceeds to step S15, and the selecting unit 85 horizontally _sets the 8 pixels in the bottom row of the upper left block B _UUL of the upper adjacent macroblock MB _U as described in FIG. 8D. The horizontal one-dimensional DCT coefficient obtained by the one-dimensional DCT conversion is acquired as the upper adjacent one-dimensional DCT coefficient.

That is, in the case where the block of interest is the block B _NUL at the upper left of the macroblock of interest, when the macroblock of interest and the upper adjacent macroblock are both field structures, the upper adjacent one-dimensional DCT coefficient is As described in 8D, the upper adjacent macroblock M
It is a horizontal one-dimensional DCT coefficient obtained by horizontally one-dimensionally DCT-transforming the eight pixels in the bottom row of the block B _{UUL in} the upper left of B _U. This horizontal one-dimensional DCT coefficient is the horizontal one-dimensional DCT coefficient of the eighth row of the block B _UUL . Is. The horizontal one-dimensional DCT coefficient of the eighth row of the block B _UUL is stored in the memory 81,
In step S15, the selection unit 85 reads out and selects the horizontal one-dimensional DCT coefficient in the eighth row of the block B _UUL from the memory 81 and outputs it as the upper adjacent one-dimensional DCT coefficient.

After the processes of steps S12 to S15, the process proceeds to step S16, and the selecting section 85 determines the DCT type of the macro block of interest.

If it is determined in step S16 that the DCT type of the macroblock of interest is the frame DCT, the process proceeds to step S17, in which the selection unit 85 selects the frame DCT shown in FIG.
As described above, the horizontal one-dimensional DCT coefficient obtained by horizontally one-dimensionally DCT-converting the uppermost eight pixels of the lower left block B _NDL of the macro block MB _N of interest is acquired as the lower adjacent one-dimensional DCT coefficient.

That is, when the target block is the block B _NUL at the upper left of the _target macro block and the _target macro block has a frame structure, the lower adjacent 1
As described with reference to FIG. 9A, the dimensional DCT coefficient is a horizontal one-dimensional DCT coefficient obtained by horizontally one-dimensionally DCT-transforming the uppermost 8 pixels of the lower left block B _NDL of the macro block MB _N of interest. The coefficient is the horizontal one-dimensional DCT coefficient in the first row of block B _NDL . Block B _NDL
The horizontal one-dimensional DCT coefficient of the first row of is stored in the memory 81, and the selecting unit 85, in step S17,
The horizontal one-dimensional DCT coefficient of the first row of the block B _NDL is read out from the memory 81, selected, and output as the lower adjacent one-dimensional DCT coefficient.

If it is determined in step S16 that the DCT type of the macroblock of interest is the field DCT, the process proceeds to step S18, and the selection unit 85 is selected.
Is the lower adjacent macroblock M as described in FIG. 9B.
A horizontal one-dimensional DCT coefficient obtained by _performing horizontal one-dimensional DCT conversion on the uppermost eight pixels of the upper left block B _DUL of B _D is acquired as a lower adjacent one-dimensional DCT coefficient.

That is, in the case where the _target block is the block B _NUL at the upper left of the _target macroblock and the target macroblock has a field structure, the lower adjacent one-dimensional DCT coefficient is lower than the lower one-dimensional DCT coefficient as described in FIG. 9B. It is a horizontal one-dimensional DCT coefficient obtained by horizontally one-dimensionally DCT-transforming the uppermost eight pixels of the upper left block B _DUL of the adjacent macroblock MB _D. The horizontal one-dimensional DCT coefficient is the block B.
It is the horizontal one-dimensional DCT coefficient of the first row of the _DUL . The horizontal one-dimensional DCT coefficient of the first row of the block B _DUL is stored in the memory 81.
In step S18, the selection unit 85 reads out and selects the horizontal 1-dimensional DCT coefficient of the first row of the block B _DUL from the memory 81, and selects the lower adjacent 1-dimensional D
Output as CT coefficient.

After the processing of steps S17 and S18,
In either case, the process proceeds to step S19 and the sampling unit 83
And the selection unit 85 determines the DCT types of the macroblock of interest and the left adjacent macroblock.

In step S19, the macro block of interest is
The DCT type of the
Is also a frame DCT, if step S
20. The selecting unit 85 proceeds to step 20 as described in FIG. 10A.
To the left adjacent macroblock MB _LBlock B on the upper right of_LUR
Vertical 1st-dimensional DCT transformation of 8 pixels in the rightmost column of
The original DCT coefficient is acquired as the left adjacent one-dimensional DCT coefficient.
It

That is, in the case where the block of interest is the block B _NUL at the upper left of the macro block of interest, when the macro block of interest and the left adjacent macro block are both frame structures, the left adjacent one-dimensional DCT coefficient is
As described in FIG. 10A, the left adjacent macroblock MB
_L is a vertical one-dimensional DCT coefficients of the 8 pixels of the rightmost column and the vertical one-dimensional DCT transform of the block B _LUR the upper right of the vertical one-dimensional DCT coefficients, eighth column block B _LUR (8 column from the left ) Is a vertical one-dimensional DCT coefficient. Block B
The vertical one-dimensional DCT coefficient of the eighth column of the _LUR is stored in the memory 81, and the selection unit 85, in step S20, selects from the memory 81 the vertical one-dimensional DCT coefficient of the eighth column of the block B _LUR .
Dimensional DCT coefficient is read and selected, left-adjacent one-dimensional DC
Output as T coefficient.

In step S19, the DCT type of the macroblock of interest is the frame DCT, and the DCT type of the left adjacent macroblock is the field DCT.
If it is determined that, the process proceeds to steps S21 to S23, and the selecting unit 85, as described in FIG. 10B,
A total of 8 pixels, that is, the upper 4 pixels of the rightmost column in the block _{BLUR on} the upper right of the left adjacent macroblock MB _{L and} the upper 4 pixels of the rightmost column in the block B _{LDR on the} lower right of the macroblock MB _L are combined into a vertical one-dimensional DC.
The vertical one-dimensional DCT coefficient that has been T-transformed is used as the left adjacent one-dimensional DC
Obtained as a T coefficient.

That is, when the target block is the block B _NUL at the upper left of the _target macroblock, and the target macroblock has a frame structure and the left adjacent macroblock has a field structure, the left adjacent one-dimensional D
CT coefficients, as described in FIG. 10B, the four pixels on the rightmost column in the top right corner of the block B _LUR left neighboring macroblock MB _L, four pixels on the rightmost column in the block B _LDR the lower right A vertical one-dimensional DCT coefficient obtained by performing a vertical one-dimensional DCT conversion on a total of eight pixels is obtained.
The CT coefficient does not exist in the memory 81.

Therefore, the sampling unit 83 uses the vertical one-dimensional DCT coefficient of the eighth column of the block B _LUR and the block B _LUR .
Vertical 1-dimensional inverse D of vertical 1-dimensional DCT coefficient in the 8th column of _LDR
The vertical one-dimensional inverse DCT transform unit 82 is requested to perform CT transform.

In response to a request from the sampling unit 83, the vertical one-dimensional inverse DCT transform unit 82, in step S21, the vertical one-dimensional DCT coefficient of the eighth column of the block B _LUR and the vertical one-dimensional DCT coefficient of the eighth column of the block B _LDR . The one-dimensional DCT coefficient is read from the memory 81 and subjected to vertical one-dimensional inverse DCT transform. As a result, the vertical one-dimensional inverse DCT conversion unit 82
Includes 8 pixels of the eighth row of the block B _LUR, block B _LDR
The eight pixels in the eighth column of are obtained and supplied to the sampling unit 83, and the process proceeds to step S22.

At step S22, the sampling section 83
Is a block B supplied from the vertical one-dimensional inverse DCT transform unit 82 while sampling the upper four pixels of the eight pixels in the eighth column of the block B _LUR supplied from the vertical one-dimensional inverse DCT transform unit 82. The upper 4 pixels of the 8 pixels in the 8th column of the _LDR are sampled, and the upper 4 pixels sampled from the 8th column of the block B _LUR are arranged in an odd row (top field) and the 8th column of the block B _LDR is arranged.
By arranging the upper 4 pixels sampled from the column in an even row (bottom field), 8 pixels arranged in the vertical direction having the same frame structure as the macroblock of interest are supplied to the vertical one-dimensional DCT conversion unit 84.

When the vertical one-dimensional DCT transform unit 84 receives the eight pixels sampled by the sampling unit 83,
In step S23, the eight pixels are converted into a vertical one-dimensional D
CT-transformed so that the left adjacent macroblock MB _L
4 pixels on the rightmost column in the block B _{LUR on} the upper right of
Vertical one-dimensional DC obtained by performing vertical one-dimensional DCT conversion on a total of eight pixels of the upper four pixels in the rightmost column in the lower right block B _LDR
The T coefficient is obtained and supplied to the selection unit 85. The selection unit 85
The vertical one-dimensional DCT coefficient supplied from the vertical one-dimensional DCT conversion unit 84 is selected and output as a left adjacent one-dimensional DCT coefficient.

In step S19, the DCT type of the macroblock of interest is the field DCT,
DCT type of left adjacent macroblock is frame DCT
If it is determined that, the process proceeds to steps S24 to S26, and the selecting unit 85, as described in FIG. 10C,
Four pixels in the rightmost odd-numbered row in the block B _LUR in the upper right of the left adjacent macroblock MB _L and the block B in the lower right thereof
A vertical one-dimensional DCT coefficient obtained by performing a vertical one-dimensional DCT conversion on a total of eight pixels of four pixels in odd-numbered rows in the _LDR is acquired as a left adjacent one-dimensional DCT coefficient.

That is, when the block of interest is the block B _NUL at the upper left of the macro block of interest, if the macro block of interest has a field structure and the macro block of the left neighbor is a frame structure, the left adjacent one-dimensional D
CT coefficients, as described in FIG. 10C, and 4 pixels in the odd rows of the rightmost column in the top right corner of the block B _LUR left neighboring macroblock MB _L, the 4 pixels in the odd rows in the block B _LDR the lower right A vertical one-dimensional DCT coefficient obtained by subjecting a total of eight pixels to the vertical one-dimensional DCT conversion is obtained, but such a vertical one-dimensional DCT coefficient does not exist in the memory 81.

Therefore, the sampling unit 83 uses the vertical one-dimensional DCT coefficient of the eighth column of the block B _LUR and the block B _LUR .
Vertical 1-dimensional inverse D of vertical 1-dimensional DCT coefficient in the 8th column of _LDR
The vertical one-dimensional inverse DCT transform unit 82 is requested to perform CT transform.

In response to the request from the sampling unit 83, the vertical one-dimensional inverse DCT conversion unit 82, in step S24, the vertical one-dimensional DCT coefficient of the eighth column of the block B _LUR and the vertical one-dimensional DCT coefficient of the block B _LDR . The one-dimensional DCT coefficient is read from the memory 81 and subjected to vertical one-dimensional inverse DCT transform. As a result, the vertical one-dimensional inverse DCT conversion unit 82
Includes 8 pixels of the eighth row of the block B _LUR, block B _LDR
The eight pixels in the eighth column of are obtained and supplied to the sampling unit 83, and the process proceeds to step S25.

At step S25, the sampling section 83
_Samples the odd-numbered four pixels of the eight pixels in the eighth column of the block B _LUR supplied from the vertical one-dimensional inverse DCT conversion unit 82, and
8 in the eighth column of the block B _LDR supplied from the conversion unit 82
Four pixels in an odd row among the pixels are sampled, four pixels in an odd row sampled from the eighth column of the block B _LUR are arranged on the upper side, and four pixels in an odd row sampled from the eighth column of the block B _LDR are arranged. Are arranged on the lower side, so that eight pixels arranged vertically in the same field (top field) structure as the macroblock of interest are arranged,
It is supplied to the vertical one-dimensional DCT conversion unit 84.

When the vertical one-dimensional DCT transform unit 84 receives the 8 pixels sampled by the sampling unit 83,
In step S26, the eight pixels are converted into a vertical one-dimensional D
CT-transformed so that the left adjacent macroblock MB _L
Obtained in the 4 pixels in the odd rows of the rightmost column in the top right corner of the block B _LUR, a vertical one-dimensional DCT coefficient a total of 8 pixels and vertical 1-dimensional DCT transform of 4 pixels in the odd rows in the block B _LDR the lower right , To the selection unit 85. Selector 85
Is the vertical 1 supplied from the vertical one-dimensional DCT conversion unit 84.
A dimensional DCT coefficient is selected and output as a left adjacent one-dimensional DCT coefficient.

In step S19, the DCT types of the macroblock of interest and the left adjacent macroblock are
If it is determined that both a field DCT, the process proceeds to step S27, the selection unit 85, as described in FIG. 10D, the eight pixels rightmost column in the upper right of the block B _LUR left neighboring macroblock MB _L The vertical one-dimensional DCT coefficient obtained by the vertical one-dimensional DCT conversion is acquired as the left adjacent one-dimensional DCT coefficient.

That is, when the target block is the block B _NUL at the upper left of the _target macroblock and both the target macroblock and the left adjacent macroblock have the field structure, the left adjacent one-dimensional DCT coefficient is As described in 10D, the 8 pixels in the rightmost column of the block _{BLUR on} the upper right of the left adjacent macroblock MB _L are vertically set to 1
It is a vertical one-dimensional DCT coefficient _obtained by the _three- dimensional DCT transformation, and this vertical one-dimensional DCT coefficient is the vertical one-dimensional DCT coefficient of the eighth column of the block B _LUR . The vertical one-dimensional DCT coefficient of the eighth column of the block B _LUR is stored in the memory 81, and the selecting unit 85 determines the memory 8 in step S27.
From 1, the vertical one-dimensional DCT coefficient in the eighth column of the block B _LUR is read and selected, and is output as the left adjacent one-dimensional DCT coefficient.

After the processing of steps S20, S23, S26, and S27, the process proceeds to step S28.
As described with reference to FIG. 11, the selection unit 85 vertically _sets 8 pixels in the leftmost column of the block B _{NUR on} the right of the block of interest B _{NU L} to the vertical direction.
The vertical one-dimensional DCT coefficient obtained by the three-dimensional DCT transformation
Obtained as the dimensional DCT coefficient.

That is, the block of interest is the macro block of interest.
Block B in the upper left corner_NULIs the right-adjacent primary if
The original DCT coefficient is the block of interest as described in FIG.
B _NULBlock B to the right of_NUR8 pixels in the leftmost column of
This is the vertical one-dimensional DCT coefficient obtained by one-dimensional DCT conversion.
The vertical one-dimensional DCT coefficient of_NURIn the first row of
It is a vertical one-dimensional DCT coefficient. Block B_NURFirst row of
The vertical one-dimensional DCT coefficient of is stored in the memory 81.
In step S28, the selection unit 85 selects the memory 8
Block B from 1_NURVertical one-dimensional DCT section of the first column of
Read out the number and select it as the right adjacent one-dimensional DCT coefficient
Output and end the upper left block processing.

Next, the lower left block processing in step S3 of FIG. 25 will be described with reference to the flowchart of FIG.

In the lower left block processing, the upper adjacent one-dimensional DC is applied to the lower left block B _NDL of the macro block of interest.
T coefficient, lower adjacent one-dimensional DCT coefficient, left adjacent one-dimensional DCT
Coefficients, right-adjacent one-dimensional DCT coefficients are obtained.

That is, in the lower left block processing, first, in step S31, the selection unit 85 determines the DCT type of the macro block of interest.

When it is determined in step S31 that the DCT type of the macroblock of interest is the frame DCT, the process proceeds to step S32, and the selecting section 85 is set to the one shown in FIG.
As described in A, the 8 pixels in the bottom row of the block B _{NUL at} the upper left of the macro block MB _N of interest are subjected to the horizontal one-dimensional DCT.
The converted horizontal one-dimensional DCT coefficient is converted into the upper adjacent one-dimensional DCT.
Get as a coefficient.

That is, in the case where the block of interest is the block B _NDL at the lower left of the macro block of interest, when the macro block of interest has a frame structure, the upper adjacent 1
As described with reference to FIG. 12A, the dimensional DCT coefficient is 8 in the bottom row of the block B _{NUL in} the upper left of the macro block MB _N of interest.
It is a horizontal one-dimensional DCT coefficient obtained by performing horizontal one-dimensional DCT conversion on a pixel, and this horizontal one-dimensional DCT coefficient is a block B _NUL.
Is the horizontal one-dimensional DCT coefficient of the eighth row of. Block B
The horizontal one-dimensional DCT coefficient of the eighth row of _NUL is stored in the memory 81, and the selecting unit 85, in step S32, reads the horizontal 1-dimensional DCT coefficient of the eighth row of block B _NUL from the memory 81.
Dimensional DCT coefficient is read out and selected, and upper one-dimensional DC
Output as T coefficient.

If it is determined in step S31 that the DCT type of the macroblock of interest is the field DCT, the process proceeds to step S33, and the selection unit 85 is selected.
As described with reference to FIG. 12B, the 8 pixels in the bottom row of the lower left block B _UDL of the upper adjacent macroblock MB _U are horizontally set to 1
The horizontal one-dimensional DCT coefficient obtained by the three-dimensional DCT transformation
Obtained as the dimensional DCT coefficient.

That is, in the case where the _target block is the block B _NDL at the lower left of the _target macroblock and the target macroblock has the field structure, the upper adjacent one-dimensional DCT coefficient is as shown in FIG. 12B. Horizontal 1-dimensional DCT obtained by horizontally 1-dimensional DCT transforming the 8 pixels in the bottom row of the lower left block B _UDL of the adjacent macroblock MB _U
Coefficient, and this horizontal one-dimensional DCT coefficient is a block B
It is the horizontal one-dimensional DCT coefficient of the eighth row of _UDL . The horizontal one-dimensional DCT coefficient of the first row of the block B _UDL is stored in the memory 81.
In step S33, the selection unit 85 reads the horizontal 1-dimensional DCT coefficient of the first row of the block B _UDL from the memory 81 and selects it, and selects the upper adjacent 1-dimensional D
Output as CT coefficient.

After the processing of steps S32 and S33,
In either case, the selection unit 85 determines the DCT types of the macro block of interest and the lower adjacent macro block in step S34.

[0307] In step S34, the macro block of interest
The DCT type of the
Is also a frame DCT, if step S
Proceeding to step 35, the selection unit 85 proceeds as described in FIG. 13A.
To the lower adjacent macroblock MB _DBlock B on the upper left of_DUL
Water obtained by horizontal 1-dimensional DCT transformation of the top 8 pixels in
Flat 1-dimensional DCT coefficient as lower adjacent 1-dimensional DCT coefficient
get.

That is, when the target block is the block B _NDL at the lower left of the _target macroblock and both the target macroblock and the lower adjacent macroblock have a frame structure, the lower adjacent one-dimensional DCT coefficient is
As described in FIG. 13A, the lower adjacent macroblock MB
It is a horizontal one-dimensional DCT coefficient obtained by horizontally one-dimensionally DCT-transforming the top eight pixels in the block B _{DUL on} the upper left of _D. This horizontal one-dimensional DCT coefficient is the horizontal one-dimensional DCT coefficient of the first row of the block B _DUL . is there. The horizontal one-dimensional DCT coefficient of the first row of the block B _DUL is stored in the memory 81, and the selection unit 85 determines the memory 8 in step S35.
From 1, the horizontal one-dimensional DCT coefficient of the first row of the block B _DUL is read and selected, and is output as the lower adjacent one-dimensional DCT coefficient.

[0309] In step S34, the DCT type of the macroblock of interest is the frame DCT, and the DCT type of the lower adjacent macroblock is the field DCT.
13B, the selection unit 85 performs horizontal one-dimensional DCT conversion on the uppermost 8 pixels of the upper left block B _DUL of the lower adjacent macroblock MB _D as described with reference to FIG. 13B. The obtained horizontal one-dimensional DCT coefficient is acquired as the lower adjacent one-dimensional DCT coefficient.

That is, in the case where the _target block is the block B _NDL at the lower left of the _target macroblock, if the _target macroblock has a frame structure and the lower adjacent macroblock has a field structure, the lower adjacent one-dimensional D
As described with reference to FIG. 13B, the CT coefficient is a horizontal one-dimensional DCT coefficient obtained by horizontally one-dimensionally DCT-transforming the eight pixels in the bottom row of the upper left block B _DUL of the lower adjacent macroblock MB _D , and this horizontal one-dimensional DCT Coefficient is block B
It is the horizontal one-dimensional DCT coefficient of the first row of the _DUL . The horizontal one-dimensional DCT coefficient of the first row of the block B _DUL is stored in the memory 81.
In step S36, the selection unit 85 reads the horizontal 1-dimensional DCT coefficient of the first row of the block B _DUL from the memory 81 and selects it, and selects the lower adjacent 1-dimensional D D coefficient.
Output as CT coefficient.

Further, in step S34, the DCT type of the macroblock of interest is the field DCT,
The DCT type of the lower adjacent macroblock is the frame DCT
13C, the selecting unit 85, as described with reference to FIG. 13C, horizontally 1-dimensionally the 8 pixels in the second row of the upper left block B _DUL of the lower adjacent macro block MB _D. The horizontal one-dimensional DCT coefficient obtained by DCT conversion is
It is acquired as the lower adjacent one-dimensional DCT coefficient.

That is, when the target block is the block B _NDL at the lower left of the _target macroblock, if the _target macroblock has a field structure and the lower adjacent macroblock has a frame structure, the lower adjacent one-dimensional D
As described with reference to FIG. 13C, the CT coefficient is a horizontal one-dimensional DCT coefficient obtained by horizontally one-dimensionally DCT-converting the eight pixels in the second row of the upper left block B _DUL of the lower adjacent macroblock MB _D. The dimensional DCT coefficient is the horizontal one-dimensional DCT coefficient in the second row of block B _DUL . Block B _DUL
The horizontal one-dimensional DCT coefficient of the second row of is stored in the memory 81, and the selecting unit 85, in step S37,
The horizontal one-dimensional DCT coefficient of the second row of the block B _DUL is read out from the memory 81, selected, and output as the lower adjacent one-dimensional DCT coefficient.

In step S34, the DCT types of the macroblock of interest and the lower adjacent macroblock are
If both are determined to be the field DCT, the process proceeds to step S38, and the selecting unit 85 horizontally sets the top 8 pixels of the lower left block B _DDL of the lower adjacent macroblock MB _D as described with reference to FIG. 13D. The horizontal one-dimensional DCT coefficient obtained by the one-dimensional DCT conversion is acquired as the lower adjacent one-dimensional DCT coefficient.

That is, when the target block is the block B _NDL at the lower left of the _target macroblock and both the target macroblock and the lower adjacent macroblock have the field structure, the lower adjacent one-dimensional DCT coefficient is As described in 13D, the uppermost 8 pixels of the lower left block B _DDL of the lower adjacent macroblock MB _D are horizontally set to 1
The horizontal one-dimensional DCT coefficient obtained by the three-dimensional DCT transformation, and the horizontal one-dimensional DCT coefficient is the horizontal one-dimensional DCT coefficient in the first row of the block B _DDL . The horizontal one-dimensional DCT coefficient of the first row of the block B _DDL is stored in the memory 81, and the selection unit 85, in step S38, the memory 8
From 1, the horizontal one-dimensional DCT coefficient in the first row of the block B _DDL is read and selected, and is output as the lower adjacent one-dimensional DCT coefficient.

After the processes of steps S35 to S38, the process proceeds to step S39, and the sampling section 83 and the selecting section 85 determine the DCT type of the macroblock of interest and the left adjacent macroblock.

In step S39, the macro block of interest
The DCT type of the
Is also a frame DCT, if step S
Proceeding to 40, the selection unit 85 proceeds to the process described in FIG. 14A.
To the left adjacent macroblock MB _LBlock B at the bottom right of_LDR
Vertical 1st-dimensional DCT transformation of 8 pixels in the rightmost column of
The original DCT coefficient is acquired as the left adjacent one-dimensional DCT coefficient.
It

That is, in the case where the block of interest is the block B _NDL at the lower left of the macro block of interest, when the macro block of interest and the left adjacent macro block are both frame structures, the left adjacent one-dimensional DCT coefficient is
As described in FIG. 14A, the left adjacent macroblock MB
It is a vertical one-dimensional DCT coefficient obtained by vertically one-dimensionally DCT-transforming the eight pixels in the rightmost column of the block B _{LDR at} the lower right of _L , and this vertical one-dimensional DCT coefficient is the vertical 1-dimensional DCT coefficient of the eighth column of the block B _LDR .
Dimensional DCT coefficient. The vertical one-dimensional DCT coefficient of the eighth column of the block B _LDR is stored in the memory 81, and the selection unit 85 selects the vertical one-dimensional DCT coefficient of the eighth column of the block B _LDR from the memory 81 in step S40. It is read, selected, and output as a left adjacent one-dimensional DCT coefficient.

Also, in step S39, the DCT type of the macroblock of interest is the frame DCT, and the DCT type of the left adjacent macroblock is the field DCT.
If it is determined that, the process proceeds to steps S41 to S43 sequentially, and the selecting unit 85, as described in FIG. 14B,
A total of 8 pixels of the lower 4 pixels of the rightmost column in the block B _{LUR on} the upper right of the left adjacent macroblock MB _L and the lower 4 pixels of the rightmost column in the block B _LDR of the lower right of the macroblock MB _L are combined into a vertical one-dimensional DC.
The vertical one-dimensional DCT coefficient that has been T-transformed is used as the left adjacent one-dimensional DC
Obtained as a T coefficient.

That is, in the case where the _target block is the block B _NDL at the lower left of the _target macroblock, if the _target macroblock has a frame structure and the left adjacent macroblock has a field structure, the left adjacent one-dimensional D
CT coefficients, as described in FIG. 14B, the four pixels on the rightmost column in the top right corner of the block B _LUR left neighboring macroblock MB _L, four pixels on the rightmost column in the block B _LDR the lower right A vertical one-dimensional DCT coefficient obtained by performing a vertical one-dimensional DCT conversion on a total of eight pixels is obtained.
The CT coefficient does not exist in the memory 81.

Therefore, the sampling unit 83 uses the vertical one-dimensional DCT coefficient of the eighth column of the block B _LUR and the block B _LUR .
Vertical 1-dimensional inverse D of vertical 1-dimensional DCT coefficient in the 8th column of _LDR
The vertical one-dimensional inverse DCT transform unit 82 is requested to perform CT transform.

In response to the request from the sampling unit 83, the vertical one-dimensional inverse DCT transform unit 82, in step S41, the vertical one-dimensional DCT coefficient of the eighth column of the block B _LUR and the vertical one-dimensional DCT coefficient of the block B _LDR . The one-dimensional DCT coefficient is read from the memory 81 and subjected to vertical one-dimensional inverse DCT transform. As a result, the vertical one-dimensional inverse DCT conversion unit 82
Includes 8 pixels of the eighth row of the block B _LUR, block B _LDR
The eight pixels in the eighth column of are obtained and supplied to the sampling unit 83, and the process proceeds to step S42.

In step S42, the sampling section 83
Is a block B supplied from the vertical one-dimensional inverse DCT transform unit 82 while sampling the lower four pixels of the eight pixels in the eighth column of the block B _LUR supplied from the vertical one-dimensional inverse DCT transform unit 82. The lower 4 pixels of the 8 pixels in the 8th column of the _LDR are sampled, the upper 4 pixels sampled from the 8th column of the block B _LUR are arranged in an odd row (top field), and the 8th column of the block B _LDR is arranged.
By arranging the upper 4 pixels sampled from the column in an even row (bottom field), 8 pixels arranged in the vertical direction having the same frame structure as the macroblock of interest are supplied to the vertical one-dimensional DCT conversion unit 84.

When the vertical one-dimensional DCT conversion unit 84 receives the 8 pixels sampled by the sampling unit 83,
In step S43, the eight pixels are converted into a vertical one-dimensional D
CT-transformed so that the left adjacent macroblock MB _L
4 pixels on the rightmost column in the block B _{LUR on} the upper right of
Vertical one-dimensional DC obtained by performing vertical one-dimensional DCT conversion on a total of eight pixels of the upper four pixels in the rightmost column in the lower right block B _LDR
The T coefficient is obtained and supplied to the selection unit 85. The selection unit 85
The vertical one-dimensional DCT coefficient supplied from the vertical one-dimensional DCT conversion unit 84 is selected and output as a left adjacent one-dimensional DCT coefficient.

In step S39, the DCT type of the macroblock of interest is the field DCT,
DCT type of left adjacent macroblock is frame DCT
If it is determined that, the process proceeds to steps S44 to S46 sequentially, and the selecting unit 85, as described with reference to FIG. 14C,
4 pixels in the right-most even-numbered row in the block B _LUR in the upper right of the left adjacent macroblock MB _L and the block B in the lower right thereof
A vertical one-dimensional DCT coefficient obtained by performing a vertical one-dimensional DCT conversion on a total of eight pixels of four pixels in an even row in the _LDR is acquired as a left adjacent one-dimensional DCT coefficient.

That is, in the case where the _target block is the block B _NDL at the lower left of the _target macroblock, if the _target macroblock has a field structure and the left adjacent macroblock has a frame structure, the left adjacent one-dimensional D
CT coefficients, as described in FIG. 14C, and 4 pixels in the even rows of the rightmost column in the top right corner of the block B _LUR left neighboring macroblock MB _L, four pixels of the even row in the block B _LDR the lower right A vertical one-dimensional DCT coefficient obtained by subjecting a total of eight pixels to the vertical one-dimensional DCT conversion is obtained, but such a vertical one-dimensional DCT coefficient does not exist in the memory 81.

Therefore, the sampling unit 83 uses the vertical one-dimensional DCT coefficient in the eighth column of the block B _LUR and the block B _LUR .
Vertical 1-dimensional inverse D of vertical 1-dimensional DCT coefficient in the 8th column of _LDR
The vertical one-dimensional inverse DCT transform unit 82 is requested to perform CT transform.

[0327] Vertical one-dimensional inverse DCT unit 82, in response to a request from the sampling unit 83, in step S44, and the vertical one-dimensional DCT coefficients of the eighth column of the block B _LUR, vertical column 8 block B _LDR The one-dimensional DCT coefficient is read from the memory 81 and subjected to vertical one-dimensional inverse DCT transform. As a result, the vertical one-dimensional inverse DCT conversion unit 82
Includes 8 pixels of the eighth row of the block B _LUR, block B _LDR
The eight pixels in the eighth column of are obtained and supplied to the sampling unit 83, and the process proceeds to step S45.

At step S45, the sampling section 83
Is a vertical one-dimensional inverse DCT converter that samples four pixels in an even row of the eight pixels in the eighth column of the block B _LUR supplied from the vertical one-dimensional inverse DCT converter.
8 in the eighth column of the block B _LDR supplied from the conversion unit 82
Even pixels of 4 pixels in even rows of the pixels are sampled, 4 pixels of even rows sampled from the 8th column of the block B _LUR are arranged on the upper side, and 4 pixels of even rows sampled from the 8th column of the block B _LDR are arranged. Is arranged on the lower side, the eight pixels arranged vertically in the same field (bottom field) structure as the macroblock of interest are arranged.
It is supplied to the vertical one-dimensional DCT conversion unit 84.

When the vertical one-dimensional DCT transform unit 84 receives the 8 pixels sampled by the sampling unit 83,
In step S46, the eight pixels are converted into a vertical one-dimensional D
CT-transformed so that the left adjacent macroblock MB _L
Obtained in the 4 pixels in the even rows of the rightmost column in the top right corner of the block B _LUR, a vertical one-dimensional DCT coefficient a total of 8 pixels and vertical 1-dimensional DCT transform of 4 pixels in the even rows in the block B _LDR the lower right , To the selection unit 85. Selector 85
Is the vertical 1 supplied from the vertical one-dimensional DCT conversion unit 84.
A dimensional DCT coefficient is selected and output as a left adjacent one-dimensional DCT coefficient.

In step S39, the DCT types of the macroblock of interest and the left adjacent macroblock are
When both are determined to be the field DCT, the process proceeds to step S47, and the selection unit 85, as described in FIG. 14D, the 8 pixels in the rightmost column of the block B _{LDR at} the lower right of the left adjacent macroblock MB _L. The vertical one-dimensional DCT coefficient obtained by performing the vertical one-dimensional DCT conversion is acquired as the left adjacent one-dimensional DCT coefficient.

That is, in the case where the block of interest is the block B _NDL at the lower left of the macro block of interest, when the macro block of interest and the left adjacent macro block are both field structures, the left adjacent one-dimensional DCT coefficient is as described in 14D, a left adjacent macroblock MB _L eight pixels of the rightmost column of the block B _LDR lower right vertical 1
It is a vertical one-dimensional DCT coefficient obtained by the three-dimensional DCT transformation, and this vertical one-dimensional DCT coefficient is the vertical one-dimensional DCT coefficient of the eighth column of the block B _LDR . The vertical one-dimensional DCT coefficient in the eighth column of the block B _LDR is stored in the memory 81, and the selection unit 85 determines the memory 8 in step S47.
From 1, the vertical one-dimensional DCT coefficient in the eighth column of the block B _LDR is read and selected, and is output as the left adjacent one-dimensional DCT coefficient.

After the processes of steps S40, S43, S46, and S47, the process proceeds to step S48.
Selecting unit 85, FIG. 15 as described in, the block of interest B _{ND L} eight pixels of the leftmost column of the block B _NDR to the right vertical 1
The vertical one-dimensional DCT coefficient obtained by the three-dimensional DCT transformation
Obtained as the dimensional DCT coefficient.

In other words, the block of interest is the macro block of interest.
Block B at the bottom left of Ku_NDLIs the right-adjacent primary if
The original DCT coefficient is the block of interest as described in FIG.
B _NDLBlock B to the right of_NDR8 pixels in the leftmost column of
This is the vertical one-dimensional DCT coefficient obtained by one-dimensional DCT conversion.
The vertical one-dimensional DCT coefficient of_NDRIn the first row of
It is a vertical one-dimensional DCT coefficient. Block B_NDRFirst row of
The vertical one-dimensional DCT coefficient of is stored in the memory 81.
In step S48, the selection unit 85 selects the memory 8
Block B from 1_NDRVertical one-dimensional DCT section of the first column of
The number is read and selected, and the right adjacent vertical one-dimensional DCT coefficient
Then, the lower left block processing is ended.

Next, the upper right block processing in step S4 of FIG. 25 will be described with reference to the flowchart of FIG.

In the upper right block processing, for the block B _NUR on the upper right of the macro block of interest, the upper adjacent one-dimensional DC
T coefficient, lower adjacent one-dimensional DCT coefficient, left adjacent one-dimensional DCT
Coefficients, right-adjacent one-dimensional DCT coefficients are obtained.

That is, in the upper right block processing, first, in step S51, the selection unit 85 determines the DCT types of the macroblock of interest and the upper adjacent macroblock.

In step S51, the macro block of interest
The DCT type of the
Is also a frame DCT, if step S
Proceeding to 52, the selection unit 85 proceeds as described in FIG. 16A.
To the upper adjacent macroblock MB _UBlock B at the bottom right of_UDR
Water obtained by horizontal 1-dimensional DCT transformation of the 8 pixels in the bottom row in
The flat one-dimensional DCT coefficient as the upper adjacent one-dimensional DCT coefficient
get.

That is, in the case where the block of interest is the block B _NUR at the upper right of the macroblock of interest, when both the macroblock of interest and the macroblock adjacent to the top have a frame structure, the top-adjacent one-dimensional DCT coefficient is
As described in FIG. 16A, the upper adjacent macroblock MB
These are horizontal 1-dimensional DCT coefficients obtained by horizontally 1-dimensionally DCT-transforming the 8 pixels in the bottom row in the lower right block B _UDR of _U. The horizontal 1-dimensional DCT coefficient is the horizontal 1-dimensional DCT coefficient of the 8th row of block B _UDR . Is. The horizontal one-dimensional DCT coefficient of the eighth row of the block B _UDR is stored in the memory 81, and the selection unit 85, in step S52, the memory 8
From 1, the horizontal one-dimensional DCT coefficient of the eighth row of the block B _UDR is read and selected, and is output as the upper adjacent one-dimensional DCT coefficient.

In step S51, the DCT type of the target macroblock is the frame DCT, and the DCT type of the upper adjacent macroblock is the field DCT.
16B, the selecting unit 85, as described with reference to FIG. 16B, selects the 8 pixels in the bottom row of the lower right block B _UDR of the upper adjacent macroblock MB _U from the horizontal one-dimensional DCT. The converted horizontal one-dimensional DCT coefficient is acquired as an upper adjacent one-dimensional DCT coefficient.

That is, in the case where the _target block is the block B _NUR at the upper right of the _target macroblock, when the _target macroblock has a frame structure and the upper adjacent macroblock has a field structure, the upper adjacent one-dimensional D
As described with reference to FIG. 16B, the CT coefficient is a horizontal one-dimensional DCT coefficient obtained by horizontal one-dimensional DCT transforming the eight pixels in the bottom row in the lower right block B _UDR of the upper adjacent macroblock MB _U. DCT coefficient is block B
It is the horizontal one-dimensional DCT coefficient of the eighth row of the _UDR . The horizontal one-dimensional DCT coefficient in the eighth row of the block B _UDR is stored in the memory 81.
In step S53, the selection unit 85 reads the horizontal 1-dimensional DCT coefficient in the eighth row of the block B _UDR from the memory 81 and selects the horizontal 1-dimensional DCT coefficient.
Output as CT coefficient.

Also, in step S51, the DCT type of the macroblock of interest is the field DCT,
The DCT type of the upper adjacent macroblock is the frame DCT
16C, the selecting unit 85 horizontally sets the 8 pixels in the seventh row of the lower right block B _UDR of the upper adjacent macroblock MB _U to horizontal 1 as described in FIG. 16C. The horizontal one-dimensional DCT coefficient obtained by the three-dimensional DCT transformation,
It is acquired as the upper adjacent one-dimensional DCT coefficient.

That is, in the case where the _target block is the block B _NUR at the upper right of the _target macroblock, if the _target macroblock has a field structure and the upper adjacent macroblock has a frame structure, the upper adjacent one-dimensional D
As described with reference to FIG. 16C, the CT coefficient is a horizontal one-dimensional DCT coefficient obtained by horizontally one-dimensional DCT-transforming the 8 pixels in the seventh row of the lower right block B _UDR of the upper adjacent macroblock MB _U , and this horizontal The one-dimensional DCT coefficient is the horizontal one-dimensional DCT coefficient in the seventh row of the block B _UDR . Block B _UDR
The horizontal one-dimensional DCT coefficient in the seventh row of is stored in the memory 81, and the selection unit 85 determines in step S54 that
The horizontal one-dimensional DCT coefficient in the seventh row of the block B _UDR is read out from the memory 81 and selected, and is output as the upper adjacent one-dimensional DCT coefficient.

Also, in step S51, the DCT types of the macroblock of interest and the upper adjacent macroblock are
When both are determined to be the field DCT, the process proceeds to step S55, and the selecting unit 85 horizontally sets the eight pixels in the bottom row of the block B _{UUR in} the upper right of the upper adjacent macroblock MB _U , as described in FIG. 16D. The horizontal one-dimensional DCT coefficient obtained by the one-dimensional DCT conversion is acquired as the upper adjacent one-dimensional DCT coefficient.

That is, when the target block is the block B _NUR at the upper right of the _target macroblock and both the _target macroblock and the upper adjacent macroblock have the field structure, the upper adjacent one-dimensional DCT coefficient is As described in 16D, the 8 pixels in the bottom row of the upper right block B _UUR of the upper adjacent macroblock MB _U are horizontally set to 1
The horizontal one-dimensional DCT coefficient _obtained by the _three- dimensional DCT conversion, and the horizontal one-dimensional DCT coefficient is the horizontal one-dimensional DCT coefficient in the eighth row of the block B _UUL . The horizontal one-dimensional DCT coefficient of the eighth row of the block B _UUR is stored in the memory 81, and the selection unit 85 determines the memory 8 in step S55.
From 1, the horizontal one-dimensional DCT coefficient in the eighth row of the block B _UUR is read and selected, and is output as the upper adjacent one-dimensional DCT coefficient.

After the processes of steps S52 to S55, the process proceeds to step S56, and the selecting section 85 determines the DCT type of the macro block of interest.

If it is determined in step S56 that the DCT type of the macroblock of interest is the frame DCT, the process proceeds to step S57, in which the selection unit 85 selects the DCT type shown in FIG.
As described in A, the eight pixels in the uppermost row of the block B _{NDR at} the lower right of the macro block MB _N of interest are _subjected to the horizontal one-dimensional DCT.
The converted horizontal one-dimensional DCT coefficient is converted to the lower adjacent one-dimensional DCT
Get as a coefficient.

That is, when the _target block is the block B _NUR at the upper right of the _target macro block and the _target macro block has a frame structure, the lower adjacent 1
As described with reference to FIG. 17A, the dimensional DCT coefficient is 8 in the uppermost row of the lower right block B _NDR of the macro block MB _N of interest.
It is a horizontal one-dimensional DCT coefficient obtained by performing horizontal one-dimensional DCT conversion on a pixel, and this horizontal one-dimensional DCT coefficient is a block B _NDR.
Is the horizontal one-dimensional DCT coefficient of the first row of. Block B
The horizontal one-dimensional DCT coefficient of the first row of the _NDR is stored in the memory 81, and the selection unit 85, in step S57, selects the horizontal 1-dimensional DCT coefficient of the first row of the block B _NDR from the memory 81.
Dimensional DCT coefficient is read and selected, and lower adjacent one-dimensional DC
Output as T coefficient.

If it is determined in step S56 that the DCT type of the macro block of interest is the field DCT, the process proceeds to step S58, and the selection unit 85 is selected.
17B, the 8 pixels in the uppermost row of the block B _{DUR at} the upper right of the lower adjacent macroblock MB _D are horizontally set to 1 as described in FIG. 17B.
The horizontal one-dimensional DCT coefficient obtained by the three-dimensional DCT transformation
Obtained as the dimensional DCT coefficient.

That is, when the _target block is the block B _NUR at the upper right of the _target macroblock and the _target macroblock has the field structure, the lower adjacent one-dimensional DCT coefficient is lower than the lower one-dimensional DCT coefficient as described in FIG. 17B. Horizontal 1-dimensional DCT obtained by horizontally 1-dimensional DCT-converting the 8 pixels in the uppermost row of the upper right block B _DUR of the adjacent macroblock MB _D
Coefficient, and this horizontal one-dimensional DCT coefficient is a block B
It is the horizontal one-dimensional DCT coefficient of the first row of the _DUR . The horizontal one-dimensional DCT coefficient of the first row of the block B _DUR is stored in the memory 81.
In step S58, the selection unit 85 reads the horizontal 1-dimensional DCT coefficient of the first row of the block B _DUR from the memory 81 and selects it, and selects the lower adjacent 1-dimensional D
Output as CT coefficient.

After the processing of steps S57 and S58,
In either case, the process proceeds to step S59 and the sampling unit 83
And the selection unit 85 determines the DCT types of the macroblock of interest and the right adjacent macroblock.

In step S59, the macro block of interest is
The DCT type of the
Is also a frame DCT, if step S
Proceeding to 60, the selection unit 85 proceeds as described in FIG. 19A.
To the right adjacent macroblock MB _RBlock B on the upper left of_RUL
Vertical 1st dimension obtained by vertical 1-dimensional DCT conversion of 8 pixels in the leftmost column of
The original DCT coefficient is acquired as the right adjacent one-dimensional DCT coefficient.
It

That is, in the case where the _target block is the block B _NUR at the upper right of the _target macroblock, when the _target macroblock and the right adjacent macroblock are both frame structures, the right adjacent one-dimensional DCT coefficient is
As described in FIG. 19A, the right adjacent macroblock MB
It is a vertical one-dimensional DCT coefficient obtained by vertically one-dimensionally DCT-transforming the eight pixels in the leftmost column of the block B _{RUL on} the upper left of _R. The vertical one-dimensional DCT coefficient is the vertical 1 of the first column of the block B _RUL .
Dimensional DCT coefficient. The vertical one-dimensional DCT coefficient of the first column of the block B _RUL is stored in the memory 81, and the selection unit 85 selects the vertical one-dimensional DCT coefficient of the first column of the block B _RUL from the memory 81 in step S60. It is read and selected, and is output as a right adjacent one-dimensional DCT coefficient.

In step S59, the DCT type of the target macroblock is the frame DCT, and the DCT type of the right adjacent macroblock is the field DCT.
If it is determined that, the process proceeds to steps S61 to S63 in sequence, and the selecting unit 85, as described in FIG. 19B,
A total of 8 pixels, that is, the upper 4 pixels of the leftmost column in the upper left block B _RUL of the right adjacent macroblock MB _{R and} the upper 4 pixels of the leftmost column in the lower left block B _RDL are combined into a vertical one-dimensional DC.
The vertical one-dimensional DCT coefficient that has been T-transformed is used as the right adjacent one-dimensional DC
Obtained as a T coefficient.

That is, in the case where the block of interest is the block B _NUR at the upper right of the macro block of interest, if the macro block of interest has a frame structure and the right adjacent macroblock has a field structure, then right adjacent one-dimensional D
As described in FIG. 19B, the CT coefficient is the _sum of the upper 4 pixels of the leftmost column in the upper left block B _RUL of the right adjacent macroblock MB _{R and} the upper 4 pixels of the leftmost column in the lower left block B _RDL . A vertical one-dimensional DCT coefficient obtained by subjecting eight pixels to a vertical one-dimensional DCT is obtained.
The CT coefficient does not exist in the memory 81.

Therefore, the sampling unit 83 uses the vertical one-dimensional DCT coefficient in the first column of the block B _RUL and the block B _RUL .
Vertical one-dimensional inverse D of vertical one-dimensional DCT coefficient in the first column of _RDL
The vertical one-dimensional inverse DCT transform unit 82 is requested to perform CT transform.

In response to a request from the sampling unit 83, the vertical one-dimensional inverse DCT conversion unit 82, in step S61, the vertical one-dimensional DCT coefficient of the first column of the block B _RUL and the vertical one-dimensional DCT coefficient of the first column of the block B _RDL . The one-dimensional DCT coefficient is read from the memory 81 and subjected to vertical one-dimensional inverse DCT transform. As a result, the vertical one-dimensional inverse DCT conversion unit 82
Is the 8 pixels in the first column of block B _RUL and block B _RDL
The 8 pixels in the first column are obtained and supplied to the sampling unit 83, and the process proceeds to step S62.

In step S62, the sampling section 83
Is a block B supplied from the vertical one-dimensional inverse DCT transform unit 82 while sampling the upper four pixels of the eight pixels in the first column of the block B _RUL supplied from the vertical one-dimensional inverse DCT transform unit 82. sampling the four pixels on of the eight pixels in the first row of _RDL, with disposing the four pixels on sampled from the first column of block B _RUL in odd rows (top field), the block B _RDL 1
By arranging the upper 4 pixels sampled from the column in an even row (bottom field), 8 pixels arranged in the vertical direction having the same frame structure as the macroblock of interest are supplied to the vertical one-dimensional DCT conversion unit 84.

When the vertical one-dimensional DCT conversion unit 84 receives the 8 pixels sampled by the sampling unit 83,
In step S63, the eight pixels are converted into a vertical one-dimensional D
CT-transformed, so that the right adjacent macroblock MB _R
Upper 4 pixels of the leftmost column in the block B _{RUL on} the upper left of
Vertical one-dimensional DC obtained by performing vertical one-dimensional DCT conversion on a total of eight pixels of the upper four pixels in the leftmost column in the lower left block B _RDL
The T coefficient is obtained and supplied to the selection unit 85. The selection unit 85
The vertical one-dimensional DCT coefficient supplied from the vertical one-dimensional DCT conversion unit 84 is selected and output as a right adjacent one-dimensional DCT coefficient.

Also, in step S59, the DCT type of the macroblock of interest is the field DCT,
DCT type of right adjacent macroblock is frame DCT
If it is determined that, the process proceeds to steps S64 to S66 in sequence, and the selecting unit 85, as described with reference to FIG. 19C,
Four pixels in the leftmost column of the odd row in the block B _RUL in the upper left of the right adjacent macroblock MB _R and the block B in the lower left thereof
A vertical one-dimensional DCT coefficient obtained by performing a vertical one-dimensional DCT conversion on a total of eight pixels of four pixels in odd rows in _RDL is acquired as a right adjacent one-dimensional DCT coefficient.

That is, in the case where the _target block is the block B _NUR at the upper right of the _target macroblock, if the _target macroblock has a field structure and the right adjacent macroblock has a frame structure, the right adjacent one-dimensional D
As described with reference to FIG. 19C, the CT coefficient is the sum of 4 pixels in the odd-numbered row in the leftmost column in the upper left block B _RUL of the right adjacent macroblock MB _L and 4 pixels in the odd-numbered row in the lower left block B _RDL . A vertical one-dimensional DCT coefficient obtained by subjecting eight pixels to a vertical one-dimensional DCT transform is not present in the memory 81.

Therefore, the sampling unit 83 uses the vertical one-dimensional DCT coefficient of the first column of the block B _RUL and the block B _RUL .
Vertical one-dimensional inverse D of vertical one-dimensional DCT coefficient in the first column of _RDL
The vertical one-dimensional inverse DCT transform unit 82 is requested to perform CT transform.

In response to the request from the sampling unit 83, the vertical one-dimensional inverse DCT conversion unit 82, in step S64, the vertical one-dimensional DCT coefficient of the first column of the block B _RUL and the vertical one-dimensional vertical DCT coefficient of the block B _RDL . The one-dimensional DCT coefficient is read from the memory 81 and subjected to vertical one-dimensional inverse DCT transform. As a result, the vertical one-dimensional inverse DCT conversion unit 82
Is the 8 pixels in the first column of block B _RUL and block B _RDL
The 8 pixels in the first column are obtained and supplied to the sampling unit 83, and the process proceeds to step S65.

In step S65, the sampling section 83
_Samples the odd-numbered 4 pixels of the 8 pixels in the first column of the block B _RUL supplied from the vertical 1-dimensional inverse DCT conversion unit 82, and
8 in the first column of the block B _RDL supplied from the conversion unit 82
Four pixels in an odd row of the pixels are sampled, four pixels in an odd row sampled from the first column of the block B _RUL are arranged on the upper side, and four pixels in an odd row sampled from the first column of the block B _RDL are arranged. Are arranged on the lower side, so that eight pixels arranged vertically in the same field (top field) structure as the macroblock of interest are arranged,
It is supplied to the vertical one-dimensional DCT conversion unit 84.

When the vertical one-dimensional DCT transform unit 84 receives the 8 pixels sampled by the sampling unit 83,
In step S66, the eight pixels are converted into a vertical one-dimensional D
CT-transformed, so that the right adjacent macroblock MB _R
To obtain a vertical one-dimensional DCT coefficient obtained by vertically one-dimensionally DCT-converting four pixels in the leftmost column of the odd-numbered row in the upper left block B _RUL and four pixels of the odd-numbered row in the lower left block B _RDL , It is supplied to the selection unit 85. Selector 85
Is the vertical 1 supplied from the vertical one-dimensional DCT conversion unit 84.
A dimensional DCT coefficient is selected and output as a right adjacent one-dimensional DCT coefficient.

Further, in step S59, the DCT types of the macroblock of interest and the right adjacent macroblock are
When both are determined to be the field DCT, the process proceeds to step S67, and the selecting unit 85 selects the eight pixels in the leftmost column of the block B _{RUL in} the upper left of the right adjacent macroblock MB _R as described in FIG. 19D. The vertical one-dimensional DCT coefficient obtained by the vertical one-dimensional DCT conversion is acquired as the right adjacent one-dimensional DCT coefficient.

That is, when the _target block is the block B _NUR at the upper right of the _target macroblock and both the _target macroblock and the right adjacent macroblock have the field structure, the right adjacent one-dimensional DCT coefficient is As described in 19D, 8 pixels in the leftmost column of the upper left block B _RUL of the right adjacent macroblock MB _R are vertically set to 1
It is a vertical one-dimensional DCT coefficient _obtained by the _three- dimensional DCT transformation, and this vertical one-dimensional DCT coefficient is the vertical one-dimensional DCT coefficient of the first column of the block B _RUL . The vertical one-dimensional DCT coefficient in the first column of the block B _RUL is stored in the memory 81, and the selection unit 85 determines the memory 8 in step S67.
The vertical one-dimensional DCT coefficient in the first column of the block B _RUL is read and selected from 1, and is output as a right adjacent one-dimensional DCT coefficient.

After the processing of steps S60, S63, S66, and S67, the process proceeds to step S68.
Selecting unit 85, as described in FIG. 18, the block of interest B _NU eight pixels of the rightmost column of the left of the block B _NUL of _R vertical 1
The vertical one-dimensional DCT coefficient obtained by the three-dimensional DCT transformation
Obtained as the dimensional DCT coefficient.

In other words, the block of interest is the macro block of interest.
Block B in the upper right corner_NURIf, then the left adjacent primary
The original DCT coefficient is the block of interest as described in FIG.
B _NURBlock B to the left of_NUL8 pixels in the rightmost column of
This is the vertical one-dimensional DCT coefficient obtained by one-dimensional DCT conversion.
The vertical one-dimensional DCT coefficient of_NULIn the 8th row of
It is a vertical one-dimensional DCT coefficient. Block B_NUR8th row of
The vertical one-dimensional DCT coefficient of is stored in the memory 81.
In step S68, the selection unit 85 selects the memory 8
Block B from 1_NULVertical one-dimensional DCT section of column 8 of
Read and select the number, and select the left adjacent vertical one-dimensional DCT coefficient
Then, the upper right block processing is ended.

Next, the lower right block processing in step S5 of FIG. 25 will be described with reference to the flowchart of FIG.

In the lower right block processing, the upper adjacent one-dimensional DC is applied to the lower right block B _NDR of the macro block of interest.
T coefficient, lower adjacent one-dimensional DCT coefficient, left adjacent one-dimensional DCT
Coefficients, right-adjacent one-dimensional DCT coefficients are obtained.

That is, in the lower right block processing, first, in step S71, the selection unit 85 determines the DCT type of the macro block of interest.

If it is determined in step S71 that the DCT type of the macroblock of interest is the frame DCT, the process proceeds to step S72, and the selection unit 85 causes the selection of FIG.
As described in A, the 8 pixels in the bottom row of the block B _{NUR at} the upper right of the macro block MB _N of interest are set to the horizontal one-dimensional DCT.
The converted horizontal one-dimensional DCT coefficient is converted into the upper adjacent one-dimensional DCT.
Get as a coefficient.

That is, in the case where the block of interest is the block B _NDR at the lower right of the macro block of interest, when the macro block of interest has a frame structure, the upper adjacent 1
As described with reference to FIG. 20A, the dimensional DCT coefficient is 8 in the bottom row of the block B _{NUR at} the upper right of the macro block MB _N of interest.
It is a horizontal one-dimensional DCT coefficient obtained by _performing horizontal one-dimensional DCT conversion on a pixel, and this horizontal one-dimensional DCT coefficient is a block B _NUR.
Is the horizontal one-dimensional DCT coefficient of the eighth row of. Block B
The horizontal one-dimensional DCT coefficient of the eighth row of _NUR is stored in the memory 81, and the selection unit 85, in step S72, selects the horizontal 1-dimensional DCT coefficient of the eighth row of block B _NUR from the memory 81.
Dimensional DCT coefficient is read out and selected, and upper one-dimensional DC
Output as T coefficient.

If it is determined at step S71 that the DCT type of the macroblock of interest is the field DCT, then the processing advances to step S73, and the selecting section 85 is selected.
20B, the 8 pixels in the bottom row of the lower right block B _UDR of the upper adjacent macroblock MB _U are horizontally set to 1
The horizontal one-dimensional DCT coefficient obtained by the three-dimensional DCT transformation
Obtained as the dimensional DCT coefficient.

That is, in the case where the _target block is the block B _NDR at the lower right of the _target macroblock and the target macroblock has a field structure, the upper adjacent one-dimensional DCT coefficient is as described in FIG. 20B. Horizontal 1-dimensional DCT obtained by horizontally 1-dimensional DCT-transforming the 8 pixels in the bottom row of the lower right block B _UDR of the upper adjacent macroblock MB _U
Coefficient, and this horizontal one-dimensional DCT coefficient is a block B
It is the horizontal one-dimensional DCT coefficient of the eighth row of the _UDR . The horizontal one-dimensional DCT coefficient of the first row of the block B _UDR is stored in the memory 81
In step S73, the selecting unit 85 reads out and selects the horizontal 1-dimensional DCT coefficient of the eighth row of the block B _UDR from the memory 81, and selects the upper adjacent 1-dimensional D D.
Output as CT coefficient.

After the processing of steps S72 and S73,
In either case, the process proceeds to step S74, and the selecting unit 85 determines the DCT type of the macroblock of interest and the lower adjacent macroblock.

In step S74, the macro block of interest
The DCT type of the
Is also a frame DCT, if step S
Proceeding to 75, the selection unit 85 is as described in FIG. 21A.
To the lower adjacent macroblock MB _DBlock B on the upper right of_DUR
Water obtained by horizontal 1-dimensional DCT transformation of the top 8 pixels in
Flat 1-dimensional DCT coefficient as lower adjacent 1-dimensional DCT coefficient
get.

That is, in the case where the block of interest is the block B _NDR at the lower right of the macro block of interest, if both the macro block of interest and the lower adjacent macro block have a frame structure, the lower adjacent one-dimensional DCT coefficient is
As described in FIG. 21A, the lower adjacent macroblock MB
It is a horizontal one-dimensional DCT coefficient obtained by _performing horizontal one-dimensional DCT conversion on the uppermost eight pixels in the block B _{DUR at} the upper right of _D. This horizontal one-dimensional DCT coefficient is the horizontal one-dimensional DCT coefficient in the first row of the block B _DUR . is there. The horizontal one-dimensional DCT coefficient of the first row of the block B _DUR is stored in the memory 81, and the selection unit 85, in step S75, the memory 8
From 1, the horizontal one-dimensional DCT coefficient of the first row of the block B _DUR is read and selected, and is output as the lower adjacent one-dimensional DCT coefficient.

In step S74, the DCT type of the macroblock of interest is the frame DCT, and the DCT type of the lower adjacent macroblock is the field DCT.
21B, the selecting unit 85 performs horizontal one-dimensional DCT conversion on the uppermost 8 pixels of the upper right block B _DUR of the lower adjacent macroblock MB _D as described in FIG. 21B. The obtained horizontal one-dimensional DCT coefficient is acquired as the lower adjacent one-dimensional DCT coefficient.

That is, when the target block is the block B _NDR at the lower right of the _target macroblock and the target macroblock has a frame structure and the lower adjacent macroblock has a field structure, the lower adjacent one-dimensional D
As described with reference to FIG. 21B, the CT coefficient is a horizontal one-dimensional DCT coefficient obtained by horizontally one-dimensionally DCT-converting the eight pixels in the bottom row of the upper right block B _DUR of the lower adjacent macroblock MB _D. Coefficient is block B
It is the horizontal one-dimensional DCT coefficient of the first row of the _DUR . The horizontal one-dimensional DCT coefficient of the first row of the block B _DUR is stored in the memory 81.
In step S76, the selection unit 85 reads the horizontal one-dimensional DCT coefficient of the first row of the block B _DUR and selects it, and selects the lower adjacent one-dimensional D
Output as CT coefficient.

Further, in step S74, the DCT type of the macroblock of interest is the field DCT,
The DCT type of the lower adjacent macroblock is the frame DCT
21C, the selecting unit 85 horizontally one-dimensionally selects the 8 pixels in the second row of the block B _DUR at the upper right of the lower adjacent macroblock MB _D as described in FIG. 21C. The horizontal one-dimensional DCT coefficient obtained by DCT conversion is
It is acquired as the lower adjacent one-dimensional DCT coefficient.

That is, when the block of interest is the block B _NDR at the lower right of the macro block of interest, when the macro block of interest has a field structure and the macro block of the lower adjacent block has a frame structure, the lower adjacent one-dimensional D
As described with reference to FIG. 21C, the CT coefficient is a horizontal one-dimensional DCT coefficient obtained by horizontally one-dimensionally DCT-converting 8 pixels in the second row of the block B _DUR at the upper right of the lower adjacent macroblock MB _D , and the horizontal 1 The dimensional DCT coefficient is the horizontal one-dimensional DCT coefficient in the second row of the block B _DUR . Block B _DUR
The horizontal one-dimensional DCT coefficient of the second row of is stored in the memory 81, and the selecting unit 85, in step S77,
The horizontal one-dimensional DCT coefficient of the second row of the block B _DUR is read out from the memory 81, selected, and output as the lower adjacent one-dimensional DCT coefficient.

In step S74, the DCT types of the macroblock of interest and the lower adjacent macroblock are
If both are determined to be the field DCT, the process proceeds to step S78, and the selecting unit 85 selects the 8 pixels in the uppermost row of the lower right block B _DDR of the lower adjacent macroblock MB _D as described in FIG. 21D. The horizontal one-dimensional DCT coefficient obtained by the horizontal one-dimensional DCT conversion is acquired as the lower adjacent one-dimensional DCT coefficient.

That is, in the case where the block of interest is the block B _NDR at the lower right of the macroblock of interest, when the macroblock of interest and the lower adjacent macroblock are both field structures, the lower adjacent one-dimensional DCT coefficient is As described with reference to FIG. 21D, the 8 pixels in the uppermost row of the lower right block B _DDR of the lower adjacent macroblock MB _D are set to 1 horizontally.
The horizontal one-dimensional DCT coefficient obtained by the three-dimensional DCT conversion, and the horizontal one-dimensional DCT coefficient is the horizontal one-dimensional DCT coefficient of the first row of the block B _DDR . The horizontal one-dimensional DCT coefficient of the first row of the block B _DDR is stored in the memory 81, and the selection unit 85, in step S78, the memory 8
From 1, the horizontal 1-dimensional DCT coefficient of the first row of the block B _DDR is read and selected, and is output as the lower adjacent 1-dimensional DCT coefficient.

After the processes of steps S75 to S78, the process proceeds to step S79 where the sampling section 83 and the selection section 85 determine the DCT type of the macroblock of interest and the left adjacent macroblock.

In step S79, the macro block of interest
The DCT type of the
Is also a frame DCT, if step S
Proceeding to 80, the selection unit 85 proceeds as described in FIG. 23A.
To the right adjacent macroblock MB _RBlock B at the bottom left of_RDL
Vertical 1st dimension obtained by vertical 1-dimensional DCT conversion of 8 pixels in the leftmost column of
The original DCT coefficient is acquired as the right adjacent one-dimensional DCT coefficient.
It

That is, in the case where the _target block is the block B _NDR at the lower right of the _target macroblock, when the _target macroblock and the right adjacent macroblock are both frame structures, the right adjacent one-dimensional DCT coefficient is
As described in FIG. 23A, the right adjacent macroblock MB
It is a vertical one-dimensional DCT coefficient obtained by vertically one-dimensionally DCT-transforming the eight pixels in the leftmost column of the block B _{RDL in} the lower left of _R. This vertical one-dimensional DCT coefficient is the vertical 1 of the first column of the block B _RDL .
Dimensional DCT coefficient. The vertical one-dimensional DCT coefficient of the first column of the block B _RDL is stored in the memory 81, and the selection unit 85 selects the vertical one-dimensional DCT coefficient of the first column of the block B _RDL from the memory 81 in step S80. It is read and selected, and is output as a right adjacent one-dimensional DCT coefficient.

In step S79, the DCT type of the macroblock of interest is the frame DCT, and the DCT type of the right adjacent macroblock is the field DCT.
23B, the selection unit 85 proceeds to steps S81 to S83 in sequence, as described in FIG. 23B.
A vertical 1-dimensional DC is obtained by _combining the lower 4 pixels of the leftmost column in the upper left block B _RUL of the right adjacent macroblock MB _R and the lower 4 pixels of the leftmost column in the lower left block B _RDL of a total of 8 pixels.
The vertical one-dimensional DCT coefficient that has been T-transformed is used as the right adjacent one-dimensional DC
Obtained as a T coefficient.

That is, when the target block is the block B _NDR at the lower right of the _target macroblock, and the target macroblock has a frame structure and the right adjacent macroblock has a field structure, the right adjacent one-dimensional D
As described in FIG. 23B, the CT coefficient is the _sum of the lower 4 pixels of the leftmost column in the upper left block B _RUL of the right adjacent macroblock MB _R and the lower 4 pixels of the leftmost column in the lower left block B _RDL . A vertical one-dimensional DCT coefficient obtained by subjecting eight pixels to a vertical one-dimensional DCT is obtained.
The CT coefficient does not exist in the memory 81.

Therefore, the sampling unit 83 uses the vertical one-dimensional DCT coefficient of the first column of the block B _RUL and the block B _RUL .
Vertical one-dimensional inverse D of vertical one-dimensional DCT coefficient in the first column of _RDL
The vertical one-dimensional inverse DCT transform unit 82 is requested to perform CT transform.

In response to a request from the sampling unit 83, the vertical one-dimensional inverse DCT transform unit 82, in step S81, the vertical one-dimensional DCT coefficient of the first column of the block B _RUL and the vertical one-dimensional DCT coefficient of the block B _RDL . The one-dimensional DCT coefficient is read from the memory 81 and subjected to vertical one-dimensional inverse DCT transform. As a result, the vertical one-dimensional inverse DCT conversion unit 82
Is the 8 pixels in the first column of block B _RUL and block B _RDL
8 pixels in the first column of are obtained and supplied to the sampling unit 83, and the process proceeds to step S82.

At step S82, the sampling section 83
Is a block B supplied from the vertical one-dimensional inverse DCT transform unit 82 while sampling the lower four pixels of the eight pixels in the first column of the block B _RUL . The lower 4 pixels of the 8 pixels in the first column of _RDL are sampled, the upper 4 pixels sampled from the first column of block B _RUL are arranged in odd rows (top fields), and the first 4 pixels of block B _RDL
By arranging the upper 4 pixels sampled from the column in an even row (bottom field), 8 pixels arranged in the vertical direction having the same frame structure as the macroblock of interest are supplied to the vertical one-dimensional DCT conversion unit 84.

When the vertical one-dimensional DCT transform unit 84 receives the 8 pixels sampled by the sampling unit 83,
In step S83, the eight pixels are converted into a vertical one-dimensional D
CT transform, which results in right adjacent macroblock MB _L
4 pixels on the lower left column in the block B _{RUL on} the upper left of
Vertical one-dimensional DC obtained by vertical one-dimensional DCT conversion of a total of eight pixels of the lower four pixels in the leftmost column in the block B _RDL in the lower left
The T coefficient is obtained and supplied to the selection unit 85. The selection unit 85
The vertical one-dimensional DCT coefficient supplied from the vertical one-dimensional DCT conversion unit 84 is selected and output as a right adjacent one-dimensional DCT coefficient.

Further, in step S79, the DCT type of the macroblock of interest is the field DCT,
DCT type of right adjacent macroblock is frame DCT
If it is determined that it is, the process proceeds to steps S84 to S86, and the selecting unit 85, as described in FIG. 23C,
4 pixels in the even-numbered row in the leftmost column in the upper left block B _RUL of the right adjacent macroblock MB _R and the lower left block B
A vertical one-dimensional DCT coefficient obtained by performing vertical one-dimensional DCT conversion on a total of eight pixels of four pixels in an even row in _RDL is acquired as a right adjacent one-dimensional DCT coefficient.

That is, when the target block is the block B _NDR at the lower right of the _target macroblock, and the target macroblock has a field structure and the right adjacent macroblock has a frame structure, the right adjacent one-dimensional D
As described with reference to FIG. 23C, the CT coefficient is the sum of 4 pixels in the even-numbered row in the leftmost column in the upper left block B _RUL of the right adjacent macroblock MB _L and 4 pixels in the even-numbered row in the lower left block B _RDL . A vertical one-dimensional DCT coefficient obtained by subjecting eight pixels to a vertical one-dimensional DCT transform is not present in the memory 81.

Therefore, the sampling unit 83 uses the vertical one-dimensional DCT coefficient in the first column of the block B _RUL and the block B _RUL .
Vertical one-dimensional inverse D of vertical one-dimensional DCT coefficient in the first column of _RDL
The vertical one-dimensional inverse DCT transform unit 82 is requested to perform CT transform.

In response to the request from the sampling unit 83, the vertical one-dimensional inverse DCT conversion unit 82, in step S84, the vertical one-dimensional DCT coefficient of the first column of the block B _RUL and the vertical one-dimensional vertical DCT coefficient of the block B _RDL . The one-dimensional DCT coefficient is read from the memory 81 and subjected to vertical one-dimensional inverse DCT transform. As a result, the vertical one-dimensional inverse DCT conversion unit 82
Is the 8 pixels in the first column of block B _RUL and block B _RDL
The 8 pixels in the first column are obtained and supplied to the sampling unit 83, and the process proceeds to step S85.

In step S85, the sampling section 83
_Of the block B _RUL supplied from the vertical one-dimensional inverse DCT conversion unit 82 samples four pixels in an even row of the eight pixels in the first column, and also _performs vertical one-dimensional inverse DCT conversion.
8 in the first column of the block B _RDL supplied from the conversion unit 82
Even pixels of 4 pixels in even rows of the pixels are sampled, 4 pixels of even rows sampled from the first column of the block B _RUL are arranged on the upper side, and 4 pixels of even rows sampled from the first column of the block B _RDL are arranged. Is arranged on the lower side, the eight pixels arranged vertically in the same field (bottom field) structure as the macroblock of interest are arranged.
It is supplied to the vertical one-dimensional DCT conversion unit 84.

When the vertical one-dimensional DCT conversion unit 84 receives the 8 pixels sampled by the sampling unit 83,
In step S86, the eight pixels are converted into a vertical one-dimensional D
CT-transformed, so that the right adjacent macroblock MB _R
To obtain a vertical one-dimensional DCT coefficient obtained by vertically one-dimensionally DCT-transforming four pixels in the leftmost column of the even-numbered row in the upper left block B _RUL and four pixels of the even-numbered row in the lower left block B _RDL , It is supplied to the selection unit 85. Selector 85
Is the vertical 1 supplied from the vertical one-dimensional DCT conversion unit 84.
A dimensional DCT coefficient is selected and output as a right adjacent one-dimensional DCT coefficient.

In step S79, the DCT types of the macroblock of interest and the right adjacent macroblock are
If both are determined to be the field DCT, the process proceeds to step S87, and the selecting unit 85 selects the 8 pixels in the leftmost column of the lower left block B _RDL of the right adjacent macroblock MB _R as described in FIG. 23D. The vertical one-dimensional DCT coefficient obtained by the vertical one-dimensional DCT conversion is acquired as the right adjacent one-dimensional DCT coefficient.

That is, in the case where the block of interest is the block B _NDR at the lower right of the macro block of interest, when the macro block of interest and the right adjacent macro block are both field structures, the right adjacent one-dimensional DCT coefficient is As described with reference to FIG. 23D, the 8 pixels in the leftmost column of the lower left block B _RDL of the right adjacent macroblock MB _R are vertically set to 1 pixel.
It is a vertical one-dimensional DCT coefficient obtained by the three-dimensional DCT transformation, and this vertical one-dimensional DCT coefficient is the vertical one-dimensional DCT coefficient in the first column of the block B _RDL . The vertical one-dimensional DCT coefficient of the first column of the block B _RDL is stored in the memory 81, and the selection unit 85, in step S87, the memory 8
The vertical one-dimensional DCT coefficient in the first column of the block B _RDL is read and selected from 1, and is output as the right adjacent one-dimensional DCT coefficient.

After the processing of steps S80, S83, S86, and S87, the process proceeds to step S88.
Selecting unit 85, as described in FIG. 22, the block of interest B _ND vertical 1 eight pixels of the rightmost column of the left of the block B _NDL of _R
The vertical one-dimensional DCT coefficient obtained by the three-dimensional DCT transformation
Obtained as the dimensional DCT coefficient.

That is, the block of interest is the macro block of interest.
Block B on the lower right_NDRIf, then the left adjacent primary
The original DCT coefficient is the block of interest as described in FIG.
B _NDRBlock B to the left of_NDL8 pixels in the rightmost column of
This is the vertical one-dimensional DCT coefficient obtained by one-dimensional DCT conversion.
The vertical one-dimensional DCT coefficient of_NDLIn the 8th row of
It is a vertical one-dimensional DCT coefficient. Block B_NDL8th row of
The vertical one-dimensional DCT coefficient of is stored in the memory 81.
In step S88, the selection unit 85 selects the memory 8
Block B from 1_NDLVertical one-dimensional DCT section of column 8 of
Read and select the number, and select the left adjacent vertical one-dimensional DCT coefficient
Then, the lower right block processing is ended.

Next, the processing of the AC power calculator 33 of FIG. 2 will be described.

As described above, the AC power calculation unit 33 obtains the power (AC power) of the AC component of the one-dimensional DCT coefficient supplied from the one-dimensional inverse DCT coefficient conversion unit 31 and selects the adjacent one-dimensional DCT coefficient. The AC power of the adjacent one-dimensional DCT coefficient supplied from the / converter 32 is obtained.

That is, as shown in FIG. 30, the AC power calculation unit 33 obtains the AC power in the horizontal direction from the horizontal one-dimensional DCT coefficient of the row at the position of the pixel of interest in the block of interest and determines the position of the pixel of interest. Vertical one-dimensional DC of row
The AC power in the vertical direction is obtained from the T coefficient.

Here, if the seven AC components of the one-dimensional DCT coefficient are represented by AC _n (n = 1, 2, ...,
7), AC power P _AC is calculated, for example, by the following equation.

P _AC = ΣAC _n ² (12)

However, in the equation (12), Σ is a variable n
Represents the summation from 1 to 7.

Also, the AC power calculation unit 33 determines that the upper adjacent one-dimensional DCT coefficient, the lower adjacent one-dimensional DCT coefficient, the left adjacent one-dimensional DCT coefficient, the right adjacent one-dimensional D
For each CT coefficient, according to the equation (12),
Find the AC power. Both the upper adjacent one-dimensional DCT coefficient and the lower adjacent one-dimensional DCT coefficient are horizontal one-dimensional DCT coefficients.
The coefficients, and therefore the AC power derived from them, is the horizontal AC power. Further, both the left adjacent one-dimensional DCT coefficient and the right adjacent one-dimensional DCT coefficient are vertical one-dimensional DCT coefficients, and therefore the AC power obtained from them is the vertical AC power.

The AC power calculation unit 33 further follows the formula (12) for the one-dimensional DCT coefficient of the target block, which is adjacent to the boundary (hereinafter, also referred to as the boundary one-dimensional DCT coefficient), according to the equation (12). Find the AC power.

That is, the AC power calculator 33 determines the horizontal 1-dimensional DCT of the first row adjacent to the upper boundary of the target block.
The AC power in the horizontal direction is obtained from the coefficient (hereinafter, appropriately referred to as the upper boundary one-dimensional DCT coefficient). Further, the AC power calculation unit 33 causes the horizontal one-dimensional DCT coefficient of the eighth row adjacent to the lower boundary of the target block (hereinafter referred to as the lower boundary 1 as appropriate).
The AC power in the horizontal direction is calculated from the dimension DCT coefficient). In addition, the AC power calculation unit 33 calculates from the vertical one-dimensional DCT coefficient of the first column (hereinafter, appropriately referred to as left boundary one-dimensional DCT coefficient) adjacent to the left boundary of the target block,
While obtaining the AC power in the vertical direction, from the vertical one-dimensional DCT coefficient of the eighth column adjacent to the right boundary of the target block (hereinafter, appropriately referred to as right boundary one-dimensional DCT coefficient),
Determine the AC power in the vertical direction.

Next, FIG. 31 shows an example of the configuration of the AC power calculator 33 of FIG. 2 for obtaining the AC power as described above.

The horizontal one-dimensional DCT coefficient extraction unit 91 and the vertical one-dimensional DCT coefficient extraction unit 92 include a one-dimensional inverse DCT conversion unit 31 (FIG. 2) and a one-dimensional DCT coefficient selection / conversion unit 32. The DCT coefficient is supplied.

The horizontal one-dimensional DCT coefficient extraction unit 91 extracts the AC power calculation target from the one-dimensional DCT coefficients supplied thereto and supplies it to the horizontal AC power calculation unit 93. That is, the horizontal one-dimensional DCT coefficient extraction unit 91
From the one-dimensional DCT coefficient supplied thereto, the horizontal one-dimensional DCT coefficient of the row at the position of the pixel of interest in the block of interest,
The upper adjacent one-dimensional DCT coefficient and the lower adjacent one-dimensional DCT coefficient of the target block, and the upper boundary one-dimensional DCT coefficient and lower boundary one-dimensional DCT coefficient of the target block are extracted and supplied to the horizontal AC power calculation unit 93.

The vertical one-dimensional DCT coefficient extracting section 92 extracts the AC power calculation object from the one-dimensional DCT coefficients supplied thereto, and the vertical AC power calculating section 94.
Supply to. That is, the vertical one-dimensional DCT coefficient extraction unit 92
Is the vertical 1D DCT coefficient of the column of the position of the pixel of interest in the block of interest, the left adjacent 1D DCT coefficient and the right adjacent 1D DCT coefficient of the block of interest, and the block of interest from the 1D DCT coefficient supplied thereto. The left boundary one-dimensional DCT coefficient and the right boundary one-dimensional DCT coefficient of are extracted and supplied to the vertical AC power calculation unit 94.

The horizontal AC power calculation unit 93 receives the horizontal one-dimensional DCT supplied from the horizontal one-dimensional DCT coefficient extraction unit 91.
The AC power in the horizontal direction is calculated and output from the coefficient according to the equation (12). That is, the horizontal AC power calculator 93 determines the horizontal one-dimensional DCT coefficient of the row at the position of the pixel of interest in the block of interest, and the upper adjacent 1 of the block of interest.
-Dimensional DCT coefficient and lower adjacent 1-dimensional DCT coefficient, and target block upper boundary 1-dimensional DCT coefficient and lower boundary 1
The horizontal AC power is calculated from each of the dimensional DCT coefficients.

The vertical AC power calculation unit 94 receives the vertical one-dimensional DCT supplied from the vertical one-dimensional DCT coefficient extraction unit 92.
From the coefficient, the AC power in the vertical direction is calculated and output according to the equation (12). That is, the vertical AC power calculation unit 94 determines the vertical one-dimensional DCT coefficient of the column at the position of the pixel of interest in the block of interest, the left adjacent 1 of the block of interest.
-Dimensional DCT coefficient and right adjacent 1-dimensional DCT coefficient, and left boundary 1-dimensional DCT coefficient and right boundary 1 of the target block
The AC power in the vertical direction is calculated from each of the dimensional DCT coefficients.

As described above, the AC power obtained from the one-dimensional DCT coefficient can be regarded as the power of the AC component of 8 pixels corresponding to the one-dimensional DCT coefficient, and thus represents the image activity. .

Next, the processing of the AC inner product calculator 34 of FIG. 2 will be described.

The AC inner product calculating section 34, as described above,
The one-dimensional DCT coefficient (boundary one-dimensional DCT coefficient) of the boundary portion of the target block supplied from the one-dimensional inverse DCT coefficient conversion unit 31
The AC component of the coefficient) and the AC component of the adjacent one-dimensional DCT coefficient supplied from the adjacent one-dimensional DCT coefficient selecting / converting unit 32 are regarded as vector components,
The inner product (AC inner product) of the two vectors is obtained.

That is, as shown in FIG. 32, the AC inner product calculating section 34 determines the upper boundary one-dimensional DCT for the target block.
The AC component of the coefficient and the AC component of the upper adjacent one-dimensional DCT coefficient are regarded as vector components,
AC inner product of the two vectors (hereinafter called upper inner product)
Is calculated according to the following equation.

I = Σ (AC _n × AC _n ') (13)

However, in the equation (13), I represents an AC inner product. Further, AC _n represents the n-th AC component of the upper boundary one-dimensional DCT coefficient for the block of interest, and A n
C _n 'is the upper adjacent one-dimensional DCT for the block of interest
It represents the nth AC component of the coefficient. Further, Σ represents the summation in which n is changed from 1 to 7.

The AC inner product calculating unit 34 determines the lower boundary one-dimensional DCT coefficient and the lower adjacent one-dimensional DCT coefficient, the left boundary one-dimensional DCT coefficient and the left adjacent one-dimensional DCT coefficient, or the right boundary one-dimensional DCT coefficient for the target block. Right adjacent one-dimensional DC
For each T coefficient, according to equation (13), A
Calculate the C inner product.

Here, hereinafter, the AC inner product obtained from the lower boundary one-dimensional DCT coefficient and the lower adjacent one-dimensional DCT coefficient for the target block is appropriately calculated as the lower inner product, the left boundary one-dimensional DCT coefficient, and the left adjacent one-dimensional DCT coefficient. The AC inner product obtained from the DCT coefficient is called the left inner product, and the AC inner product obtained from the right boundary one-dimensional DCT coefficient and the right adjacent one-dimensional DCT coefficient is called the right inner product.

The AC inner product is adjacent to a vector having the AC component of the boundary 1-dimensional DCT coefficient as a component, when the AC component of the boundary 1-dimensional DCT coefficient and the AC component of the adjacent 1-dimensional DCT coefficient are similar to each other. One-dimensional DC
When the angle formed by the vector having the AC component of the T coefficient as a component is less than 90 degrees (or more), it has a positive value (value of 0 or more). Therefore, the positive value of the AC inner product means that the boundary one-dimensional DCT that sandwiches the boundary of the block of interest.
It shows that the waveform pattern of 8 pixels corresponding to the coefficient is similar to the waveform pattern of 8 pixels corresponding to the adjacent one-dimensional DCT coefficient. For example, at the boundary between the block of interest and the block adjacent to it, the waveform crosses the boundary. Indicates that there are continuous edges.

Next, FIG. 33 shows the target block.
3 shows a configuration example of the AC inner product calculating unit 34 of FIG. 2 for calculating the AC inner product (upper inner product, lower inner product, left inner product, right inner product) as described above.

One-dimensional inverse DCT transform unit 31 and adjacent one-dimensional D
The one-dimensional DCT coefficients output by the CT coefficient selecting / converting unit 32 are the one-dimensional DCT coefficient extracting unit for upper inner product 101, the one-dimensional DCT coefficient extracting unit for lower inner product 102, and the one-dimensional DCT for left inner product.
It is adapted to be supplied to the coefficient extracting unit 103 and the one-dimensional DCT coefficient extracting unit for right inner product 104.

One-dimensional DCT coefficient extraction unit 101 for upper inner product
Extracts the upper boundary one-dimensional DCT coefficient and the upper adjacent one-dimensional DCT coefficient used to calculate the upper inner product for the block of interest from the one-dimensional DCT coefficients supplied thereto, and supplies the upper boundary one-dimensional DCT coefficient to the upper inner product calculating unit 105. .

The upper inner product computing unit 105 determines the one-dimensional D for the upper inner product.
Upper boundary one-dimensional DC supplied from the CT coefficient extraction unit 101
From the T coefficient and the upper adjacent one-dimensional DCT coefficient, the upper inner product is calculated and output according to Expression (13).

One-dimensional DCT coefficient extracting unit for lower inner product 102
Extracts the lower boundary one-dimensional DCT coefficient and the lower adjacent one-dimensional DCT coefficient used for calculating the lower inner product of the block of interest from the one-dimensional DCT coefficients supplied thereto, and supplies the lower boundary one-dimensional DCT coefficient to the lower inner product calculation unit 106. .

The lower inner product calculating unit 106 determines the one-dimensional D for lower inner product.
Lower boundary one-dimensional DC supplied from the CT coefficient extraction unit 102
The lower inner product is calculated and output from the T coefficient and the lower adjacent one-dimensional DCT coefficient according to the equation (13).

Left one-dimensional DCT coefficient extracting unit 103 for inner product
Extracts the left boundary one-dimensional DCT coefficient and the left adjacent one-dimensional DCT coefficient used to calculate the left inner product for the block of interest from the one-dimensional DCT coefficients supplied thereto, and supplies the left boundary one-dimensional DCT coefficient to the left inner product computing unit 107. .

The left inner product computing unit 107 determines the one-dimensional D for left inner product.
Left boundary one-dimensional DC supplied from the CT coefficient extraction unit 103
A left inner product is calculated and output from the T coefficient and the left adjacent one-dimensional DCT coefficient according to the equation (13).

Right-dimensional inner product one-dimensional DCT coefficient extraction unit 104
Extracts the right boundary one-dimensional DCT coefficient and the right adjacent one-dimensional DCT coefficient used for calculating the right inner product of the block of interest from the one-dimensional DCT coefficients supplied thereto, and supplies the right boundary one-dimensional DCT coefficient to the right inner product computing unit 108. .

The right inner product computing unit 108 calculates the right inner product one-dimensional D
Right boundary one-dimensional DC supplied from the CT coefficient extraction unit 104
The right inner product is calculated and output from the T coefficient and the right adjacent one-dimensional DCT coefficient according to the equation (13).

Next, FIG. 34 shows a configuration example of the class code generation unit 36 of FIG.

The class code generator 36 is adapted to classify the pixels constituting the block for the block of the luminance signal.

That is, the comparison units 111 and 112 are provided with A
The AC power output by the C power calculation unit 33 (FIG. 2) is supplied. In the flatness condition determination unit 113, the AC power output from the AC power calculation unit 33 (FIG. 2), the one-dimensional inverse DCT conversion unit 31, and the adjacent one-dimensional DCT coefficient selection /
The one-dimensional DCT coefficient output by the conversion unit 32 is supplied.
The continuity determination unit 114 includes an AC inner product calculation unit 34 (FIG. 2).
The AC inner product output by is supplied. The boundary edge condition determining unit 115 outputs the one-dimensional D output from the one-dimensional inverse DCT transforming unit 31 and the adjacent one-dimensional DCT coefficient selecting / transforming unit 32.
CT coefficients are provided.

The comparing section 111 includes an AC power calculating section 33.
Of the AC power output by (FIG. 2), the horizontal AC obtained from the horizontal one-dimensional DCT coefficient of the row of the pixel of interest.
The power is compared with a predetermined threshold value A, and the comparison result is supplied to the class code creating unit 116.

The comparing unit 112 is the AC power calculating unit 33.
Of the AC power output by (FIG. 2), the AC in the vertical direction obtained from the vertical one-dimensional DCT coefficient of the column of the pixel of interest.
The power is compared with a predetermined threshold value A, and the comparison result is supplied to the class code creating unit 116.

For the target block, the flatness condition judging unit 113 determines the AC power obtained from the boundary one-dimensional DCT coefficient, the AC power obtained from the adjacent one-dimensional DCT coefficient, and the DC component of the boundary one-dimensional DCT coefficient. , Based on the DC component of the adjacent one-dimensional DCT coefficient, the flatness of the image at each boundary portion of the upper, lower, left, and right boundaries of the block of interest is determined, and the determination result is supplied to the class code generation unit 116.

The continuity condition determining unit 114 determines the continuity of the image at each boundary part for each of the upper, lower, left, and right boundaries of the target block based on the upper inner product, the lower inner product, the left inner product, and the right inner product obtained for the target block. It is determined and the determination result is supplied to the class code generation unit 116.

The boundary edge condition determining unit 115 determines the DC component of the boundary one-dimensional DCT coefficient for the target block,
Based on the DC component of the adjacent one-dimensional DCT coefficient, it is determined whether or not there is an edge along each of the upper, lower, left, and right boundaries of the block of interest, and the determination result is supplied to the class code generation unit 116. .

The class code creating section 116 includes the comparing section 11
1 and 112, the flatness condition determination unit 113, the continuity condition determination unit 114, and the boundary edge condition determination unit 115, the target pixel is classified into classes, and a class code (luminance class code) representing the class is assigned. Create (generate)
And output.

Here, FIG. 35 shows the class code creating section 1
16 shows the format of the class code output by 16.

In the embodiment of FIG. 35, the class code is set to 10 bits, and from the beginning, a 2-bit AC power class code, a 4-bit block flatness class code, and a 4-bit inter-block continuity class. The cords are sequentially arranged and configured.

The 2-bit AC power class code indicates that the pixel of interest has AC power in the vertical direction and AC power in the horizontal direction.
The first bit is determined by the vertical AC power obtained from the vertical one-dimensional DCT coefficient of the column of the pixel of interest, and the second bit is the horizontal 1 of the row of the pixel of interest. It is determined by the horizontal AC power obtained from the dimensional DCT coefficient. Therefore, the AC power class code is determined for each pixel.

The 4-bit block flatness class code is used to determine the flatness of the boundary (upper, lower, left, right) of the block containing the target pixel (the flatness of the image between the target block and its adjacent blocks). )
The first to fourth bits are determined by the flatness of the boundary of each of the upper, lower, left, and right blocks of interest. Therefore, the block flatness class code is determined for each block.

The 4-bit inter-block continuity class code is used to determine the continuity of the boundaries (upper, lower, left, and right) of the block including the target pixel (the target block and the image connection between the target block and its adjacent blocks). The first to fourth bits are determined by the continuity of the upper, lower, left, and right boundaries of the target block. Therefore, the block continuity class code is also determined for each block similarly to the block flatness class code.

From the above, the AC power class code is basically different for each pixel, but the block flatness class code and the inter-block continuity class code are the same for the pixels of the same block.

Next, the processing (class classification processing) of the class code generation unit 36 of FIG. 34 will be described with reference to the flowchart of FIG.

In the class code generation unit 36, first, in step S91, the comparison unit 111 causes the horizontal one-dimensional DCT of the row of the pixel of interest to be included in the AC power output by the AC power calculation unit 33 (FIG. 2). The horizontal AC power obtained from the coefficient is compared with a predetermined threshold value A, and the comparison result is supplied to the class code creating unit 116.
Further, in step S91, the comparison unit 112 calculates the vertical AC power obtained from the vertical one-dimensional DCT coefficient of the column of the pixel of interest among the AC power output by the AC power calculation unit 33 (FIG. 2). The result is compared with a predetermined threshold value A, and the comparison result is supplied to the class code creating unit 116.

Then, the class code creating section 116, based on the outputs of the comparing sections 111 and 112, outputs the AC of the pixel of interest.
Determine the power class code.

That is, when the vertical AC power obtained from the vertical one-dimensional DCT coefficient of the column of the pixel of interest is larger than the predetermined threshold A, the class code creating unit 116 outputs the 2-bit AC power class code. The first bit among them is set to 1, for example, and when the AC power in the vertical direction is not larger than the predetermined threshold value A, the first bit of the AC power class code is set to 0, for example. Further, the class code creating unit 116 determines that the horizontal one-dimensional D of the row of the pixel of interest is
When the horizontal AC power obtained from the CT coefficient is larger than the predetermined threshold value A, the second bit of the 2-bit AC power class code is set to 1, for example, and the horizontal AC power is When it is not larger than the predetermined threshold A, the second bit of the AC power class code is set to 0, for example.

After that, proceeding to step S92, the boundary edge condition determining unit 115 determines the boundary of the target block based on the DC component of the boundary one-dimensional DCT coefficient and the DC component of the adjacent one-dimensional DCT coefficient for the target block. ,
It is determined whether a boundary edge condition that an edge exists along the boundary is satisfied.

That is, the boundary edge condition judging unit 115
For example, the DC component DC of the boundary first-order DCT coefficient and the DC component DC ′ of the adjacent first-order DCT coefficient for the block of interest.
And a difference absolute value | DC-DC '| is larger than (or more than) a predetermined threshold value E is determined as a boundary edge condition, and it is determined whether or not such a boundary edge condition is satisfied. To do.

If it is determined in step S92 that the boundary edge condition is satisfied, that is, the DC component DC of the boundary first-order DCT coefficient for the target block and the adjacent 1
The difference from the DC component DC 'of the next DCT coefficient is very large,
Therefore, when it is considered that there is an edge at the boundary portion of the block of interest and the image patterns of the block of interest and the blocks adjacent thereto are not connected, the boundary edge condition determination unit 115 notifies the effect to that effect. Output to step S97. In step S97,
The class code creation unit 116 sets both the block flatness class code and the inter-block continuity class code to 0, for example, and proceeds to step S98.

If it is determined in step S92 that the boundary edge condition is not satisfied, step S92.
Proceeding to 93, the flatness condition determining unit 113 determines, for the target block, the AC power obtained from the boundary one-dimensional DCT coefficient, the AC power obtained from the adjacent one-dimensional DCT coefficient, and further the direct current of the boundary one-dimensional DCT coefficient. Based on the component and the DC component of the adjacent one-dimensional DCT coefficient, it is determined whether the flatness condition that the boundary portion of the target block is flat is satisfied.

That is, the flatness condition determining unit 113 determines, for example, whether the flatness condition is satisfied by using the condition represented by the following equation as the flatness condition.

(P _AC ≤ B) ∩ (P _AC '≤ B) ∩ (| DC-DC' | ≤ D) (14)

P _AC '≦ C (15)

However, in the equations (14) and (15), B, C, and D are predetermined threshold values, and the threshold value C is sufficiently smaller than the threshold value B. Further, in equations (14) and (15), P _AC represents the AC power obtained from the boundary one-dimensional DCT coefficient for the target block, and P _AC ′ is the adjacent one-dimensional DC for the target block.
It represents the AC power obtained from the T coefficient. Further, in Expression (14), DC represents the DC component of the boundary one-dimensional DCT coefficient for the target block, and DC ′ represents the DC component of the adjacent one-dimensional DCT coefficient for the target block. Further, ∩ represents a logical product.

Equation (14) shows that AC powers P _AC and P _AC ′ obtained from the boundary one-dimensional DCT coefficient and the adjacent one-dimensional DCT coefficient for the target block are both below (or below) the threshold value B, Also, the absolute difference | DC−DC ′ | between the DC components DC and DC ′ is the threshold D
True if (or less than): Also, the formula (1
5) is true when the AC power P _AC ′ obtained from the adjacent one-dimensional DCT coefficient for the block of interest is less than (or less than) the threshold C.

Here, as described above, the threshold value C is sufficiently smaller than the threshold value B, for example, a value close to 0. Therefore, the expression (15) is obtained from the adjacent one-dimensional DCT coefficient for the target block. True if the applied AC power P _AC 'is close to zero.

Here, the flatness condition is, for example,
If either one of equations (14) and (15) is true, it is satisfied.

If it is determined in step S93 that the flatness condition is satisfied, the flatness condition determining unit 113 is used.
Supplies that effect to the class code creating unit 116,
It proceeds to step S94.

At step S94, the class code creating section 116 sets both the block flatness class code and the inter-block continuity class code to, for example, 1 and proceeds to step S98.

If it is determined in step S93 that the flatness condition is not satisfied, the process proceeds to step S95, in which the continuity condition determination unit 114 determines the boundary portion of the target block based on the AC inner product obtained for the target block. It is determined whether the continuity condition that there is continuity is satisfied.

That is, the continuity condition judging unit 114 determines, for example, that the AC inner product I of the target block is a positive value (or 0 or more) as the continuity condition.
Determine whether such continuity conditions are met.

If it is determined in step S95 that the continuity condition is satisfied, the continuity condition determination unit 114
Supplies that effect to the class code creating unit 116,
It proceeds to step S96.

In step S96, the class code creating section 116 sets the block flatness class code to 0, for example.
And the inter-block continuity class code is set to 1, for example, and the process proceeds to step S98.

On the other hand, if it is determined in step S96 that the continuity condition is not satisfied, the continuity condition determination unit 114 supplies a message to that effect to the class code creation unit 116, and the flow advances to step S97. In step S97, as described above, the class code creating unit 116 sets both the block flatness class code and the inter-block continuity class code to 0, and proceeds to step S98.

The processing of steps S92 to S97 is performed for each of the upper, lower, left and right boundaries of the target block.
As a result, the block flatness class code and the inter-block continuity class code are obtained for each of the upper, lower, left, and right boundaries of the block of interest.

At step S98, the class code generator 116 uses the AC power class code, the block flatness class code, and the inter-block continuity class code obtained by the processing of steps S91 to S97,
The 10-bit class code shown in FIG. 35 is created, and the process ends.

The processing shown in the flowchart of FIG. 36 is performed every time a new pixel is set as the target pixel.
However, as described above, since the block flat raw class code and the inter-block continuity class code are the same for the pixels of the same block, for the pixels configuring the same block, only the step for the first pixel S9
1 to 98 are performed, and for other pixels, only the processes of steps S91 and S98 are performed, and the block flat raw class code and the interblock continuity class code are obtained for the first pixel. Can be diverted.

In this embodiment, the class code is 10 bits as shown in FIG.
According to such a 10-bit class code, 102
It can represent 4 (= 2 ¹⁰ ) different classes.

However, in the class classification processing shown in FIG. 36, there is no case where the block flatness class code is 1 and the inter-block continuity class code is 0. That is, here, it is assumed that there is no continuity even if the block boundary is flat, and if the block flatness class code is 1, the interblock continuity class code is also always 1. I am trying to do it.

Therefore, for example, the set (b1, b2) of the block flatness class code and the inter-block continuity class code for the upper boundary of the block is (0, 0),
There are only three possibilities (0,1) and (1,1). As a result, the number of classes represented by the 4-bit block flatness class code and the 4-bit inter-block continuity class code for all the four boundaries of the upper, lower, left and right of the block is 81 (= 3 ⁴ ). .

2-bit AC power class code
The number of classes represented by is 4 (= 2 ²) It is the street.

Therefore, here, the number of classes represented by the 10-bit class code in FIG. 35 is 324 (= 81).
× 4) It becomes as follows.

In the above case, the set (b1, b2) of the block flatness class code and the inter-block continuity class code is (0, 0) in the case where the block of interest and its adjacent neighbors. The images in the block are not connected to each other, and the block of interest and the adjacent block are, so to speak, “unrelated”. Also, (b
In the case where (1, b2) is (0, 1), the images in the target block and adjacent blocks are not flat,
Represents "continuous". Furthermore, (b1, b2)
However, in the case of (1, 1), the images in the block of interest and the adjacent block are “flat” (therefore,
It is also "continuous").

In the above case, the set of the block flatness class code and the inter-block continuity class code (b
1, b2) is set to 3 types to represent 324 classes by a 10-bit class code, but (b1, b2) is not 3 types but (0, 0),
It is possible to take four ways of (0,1), (1,0), and (1,1).
It is also possible to be able to represent 24 (= 2 ¹⁰ ) different classes.

That is, in the above-mentioned case, if only one of the equations (14) or (15) is satisfied, the flatness condition is satisfied, and (b1, b2) is set to (1,
Although 1) is assigned, for example, the case where both equations (14) and (15) are satisfied and the case where only equation (14) is satisfied are distinguished, and equations (14) and (15) are assigned. ) Are both satisfied, (b1, b2)
Is assigned to (1,1), and when only equation (14) is satisfied, (1,0) is assigned to (b1, b2).
Can be assigned.

In this case, (b1, b2) becomes (1,1)
In this case, the images in the target block and the adjacent block are “flat on both the target block side and the adjacent block side”, and (b1, b2) is (1, 0).
The case where the image of the target block and the adjacent block is “flat” only on the side of the adjacent block.

Further, in the flowchart of FIG. 36, (b1, b2) is set to (b1, b2) regardless of whether the boundary edge condition is satisfied or the flatness condition and the continuity condition are not satisfied. 0,0), but either the case where the boundary edge condition is satisfied or the case where neither the flatness condition nor the continuity condition is satisfied is assigned to (1,0). Due to
It is also possible to allow four combinations (b1, b2) of the block flatness class code and the inter-block continuity class code.

Furthermore, the case where both equations (14) and (15) are satisfied and the case where only equation (14) are satisfied are distinguished, and the case where the boundary edge condition is satisfied and the flatness condition and the continuity condition are continuous. It is also possible to distinguish from the case where none of the sex conditions are met.
However, in this case, in combination with the case where the images in the target block and the adjacent block are “continuous” and the case where the images in the target block and the adjacent block are “flat”, one of the target blocks (one side) It is necessary to divide the boundary into five cases. Therefore, in this case, the number of classes in the boundary block boundary is 6
There are 25 types, and as a result, the total number of classes is 2500, considering the 2-bit AC power class code.

Next, FIG. 37 shows a configuration example of the class code generation unit 37 of FIG.

The class code generator 37 classifies a pixel of a block of color difference signals into classes.

Therefore, the class code generation unit 37 can be configured in the same manner as the class code generation unit 36 which outputs the luminance class code.

However, the color difference signal block generally has a lower image activity than the luminance signal block, and the AC component value of the one-dimensional DCT coefficient is smaller, so that it is the same as the class code generation unit 36. When processed, effective classification may be difficult.

Therefore, here, the class code generator 3
7 also uses the luminance class code obtained by the class code generator 36 to classify the pixels of the color difference signals.

That is, in the embodiment of FIG. 37, the AC power output from AC power calculation unit 33 (FIG. 2) is supplied to comparison units 121 and 122.

Similar to the comparison unit 111 in FIG. 34, the comparison unit 121 outputs the AC output from the AC power calculation unit 33 (FIG. 2).
Of the power, the horizontal AC power obtained from the horizontal one-dimensional DCT coefficient of the row of the pixel of interest is set to a predetermined threshold A
The class code creating unit 123
Supply to.

Similar to the comparison unit 112 in FIG. 33, the comparison unit 122 outputs the AC output from the AC power calculation unit 33 (FIG. 2).
Of the power, the AC power in the vertical direction obtained from the vertical one-dimensional DCT coefficient of the column of the pixel of interest is set to a predetermined threshold A
The class code creating unit 123
Supply to.

In the class code generator 37, the block of the color difference signal is processed, but as described above, the value of the AC component of the one-dimensional DCT coefficient of the color difference signal becomes small. , AC power also decreases. Therefore, it is desirable that the threshold value A used in the comparison units 121 and 122 has a value smaller than that used in the comparison units 111 and 112 in FIG.

The class code creating section 123 includes a comparing section 1
In addition to the outputs of 21 and 122, the class code creating unit 36
35 is also supplied, and the class code creating unit 123 classifies the pixel of interest into classes based on the outputs of the comparing units 121 and 122 and the brightness class code. Create (generate) a class code (color difference class code) that represents that class
And output.

Here, the class code creating section 123 is adapted to create a color difference class code having the same format as the luminance class code shown in FIG.

That is, the class code creating unit 123 determines the AC power class code of the pixel of interest with respect to the comparing unit 12
Based on the outputs of 1 and 122, it is created in the same manner as the class code creating unit 116 of FIG.

Further, the class code creating unit 123 spatially co-locates the block flatness class code of the pixel of interest and the block continuity class code with the block of the color difference signal including the pixel of interest (block of interest). It is created using the brightness class code of the pixel in the block of the brightness signal in.

That is, for example, now, the image data is YUV.
And the image format is
If it is 4: 2: 2, as shown in FIG. 38A, two luminance blocks Y1 and Y2 arranged side by side, one color difference block U, and one color difference block V correspond to each other.

In this case, the upper boundary of the color difference block U is the boundary a on the left luminance block Y1 and the upper boundary b of the right luminance block Y2, and the lower boundary of the color difference block U is the luminance. The boundary e below the block Y1 and the luminance block Y
2, the left boundary of the color difference block U corresponds to the left boundary c of the luminance block Y1, and the right boundary of the color difference block U corresponds to the right boundary d of the luminance block Y2.

Therefore, as shown by the dotted line in FIG. 38A, when the color difference block U is divided into two small blocks Ul and Ur that are adjacent to each other in the horizontal and vertical directions and have 4 × 8 pixels, the result is shown in FIG. 38B. As described above, the upper boundary of the left small block Ul is located at the boundary a of the luminance block Y1 at the small block Ul.
The lower boundary is the lower boundary e of the luminance block Y1, the left boundary of the small block Ul is the left boundary c of the luminance block Y1, and the right boundary of the small block Ul is the luminance block Y2.
To the right boundary d of each. The boundary on the right small block Ur is the boundary b on the luminance block Y2.
The lower boundary of the small block Ur is the lower boundary f of the luminance block Y2, the left boundary of the small block Ur is the left boundary c of the luminance block Y1, and the right boundary of the small block Ur is the luminance. Each corresponds to the right boundary d of the block Y2.

Here, in the class code of FIG.
The block flatness class codes obtained for the upper, lower, left, and right boundaries of the block are hereinafter referred to as the upper boundary flatness code, the lower boundary flatness code, the left boundary flatness code, and the right boundary flatness code, respectively. Further, the inter-block continuity class codes obtained for the upper, lower, left, and right boundaries of the block are referred to below as the upper boundary continuity code, the lower boundary continuity code, the left boundary continuity code, and the right boundary continuity code, respectively. Say. Furthermore, below, as appropriate, the upper boundary flatness code and the upper boundary continuity code are combined, the upper boundary code, the lower boundary flatness code and the lower boundary continuity code are combined, and the lower boundary code and the left boundary flatness code are combined. The code and the left boundary continuity code are collectively referred to as the left boundary code, and the right boundary flatness code and the right boundary continuity code are collectively referred to as the right boundary code. Also, below, as appropriate, the upper boundary code,
The lower boundary code, the left boundary code, and the right boundary code are collectively referred to as a boundary code.

Since the boundaries between the luminance blocks Y1 and Y2 and the boundaries between the small blocks Ul and Ur have the above-described correspondence relationship, the class code creating unit 123
As the boundary code in the color difference class code of the pixels of each of the small blocks Ul and Ur, the one obtained for the corresponding boundaries of the luminance blocks Y1 and Y2 is used as it is.

That is, the class code creating unit 123 determines that the pixels of the small block Ul are the upper boundary code, the lower boundary code, the left boundary code, and the right boundary code in the color difference class code, respectively. The code, the lower boundary code of the luminance block Y1, the left boundary code of the luminance block Y1, and the right boundary of the luminance block Y2 are set.

Also, the class code creating unit 123, for the pixels of the small block Ur, sets the upper boundary code, the lower boundary code, the left boundary code, and the right boundary code in the color difference class code to the upper boundary of the luminance block Y2, respectively. The code, the lower boundary code of the luminance block Y2, the left boundary code of the luminance block Y1, and the right boundary code of the luminance block Y2 are set.

The correspondence relationship between the color difference block V and the luminance blocks Y1 and Y2 is also the same as the correspondence relationship between the color difference block U and the luminance blocks Y1 and Y2. Similarly, the color difference block V is divided into two small blocks Vl and Vr each having a size of 4 × 8 pixels in the horizontal and vertical directions, and the small blocks Vl and Vr are divided.
The boundary code of the luminance blocks Y1 and Y2 is used as the boundary code in the color difference class code of
Set in the same way as in.

As a result, the class code creating section 123 creates the following color difference class code for the pixels of the small blocks Ul, Ur, Vl, Vr.

That is, for example, as shown in FIG. 39A, for example, the i-th bit (maximum bit) of the 10-bit luminance class code obtained for the pixels of the two luminance blocks Y1 or Y2 corresponding to the color difference blocks U and V. I from the least significant bit
(Bit bit) is represented as BL # i-1 or BR # i-1, respectively, and small blocks Ul, Ur, V
For the l and Vr pixels, a class code as shown in FIG. 39B is created.

That is, BR0, BL are set in the first to eighth bits of the color difference class code of the pixels of the small blocks Ul, Vl.
1, BL2, BL3, BR4, BL5, BL6, BL7
Are arranged respectively. In addition, B is included in the first to eighth bits of the color difference class code of the pixels of the small blocks Ur and Vr.
R0, BL1, BR2, BR3, BR4, BL5, BR
6 and BR7 are arranged respectively.

Here, the 9th bit and the 10th bit of the color difference class code are determined based on the AC power of the pixel of interest, as in the case of the luminance class code.

Next, the image format is, for example, 4:
If it is 2: 0, as shown in FIG.
Four adjacent luminance blocks Y1, Y2, Y3, and Y4 that are in the positional relationship of lower left, upper right, and lower right, one color difference block U, and one color difference block V correspond to each other.

Then, in this case, as indicated by the dotted line in FIG. 40A, the color difference block U is defined as four small blocks having a horizontal × vertical 4 × 4 pixel positional relationship of upper left, lower left, upper right, and lower right. When divided into Uul, Ull, Uur, and Ulr,
As shown in FIG. 40B, the boundary above the small block Uul is located at the boundary a above the luminance block Y1 and the small block Uu.
The lower boundary of l is the lower boundary g of the luminance block Y3, the left boundary of the small block Uul is the left boundary c of the luminance block Y1, and the right boundary of the small block Uul is the right boundary of the luminance block Y2. Respectively corresponding to the boundaries d of. Further, the upper boundary of the small block Ull is the boundary a above the luminance block Y1, the lower boundary of the small block Ull is the lower boundary g of the luminance block Y3, and the left boundary of the small block Ull is
The left boundary e of the luminance block Y3 and the right boundary of the small block Ull correspond to the right boundary f of the luminance block Y4, respectively. Furthermore, the boundary above the small block Uur is
On the upper boundary b of the luminance block Y2, the lower boundary of the small block Uur is the lower boundary h of the luminance block Y4, and the left boundary of the small block Uur is the left boundary c of the luminance block Y1. The right border of Uur is the luminance block Y
It corresponds to the boundary d on the right of 2. Further, the boundary above the small block Ulr is the boundary b above the luminance block Y2, and the boundary below the small block Ulr is the luminance block Y.
4, the left boundary of the small block Ulr corresponds to the left boundary e of the luminance block Y3, and the right boundary of the small block Ulr corresponds to the right boundary f of the luminance block Y4.

Since the boundaries between the luminance blocks Y1 to Y4 and the boundaries between the small blocks Uul and Ull, Uur, and Ulr have the above-described correspondences, the class code creating unit 123 uses the image format. Is 4: 2: 2, small blocks Uul, Ull, Uu
As the boundary code in the color difference class code of each pixel of r and Ulr, the one obtained for the corresponding boundary of the luminance blocks Y1 to Y4 is used as it is.

That is, the class code creating unit 123, for the pixels of the small block Uul, the upper boundary code, the lower boundary code, the left boundary code,
As the right boundary code, the upper boundary code of the luminance block Y1, the lower boundary code of the luminance block Y3, the left boundary code of the luminance block Y1, and the right boundary of the luminance block Y2 are set, respectively.

Also, the class code creating unit 123, for the pixels of the small block Ull, the upper boundary code, the lower boundary code, the left boundary code in the color difference class code,
As the right boundary code, the upper boundary code of the luminance block Y1, the lower boundary code of the luminance block Y3, the left boundary code of the luminance block Y3, and the right boundary of the luminance block Y4 are set, respectively.

Further, with respect to the pixels of the small block Uur, the class code creating unit 123 sets the luminance block Y2 as the upper boundary code, the lower boundary code, the left boundary code and the right boundary code in the color difference class code.
The upper boundary code, the lower boundary code of the luminance block Y4, the left boundary code of the luminance block Y1, and the right boundary code of the luminance block Y2 are set.

Also, the class code creating unit 123, for the pixels of the small block Ulr, the upper boundary code, the lower boundary code, the left boundary code,
As the right boundary code, the upper boundary code of the luminance block Y2, the lower boundary code of the luminance block Y4, the left boundary code of the luminance block Y3, and the right boundary code of the luminance block Y4 are set, respectively.

The correspondence relationship between the color difference block V and the luminance blocks Y1 to Y4 is the same as the correspondence relationship between the color difference block U and the luminance blocks Y1 to Y4. Similarly, the color difference block V is divided into four small blocks Vul, Vll, Vur, and Vlr each having 4 × 4 pixels in the horizontal and vertical directions, and as a boundary code in the color difference class code of the small blocks Vul, Vll, Vur, and Vlr , The boundary codes of the luminance blocks Y1 to Y4 are set in the same manner as in the color difference block U.

As a result, the class code creating section 123 creates the following color difference class code for the pixels of the small blocks Uul, Ull, Uur, Ulr and Vul, Vll, Vur, Vlr.

That is, for example, as shown in FIG. 41A, the i-th bit of the 10-bit luminance class code obtained for the pixels of the two luminance blocks Y1 to Y4 corresponding to the color difference blocks U and V is now set to BUL # i-1, B
UR # i-1, BDL # i-1, and BDR # i-1, respectively, to represent small blocks Uul, Ull,
Uur, Ulr, and Vul, Vll, Vur, Vl
For the pixel of r, a class code as shown in FIG. 41B is created.

That is, BUR is set in the first to eighth bits of the color difference class code of the pixels of the small blocks Uul and Vul.
0, BUL1, BDL2, BUL3, BUR4, BUL
5, BDL6 and BUL7 are arranged respectively. Also,
The first to eighth bits of the color difference class code of the pixels of the small blocks Ull and Vll include BDR0, BDL1, and BDL.
2, BUL3, BDR4, BDL5, BDL6, BUL
7 are arranged respectively. In addition, small block Uur,
The first to eighth bits of the color difference class code of the pixel of Vur include BUR0, BUL1, BDR2, BUR3, BU.
R4, BUL5, BDR6 and BUR7 are arranged respectively. In addition, BDR0, BD are included in the first to eighth bits of the color difference class code of the pixels of the small blocks Ulr, Vlr.
L1, BDR2, BUR3, BDR4, BDL5, BD
R6 and BUR7 are arranged respectively.

Here, the 9th bit and the 10th bit of the color difference class code are determined based on the AC power of the pixel of interest, as in the case of the luminance class code.

Here, the image data is Y, U,
Although the description has been given assuming that the image data is represented by the V format, the same class classification can be performed when the image data is represented by other formats such as Y, Cb, and Cr.

Further, here, the image format is
The case of 4: 2: 2 and the case of 4: 2: 0 have been described, but the case where the image format is 4: 4: 4 (for example, image data is R (red), G (green), B (in the case of (blue) format), the same class classification may be performed for each signal, or the class code obtained by the class classification performed for a certain signal may be used as it is for other signals. good.

Next, FIG. 42 shows a configuration example of the adaptive processing unit 51 of FIG.

The prediction tap generation unit 131 reads a two-dimensional DCT coefficient from the buffer memory 12 (FIG. 2) to generate a prediction tap for performing the linear primary prediction calculation of the formula (1) for predicting the pixel of interest. And supplies it to the product-sum calculation unit 133.

Here, FIG. 43 shows a configuration example of the prediction tap generation unit 131 of FIG.

[0531] The reading unit 136 uses the buffer memory 12
The necessary two-dimensional DCT coefficient is read from (FIG. 2) and supplied to the array unit 137. The array unit 137 includes the reading unit 13
The prediction taps are formed by arranging (arranging) the two-dimensional DCT coefficients from 6 in a predetermined order, and the prediction taps are supplied to the product-sum calculation unit 133 (FIG. 42).

[0532] Here, the reading unit 136 is, for example, the two-dimensional D of the target block stored in the buffer memory 12.
All the CT coefficients are read and supplied to the array unit 137 as prediction taps. In this case, the prediction tap is 64 (=
It will be composed of 8 × 8) taps.

By the way, the spatial frequency component of the pixel of interest is
Since the two-dimensional DCT coefficients of the block of interest are dispersed, so to speak, it is desirable to use at least all of the two-dimensional DCT coefficients of the block of interest as prediction taps when predicting the pixel of interest.

However, the pixel of interest generally has a correlation not only with the pixel of the block of interest but also with the pixels of neighboring blocks around the block of interest, such as the adjacent block adjacent to it, and therefore the prediction tap is In general, it is more accurate to decode (predict) the pixel of interest by configuring it in a form that also includes information on peripheral blocks (peripheral information).
It becomes possible to do.

Therefore, in the reading section 136, for example,
As shown in FIG. 44, all the two-dimensional DCT coefficients of the block of interest stored in the buffer memory 12 and all the two-dimensional DCT coefficients of four adjacent blocks that are adjacent to the block of interest in the vertical and horizontal directions are read out, and the prediction tap Can be supplied to the array unit 137. In this case, the prediction taps are composed of 320 (= 8 × 8 × 5) taps.

Further, in the reading section 136, as shown in FIG. 45, for example, all the two-dimensional DCT coefficients of the target block stored in the buffer memory 12 and the upper, lower, left, right, upper left of the target block, All of the two-dimensional DCT coefficients of the eight adjacent blocks adjacent to the lower left, upper right, and lower right can be read and supplied to the array unit 137 as prediction taps. In this case, the prediction tap is 57
It consists of 6 (= 8 × 8 × 9) taps.

By the way, as shown in FIGS. 44 and 45, two-dimensional DC of a plurality of blocks adjacent to the target block
If all the T coefficients are included in the prediction taps, the number of taps of the prediction taps becomes large, and the product-sum calculation unit 1 described later
The number of tap coefficients used in 33 (FIG. 42) also increases, and the amount of calculation of the product-sum calculation unit 133 also increases.

Therefore, in the reading unit 136, for example,
As shown in FIG. 46, all the two-dimensional DCT coefficients of the block of interest stored in the buffer memory 12 and eight adjacent adjacent blocks above, below, left, right, upper left, lower left, upper right, and lower right of the block of interest. Only the DC component of the two-dimensional DCT coefficient of the block can be read as the prediction tap. In this case, the prediction tap is 72
(= 8 × 8 + 8) taps, as shown in FIG.
4 and the case of FIG. 45, the prediction tap including the information of the adjacent block, that is, the prediction including the information of the gradient of the pixel value near the target block, which is represented by the DC component of the two-dimensional DCT coefficient here. The taps can be configured with a small number of taps.

Returning to FIG. 42, the tap coefficient buffer 132
Temporarily stores the tap coefficient supplied from the tap coefficient storage unit 14 (FIG. 2).

The product-sum operation unit 131 is the prediction tap generation unit 1
Using the prediction taps supplied from 31 and the tap coefficients stored in the tap coefficient buffer 132, the linear primary prediction calculation of Expression (1) is performed, and the decoded value of the pixel of interest is output.

Next, the processing (adaptive processing) of the adaptive processing unit 51 of FIG. 42 will be described with reference to the flowchart of FIG.

First, in step S101,
The prediction tap generation unit 131 uses prediction taps for performing the linear first-order prediction calculation of Expression (1) that predicts each pixel of the target block, for example, all the two-dimensional DCTs of the target block stored in the buffer memory 12. It is generated by using the coefficient and other necessary two-dimensional DCT coefficient, and the product-sum operation unit 133
And the process proceeds to step S102.

At step S102, the product-sum operation unit 133
, The variable i representing the pixel position mode is initialized to 1, for example.

Here, in the case of inverse DCT transforming the two-dimensional DCT coefficient of a block, the position information indicating the spatial position of each pixel in the block is the component c _{ij of} the transform matrix C shown in equation (11). Cos ((2
It is considered in the form of a phase of j + 1) × i × π / 16).

On the other hand, in the adaptive processing, the linear first-order prediction operation of the equation (1) using the DCT coefficient forming the prediction tap and the tap coefficient is performed. The position information of the existing pixel (pixel of interest) is not considered. Therefore, in order to perform the linear primary prediction calculation considering the position information of the pixel of interest, different tap coefficients are used here depending on the position of the pixel of interest. That is, even if the pixels are classified into the same class, if the positions in the blocks are different, the linear primary prediction calculation is performed using different tap coefficients. In this case, the tap coefficient used for the linear primary prediction calculation is switched depending on the position of the pixel of interest in the block, and the information indicating the position of this pixel of interest is the pixel position mode.

In this case, since the block is composed of 8 × 8 pixels, there are 64 positions, and here, for example, the i-th position in the raster scan order is represented as pixel position mode #i. I will.

After that, the pixel in the pixel position mode #i is set as the target pixel, and the class classification unit 13 (FIG. 2) classifies the target pixel. After waiting for the tap coefficient corresponding to the class code to be supplied, the process proceeds to step S103.

In step S103, the tap coefficient from the tap coefficient storage unit 14 is supplied to and stored in the tap coefficient buffer 132.

Here, as described above, even if the pixels are classified into the same class, different tap coefficients are used when the position in the block (pixel position mode) is different. Therefore, from the tap coefficient storage unit 14, 64 sets of tap coefficients, which are the total number of pixel position modes, are supplied as the tap coefficients of the class of the pixel of interest. 6
Store 4 sets of tap coefficients.

[0550] After that, proceeding to step S104, the sum-of-products operation unit 133 sets the set of tap coefficients (w ₁ , w ₂ , ... in equation (1)) corresponding to the pixel position mode #i,
It is acquired by reading from the tap coefficient buffer 132, and the process proceeds to step S105.

In step S105, the sum of products operation unit 133
Uses the prediction tap from the prediction tap generation unit 131 and the tap coefficient read from the tap coefficient buffer 132 to perform the linear primary prediction calculation of Expression (1), thereby decoding the pixel value of the pixel of interest. To do.

[0552] Then, proceeding to step S106, the product-sum operation unit 133 determines whether or not the pixel position mode #i is equal to 64 (= 8x8) which is the number of pixels of the target block.

If it is determined in step S106 that the pixel position mode #i is not equal to 64, the operation proceeds to step S107, where the product-sum calculation unit 133 increments the pixel position mode #i by 1 and advances to step S103. After returning, the same processing is repeated thereafter.

If it is determined in step S106 that the pixel position mode #i is equal to 64, that is,
When all the pixel values of the block of interest have been decoded, the process ends.

The adaptive processing unit 51 repeats the adaptive processing of FIG. 47 by sequentially setting the blocks stored in the buffer memory 12 (FIG. 2) as target blocks.

The product-sum operation unit 133 temporarily stores the decoded block until a decoded image of one frame (or field) is obtained, and when a decoded image of one frame is obtained, the decoded image of that one frame is acquired. Output the decoded image.

In the MPEG coding, when 8-bit image data is coded, the pixel value and the two-dimensional DC
In order to evenly distribute the T coefficient on the positive side and the negative side around 0, the pixel value obtained by subtracting 128 (= 2 ⁷ ) from the pixel value of the original image is encoded.

Therefore, the product-sum operation unit 133 can obtain a decoded image in which the level of each pixel value is lower by 128. Therefore, the product-sum calculation unit 133 adds 128 to each pixel value of the decoded image and outputs the result.

Next, learning of tap coefficients stored in the coefficient memories 43 and 44 of FIG. 2 will be described.

[0560] In MPEG, I
Since there are three picture types of picture, P picture, and B picture, learning of tap coefficients is also performed for each picture type.

[0561] Fig. 48 shows a configuration example of an embodiment of a learning device in the case of learning tap coefficients for I pictures.

The teacher data storage 141 stores image data for learning as teacher data.

The student data generator 142 includes an MPEG encoder 151, a separator 152, and a DCT coefficient extractor / extractor.
The inverse quantization unit 153 is configured to generate student data from the learning image data (also teacher data here) stored in the teacher data storage 141.

That is, the MPEG encoder 151 reads the learning image data stored in the teacher data storage 141, MPEG-encodes it, and supplies the encoded data obtained as a result to the separation unit 152. Separation unit 15
2 and the DCT coefficient extraction / inverse quantization unit 153 are configured similarly to the separation unit 1 and the DCT coefficient extraction / inverse quantization unit 2 in FIG. 2, respectively, and separate the quantized DCT coefficient from the encoded data, Extract, dequantize, and output.

The separating section 152 and the DCT coefficient extracting / inverse quantizing section 153 perform processing only on I pictures. Further, the separation unit 152 separates not only the quantized DCT coefficient but also so-called side information such as the quantization scale and the DCT type from the encoded data as necessary.

Therefore, the DCT coefficient extraction / inverse quantization unit 15
From 3, the necessary side information such as the DCT type is output in addition to the two-dimensional DCT coefficient for the I picture.

The two-dimensional DCT coefficient and DCT type for the I picture output by the DCT coefficient extraction / inverse quantization unit 153 are used as student data in the student data storage 1
43.

The student data storage 143 includes the DCT coefficient extraction / inverse quantization unit 15 of the student data generation unit 142.
3) Store the student data supplied from.

The prediction tap generation unit 144 generates the same prediction tap as the prediction tap generation unit 131 of FIG. 42 generates from the student data stored in the student data storage 143 and supplies it to the addition unit 146. . Therefore, here, the prediction tap generation unit 144 uses the two-dimensional DC as the student data stored in the student data storage 143.
Of the T coefficients, all the two-dimensional DCT coefficients forming the block of interest and the necessary two-dimensional DCT coefficients are read out,
Use as a prediction tap.

When the prediction tap generation unit 131 in FIG. 42 generates a prediction tap using the two-dimensional DCT coefficient of the block adjacent to the target block in addition to the two-dimensional DCT coefficient of the target block, the prediction tap generation is performed. The unit 144 also uses the two-dimensional DCT coefficient of the target block and the two-dimensional DCT coefficient adjacent to the target block, and uses the prediction tap generation unit 1
Generates a prediction tap having the same structure as that generated by 31.
In this case, the prediction tap generation unit 144 acquires the two-dimensional DCT coefficient of the block adjacent to the target block from the student data storage 143.

The class classification unit 145 is similar to the class classification unit 13 in FIG. 2, and calculates a one-dimensional DCT coefficient from the two-dimensional DCT coefficient as the student data stored in the student data storage 143, and further the one-dimensional DCT coefficient. Based on the DCT coefficient, the target pixel in the target block is classified into the same class as in the class classification unit 13 of FIG.
Add a class code that represents the class of the pixel of interest
Output to 46.

[0572] The add-in unit 146 uses the two-dimensional DCT as the student data stored in the student data storage 143.
The coefficient blocks are sequentially set as the block of interest, and
The pixels of the block of interest are sequentially used as the pixel of interest (focused teacher data), and prediction taps (two-dimensional DCT coefficients constituting the prediction taps) as student data from the prediction tap generation unit 144,
And addition is performed for the target pixel.

That is, the adding unit 146 uses the prediction tap (student data) for each class corresponding to the class code supplied from the class classifying unit 145, and becomes each component in the matrix A of the equation (8). The multiplication (x _in x _im ) of the student data that is present and the calculation corresponding to the summation (Σ) are performed.

Furthermore, the adding unit 146 also uses the prediction tap (student data) and the pixel of interest (teacher data) for each class corresponding to the class code supplied from the class classification unit 145, and formula (8) The calculation corresponding to the multiplication (x _in y _i ) of the student data and the teacher data and the summation (Σ), which are the respective components in the vector v, is performed.

The above-described addition in the adding unit 146 is performed for each class for each pixel position mode for the target pixel.

[0577] The padding portion 146
All the blocks forming the two-dimensional DCT coefficient as the student data stored in the student data storage 143 are performed as the attention block, whereby the normal equation shown in the equation (8) is obtained for each class for each pixel position mode. Build up.

[0577] The tap coefficient calculation unit 147 has the addition unit 1
By solving the normal equations generated for each class and for each pixel position mode in 46, 64 sets of tap coefficients corresponding to 64 pixel position modes are obtained for each class.

[0578] Depending on the number of images prepared as learning images and the contents of the images, the adding section 14 may be used.
6, a class in which the number of normal equations required to obtain the tap coefficient cannot be obtained, and further, a pixel position mode may occur. However, the tap coefficient calculation unit 147 determines the class and the pixel position mode. Is, for example,
Output the default tap coefficient.

Next, the processing (learning processing) of the learning device in FIG. 48 will be described with reference to the flowchart in FIG.

First, in step S111,
As described above, the student data generation unit 142 generates student data for the I picture from the learning image data stored in the teacher data storage 141 and supplies the student data to the student data storage 143 for storage.

[0580] Then, proceeding to step S112, the adding unit 146 selects one of the blocks of the two-dimensional DCT coefficient as the student data stored in the student data storage 143, which is not the block of interest yet. , The block of interest, and proceeds to step S113.

[0582] In step S113, the prediction tap generation unit 144 causes the two-dimensional D as the student data of the block of interest.
All the CT coefficients and the required two-dimensional DCT coefficients are read from the student data storage 143 to generate prediction taps, and the process proceeds to step S114.

[0583] In step S114, the adding portion 146 is added.
, The variable i representing the pixel position mode is initialized to 1, for example, and the process proceeds to step S115. In step S115,
The class classification unit 145 classifies the pixel of interest as a pixel of interest, using the pixel at the position represented by the pixel position mode #i in the block of interest as the pixel of interest, and obtains the result. The class code is output to the adding unit 146.

In step S116, the adding unit 146 reads out the teacher data (pixel value) that is the pixel of interest from the teacher data storage 141, and predicts taps (two-dimensional DCT coefficients constituting the) as student data, Also, the above-described addition of the matrix A and the vector v in Expression (8) is performed on the target pixel as the teacher data. It should be noted that this addition is performed for each class corresponding to the class code from the class classification unit 145 and for each pixel position mode #i for the pixel of interest.

[0585] Then, proceeding to step S117, the adding section 146 determines whether or not the pixel position mode #i is equal to 64 which is the number of pixels of the target block.

If it is determined in step S117 that the pixel position mode #i is not equal to 64, the operation proceeds to step S118, where the adder 146 increments the pixel position mode #i by 1 and returns to step S115. After that, similar processing is repeated.

If it is determined in step S117 that the pixel position mode #i is equal to 64, that is,
If all pixels of the target block are added as target pixels, the process proceeds to step S119, and the adding unit 1
Among the blocks of the two-dimensional DCT coefficient as the student data stored in the student data storage 143, 46 determines whether or not there is a block that is not yet a target block (hereinafter, appropriately referred to as an unprocessed block).

If it is determined in step S119 that there is an unprocessed block, the process returns to step S112.
From among the unprocessed blocks, a new block of interest is selected, and the same process is repeated thereafter.

If it is determined in step S119 that there is no unprocessed block, that is, the adding unit 1
46, if a normal equation for each pixel position mode is obtained for each class, the process proceeds to step S120, where the tap coefficient calculation unit 147 solves the normal equation generated for each pixel position mode of each class, For each class, 64 sets of tap coefficients corresponding to the 64 pixel position modes of the class are obtained, and the learning process ends.

Next, FIG. 50 shows an example of the configuration of an embodiment of a learning device in the case of learning tap coefficients for P pictures. Note that, in the figure, parts corresponding to those in FIG. 48 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the learning device of FIG. 50 is basically configured similarly to the case in FIG. 48 except that the student data generation unit 162 is provided in place of the student data generation unit 151.

Here, since the P picture is predictively coded (non-intra coded) with the I or P picture decoded (encoded) preceding in time as the reference image, that is, predicted from the original image. The residual image obtained by subtracting the image is a two-dimensional D
Since CT conversion is performed, in the image processing apparatus of FIG. 2, the pre-processing unit 11 performs motion compensation on the previously decoded I or P picture, and the resulting two-dimensional D of the predicted image is obtained.
After adding the CT coefficient and the two-dimensional DCT coefficient of the residual image, the adaptive processing unit 51 performs adaptive processing.

Therefore, in learning the tap coefficient for the P picture, it is necessary to use, as the student data, the one obtained by adding the two-dimensional DCT coefficient of the prediction image to the two-dimensional DCT coefficient of the residual image.

By the way, the predicted picture of the P picture can be obtained by using the previously decoded I or P picture as a reference picture and applying motion compensation to the reference picture. There are no tap coefficients for P-pictures because we are trying to learn the tap coefficients.

On the other hand, tap coefficients for I-picture are shown in FIG.
In the learning device of No. 8, it can be obtained in advance.

Therefore, in the learning apparatus shown in FIG. 50, the student data generation unit 162 generates student data for a block of a P picture that is predictively coded using an I picture as a reference image.

That is, the MPEG encoder 171 reads out the learning image data stored in the teacher data storage 141, MPEG-encodes it, and supplies the encoded data obtained as a result to the separation unit 172. Separation unit 17
2 and the DCT coefficient extraction / inverse quantization unit 173 are respectively configured similarly to the separation unit 1 and the DCT coefficient extraction / inverse quantization unit 2 in FIG. 2, and separate the quantized DCT coefficient from the encoded data, Extract and dequantize.

The separating unit 172 and the DCT coefficient extracting / inverse quantizing unit 173 perform processing only on the predictively coded blocks of the I picture and P picture. The separating unit 172 also uses the encoded data to quantize the quantized DCT.
In addition to the coefficient, side information such as a quantization scale, a DCT type, a motion vector, etc. is separated as necessary.

The DCT coefficient extraction / inverse quantization unit 173 uses the I
When the two-dimensional DCT coefficient of the picture is obtained, the two-dimensional DCT coefficient of the I picture is supplied to the class classification unit 174 and the adaptive processing unit 176.

Also, the DCT coefficient extraction / inverse quantization unit 173
Is the two-dimensional D of the predictively coded block of the P picture.
When the CT coefficient, that is, the two-dimensional DCT coefficient of the residual image is obtained, the two-dimensional DCT coefficient of the residual image is supplied to the frequency domain motion compensation addition unit 181.

Further, the DCT coefficient extraction / inverse quantization unit 17
3 obtains the motion vector of the prediction-coded block of the P picture, the motion vector is set to the motion compensation unit 1
Supply to 78.

The class classification unit 174 extracts the DCT coefficient /
The pixels of the block of the I picture supplied from the dequantization unit 173 are sequentially set as the target pixel, and the target pixel is classified as in the case of the class classification unit 13 in FIG. 2 to obtain the result. The class code
It is supplied to the tap coefficient storage unit 175. The tap coefficient storage unit 175 stores the tap coefficient for the I picture obtained by the learning device in FIG. 48, and the tap coefficient storage unit 1 in FIG.
4, the tap coefficient corresponding to the class code supplied from the class classification unit 174 is acquired, and the adaptive processing unit 1
Supply to 76.

The adaptive processing unit 176 is the adaptive processing unit 5 of FIG.
Similar to 1, all the two-dimensional DCT coefficients of the block including the pixel of interest, which are supplied from the DCT coefficient extraction / inverse quantization unit 173, and the required two-dimensional DCT coefficients are used as prediction taps, and the prediction taps and tap coefficients Linear primary prediction calculation using the tap coefficient supplied from the storage unit 175, that is, adaptive processing is performed.

The decoded image of the I picture obtained by performing the adaptive processing in the adaptive processing unit 176 is supplied to and stored in the I picture storage 177.

After that, the motion compensating section 178 reads out from the I picture storage 177 an I picture as a reference picture to be motion compensated by the motion vector supplied from the DCT coefficient extracting / inverse quantizing section 173, and stores the I picture.
A predicted image is generated by performing motion compensation on the picture. This predicted image is supplied to and stored in the image memory 179. The predicted image stored in the image memory 179 is
The DCT conversion unit 180 converts the two-dimensional DCT coefficient into a two-dimensional DCT coefficient, which is supplied to the frequency domain motion compensation addition unit 181. The frequency domain motion compensation addition unit 181 has a two-dimensional DCT coefficient of the P-picture residual image supplied from the DCT coefficient extraction / inverse quantization unit 173 and a two-dimensional DCT coefficient of the predicted image supplied from the DCT conversion unit 180. And are added.

That is, the motion compensation unit 178 and the image memory 17
9, the DCT transform unit 180, and the frequency domain motion compensation addition unit 181 are the motion compensation unit 4, the image memory 5, and D in FIG.
The CT transform unit 21 and the frequency domain motion compensation addition unit 22 are configured in the same manner. Therefore, in the frequency domain motion compensation addition unit 181, the DCT coefficient extraction / inverse quantization unit 1
2D DCT coefficient of the P-picture residual image from 73 and 2D DCT of the prediction image from the DCT transform unit 180
By adding the coefficient and the coefficient, a two-dimensional DCT coefficient obtained by two-dimensional DCT transforming the original image of the P picture (not the original image as described above) is obtained.

The two-dimensional DCT coefficient of the original image of the P picture obtained by the frequency domain motion compensation addition unit 181 is supplied to the student data storage 143 and stored as student data.

Thereafter, the same processing as in the case of the learning device of FIG. 48 is performed, whereby the tap coefficient for P picture (more accurately, for the predictively coded block of P picture) is obtained.

Next, FIG. 51 shows a configuration example of an embodiment of a learning device for learning tap coefficients for B pictures. As with the learning device for learning tap coefficients for P pictures in FIG. 50, the learning device for learning tap coefficients for B pictures has only the student data generation unit for generating student data, and the learning device for I pictures in FIG. Only the student data generation unit of the learning device for learning the tap coefficient for the B picture is shown in FIG. 51 because it is different from the learning device for learning the tap coefficient for B picture.

Like the P picture, the B picture is also predictively coded (non-intra coded) with the I or P picture decoded earlier in time as the reference image.
That is, since the residual image obtained by subtracting the predicted image from the original image is two-dimensionally DCT-transformed, in the image processing apparatus of FIG. 2, the pre-processing unit 11 performs motion compensation on the previously decoded I or P picture. After adding the two-dimensional DCT coefficient of the predicted image and the two-dimensional DCT coefficient of the residual image obtained as a result, the adaptive processing unit 51 performs adaptive processing.

Therefore, in learning the tap coefficient for B picture, it is necessary to use, as student data, the one obtained by adding the two-dimensional DCT coefficient of the prediction image to the two-dimensional DCT coefficient of the residual image.

Therefore, the learning device shown in FIG. 51 is designed to generate such student data for learning.

That is, the MPEG encoder 191 is supplied with the image data for learning (here, it is equal to the teacher data).
1 MPEG-encodes the learning image data, and supplies the encoded data obtained as a result to the separation unit 192. Separation unit 192 and DCT coefficient extraction / inverse quantization unit 193
Are configured in the same manner as the separation unit 1 and the DCT coefficient extraction / dequantization unit 2 in FIG. 2, respectively, and separates and extracts quantized DCT coefficients from encoded data, dequantizes and outputs them.

The two-dimensional DCT coefficient of the I picture output by the DCT coefficient extraction / inverse quantization section 193 is the class classification section 1
94 and the adaptive processing unit 196. The two-dimensional DCT coefficient of the P picture output from the DCT coefficient extraction / inverse quantization section 193 is supplied to the frequency domain motion compensation addition section 201. Further, the DCT coefficient extraction / inverse quantization unit 193
The two-dimensional DCT coefficient of the B picture output by is supplied to the frequency domain motion compensation addition unit 209.

[0614] Here, the separation unit 192 extracts, from the encoded data, not only the quantized DCT coefficient, but also side information such as the quantization scale, the DCT type, and the moving vector.
The DCT coefficient extraction / inverse quantization unit 19 is separated if necessary.
Via 3 to the required blocks. Regarding the side information, in the embodiment of FIG. 51, the motion vector of the P picture is supplied from the DCT coefficient extraction / inverse quantization unit 193 to the motion compensation unit 198, and the DCT
Only the supply of B-picture motion vectors from the coefficient extractor / inverse quantizer 193 to the motion compensator 206 is shown.

Also, the DCT coefficient extraction / inverse quantization unit 193
Is the two-dimensional D of the predictively coded block of the B picture.
Only the CT coefficient, that is, the two-dimensional DCT coefficient of the residual image,
It is supplied to the frequency domain motion compensation addition unit 209.

Further, the DCT coefficient extraction / inverse quantization unit 19
For the P picture, 3 supplies only the two-dimensional DCT coefficient of the predictively coded block to the frequency domain motion compensation addition unit 201. Further, in the DCT coefficient extraction / inverse quantization unit 193, the two-dimensional DCT coefficient of the intra-coded block of the P picture is supplied to the class classification unit 194 and the adaptive processing unit 196, and hereinafter, the intra-coded I It is processed like a picture.

That is, the class classification unit 194 sequentially sets the pixels of the intra-coded blocks of the I picture and P picture supplied from the DCT coefficient extraction / inverse quantization unit 193 as the target pixel, and regards the target pixel. The class classification is performed in the same manner as in the class classification unit 13 of FIG. 2, and the class code obtained as a result is supplied to the tap coefficient storage unit 195. Tap coefficient storage unit 195
Stores tap coefficients for I-pictures obtained by the learning device in FIG. 48, and like the tap coefficient storage unit 14 in FIG. 2, tap coefficients corresponding to the class code supplied from the class classification unit 194. Is acquired and supplied to the adaptive processing unit 196.

The adaptive processing unit 196 is the adaptive processing unit 5 of FIG.
As in the case of 1, the two-dimensional DCT coefficients of the block including the pixel of interest and the required two-dimensional DCT coefficients are used as prediction taps, and the prediction taps and the tap coefficients supplied from the tap coefficient storage unit 195 are used for linearity. Primary prediction calculation, that is, adaptive processing is performed.

The decoded image of the I picture obtained by performing the adaptive processing in the adaptive processing unit 196 is supplied to and stored in the I picture storage 197. Note that the adaptive processing unit 196 can also obtain a decoded image of a block of intra-coded P picture, but the decoded image of this P picture is supplied from the adaptive processing unit 196 to the P picture storage 205 and stored therein.

After that, the motion compensating unit 198 reads from the I picture storage 197 an I picture as a reference image to be motion compensated by the motion vector supplied from the DCT coefficient extracting / dequantizing unit 193, and the I picture is read.
By performing motion compensation on the picture, a predicted image of the P picture is generated. The predicted image of the P picture is supplied to and stored in the image memory 199. The predicted image stored in the image memory 199 is converted into 2 by the DCT conversion unit 200.
It is converted into a three-dimensional DCT coefficient and supplied to the frequency domain motion compensation addition unit 201. Frequency domain motion compensation adder 201
Adds the two-dimensional DCT coefficient of the P-picture residual image supplied from the DCT coefficient extraction / inverse quantization section 193 and the two-dimensional DCT coefficient of the predicted image supplied from the DCT conversion section 200.

That is, the motion compensation unit 198 and the image memory 19
9, the DCT transform unit 200, and the frequency domain motion compensation addition unit 201 are the motion compensation unit 4, the image memory 5, and D in FIG.
The CT transform unit 21 and the frequency domain motion compensation addition unit 22 have the same configuration. Therefore, in the frequency domain motion compensation addition unit 201, the DCT coefficient extraction / inverse quantization unit 1
Two-dimensional DCT coefficient of the P-picture residual image from 93 and two-dimensional DCT of the predicted image from the DCT transform unit 200
By adding the coefficient and the coefficient, a two-dimensional DCT coefficient obtained by two-dimensional DCT transforming the original image of the P picture (not the original image as described above) is obtained.

The two-dimensional DCT coefficient of the original image of the P picture obtained by the frequency domain motion compensation addition unit 201 is supplied to the class classification unit 202 and the adaptation processing unit 204.

The class classification unit 202 sequentially designates the pixels of the predictively coded block of the P picture supplied from the frequency domain motion compensation addition unit 201 as the attention pixel, and regarding the attention pixel, the class classification unit of FIG. Class classification is performed in the same manner as in the case of 13, and the class code obtained as a result is supplied to the tap coefficient storage unit 203. The tap coefficient storage unit 203 stores the tap coefficient for the P picture obtained by the learning device in FIG. 50, and like the tap coefficient storage unit 14 in FIG. 2, the class code supplied from the class classification unit 202. The tap coefficient corresponding to is acquired and supplied to the adaptive processing unit 204.

The adaptive processing unit 204 is the adaptive processing unit 5 of FIG.
Similarly to 1, linear two-dimensional prediction calculation using all the two-dimensional DCT coefficients of the block including the pixel of interest as prediction taps and the tap coefficients supplied from the tap coefficient storage unit 203, that is, adaptation Perform processing.

[0625] The decoded image of the predictively coded P picture block obtained by performing the adaptive processing in the adaptive processing unit 204 is the P picture storage 205.
Are stored and stored in. As described above, the P picture storage 205 also stores the decoded image of the intra-coded P picture block supplied from the adaptation processing unit 196.

Thereafter, the motion compensating unit 206 stores the I or P picture as the reference image to be motion compensated by the motion vector of the B picture supplied from the DCT coefficient extracting / inverse quantizing unit 193, in the I picture storage 197.
Alternatively, a predicted image of a B picture is generated by reading from the P picture storage 205 and subjecting the I or P picture to motion compensation. The predicted image of the B picture is supplied to and stored in the image memory 207. The predicted image stored in the image memory 207 is the DCT conversion unit 20.
In step 8, it is converted into a two-dimensional DCT coefficient and supplied to the frequency domain motion compensation addition section 209. The frequency domain motion compensation addition unit 209 includes a two-dimensional DCT coefficient of the residual image of the B picture supplied from the DCT coefficient extraction / inverse quantization unit 193 and a two-dimensional DCT coefficient of the prediction image supplied from the DCT conversion unit 208. And are added.

That is, the motion compensation unit 206 and the image memory 20.
7, the DCT transform unit 208, and the frequency domain motion compensation addition unit 209 are the motion compensation unit 4, the image memory 5, and D in FIG.
The CT transform unit 21 and the frequency domain motion compensation addition unit 22 are configured in the same manner. Therefore, in the frequency domain motion compensation addition unit 209, the DCT coefficient extraction / inverse quantization unit 1
Two-dimensional DCT coefficient of B picture residual image from 93 and two-dimensional DCT of prediction image from DCT transform unit 208
By adding the coefficient and the coefficient, a two-dimensional DCT coefficient obtained by two-dimensional DCT transforming the original image of the B picture (not the original image as described above) is obtained.

The two-dimensional DCT coefficient of the original image of the B picture obtained by the frequency domain motion compensation addition unit 209 is output as student data. Then, after that, the same processing as in the case of the learning device in FIG. 48 is performed, whereby
The tap coefficient for the B picture (more precisely, for the predictively coded block of the B picture) is obtained.

In the coefficient memories 43 and 44 of FIG. 2, 6 obtained for each class by the above learning.
The tap coefficients for I, P, and B pictures for each of the four pixel position modes are stored.

Therefore, the tap coefficients stored in the coefficient memories 43 and 44 have the prediction error (here, the prediction error of the prediction value of the original pixel value obtained by performing the linear primary prediction calculation).
Square error) is obtained by performing learning so as to be statistically minimized. As a result, according to the adaptive processing unit 51 of FIG. 2, the MPEG encoded image is converted into the original image. It is possible to decode an image as close as possible to the image, that is, an image with good image quality in which various distortions such as block distortion and mosquito noise are sufficiently reduced.

Here, in the learning device, the normal equations are set separately for the luminance signal and the color difference signal, and the tap coefficient obtained by solving the normal equation made from the luminance signal is stored in the coefficient memory 43. The tap coefficient obtained by solving the normal equation created from the color difference signal is stored in the coefficient memory 44.

Also, as described above, in MPEG encoding, the pixel value obtained by subtracting 128 (= 2 ⁷ ) from the pixel value of the original image is encoded. For this reason, the push-in section 1
46 (FIGS. 48 and 50) uses a value obtained by subtracting 128 from the pixel value as the teacher data stored in the teacher data storage 141 in the addition.

In the image processing apparatus of FIG. 2, adaptive processing is performed on blocks of I pictures using tap coefficients for I pictures. Further, for a block of P picture or B picture, if the block is predictively coded (non-intra coded),
Adaptive processing is performed using tap coefficients for P pictures or B pictures, but when a block is intra-coded, adaptive processing is performed using tap coefficients for I pictures.

Also, in the above case, the tap coefficients for I, P, and B pictures are learned, but the learning of tap coefficients for B pictures can be omitted, for example. In this case, in the image processing apparatus of FIG.
For both predictive-coded blocks of pictures and B pictures, for example, adaptive processing is performed using tap coefficients for P pictures. Further, in the image processing apparatus of FIG. 2, it is possible to perform all the adaptive processing for I, P, B pictures by using tap coefficients for I pictures.

Next, FIGS. 52 and 53 show simulation results obtained by performing a simulation for decoding an MPEG encoded image in the image processing apparatus of FIG.

In the simulation, 4:
The coded data obtained by coding the image of the 2: 2 format by the MPEG2 system at the data rate of about 3.3 Mbps was used. In the simulation, I
The picture sequence in which the picture and the P picture are even frames and odd frames, respectively, and alternating every frame is used. Furthermore, in the simulation, the thresholds A, B, C, D, and E described above are set to 700 and 7 respectively.
00,350,120,120 was used. However, for the color difference signal, 80 was used as the threshold value A.

FIG. 52 shows the S / N (Signal to N) for the luminance signal of the decoded image obtained by the simulation.
oise ratio).

In FIG. 52, the S / N indicated by the solid line indicates that of the decoded image by the image processing apparatus of FIG.
The S / N indicated by the dotted line indicates that of an image decoded by a conventional MPEG software decoder conforming to the MPEG standard. From FIG. 52, it can be seen that the S / N of the decoded image by the image processing apparatus of FIG. 2 is improved by about 1 dB as compared with the decoded image by the conventional MPEG software decoder.

FIG. 53 shows a decoded image obtained by the simulation.

That is, FIG. 53A shows the original image as shown in FIG.
Shows a decoded image by the conventional MPEG software decoder, and FIG. 53C shows a decoded image by the image processing apparatus of FIG. 53A to 53C
In, about 1/3 of the right side shows the entire image of the entire bottle, about 2/3 of the left side
Indicates the magnified image of the label portion of the bottle. The image shown in FIG. 53 is an I-picture image.

The original image of FIG. 53A and the conventional M of FIG. 53B.
Comparing the images decoded by the PEG software decoder, the decoded image in FIG. 53B has block distortion in which the boundaries of the blocks are conspicuous, and further, mosquito noise is conspicuous in the portion of the letter “Z” in the label of the bottle. Is appearing.

On the other hand, in the decoded image by the image processing apparatus of FIG. 53C, block distortion is sufficiently reduced, and mosquito noise is also reduced.

Next, FIG. 54 shows a second configuration example of the prediction tap generation unit 131 of FIG. Note that, in the figure, parts corresponding to those in FIG. 43 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, in the embodiment of FIG. 54, the inverse DCT transform unit 2
21 and a selection unit 222 are newly provided,
Basically, the configuration is similar to that in the case of FIG.

Here, only the prediction taps configured by the prediction tap generation unit 131 will be described below. However, when the prediction tap generation unit 131 configures the prediction taps described below, FIGS. The prediction tap generation unit 144 of the learning device illustrated in (1) also configures prediction taps having the same tap structure. Also, the adaptation processing unit 176 of FIG. 50 and the adaptation processing units 196 and 20 of FIG.
4, the same processing as the adaptive processing unit 51 of FIG. 42 is performed, but the prediction tap generation unit 131
A prediction tap having the same tap structure as that generated by is created (generated).

The inverse DCT transform unit 221 has the read unit 13
Inverse DCT transformation is performed on four adjacent blocks that are supplied from 6 and are adjacent to the target block in the upper, lower, left, and right directions. That is, in the embodiment of FIG. 54, the reading unit 136 reads from the buffer memory 12 (FIG. 2) the two-dimensional DCT coefficient of the block of interest and the two adjacent blocks of four adjacent blocks that are adjacent to the block of interest. The dimensional DCT coefficient is also read out. Then, the reading unit 136
Is the array unit 137 for the two-dimensional DCT coefficient of the block of interest.
2D DC of 4 adjacent blocks
The T coefficient is supplied to the inverse DCT conversion unit 221. Inverse DCT
In this way, the conversion unit 221 performs two-dimensional inverse DCT conversion on the four adjacent blocks that are supplied from the reading unit 136 and that are adjacent to the target block in the upper, lower, left, and right directions, respectively, and thereby the pixel values of the four adjacent blocks are obtained. To decrypt
It is supplied to the selection unit 222.

The selecting section 222 selects the pixel value of a predetermined pixel from the pixel values of the 8 × 8 pixels of the four adjacent blocks supplied from the inverse DCT converting section 221, and supplies it to the array section 137. In this case, in the array unit 137, the reading unit 1
The prediction tap is configured by arranging the two-dimensional DCT coefficient of the block of interest supplied from 36 and the pixel values of the pixels supplied from the selection unit 222 in a predetermined order to form a prediction tap, and the sum of products operation unit 133 ( 42).

[0647] That is, the selection unit 222 is, for example, as shown in Fig. 55A.
As shown in FIG. 2, among the pixels of the adjacent block, two pixels (block adjacent pixels) adjacent to the target block, which are on the left and right of the same row as the target pixel (hereinafter, each of them is referred to as the left adjacent same pixel). Pixel value of row pixel and right adjacent same row pixel) and two pixels above and below the same column as the pixel of interest (hereinafter, appropriately referred to as upper adjacent same column pixel and lower adjacent same column pixel) Select the value, and the array part 13
Supply to 7. In this case, in the array unit 137, a total of 68 (= 8) of the 8 × 8 two-dimensional DCT coefficient of the target block supplied from the reading unit 136 and the pixel values of the four pixels supplied from the selection unit 222. A prediction tap composed of × 8 + 4) taps is configured.

[0648] Also, the selection unit 222 is, for example, as shown in Fig. 55B.
As shown in, the left adjacent same row pixel, the right adjacent same row pixel,
In addition to the upper-adjacent same-column pixel and the lower-adjacent same-column pixel, the pixel adjacent to the left of the left-adjacent same-row pixel, the pixel adjacent to the right of the right-adjacent same-row pixel, and the upper-adjacent same-column pixel A pixel and a pixel adjacent below the pixel in the same column adjacent to the lower pixel are selected and supplied to the array unit 137. In this case, the array unit 13
7, the total of the 8 × 8 two-dimensional DCT coefficient of the target block supplied from the reading unit 136 and the pixel values of the eight pixels supplied from the selection unit 222 is 72 (= 8 × 8).
A prediction tap composed of +8) taps is configured.

Alternatively, for example, as shown in FIG. 55C, the selection unit 222 selects one of the left-adjacent same-row pixels and the right-adjacent same-row pixels that is closer to the pixel of interest, and the upper-adjacent same column. Of the pixels in the same column adjacent to the pixel below,
The one closer to the pixel of interest is selected and supplied to the array unit 137.
In this case, in the array unit 137, the 8 × 8 two-dimensional DCT coefficient of the target block supplied from the reading unit 136,
With the pixel values of the two pixels supplied from the selection unit 222,
A prediction tap composed of 66 (= 8 × 8 + 2) taps in total is configured.

[0650] Also, the selection unit 222 may be, for example, as shown in Fig. 55D.
As shown in (4), four sets of eight pixels (block adjacent pixels) adjacent to the target block in each of the four adjacent blocks adjacent to the target block in the vertical and horizontal directions are selected and supplied to the array unit 137. In this case, the array unit 137
Then, a total of 96 (= 8 × 8) of the 8 × 8 two-dimensional DCT coefficient of the target block supplied from the reading unit 136 and the pixel value of the 8 × 4 pixel supplied from the selecting unit 222.
A prediction tap composed of + 8 × 4) taps is configured.

Here, it is possible for the selecting section 222 to select, for example, all the pixels of four adjacent blocks which are adjacent to the target block in the vertical and horizontal directions. However, in this case, as in the case of FIG. 44, the number of taps of the prediction taps configured by the array unit 137 becomes large, and the calculation amount of the product-sum calculation unit 133 (FIG. 42) and the coefficient for storing the tap coefficient are stored. The memory capacity of the memories 43 and 44 (FIG. 2) also becomes large.

Therefore, for example, as shown in FIG.
From all the two-dimensional DCT coefficients of the block of interest and the DC components of the two-dimensional DCT coefficients of the eight adjacent blocks adjacent to the block of interest, above, below, left, right, upper left, lower left, upper right, and lower right, respectively. By configuring the prediction taps, it is possible to configure the prediction taps with a small number of taps in a form including the information of the adjacent block having the correlation with the target pixel.

By the way, the target pixel and the adjacent block 8
The correlation with each of the x8 pixels is generally not uniform. That is, the correlation between the pixel of the adjacent block and the pixel of interest is generally high as the pixel closer to the pixel of interest and lower as the pixel farther away. Further, for example, when there is an edge between the pixel of the adjacent block and the target pixel, or in the case of a complicated image, there is no correlation (almost no) between the pixel of the adjacent block and the target pixel. Sometimes.

On the other hand, the DC component of the two-dimensional DCT coefficient of the adjacent block represents the average value of 8 × 8 pixels (average pixel value) of the adjacent block, and the information of all the 8 × 8 pixels is reflected. There is. Therefore, the two-dimensional D of the adjacent block
The DC component of the CT coefficient includes information on a pixel having a high correlation with the pixel of interest, but may also include information on a pixel having no correlation with the pixel of interest. In this way, when the prediction tap is configured so as to include the information of the pixel having no correlation (or almost no correlation) with the pixel of interest, the product-sum operation unit 133 (FIG. 42).
This adversely affects the decoding accuracy of the decoded value of the pixel of interest obtained in.

Therefore, as shown in FIG. 55, the prediction tap may be configured not by selecting all the pixels of the adjacent block but by selecting only the pixels close (or adjacent) to the target block. it can. In this case, the decoding accuracy can be improved by the prediction taps having a small number of taps.

55A and 55B, the left adjacent same row pixel, the right adjacent same row pixel, the upper adjacent same column pixel, the lower adjacent same column pixel, and the pixel adjacent to the left of the left adjacent same row pixel shown in FIG. 55A and FIG. 55B. , A pixel adjacent to the right of the pixel adjacent to the right adjacent same row, a pixel adjacent to the upper adjacent same column pixel, and a pixel adjacent below the lower adjacent same column pixel are often highly correlated with the pixel of interest, Therefore, configuring the prediction taps including these is for improving the decoding accuracy of the pixel of interest, that is, for decoding a high-quality image that is closer to the original image and has various distortions sufficiently reduced. , Basically effective.

Also, for example, when a pixel near the block boundary such as the left end of the target block is set as the target pixel, the target pixel at the left end of the target block is correlated with the pixel of the adjacent block adjacent to the left. Is large, but there may be little correlation with the pixels of the adjacent block adjacent to the right. Therefore, as shown in FIG. 55C, of the left-adjacent same-row pixels and the right-adjacent same-row pixels, whichever is closer to the target pixel is selected, and among the upper-adjacent same-column pixels and the lower-adjacent same-column pixels, It is effective to improve the decoding accuracy of the pixels at the block boundary of the target block by selecting the one close to the target pixel to configure the prediction tap. In addition,
For example, even when the vicinity of the target block has a complicated image pattern or has an edge, the pixel adjacent to the same pixel on the left side adjacent to the same row or the right adjacent pixel on the same row, or the upper adjacent same column pixel and the lower adjacent same The column pixel may have little correlation with the pixel of interest. Therefore, even in such a case, configuring the prediction tap as shown in FIG. 55C is effective in improving the decoding accuracy of the pixel of interest. Is.

On the other hand, when the classification of the pixel of interest is properly performed, the decoding accuracy improves as the amount of information forming the prediction tap increases. Therefore, in such a case, the prediction tap shown in FIG. 55D is effective.

Here, the prediction taps are shown in FIG. 55A to FIG.
Although it is possible to fix it to any one of the tap structures shown in FIG. 5D, it is also possible to selectively adopt one of the four tap structures shown in FIGS. 55A to 55D. That is, the prediction tap is based on the pixel position mode of the pixel of interest, the presence / absence of an edge near the pixel of interest, and the complexity of the image near the pixel of interest, and the like, and the four tap structures shown in FIGS. 55A to 55D. It is possible to selectively adopt the ones.

Next, with reference to the flowchart in FIG. 56, the processing performed by the prediction tap generation unit 131 in FIG. 54 in step S101 in FIG. 47 will be described.

In the prediction tap generation unit 131, first, in step S131, the reading unit 136
The two-dimensional DCT coefficients of the block of interest and the four adjacent blocks adjacent to each of the upper, lower, left and right sides are read from the buffer memory 12 (FIG. 2). Furthermore, step S131
Then, the reading unit 136 determines that the two-dimensional DC of the target block is
The T coefficient is supplied to the array unit 137, and the two-dimensional DCT coefficients of the four adjacent blocks are supplied to the inverse DCT transform unit 22.
1, and the process proceeds to step S132.

In step S132, the inverse DCT transform unit 2
21 performs the inverse DCT conversion on the four adjacent blocks supplied from the reading unit 136, thereby decoding the pixel values of the four adjacent blocks and supplying the decoded pixel values to the selection unit 222.
It proceeds to step S133.

In step S133, the selection unit 222
The 8 × 8 pixels of the target block are sequentially set as target pixels,
For each pixel of interest, from the pixel values of the 8 × 8 pixels of the four adjacent blocks supplied from the inverse DCT conversion unit 221,
As described with reference to FIG. 55, the pixel value of a predetermined pixel is selected, supplied to the array unit 137, and the process proceeds to step S134.

In step S134, the array unit 137
For each 8 × 8 pixel of the target block, the two-dimensional DCT coefficient of the target block supplied from the reading unit 136 and the pixel value of the pixel supplied from the selection unit 222 are
The prediction taps shown in FIG. 55 are formed by arranging them in a predetermined order, and the prediction taps are supplied to the product-sum operation unit 133 (FIG. 42), and the process is terminated.

Next, the prediction tap generation unit 131 of FIG. 54 generates a prediction tap including a pixel value of a pixel adjacent to the target block (block adjacent pixel) in the adjacent block. It must be spatially adjacent to the block of interest.

However, in MPEG2, as described above, since it is possible to select the frame structure and the field structure in macroblock units, the structure of the macroblock of interest including the block of interest and the adjacency adjacent to the macroblock of interest. Depending on the structure of the macroblock, the pixel adjacent to the target block in the adjacent block adjacent to the target block may not be the pixel (block adjacent pixel) adjacent to the boundary of the target block in the spatial region.

Then, the reading unit 136 (FIG. 4) recognizes the structure of the target block and the adjacent block by referring to the DCT type stored in the buffer memory 12, and uses the structure of the target block as a reference to determine the spatial region. 2, the two-dimensional DCT coefficient of the block having the block adjacent pixel adjacent to the boundary of the target block is read from the buffer memory 12. Therefore, in the above-mentioned case, for simplification of description, it is described that the reading unit 136 in FIG. 54 reads the two-dimensional DCT coefficients of four blocks that are adjacent to the target block in the upper, lower, left, and right directions. The reading unit 136 reads out the two-dimensional DCT coefficients of blocks other than the above-mentioned four blocks, if necessary, in order to obtain pixels adjacent to the target block (block adjacent pixels) in the spatial domain. There is.

57 to 72, block adjacent pixels adjacent to the boundary of the target block in the spatial region will be described with reference to the structure of the target block.

Note that in FIGS. 57 to 72, the MB of interest is assigned to the MB as in the case of FIGS. 8 to 23.
_The macroblocks _N and the macroblocks MB _N adjacent to each other in the vertical and horizontal directions are denoted as MB _U , MB _D , MB _L , and MB _R , respectively. Further, the upper left, lower left, upper right, and lower right blocks of the macro block MB _N of interest are respectively B _NUL ,
B _NDL , B _NUR , and B _NDR, and the upper adjacent macroblock M
The upper left, lower left, upper right, and lower right blocks of B _U are represented as B _UUL , B _UDL , B _UUR , and B _UDR , respectively. Also, the upper left of the lower adjacent macroblocks MB _D, lower left, upper right, a block in the lower right, _{respectively, B DUL, B DDL, B} DUR, B DDR and represents the upper left of the left neighboring macroblock MB _L, lower left, upper right,
The blocks at the lower right are B _LUL , B _LDL , B _LUR , and
It is expressed as B _LDR . Further, the upper left, lower left, upper right, and lower right blocks of the right adjacent macroblock MB _R are respectively
_Represented by B _RUL , B _RDL , B _RUR , and B _RDR .

Further, in FIGS. 57 to 72, the shaded lines represent block adjacent pixels, and the black blocks represent the target block.

When the structure of the target block is used as a reference,
In the spatial area, the block adjacent pixels adjacent to the boundary of the block of interest are the same as those in the case of FIGS. 8 to 23 described for the adjacent one-dimensional DCT coefficient.

That is, FIG. 57 shows the macroblock MB of interest.
_When the block B _{NUL at} the upper left of _N is the block of interest, the block adjacent pixels adjacent to the upper boundary of the block of interest B _NUL are shown in the spatial region.

[0673] The DCT type of the target macroblock MB _N and the upper adjacent macroblock MB _U are both frame D
In the case of CT, the block adjacent pixel adjacent to the upper boundary of the target block B _NUL of the frame structure has an upper adjacent macroblock MB _U of the frame structure as shown in FIG. 57A.
8 pixels in the bottom row of the block B _{UDL at} the lower left of (the shaded portion).

The DCT type of the macro block MB _N of interest is the frame DCT, and the upper adjacent macro block MB _N
When the DCT type of _U is the field DCT, the block adjacent pixel adjacent to the upper boundary of the target block B _NUL of the frame structure is located at the lower left of the upper adjacent macroblock MB _U of the field structure as shown in FIG. 57B. It is the bottom 8 pixels of the block B _UDL (hatched portion).

The DCT type of the macroblock MB _N of interest is the field DCT, and the upper adjacent macroblock M
When the DCT type of B _U is the frame DCT, the block adjacent pixel adjacent to the upper boundary of the target block B _NUL of the field structure is, as shown in FIG. 57C, the lower left corner of the upper adjacent macroblock MB _U of the frame structure. Block B
_It is 8 pixels in the 7th line of _UDL (7th line from the top).

The DCT type of the target macroblock MB _N is the field DCT, and the upper adjacent macroblock M
When the DCT type of B _U is also the field DCT, the block adjacent pixel adjacent to the upper boundary of the target block B _NUL of the field structure is, as shown in FIG. 57D, the upper left corner of the upper adjacent macroblock MB _U of the field structure. 8 pixels in the bottom row of block B _UUL (hatched area)
Becomes

Next, FIG. 58 shows the macroblock MB of interest.
_When the block B _{NUL at} the upper left of _N is the block of interest, the block adjacent pixels adjacent to the lower boundary of the block of interest B _NUL are shown in the spatial region.

When the DCT type of the macro block MB _N of interest is the frame DCT, the block adjacent pixels adjacent to the lower boundary of the block B _NUL of interest of the frame structure are the pixel of interest of the frame structure as shown in FIG. 58A. It is the uppermost 8 pixels (hatched portion) of the block B _{NDL at} the lower left of the macro block MB _N.

When the DCT type of the macro block MB _N of interest is the field DCT, the block adjacent pixels adjacent to the lower boundary of the block B _NUL of interest of the field structure are of the field structure as shown in FIG. 58B. It is the uppermost 8 pixels (hatched portion) of the upper left block B _DUL of the lower adjacent macroblock MB _D that is below the macroblock MB _N of interest.

Next, FIG. 59 shows the macroblock MB of interest.
_When the block B _{NUL at} the upper left of _N is the block of interest, the block adjacent pixels adjacent to the left boundary of the block of interest B _NUL are shown in the spatial region.

[0681] The DCT types of the target macroblock MB _N and the left adjacent macroblock MB _U are both frame D
In the case of CT, the block adjacent pixel adjacent to the left boundary of the target block B _NUL of the frame structure is the left adjacent macroblock MB _L of the frame structure as shown in FIG. 59A.
8 pixels in the rightmost column of the block B _{LUR at} the upper right of (the shaded portion).

[0682] The DCT type of the macro block MB _N of interest is the frame DCT, and the left adjacent macro block MB
_When the DCT type of _L is the field DCT, the block adjacent pixel adjacent to the left boundary of the target block B _NUL of the frame structure is at the upper right of the left adjacent macroblock MB _L of the field structure as shown in FIG. 59B. The upper 4 pixels of the rightmost column in the block B _LUR and the upper 4 pixels of the rightmost column in the block B _{LDR at the} lower right (hatched portion). That is, the left block adjacent pixels adjacent to the first row to the eighth row of the block of interest B _NUL are the pixels on the first row to the fourth row in the rightmost column of the block B _LUR and the rightmost pixel of the block B _LDR , respectively. The pixels are the first to fourth rows of the column.

Macro block MB of interest_NDCT type
Is the field DCT and the left adjacent macroblock M
B_LIf the DCT type of is frame DCT,
Block B of the field structure_NULAdjacent to the left border of
The adjacent pixels in the block are
Left adjacent macroblock MB with frame structure_LUpper right block
B_LUR4 pixels in the odd row of the rightmost column in
Block B_LDR4 pixels in odd rows in (marked with diagonal lines
There is a part). That is, attention block B_NULFirst row of
To the block adjacent pixel on the left side adjacent to the eighth row is
Block B _LURIn the rightmost column of rows 1, 3, 5, and 7
Pixel, block B_LDRIn the rightmost column of rows 1, 3, 5, and 7
It becomes a pixel.

The DCT type of the target macroblock MB _N is the field DCT, and the left adjacent macroblock M
When the DCT type of _BL is also the field DCT, the block adjacent pixel adjacent to the left boundary of the target block B _NUL of the field structure is the upper right of the left adjacent macroblock MB _L of the field structure as shown in FIG. 59D. 8 pixels in the rightmost column of block B _LUR (hatched area)
Becomes

Next, FIG. 60 shows the macroblock MB of interest.
_When the block B _{NUL at} the upper left of _N is the block of interest, the block adjacent pixels adjacent to the right boundary of the block B _NUL of interest are shown in the spatial region.

When the block B _{NUL at} the upper left of the macro block MB _N of interest is the block of interest, the block adjacent pixel adjacent to the right boundary of the block of interest B _NUL is adjacent to its right side as shown in FIG. 8 pixels in the leftmost column of the block B _NUR (hatched portion).

Next, FIG. 61 shows the macroblock MB of interest.
_When the block B _{NDL at} the lower left of _N is the block of interest, the block adjacent pixels adjacent to the upper boundary of the block of interest B _NDL are shown in the spatial domain.

When the DCT type of the target macroblock MB _N is the frame DCT, the block adjacent pixel adjacent to the upper boundary of the target block B _{NDL is} the upper left block of the target macroblock MB _N as shown in FIG. 61A. B
_{It will be} the bottom 8 pixels of the _NUL (the shaded area).

When the DCT type of the macro block MB _N of interest is the field DCT, the block B of interest
Block adjacent pixels adjacent to the upper boundary of the _NDL are shown in FIG.
As shown in FIG. 1B, the lower left block B of the upper adjacent macroblock MB _{U which} is adjacent above the macroblock MB _N of interest.
_{It is} the bottom 8 rows of _UDL (hatched area).

Next, FIG. 62 shows the macroblock MB of interest.
_When the block B _{NDL at} the lower left of _N is the block of interest, the block adjacent pixels adjacent to the lower boundary of the block of interest B _NDL are shown in the spatial region.

[0692] The DCT types of the target macroblock MB _N and the lower adjacent macroblock MB _D are both frame D
In the case of CT, the block adjacent pixels adjacent to the lower boundary of the target block B _NDL are, as shown in FIG. 62A, the 8 pixels in the uppermost row of the upper left block B _DUL of the lower adjacent macroblock MB _D (hatched lines It will be the part attached).

The DCT type of the macro block MB _N of interest is the frame DCT, and the lower adjacent macro block MB _N
When the DCT type of _D is the field DCT, the block adjacent pixel adjacent to the lower boundary of the target block B _NDL is the lower adjacent macroblock M as shown in FIG. 62B.
It is the uppermost 8 pixels (hatched portion) of the block B _{DUL on} the upper left of B _D.

The DCT type of the macroblock MB _N of interest is the field DCT, and the lower adjacent macroblock M
When the DCT type of B _D is the frame DCT, the block adjacent pixel adjacent to the lower boundary of the target block B _NDL is, as shown in FIG. 62C, the lower adjacent macroblock M.
It becomes 8 pixels (hatched portion) in the second row of the block B _{DUL on} the upper left of B _D.

The DCT type of the target macroblock MB _N is the field DCT, and the lower adjacent macroblock M
In the case of the DCT type of B _D as well, in the case of the field DCT, the block adjacent pixels adjacent to the lower boundary of the target block B _NDL are the 8 pixels in the uppermost row of the lower left block B _DDL of the lower adjacent macroblock MB _D (hatched line The part marked with).

Next, FIG. 63 shows the macroblock MB of interest.
_When the block B _{NDL at} the lower left of _N is the block of interest, the block adjacent pixels adjacent to the left boundary of the block of interest B _NDL are shown in the spatial region.

[0696] The DCT type of the target macroblock MB _N and the left adjacent macroblock MB _L are both frame D
In the case of CT, the block adjacent pixel adjacent to the left boundary of the block of interest B _NDL is, as shown in FIG. 63A, 8 pixels in the rightmost column of the block B _{LDR at} the lower right of the left adjacent macroblock MB _L (hatched line). The part marked with).

[0697] The DCT type of the macro block MB _N of interest is the frame DCT, and the left adjacent macro block MB
_When the DCT type of _L is the field DCT, the block adjacent pixel adjacent to the left boundary of the target block B _NDL is the left adjacent macroblock M as shown in FIG. 63B.
And the lower 4 pixels of the rightmost column in the top right corner of the block B _LUR of B _L, the lower 4 pixels of the rightmost column (portion that is hatched) in the block B _LDR of the lower right. That is, the left block adjacent pixels adjacent to the first row to the eighth row of the block of interest B _NDL are respectively the fifth _pixels in the rightmost column of the block B _LUR .
The pixels in the rows to the eighth row and the pixels in the fifth row to the eighth row in the rightmost column of the block B _LDR .

The DCT type of the target macroblock MB _N is the field DCT, and the left adjacent macroblock M
When the DCT type of B _L is the frame DCT, the block adjacent pixel adjacent to the left boundary of the block of interest B _NDL is the left adjacent macroblock M as shown in FIG. 63C.
4 of the even rows of the rightmost column in the top right corner of the block B _LUR of B _L
Pixel and 4 in the even row in the block B _{LDR at the} lower right of the pixel
It becomes a pixel (hatched part). That is, the left block adjacent pixels adjacent to the first row to the eighth row of the block of interest B _NDL are the pixels of the second, fourth, sixth, and eighth rows in the rightmost column of the block B _LUR and the blocks of the block B _LDR , respectively. The pixels on the rightmost column are the second, fourth, sixth, and eighth rows.

The DCT type of the target macroblock MB _N is the field DCT, and the left adjacent macroblock M
When the DCT type of B _L is also the field DCT, the block adjacent pixel adjacent to the left boundary of the target block B _NDL is, as shown in FIG. 63D, the block B _{LDR at} the lower right of the left adjacent macroblock MB _L. There are 8 pixels in the rightmost column (hatched portion).

Next, FIG. 64 shows the macroblock MB of interest.
_When the block B _{NDL at} the lower left of _N is the block of interest, the block adjacent pixels adjacent to the right boundary of the block of interest B _NDL are shown in the spatial region.

When the lower left block B _NDL of the target macro block MB _N is the target block, the block adjacent pixel adjacent to the right boundary of the target block B _NDR is
As shown in FIG. 64, the 8 pixels in the leftmost column of the block B _{NDR to} the right of the block of interest B _NDL (hatched portion)
Becomes

FIG. 65 shows block adjacent pixels adjacent to the upper boundary of the block of interest B _NUR in the spatial region when the block B _{NUR at} the upper right of the block of macro block MB _N of interest is the block of interest. .

The DCT type of the target macroblock MB _N and the upper adjacent macroblock MB _U are both frame D
In the case of CT, the block adjacent pixels adjacent to the upper boundary of the target block B _NUR are, as shown in FIG. 65A, the 8 pixels in the bottom row of the block B _{UDR at} the lower right of the upper adjacent macro block MB _U (shaded lines). It will be the part attached).

The DCT type of the macro block MB _N of interest is the frame DCT, and the upper adjacent macro block MB _N
When the DCT type of _U is the field DCT, the block adjacent pixels adjacent to the upper boundary of the target block B _NUR are the upper adjacent macroblock M as shown in FIG. 65B.
The lower right block B _UDR of B _U is the lowermost row of 8 pixels (hatched portion).

The DCT type of the target macroblock MB _N is the field DCT, and the upper adjacent macroblock M
DCT type B _U is the case of frame DCT, block adjacent pixel adjacent to the upper boundary of the target block B _NUR, as shown in FIG. 65C, upper adjacent macroblock M
It is 8 pixels (hatched portion) in the seventh row of the block B _UDR at the lower right of B _U.

The DCT type of the macro block MB _N of interest is the field DCT, and the upper adjacent macro block M
When the DCT type of B _U is also the field DCT, the block adjacent pixel adjacent to the upper boundary of the target block B _NUR is the highest block B _UUR of the upper right macro block MB _U , as shown in FIG. 65D. There are 8 pixels in the lower row (hatched portions).

Next, FIG. 66 shows the macroblock MB of interest.
_When the block B _{NUR at} the upper right of _N is the block of interest, the block adjacent pixels adjacent to the lower boundary of the block of interest B _NUR in the spatial region are shown.

When the DCT type of the target macroblock MB _N is the frame DCT, the block adjacent pixel adjacent to the lower boundary of the target block B _NUR is the right side of the target macroblock MB _N as shown in FIG. 66A. The uppermost 8 pixels of the lower block B _NDR (hatched portion).

When the DCT type of the macro block MB _N of interest is the field DCT, the block B of interest
Block adjacent pixels adjacent to the lower boundary of the _NUR are shown in FIG.
As shown in FIG. 6B, the uppermost 8 pixels of the block B _{DUR at} the upper right of the lower adjacent macroblock MB _D (the hatched portion).

Next, FIG. 67 shows the macroblock MB of interest.
_When the block B _{NUR at} the upper right of _N is the block of interest, the block adjacent pixels adjacent to the left boundary of the block of interest B _NUR are shown in the spatial region.

When the upper right block B _{NUR of the target} macro block MB _N is the target block, the block adjacent pixels adjacent to the left boundary of the target block B _NUR are the target block B _NUR as shown in FIG. The block B _{NUL on} the left side of the _NUR has 8 pixels in the rightmost column (hatched portion).

Next, FIG. 68 shows the macroblock MB of interest.
_When the block B _{NUR at} the upper right of _N is the block of interest, the block adjacent pixels adjacent to the boundary on the right side of the block of interest B _NUR in the spatial region are shown.

[0713] The DCT types of the target macroblock MB _N and the right adjacent macroblock MB _R are both frame D
In the case of CT, the block adjacent pixels adjacent to the right boundary of the target block B _NUR are, as shown in FIG. 68A, 8 pixels in the leftmost column of the block B _{RUL in} the upper left of the right adjacent macroblock MB _R (shaded lines It will be the part attached).

[0714] The DCT type of the macro block MB _N of interest is the frame DCT, and the right adjacent macro block MB _N
When the DCT type of _R is the field DCT, the block adjacent pixel adjacent to the right boundary of the target block B _NUR is the right adjacent macroblock M as shown in FIG. 68B.
And four pixels on the leftmost column in the upper left block B _RUL of B _L, the four pixels on the leftmost column (the portion that is shaded) in the block B _RDL of the lower left.

The DCT type of the target macroblock MB _N is the field DCT, and the right adjacent macroblock M
When the DCT type of B _R is the frame DCT, the block adjacent pixel adjacent to the right boundary of the block of interest B _NUR is the right adjacent macroblock M as shown in FIG. 68C.
4 in the leftmost odd-numbered row in the block B _RUL in the upper left of B _R
Pixel and 4 in odd row in block B _{RDL at} the lower left of the pixel
It becomes a pixel (hatched part).

The DCT type of the target macroblock MB _N is the field DCT, and the right adjacent macroblock M
In the case of the DCT type of B _R as well, in the case of the field DCT, the block adjacent pixel adjacent to the right boundary of the block of interest B _NUR is the highest block B _RUL of the upper left block of the right adjacent macroblock MB _R as shown in FIG. 68D. There are 8 pixels in the left column (hatched portion).

Next, FIG. 69 shows the macroblock MB of interest.
_When the block B _{NDR at the} lower right of _N is the block of interest, the block adjacent pixels adjacent to the upper boundary of the block of interest B _NDR are shown in the spatial region.

When the DCT type of the target macroblock MB _N is the frame DCT, the block adjacent pixel adjacent to the upper boundary of the target block B _NDR is the lower right corner of the target macroblock MB _N as shown in FIG. 69A. Of the block B _{NDR in} the top row of 8 pixels (hatched portion).

When the DCT type of the macro block MB _N of interest is the field DCT, the block B of interest
Block adjacent pixels adjacent to the upper boundary of the _NDR are shown in FIG.
As shown in FIG. 9B, the lower right block B of the upper adjacent macroblock MB _{U which} is adjacent above the macroblock MB _N of interest.
_{It is} the bottom 8 rows of _UDR (hatched area).

Next, FIG. 70 shows the macroblock MB of interest.
_When the block B _{NDR at the} lower right of _N is the block of interest, the block adjacent pixels adjacent to the lower boundary of the block of interest B _NDR are shown in the spatial region.

[0721] The DCT type of the target macroblock MB _N and the lower adjacent macroblock MB _D are both frame D
In the case of CT, the block adjacent pixels adjacent to the lower boundary of the block of interest B _NDR are, as shown in FIG. 70A, the 8 pixels in the uppermost row of the block B _{DUR at} the upper right of the lower adjacent macroblock MB _D (hatched lines It will be the part attached).

The DCT type of the target macroblock MB _N is the frame DCT, and the lower adjacent macroblock MB
_When the DCT type of _D is the field DCT, the block adjacent pixel adjacent to the lower boundary of the target block B _NDR is the lower adjacent macroblock M as shown in FIG. 70B.
The uppermost 8 pixels of the block B _{DUR on} the upper right of B _D (the shaded portion).

The DCT type of the target macroblock MB _N is the field DCT, and the lower adjacent macroblock M
When the DCT type of B _D is the frame DCT, the block adjacent pixel adjacent to the lower boundary of the block of interest B _NDR is the lower adjacent macroblock M as shown in FIG. 70C.
This is 8 pixels (hatched portion) in the second row of the block B _{DUR on} the upper right of B _D.

The DCT type of the macroblock MB _N of interest is the field DCT, and the lower adjacent macroblock M
When the DCT type of B _D is also the field DCT, the block adjacent pixel adjacent to the lower side of the target block B _NDR is the lower adjacent macroblock MB as shown in FIG. 70D.
_{It is} the uppermost 8 pixels of the block B _{DDR at} the lower right of _D (hatched portion).

Next, FIG. 71 shows the macroblock MB of interest.
_When the block B _{NDR at the} lower right of _N is the block of interest, the block adjacent pixels adjacent to the left boundary of the block of interest B _NDR are shown in the spatial region.

When the lower right block B _{NDR of the} target macro block MB _N is the target block, the block adjacent pixel adjacent to the left boundary of the target block B _{NDR is} the target block as shown in FIG. It becomes 8 pixels (hatched portion) in the rightmost column of the block B _{NDL on} the left side of B _NDR .

Next, FIG. 72 shows the macroblock MB of interest.
_When the block B _{NDR at the} lower right of _N is the block of interest, the block adjacent pixels adjacent to the right boundary of the block of interest B _NDR are shown in the spatial region.

[0728] The DCT types of the target macroblock MB _N and the right adjacent macroblock MB _R are both frame D
In the case of CT, the block adjacent pixels adjacent to the right boundary of the block of interest B _NDR are, as shown in FIG. 72A, 8 pixels in the leftmost column of the lower left block B _RDL of the right adjacent macroblock MB _R (hatched lines It will be the part attached).

The DCT type of the macro block MB _N of interest is the frame DCT, and the right adjacent macro block MB _N
When the DCT type of _R is the field DCT, the block adjacent pixel adjacent to the right boundary of the target block B _NDR is the right adjacent macroblock M as shown in FIG. 72B.
It is the lower 4 pixels of the leftmost column in the upper left block B _RUL of B _R and the lower 4 pixels of the leftmost column in the lower left block B _RDL thereof (hatched portion).

The DCT type of the macroblock MB _N of interest is the field DCT, and the right adjacent macroblock M
When the DCT type of B _R is the frame DCT, the block adjacent pixel adjacent to the right boundary of the block of interest B _NDR is the right adjacent macroblock M as shown in FIG. 72C.
4 in the leftmost even-numbered row in the block B _RUL in the upper left of B _R
Pixels and 4 pixels in the even-numbered row in the leftmost column in the block B _{RDL at} the lower left (hatched portion).

The DCT type of the macro block MB _N of interest is the field DCT, and the right adjacent macro block M
When the DCT type of BR is also the field DCT, the block adjacent pixel adjacent to the right boundary of the target block B _NDR is, as shown in FIG. 72D, the leftmost block of the lower left block B _RDL of the right adjacent macroblock MB _R. There are 8 pixels in the column (the shaded portion).

Next, FIG. 73 shows a third configuration example of the prediction tap generation unit 131 of FIG. Note that, in the figure, parts corresponding to those in FIG. 54 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, in the embodiment of FIG. 73, the inverse DCT transform unit 2
54, except that an MPEG decoder 231 and a buffer memory 232 are provided instead of 21.
The configuration is the same as in the case of.

The MPEG decoder 231 is, for example, as shown in FIG.
It has the same structure as the MPEG decoder shown in FIG. Further, in the embodiment of FIG. 73, the video stream supplied to the image processing apparatus of FIG.
MPEG decoder 2
The MPEG 31 MPEG-decodes the video stream supplied thereto as described in FIG. 1, and supplies the decoded image obtained as a result to the buffer memory 232.

The buffer memory 232 temporarily stores the decoded image supplied from the MPEG decoder 231. It should be noted that the buffer memory 232 stores the pixel of interest in FIG.
It has at least a storage capacity capable of storing a decoded image including pixels forming the prediction tap described in 5.

[0735] In the predictive tap generation unit 131 configured as described above, the MPEG decoder 231 MPEG-decodes the video stream supplied thereto,
The decoded image obtained as a result is supplied to and stored in the buffer memory 232.

Then, the selecting unit 222 selects the pixel forming the prediction tap described in FIG. 55 from the pixels forming the decoded image stored in the buffer memory 232, and sets the pixel value of the pixel to the buffer memory. The data is read from 232 and supplied to the array unit 137.

Therefore, also in the embodiment of FIG. 73,
The same prediction tap as in the case of FIG. 54 is generated.

Note that the decoding order in the MPEG decoder 231 does not necessarily match the display order. That is, MP
The decoding order in the EG decoder 231 is, for example, GO
It changes depending on the P (Group Of Picture) structure. Therefore, the MPEG decoder 231 needs to perform MPEG decoding so that the pixels that can be selected by the selection unit 222 are in the state stored in the buffer memory 232.

Next, FIG. 74 shows a fourth configuration example of the prediction tap generation unit 131 of FIG. In the figure, parts corresponding to those in FIG. 73 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the embodiment of FIG. 74 is basically configured in the same manner as in the case of FIG. 73 except that the image processing device 240 is provided instead of the MPEG decoder 231.

The image processing device 240 is the preprocessing unit 1 of FIG.
1, a buffer memory 12, a class classification unit 13, a tap coefficient storage unit 14, and an image reconstruction unit 15, respectively, a preprocessing unit 241, a buffer memory 242, a class classification unit 243, a tap coefficient storage unit 244, and an image. It is composed of the reconstructing unit 245. Further, in the embodiment of FIG. 74, as in the case of FIG. 73, the video stream supplied to the image processing apparatus of FIG.
Image processing device 240.
Decodes the video stream supplied thereto by processing in the same manner as in the image processing apparatus of FIG. 2, and the decoded image obtained as a result is buffer memory 23.
Supply to 2.

That is, in the embodiment of FIG. 73, MPEG
By the decoder 231, the video stream is MPEG
Although it was designed to be decoded, in the embodiment of FIG. 74, the video stream is adapted to be decoded by the image processing device 240.

Therefore, also in the embodiment of FIG. 74,
The same prediction tap as in the case of FIG. 54 is generated.

Here, the image reconstructing unit 245 in the image processing device 240 has an adaptive processing unit 251 which performs the same processing as the adaptive processing unit 51 of the image reconstructing unit 15 of FIG. The processing unit 251 is configured to generate a prediction tap in the same manner as, for example, the case of the prediction tap generation unit 131 of FIG. 43 (or FIG. 54 or FIG. 73) described above.

When the image processing device 240 decodes the video stream, the non-intra-coded block has the difference value (residual image) between the image of the block and the predicted image as described above. Two-dimensional DCT coefficient obtained by three-dimensional DCT transformation (hereinafter, as appropriate, residual DCT
Therefore, in the image processing device 240, as in the image processing device of FIG. 2, for the non-intra coded block, as shown in FIG. 75A, the residual DCT coefficient and the predicted image 2 obtained by DCT transform of
A two-dimensional DCT coefficient is added, and a prediction tap is formed from the two-dimensional DCT coefficient obtained as a result of the addition. Then, an image is decoded by performing an adaptive process using the prediction tap.

However, in the image processing device 240, for the non-intra-coded blocks, other than the residual image obtained by two-dimensional inverse DCT transform of the residual DCT coefficient, for example, as shown in FIG. 75B. , It is possible to decode the image by adding the predicted image and the image obtained as a result of the addition to configure a prediction tap and perform adaptive processing. In addition, the image processing device 240
Then, for non-intra coded blocks,
For example, as shown in FIG. 75C, it is possible to decode the image by forming a prediction tap from the residual DCT coefficient and the prediction image and performing an adaptive process.

The prediction tap shown in FIG.
When decoding an image, it is necessary to configure a prediction tap having the same structure as that in the case of FIG. 75A and obtain tap coefficients in learning. When the prediction tap of FIG. 75B or 75C is configured to decode an image, similarly, it is necessary to configure the prediction tap having the same structure as that in FIG. 75B or 75C to obtain the tap coefficient. is there.

Next, FIG. 76 shows a fifth configuration example of the prediction tap generation section 131 of FIG. Note that, in the figure, parts corresponding to those in FIG. 43 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, in the embodiment of FIG. 76, the one-dimensional inverse DCT transformation unit 261 and the adjacent one-dimensional DCT coefficient selection / transformation unit 2
Basically, the configuration is the same as in the case of FIG. 43, except that 62 is newly provided.

The one-dimensional inverse DCT transform unit 261 has the same structure as the one in FIG.
The two-dimensional DCT coefficient of the block, which is configured similarly to the three-dimensional inverse DCT transformation unit 31 and is supplied from the reading unit 136,
By performing the one-dimensional inverse DCT transform, one-dimensional DCT coefficients (horizontal one-dimensional DCT coefficient and vertical one-dimensional DCT coefficient) are obtained and supplied to the adjacent one-dimensional DCT coefficient selecting / transforming unit 262.

Adjacent one-dimensional DCT coefficient selecting / converting section 262
Is configured similarly to the adjacent one-dimensional DCT coefficient selecting / converting unit 32 of FIG. 2, and based on the DCT type of the block (macro block) supplied from the reading unit 136, the one-dimensional DCT of the pixel column adjacent to the target block. The coefficient (adjacent one-dimensional DCT coefficient) is acquired from the one-dimensional DCT coefficient supplied from the one-dimensional inverse DCT transform unit 261, and is supplied to the array unit 137.

Next, with reference to the flowchart in FIG. 77, the processing performed by the prediction tap generation unit 131 in FIG. 76 in step S101 in FIG. 47 will be described.

In the predictive tap generation unit 131, first, in step S141, the read unit 136
The buffer memory 12 stores the two-dimensional DCT coefficients of the block of interest and four adjacent macroblocks that are adjacent to the macroblock of interest including the block of interest.
(FIG. 2), the DCT type of each macro block is read from the buffer memory 12. Further, in step S141, the reading unit 136
, The two-dimensional DCT coefficient of the block of interest,
And the two-dimensional DCT coefficients of four adjacent macroblocks are supplied to the one-dimensional inverse DCT transform unit 261. Further, in step S141, the reading unit 136
Supplies the DCT type read from the buffer memory 12 to the adjacent one-dimensional DCT coefficient selecting / converting unit 262, and proceeds to step S142.

At step S142, the one-dimensional inverse DCT transform unit 261 transforms the blocks constituting the four adjacent macroblocks supplied from the read unit 136 into the one-dimensional inverse DCT.
The one-dimensional DCT coefficient obtained as a result of the T-transform is supplied to the adjacent one-dimensional DCT coefficient selecting / converting unit 262, and the process proceeds to step S143.

In step S143, the adjacent one-dimensional DCT
Like the adjacent one-dimensional DCT coefficient selecting / converting unit 32 of FIG. 2, the coefficient selecting / converting unit 262 is based on the DCT type supplied from the reading unit 136 and is adjacent to the target block described in FIGS. 8 to 23. The one-dimensional DCT coefficient (adjacent one-dimensional DCT coefficient) of the pixel row to be acquired is acquired from the one-dimensional DCT coefficient supplied from the one-dimensional inverse DCT conversion unit 261, and is supplied to the array unit 137.

[0754] Then, proceeding to step S144, the array section 137 selects the two-dimensional DCT coefficient of the target block supplied from the reading section 136 and the adjacent one-dimensional DCT coefficient as prediction taps for each 8x8 pixel of the target block. The adjacent one-dimensional DCT coefficients supplied from the / conversion unit 262 are arranged in a predetermined order to form a prediction tap, and the prediction taps are supplied to the product-sum operation unit 133 (FIG. 42), and the process ends. .

Therefore, the prediction tap generation unit 131 of FIG.
According to FIG. 78, as shown in FIG.
Two-dimensional DCT coefficient and adjacent one-dimensional DCT coefficients adjacent to the target block in the upper, lower, left, and right directions (upper adjacent one-dimensional DCT coefficient, lower adjacent one-dimensional DCT coefficient, left adjacent one-dimensional D
A prediction tap composed of a total of 96 (= 8 × 8 + 8 × 4) taps including the CT coefficient and the right adjacent one-dimensional DCT coefficient) is configured.

In the taps of the tap structure shown in FIG. 78, the upper adjacent one-dimensional DCT coefficient, the lower adjacent one-dimensional DCT coefficient, the left adjacent one-dimensional DCT coefficient, and the right adjacent one-dimensional DCT coefficient are the upper, lower, and left of the target block. , From the fact that it represents the spatial frequency components in the horizontal direction or the vertical direction of the eight block adjacent pixels of each of the adjacent blocks adjacent to the right, that is, from the spatial frequency of the position close to the pixel of interest.
Also in the prediction tap generation unit 131 of FIG.
Compared with the case of 5, the predicted taps with a small number of taps are generated, which includes information having a large correlation with the pixel of interest.

The one-dimensional inverse DCT transform unit 261 and the adjacent one-dimensional DCT coefficient selecting / transforming unit 262 are also used by the one-dimensional inverse DCT transforming unit 31 and the adjacent one-dimensional DCT coefficient selecting / transforming unit 32 in FIG. It is possible to do so.

In the above case, the one-dimensional DCT coefficient to be used as the prediction tap is obtained by performing the one-dimensional inverse DCT conversion of the two-dimensional DCT coefficient. , Decodes the bitstream into a decoded image by MPEG decoding,
It is also possible to obtain the decoded image by one-dimensional DCT conversion.

Next, FIG. 79 shows a sixth configuration example of the prediction tap generation unit 131 of FIG. 42. In the figure, parts corresponding to those in FIG. 73 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, in the embodiment of FIG. 79, the predicted tap generation unit 131 is basically configured in the same manner as in the case of FIG. 73.

However, the selection unit 222 uses the buffer memory 2
80 from the pixels forming the decoded image stored in 32.
As shown in, a pixel at the same position as the pixel of interest (hereinafter, appropriately referred to as a pixel of the same position) is selected in each of frames one frame before and one frame after the frame of the target pixel, and the pixel value of the pixel at the same position is selected. , And reads them from the buffer memory 232 and supplies them to the array unit 137.

Next, with reference to the flowchart in FIG. 81, the processing performed by the prediction tap generation unit 131 in FIG. 79 in step S101 in FIG. 47 will be described.

[0762] In the prediction tap generation unit 131, first, in step S151, the reading unit 136
The two-dimensional DCT coefficient of the block of interest is stored in the buffer memory 1
2 (FIG. 2), the data is supplied to the array unit 137, and the process proceeds to step S152.

At step S152, the selection unit 222
A pixel (same position pixel) located at the same position as each pixel of the target block in each of the frame before and after the frame of the target pixel is selected, and the pixel value of the same position pixel is read from the buffer memory 232. Array part 1
Supply to 37.

[0764] Then, proceeding to step S153, the array section 137 sequentially sets the 8x8 pixels of the target block as the target pixel, and for each target pixel, the two-dimensional DCT coefficient of the target block supplied from the reading section 136. , Two same-position pixels for the pixel of interest, which are supplied from the selection unit 222, are arranged in a predetermined order, thereby forming a prediction tap, and supplied to the product-sum calculation unit 133 (FIG. 42). The process ends.

Therefore, the prediction tap generation unit 131 of FIG.
According to the above, for the pixel of interest, a prediction tap consisting of a total of 66 (= 8 × 8 + 2) taps of the 8 × 8 two-dimensional DCT coefficient of the block of interest and two pixels at the same position with respect to the pixel of interest is configured. It

Since the pixels at the same position in frames one frame before and one frame after the frame of the pixel of interest generally have a large correlation with the pixel of interest, the prediction tap generation unit 131 of FIG.
A prediction tap having a small number of taps is generated that includes information having a large correlation with the pixel of interest.

[0767] In addition, in the tap generation unit 131 of FIG.
A prediction tap having a shape including the same position pixel in the frame one frame before and after the frame of the target pixel and the same position pixel in the frame two or more frames before and after the frame of the target pixel is generated. Is also possible.

Further, in the tap generation unit 131 of FIG. 79,
It is also possible to generate a prediction tap in a form that includes pixels at the same position in the frames one frame before and one frame after the frame of the pixel of interest, and pixels (spatially) adjacent to the pixels at the same position. is there.

Next, FIG. 82 shows a seventh configuration example of the prediction tap generation unit 131 of FIG. It should be noted that in the figure, parts corresponding to those in FIG. 79 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, in the embodiment of FIG. 82, the predicted tap generation unit 131 is basically configured similarly to the case in FIG. 79.

However, the MPEG decoder 231 uses the MPE
In addition to the G-decoded decoded image, the motion vector used for the MPEG decoding is also supplied to the buffer memory 232. In addition, the buffer memory 232 is
The decoded image and motion vector output from the MPEG decoder 231 are temporarily stored.

Further, the selecting section 222 selects the pixels constituting the decoded image stored in the buffer memory 232 from the pixels shown in FIG.
As shown in FIG. 3, a pixel at a position moved by a position corresponding to the motion vector of the target block from a position corresponding to the target pixel in a frame before or after the frame of the target block (hereinafter, referred to as a motion-corresponding pixel as appropriate). Is called), and the pixel value of the pixel corresponding to the motion is stored in the buffer memory 2
It is read from 32 and supplied to the array unit 137.

When the target block is an image of a B picture that is bidirectionally predicted, as shown in FIG. 83A, the target block is forward predicted using the frame preceding the frame of the target block as a reference image. And a motion vector for performing backward prediction using a frame after the frame of the target block as a reference image.

Here, in the embodiment of FIG. 83A, the nth
The frame is the frame of the block of interest and is a B picture. The block of interest includes a motion vector v ₁ for performing forward prediction using the n-2nd frame, which is an I picture, as a reference image, and a motion vector v ₁ for performing backward prediction using the n + 1th frame, which is a P picture, as a reference image. have v ₂ .

[0774] In this case, the selecting section 222 selects the motion vector v from the position corresponding to the pixel of interest in the (n-2) th frame.
The pixel at the position moved by ₁ and the pixel at the position moved by the motion vector v ₂ from the position corresponding to the target pixel in the (n + 2) th frame are selected as the motion corresponding pixels.

On the other hand, when the block of interest is an I-picture that is not predictively coded or an image of a P-picture that is forward-predicted, there is no motion vector for prediction, or if there is, a block of interest There is only a motion vector for performing forward prediction using the frame preceding the frame of No. as a reference image.

Here, in the embodiment of FIG. 83B, the nth
The frame is the frame of the block of interest and is a P picture. Then, the block of interest has only the motion vector v ₃ for performing forward prediction using the n-2nd frame which is the I picture as a reference image.

By the way, I picture and P picture are
It can be a reference image when predicting a predicted image of a picture or a B picture. In the embodiment of FIG. 83B, the block of the (n + 1) th frame, which is a B picture, has a motion vector v ₄ for performing forward prediction using the image including the pixel of interest of the frame of the pixel of interest as a reference image.

In this case, the selection unit 222 instructs the selection unit 222 to start the motion vector v
The pixel at the position moved by ₃ and the pixel at the position moved by the motion vector v ₄ from the position corresponding to the target pixel in the (n + 1) th frame can be selected as the motion corresponding pixel.

Here, a frame of an image predicted by a predicted image obtained by motion-compensating a reference image based on a motion vector will be hereinafter referred to as a non-reference frame.
Further, hereinafter, a frame of an image referred to for predicting an image of a non-reference frame will be referred to as a reference frame, as appropriate.
Furthermore, both the motion vector of the block of interest and the motion vector of another block that is used for motion compensation with the image containing the pixel of interest as the reference image are collected,
Hereinafter, it will be referred to as a motion vector related to the block of interest as appropriate.

There may be a plurality of blocks in other frames (non-reference frames) that have a motion vector for performing forward prediction using the image including the pixel of interest as a reference image in the frame of the pixel of interest. It is possible to select the motion corresponding pixel corresponding to each of the one or more motion vectors based on any one or more of the plurality of motion vectors.

Also, in the above-mentioned case, one motion corresponding pixel is selected from the reference frame (or non-reference frame). However, the motion corresponding pixel is the position corresponding to the pixel of interest in the reference frame. From the above, a total of 2 or more pixels of a pixel at a position moved by an amount corresponding to the motion vector of the block of interest and one or more pixels around the pixel,
It is also possible to select it as a motion corresponding pixel. Further, in the selection unit 222, when the motion corresponding pixel for the target pixel is set as the temporary target pixel, the pixel that becomes the motion corresponding pixel for the temporary target pixel is also selected as the motion corresponding pixel for the target pixel. It is possible to do so.

Next, with reference to the flowchart in FIG. 84, the processing performed by the prediction tap generation unit 131 in FIG. 82 in step S101 in FIG. 47 will be described.

In the prediction tap generation unit 131, first, in step S161, the reading unit 136
The two-dimensional DCT coefficient of the block of interest is stored in the buffer memory 1
2 (FIG. 2), the data is supplied to the array unit 137, and the process proceeds to step S162.

[0784] In step S162, the selection unit 222
The motion vector regarding the block of interest is read from the buffer memory 232, and the process proceeds to step S163.

[0785] In step S163, the selection unit 222
In the reference frame or the non-reference frame, a pixel located at a position corresponding to the motion vector related to the target block from the position of each pixel of the target block is selected as the motion corresponding pixel, and the pixel value of the motion corresponding pixel is selected. , The buffer memory 232, and the array unit 13
Supply to 7.

[0786] Then, proceeding to step S164, the array unit 137 sequentially sets the 8x8 pixels of the target block as the target pixel, and for each target pixel, the two-dimensional DCT coefficient of the target block supplied from the reading unit 136. , The motion corresponding pixels supplied from the selection unit 222 are arranged in a predetermined order, thereby forming a prediction tap, supplied to the product-sum calculation unit 133 (FIG. 42), and the processing ends.

Since the motion corresponding pixel generally has a large correlation with the pixel of interest, the prediction tap generation unit 131 of FIG. 82 has a prediction tap of a form including information having such a large correlation with the pixel of interest. , With a small number of taps will be generated.

Here, constructing a prediction tap including a motion corresponding pixel is effective in improving the decoding accuracy of an image in which a moving object is displayed.

Incidentally, in the embodiment of FIGS. 79 and 82,
Although the pixel value of the same position pixel or the motion corresponding pixel is included in the prediction tap, the prediction tap does not include the pixel value of the same position pixel or the motion corresponding pixel but its one-dimensional DCT coefficient. Is possible. In addition, the prediction tap may include one-dimensional DCT coefficients of a predetermined number of pixel rows arranged in the vertical or horizontal direction, including the same-position pixels and pixels corresponding to motion.

Next, the series of processes described above can be performed by hardware or software. When performing a series of processing by software, the program that constitutes the software,
It is installed on a general-purpose computer or the like.

Therefore, FIG. 85 shows a configuration example of an embodiment of a computer in which a program for executing the series of processes described above is installed.

The program is stored in a hard disk 405 or ROM 4 as a recording medium built in the computer.
03 can be recorded in advance.

Alternatively, the program is a flexible disk, CD-ROM (Compact Disc Read Only Memory),
MO (Magneto Optical) disc, DVD (Digital Versatile)
Disc), magnetic disk, semiconductor memory, or other removable recording medium 411 can be stored (recorded) temporarily or permanently. Such removable recording medium 411 can be provided as so-called package software.

The program is installed in the computer from the removable recording medium 411 as described above, and is also wirelessly transferred from the download site to the computer via an artificial satellite for digital satellite broadcasting or LAN (Local Area). It is possible to transfer the program by wire to a computer via a network such as Network) and the Internet. In the computer, the program thus transferred can be received by the communication unit 408 and installed in the built-in hard disk 405.

A computer is a CPU (Central Processing).
Unit) 402 is built in. The CPU 402 has a bus 4
The input / output interface 410 is connected via 01, and the CPU 402 operates the input / output interface 410 via the input unit 407 including a keyboard, a mouse, a microphone, etc., by the user. When a command is input, the ROM (Read O
The program stored in the nly memory) 403 is executed. Alternatively, the CPU 402 also reads from a program stored in the hard disk 405, a program transferred from a satellite or a network, received by the communication unit 408 and installed in the hard disk 405, or a removable recording medium 411 mounted in the drive 409. The programs read and installed in the hard disk 405 are loaded into RAM (Random Access Memor
y) Load in 404 and execute. This allows the CPU 40
2 performs processing according to the above-described flowchart or processing performed by the configuration of the above-described block diagram. Then, the CPU 402 displays the processing result as needed, for example, via the input / output interface 410.
The data is output from the output unit 406 including an LCD (Liquid CryStal Display), a speaker, or the like, or transmitted from the communication unit 408, and further recorded on the hard disk 405.

Here, in the present specification, the processing steps for writing a program for causing a computer to perform various kinds of processing do not necessarily have to be processed in time series in the order described in the flow chart, but in parallel. Alternatively, it also includes processes that are executed individually (for example, parallel processes or processes by objects).

Also, the program may be processed by one computer or may be processed by a plurality of computers in a distributed manner. Further, the program may be transferred to a remote computer and executed.

As described above, the two-dimensional DC of the target block
In addition to the T coefficient, the prediction tap may be configured by a pixel value or a one-dimensional DCT coefficient obtained from a two-dimensional DCT coefficient of a block of a frame spatially or temporally close to the frame of the block of interest. Therefore, it is possible to obtain a high-quality decoded image with sufficiently reduced distortion such as block distortion and mosquito noise by using a small number of prediction taps.

In this embodiment, the predicted value of the teacher data is obtained by the linear first-order prediction calculation of the equation (1). However, the predicted value of the teacher data may be, for example,
It is possible to obtain it by using a higher-order equation of second or higher order.

In this embodiment, the class classification is
One-dimensional DCT of block of interest and adjacent block
Although the coefficient classification is performed using the coefficient, the class classification can be performed using the one-dimensional DCT coefficient of the block spatially or temporally close to the block of interest.
Furthermore, the class classification can be performed using not only the one-dimensional DCT coefficient but also the two-dimensional DCT coefficient. Further, the class classification can be performed by using the one-dimensional DCT coefficient of a block which is close to the block of interest in the time direction.

Further, in the class classification, for example, the power (sum of squares) of 63 two-dimensional DCT coefficients excluding the DC component among the 8 × 8 two-dimensional DCT coefficients of the target block and the adjacent block is calculated, It is also possible to perform it based on the magnitude relationship between the power and the predetermined threshold value, or the magnitude relationship between the power obtained from the target block and the power obtained from the adjacent block. Further, the class classification is, for example, the magnitude relationship between the DC component of the two-dimensional DCT coefficients of the eight adjacent blocks adjacent to the target block shown in FIG. 46 and a predetermined threshold, and the two-dimensional relationship of the eight adjacent blocks. DCT
It is also possible to perform it based on the magnitude relationship between each DC component of the coefficients and the DC component of the two-dimensional DCT coefficient of the block of interest.

Further, the class classification is performed by, for example, configuring a class tap in a form including a plurality of pixels of a target block or an adjacent block, and defining the class tap as ADRC (Adaptive Dynami
c Range Coding) processing is also possible.

Here, according to the K-bit ADRC processing, for example, the maximum value MAX of the pixel values of the pixels forming the class tap
And the minimum value MIN is detected, DR = MAX-MIN is set as the local dynamic range of the set, and this dynamic range DR
, The pixel values that make up the class tap are requantized into K bits. That is, the minimum value MIN is subtracted from the pixel value of each pixel forming the class tap, and the subtracted value is D
In R / 2 ^K is divided (quantized). Then, the bit string obtained by arranging the pixel values of the K-bit pixels forming the class tap obtained in the above manner in a predetermined order is output as an ADRC code. Therefore, when the class tap is subjected to, for example, 1-bit ADRC processing, the pixel value of each pixel forming the class tap is the average of the maximum value MAX and the minimum value MIN after the minimum value MIN is subtracted. The pixel value of each pixel is divided into 1 bit (binarized) by dividing by the value (rounding down after the decimal point). And that one
AD that is a bit string in which pixel values of bits are arranged in a predetermined order
The class of the pixel of interest is determined based on the RC code.

[0804] Furthermore, the classification can be performed using, for example, side information included in the video stream. That is, the class classification can be performed based on, for example, the quantization step which is one of the side information. Also,
Class classification is, for example, side information 1
It is also possible to recognize the combination of the structures (field structure or frame structure) of the block of interest and the adjacent block based on the DCT type, which is one of the three, and perform the recognition based on the recognition result. . further,
The class classification can be performed based on the presence / absence of motion compensation recognized from the side information such as the motion compensation type and CBP, and the presence / absence of a two-dimensional DCT coefficient.

Also, the classification can be performed based on the above-mentioned pixel position mode, for example.

Further, in the present embodiment, for the block of interest, only the two-dimensional DCT coefficient is included in the prediction tap, but the prediction tap may also include, for example, the pixel value of the block of interest or the one-dimensional block. It is possible to include DCT coefficients.

Furthermore, although the present embodiment is directed to MPEG-encoded moving images, the present invention is a two-dimensional DC.
It can also be applied to the case of decoding a moving image encoded by an encoding method other than MPEG using T conversion, a JPEG encoded still image, or the like.

[0808]

As described above, according to the present invention, image data is converted into at least two-dimensional DCT (Discrete Cosine Trans).
It becomes possible to decode a high-quality image of encoded data including a two-dimensional DCT coefficient obtained by performing form) conversion.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration example of an MPEG decoder.

FIG. 2 is a block diagram showing a configuration example of a first embodiment of an image processing apparatus to which the present invention has been applied.

FIG. 3 is a block diagram showing a configuration example of a DCT transform unit 21 and a frequency domain motion compensation addition unit 22.

FIG. 4 is a diagram illustrating a process of a sampling unit 61.

FIG. 5 is a diagram for explaining a process of a sampling unit 61.

FIG. 6 is a two-dimensional DCT transform and a two-dimensional inverse DCT transform,
It is a figure for demonstrating one-dimensional DCT transformation and one-dimensional inverse DCT transformation.

FIG. 7: Original image, horizontal one-dimensional DCT coefficient, vertical one-dimensional D
3 is a half-tone photograph displayed on a display showing a CT coefficient and a two-dimensional DCT coefficient.

FIG. 8 is a diagram for explaining a pixel row of an adjacent block that is spatially adjacent to a pixel row on the boundary of the target block.

FIG. 9 is a diagram for explaining a pixel row of an adjacent block that is spatially adjacent to a pixel row on the boundary of the block of interest.

FIG. 10 is a diagram for explaining a pixel row of an adjacent block that is spatially adjacent to a pixel row on the boundary of the target block.

FIG. 11 is a diagram for explaining a pixel row of an adjacent block that is spatially adjacent to a pixel row on the boundary of the target block.

FIG. 12 is a diagram for explaining a pixel row of an adjacent block that is spatially adjacent to a pixel row at the boundary of the block of interest.

FIG. 13 is a diagram for explaining a pixel row of an adjacent block that is spatially adjacent to a pixel row on the boundary of the target block.

FIG. 14 is a diagram for explaining a pixel row of an adjacent block that is spatially adjacent to a pixel row on the boundary of the target block.

FIG. 15 is a diagram for explaining a pixel row of an adjacent block that is spatially adjacent to a pixel row at the boundary of the target block.

FIG. 16 is a diagram for explaining a pixel row of an adjacent block that is spatially adjacent to a pixel row on the boundary of the block of interest.

FIG. 17 is a diagram for explaining a pixel row of an adjacent block that is spatially adjacent to a pixel row on the boundary of the target block.

FIG. 18 is a diagram for explaining a pixel row of an adjacent block that is spatially adjacent to a pixel row at the boundary of the block of interest.

FIG. 19 is a diagram for explaining a pixel row of an adjacent block that is spatially adjacent to a pixel row on the boundary of the block of interest.

FIG. 20 is a diagram for explaining a pixel row of an adjacent block that is spatially adjacent to a pixel row on the boundary of the block of interest.

FIG. 21 is a diagram for explaining a pixel row of an adjacent block that is spatially adjacent to a pixel row at the boundary of the block of interest.

FIG. 22 is a diagram for explaining a pixel row of an adjacent block that is spatially adjacent to a pixel row at the boundary of the block of interest.

FIG. 23 is a diagram for explaining a pixel row of an adjacent block that is spatially adjacent to a pixel row at the boundary of the block of interest.

FIG. 24 is a block diagram showing a configuration example of an adjacent one-dimensional DCT coefficient selection / transformation unit 32.

FIG. 25 is a flowchart illustrating a process of an adjacent one-dimensional DCT coefficient selection / transformation unit 32.

FIG. 26 is a flowchart illustrating details of upper left block processing.

FIG. 27 is a flowchart illustrating details of lower left block processing.

FIG. 28 is a flowchart illustrating details of upper right block processing.

FIG. 29 is a flowchart illustrating details of lower right block processing.

FIG. 30 is a diagram for explaining AC power.

FIG. 31 is a block diagram showing a configuration example of an AC power calculation unit 33.

FIG. 32 is a diagram for explaining an AC inner product.

FIG. 33 is a block diagram showing a configuration example of an AC inner product calculation unit 34.

FIG. 34 is a block diagram showing a configuration example of a class code generation unit 36.

FIG. 35 is a diagram showing a format of a class code.

FIG. 36 is a flowchart illustrating a process of a class code generator 36.

FIG. 37 is a block diagram showing a configuration example of a class code generation unit 37.

FIG. 38 is a diagram showing a correspondence relationship between a luminance block and a color difference block in the 4: 2: 2 format.

FIG. 39 is a diagram illustrating generation of a color difference class code using a luminance class code.

FIG. 40 is a diagram showing a correspondence relationship between a luminance block and a color difference block in the 4: 2: 0 format.

FIG. 41 is a diagram illustrating generation of a color difference class code using a luminance class code.

42 is a block diagram showing a configuration example of an adaptive processing unit 51. FIG.

FIG. 43 is a block diagram showing a first configuration example of a prediction tap generation unit 131.

FIG. 44 is a diagram showing a tap structure of a prediction tap generated by the prediction tap generation unit 131.

FIG. 45 is a diagram showing a tap structure of a prediction tap generated by the prediction tap generation unit 131.

FIG. 46 is a diagram showing a tap structure of a prediction tap generated by the prediction tap generation unit 131.

FIG. 47 is a flowchart illustrating processing of the adaptive processing section 51.

FIG. 48 is a block diagram showing a configuration example of a first embodiment of a learning device for learning tap coefficients.

FIG. 49 is a flowchart illustrating a learning process performed by the learning device.

FIG. 50 is a block diagram showing a configuration example of a second embodiment of a learning device for learning tap coefficients.

FIG. 51 is a block diagram showing a configuration example of a third embodiment of a learning device for learning tap coefficients.

FIG. 52 is a diagram showing S / N of a decoded image obtained by simulation.

FIG. 53 is a half-tone photograph displayed on the display showing the decoded image obtained by the simulation.

FIG. 54 is a block diagram showing a second configuration example of the prediction tap generation unit 131.

[Fig. 55] Fig. 55 is a diagram illustrating a tap structure of a prediction tap generated by the prediction tap generation unit 131.

[Fig. 56] Fig. 56 is a flowchart illustrating a process of an expected tap generation unit 131.

[Fig. 57] Fig. 57 is a diagram for describing block adjacent pixels spatially adjacent to the boundary of the target block.

FIG. 58 is a diagram for explaining block adjacent pixels spatially adjacent to the boundary of the target block.

[Fig. 59] Fig. 59 is a diagram for describing block adjacent pixels spatially adjacent to the boundary of the target block.

FIG. 60 is a diagram for explaining block adjacent pixels spatially adjacent to the boundary of the target block.

FIG. 61 is a diagram for explaining block adjacent pixels spatially adjacent to the boundary of the target block.

FIG. 62 is a diagram for explaining block adjacent pixels spatially adjacent to the boundary of the target block.

[Fig. 63] Fig. 63 is a diagram for describing block adjacent pixels spatially adjacent to the boundary of the target block.

[Fig. 64] Fig. 64 is a diagram for describing block adjacent pixels spatially adjacent to the boundary of the target block.

[Fig. 65] Fig. 65 is a diagram for describing block adjacent pixels spatially adjacent to the boundary of the target block.

[Fig. 66] Fig. 66 is a diagram for describing block adjacent pixels spatially adjacent to the boundary of the target block.

[Fig. 67] Fig. 67 is a diagram for describing block adjacent pixels spatially adjacent to the boundary of the target block.

FIG. 68 is a diagram for explaining block adjacent pixels spatially adjacent to the boundary of the target block.

FIG. 69 is a diagram for explaining block adjacent pixels spatially adjacent to the boundary of the target block.

FIG. 70 is a diagram for explaining block adjacent pixels spatially adjacent to the boundary of the target block.

FIG. 71 is a diagram for explaining block adjacent pixels spatially adjacent to the boundary of the target block.

[Fig. 72] Fig. 72 is a diagram for describing block adjacent pixels spatially adjacent to the boundary of the target block.

[Fig. 73] Fig. 73 is a block diagram illustrating a third configuration example of the prediction tap generation unit 131.

[Fig. 74] Fig. 74 is a block diagram illustrating a fourth configuration example of the prediction tap generation unit 131.

FIG. 75 is a diagram for explaining the processing of the image processing device 240.

[Fig. 76] Fig. 76 is a block diagram illustrating a fifth configuration example of the prediction tap generation unit 131.

FIG. 77 is a flowchart illustrating a process of the prediction tap generation unit 131.

[Fig. 78] Fig. 78 is a diagram illustrating a tap structure of a prediction tap generated by the prediction tap generation unit 131.

[Fig. 79] Fig. 79 is a block diagram illustrating a sixth configuration example of the prediction tap generation unit 131.

FIG. 80 is a diagram showing a tap structure of a prediction tap generated by the prediction tap generation unit 131.

FIG. 81 is a flowchart illustrating a process of the prediction tap generation unit 131.

[Fig. 82] Fig. 82 is a block diagram illustrating a seventh configuration example of the prediction tap generation unit 131.

[Fig. 83] Fig. 83 is a diagram illustrating a tap structure of a prediction tap generated by the prediction tap generation unit 131.

FIG. 84 is a flowchart illustrating a process of the prediction tap generation unit 131.

FIG. 85 is a block diagram showing a configuration example of an embodiment of a computer to which the present invention has been applied.

[Explanation of symbols]

1 separation unit, 2 DCT coefficient extraction / inverse quantization unit, 4
Motion compensation unit, 5 image memory, 11 pre-processing unit,
12 buffer memory, 13 class classification unit, 14
Tap coefficient storage unit, 15 Image reconstruction unit, 21
DCT transform unit, 22 frequency domain motion compensation addition unit,
31 1-dimensional inverse DCT transform unit, 32 adjacent 1-dimensional DCT
Coefficient selection / conversion unit, 33 AC power calculation unit, 34
AC inner product calculation unit, 36, 37 class code generation unit, 41, 42 tap coefficient selection unit, 43, 44
Coefficient memory, 51 adaptive processing unit, 52 inverse DCT conversion unit, 61 sampling unit, 62 DCT unit,
71 DCT coefficient selection unit, 72 addition unit, 73 selection unit, 80 control unit, 81 memory, 82 vertical 1
Dimensional inverse DCT transform unit, 83 sampling unit, 84
Vertical one-dimensional DCT conversion unit, 85 selection unit, 91
Horizontal one-dimensional DCT coefficient extraction unit, 92 vertical one-dimensional DCT
Coefficient extraction unit, 93 Horizontal AC power calculation unit, 94
Vertical AC power calculation unit, 101 One-dimensional DC for upper inner product
T coefficient extracting unit, 102 one-dimensional DCT coefficient extracting unit for lower inner product, 103 one-dimensional DCT coefficient extracting unit for left inner product, 1
04 one-dimensional DCT coefficient extracting unit for right inner product, 105 upper inner product calculating unit, 106 lower inner product calculating unit, 107 left inner product calculating unit, 108 right inner product calculating unit, 111, 112
Comparison unit, 113 flatness condition determination unit, 114 continuity condition determination unit, 115 boundary edge condition determination unit,
116 class code creation unit, 121, 122 comparison unit, 123 class code creation unit, 131 prediction tap generation unit, 132 tap coefficient buffer, 133
Sum-of-products operation unit, 136 reading unit, 137 array unit, 141 teacher data storage, 142 student data generation unit, 143 student data storage, 1
44 prediction tap generation unit, 145 class classification unit,
146 addition section, 147 tap coefficient calculation section,
151 MPEG encoder, 152 separation unit, 1
53 DCT coefficient extraction / inverse quantization unit, 171 MPE
G encoder, 172 separation unit, 173 DCT coefficient extraction / inverse quantization unit, 174 class classification unit, 17
5 tap coefficient storage unit, 176 adaptive processing unit, 17
7 I picture storage, 178 motion compensation unit,
179 image memory, 180 DCT converter, 18
1 frequency domain motion compensation addition unit, 191 MPEG encoder, 192 separation unit, 193 DCT coefficient extraction / dequantization unit, 194 class classification unit, 195 tap coefficient storage unit, 196 adaptation processing unit, 197 I picture storage, 198 motion compensation Department, 199
Image memory, 200 DCT transform unit, 201 frequency domain motion compensation addition unit, 202 class classification unit, 20
3 tap coefficient storage unit, 204 adaptive processing unit, 20
5 P picture storage, 206 motion compensation unit,
207 image memory, 208 DCT converter, 20
9 frequency domain motion compensation addition unit, 221 inverse DCT conversion unit, 222 selection unit, 231 MPEG decoder,
232 buffer memory, 240 image processing device,
241 pre-processing unit, 242 buffer memory, 24
3 class classification unit, 244 tap coefficient storage unit, 2
45 Image Reconstructing Unit, 251 Adaptive Processing Unit, 261
One-dimensional inverse DCT transform unit, 262 adjacent one-dimensional DCT
Coefficient selection / conversion unit, 401 bus, 402 CPU,
403 ROM, 404 RAM, 405 hard disk, 406 output section, 407 input section, 408 communication section, 409 drive, 410 input / output interface, 411 removable recording medium

Continued front page    (72) Inventor Toshihiko Hamamatsu             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation (72) Inventor Hideki Otsuka             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation (72) Inventor Takeharu Nishikata             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation (72) Inventor Takeshi Kunihiro             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation (72) Inventor Takafumi Morito             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation F term (reference) 5C059 MA00 MA14 MA23 PP05 PP06                       PP07 PP16 SS20 TA00 TA23                       TA29 TB07 TC03 TC34 TC43                       TD13 UA05 UA38 UA39                 5C078 AA04 BA57 CA21 DA02                 5J064 AA01 BA16 BB14 BC01 BC08                       BC14 BC16 BD03

Claims (33)

[Claims] 1. At least a two-dimensional DCT for image data
(Discrete Cosine Transform) An image processing device for decoding coded data including a two-dimensional DCT coefficient obtained by transforming, wherein pixels constituting the image data are classified into one of a plurality of classes. A class classifying unit for classifying the class, an acquiring unit for acquiring a tap coefficient corresponding to the class of the pixel of interest, which is the pixel of interest, among the tap coefficients obtained by predetermined learning; The prediction tap used in a predetermined prediction calculation with the tap coefficient acquired by the means is the two-dimensional DCT.
By performing the predetermined prediction calculation using a coefficient, a prediction tap generation unit that is generated from information obtained from the two-dimensional DCT coefficient, a tap coefficient of the class of the pixel of interest, and the prediction tap, An image processing apparatus comprising: a prediction calculation unit that obtains a pixel value of a pixel of interest. 2. The tap coefficient is learned so that a prediction error of a prediction value of a pixel value of the pixel of interest obtained by performing a prediction calculation using the prediction tap and the tap coefficient is statistically minimized. The image processing apparatus according to claim 1, wherein the image processing apparatus is obtained by performing. 3. The encoded data is the image data obtained by subjecting the image data to two-dimensional DCT conversion in predetermined block units.
A two-dimensional DCT coefficient, and the prediction tap generation means obtains it from the two-dimensional DCT coefficient of the target block that is the block including the target pixel and the two-dimensional DCT coefficient of a peripheral block that is a block around the target block. The image processing apparatus according to claim 1, wherein the prediction tap is generated using peripheral information that is information that is provided. 4. The image processing apparatus according to claim 3, wherein the peripheral block is a block that is spatially or temporally adjacent to the block of interest. 5. The image processing apparatus according to claim 3, wherein the peripheral information is a pixel value obtained from the two-dimensional DCT coefficient. 6. The image processing apparatus according to claim 5, wherein the peripheral information is a pixel value of a pixel of a block adjacent to the target block. 7. The image processing apparatus according to claim 5, wherein the peripheral information is a pixel value of a block adjacent pixel that is a pixel adjacent to the target block in a block adjacent to the target block. . 8. The peripheral information includes pixel values of two pixels on the left and right of the same row as the pixel of interest and pixel values of two pixels above and below the same column of the pixel of interest in the block adjacent pixel. The image processing apparatus according to claim 7, wherein 9. The peripheral information includes the pixel value of the pixel adjacent to the pixel of interest among two pixels on the left and right of the same row of the pixel of interest in the block adjacent pixel, and the same column as the pixel of interest. The image processing apparatus according to claim 7, wherein the pixel value of one of the two pixels above and below the one that is closer to the pixel of interest. 10. The image processing apparatus according to claim 3, wherein the peripheral information is a one-dimensional DCT coefficient obtained from the two-dimensional DCT coefficient. 11. The image processing apparatus according to claim 10, wherein the peripheral information is the one-dimensional DCT coefficient adjacent to the target block in a block adjacent to the target block. 12. The peripheral information is a pixel value of one or more pixels in a frame before or after the frame of the block of interest, or a one-dimensional DCT coefficient of the one or more pixels. The image processing device according to item 3. 13. The peripheral information is a pixel value of one or more pixels including a pixel at the same position as the target pixel in a frame before or after the frame of the target block,
13. The image processing apparatus according to claim 12, wherein the image processing apparatus is a one-dimensional DCT coefficient of one or more pixels. 14. The encoded data includes a motion vector representing a motion of the image data, and the peripheral information is from a position corresponding to the pixel of interest in a frame before or after a frame of the block of interest. 13. The image processing apparatus according to claim 12, wherein the pixel value of one or more pixels including a pixel at a position moved by an amount corresponding to the motion vector, or a one-dimensional DCT coefficient of the one or more pixels. . 15. Image data at least two-dimensional DC
An image processing method for decoding coded data including a two-dimensional DCT coefficient obtained by T (Discrete Cosine Transform) transformation, wherein pixels constituting the image data are classified into one of a plurality of classes. A class classification step of performing class classification for dividing, an acquisition step of acquiring a tap coefficient corresponding to a class of a pixel of interest, which is a pixel of interest, of tap coefficients for each class obtained by predetermined learning; The prediction tap used in a predetermined prediction calculation with the tap coefficient obtained in step is the two-dimensional D
By performing a predetermined prediction calculation using a CT coefficient, a prediction tap generation step that is generated from information obtained from the two-dimensional DCT coefficient, a tap coefficient of the class of the pixel of interest, and the prediction tap, An image processing method, comprising: a prediction calculation step of obtaining a pixel value of the pixel of interest. 16. At least two-dimensional DC for image data
Image processing for decoding coded data including two-dimensional DCT coefficients obtained by T (Discrete Cosine Transform) transformation
A program to be executed by a computer, which comprises a class classification step of classifying pixels constituting the image data into one of a plurality of classes, and a class classification step for each class obtained by predetermined learning. Of the tap coefficients, an acquisition step of acquiring one corresponding to the class of the pixel of interest, which is the pixel of interest, and a prediction tap used for a predetermined prediction calculation with the tap coefficient acquired in the acquisition step, The two-dimensional D
By performing a predetermined prediction calculation using a CT coefficient, a prediction tap generation step that is generated from information obtained from the two-dimensional DCT coefficient, a tap coefficient of the class of the pixel of interest, and the prediction tap, A prediction calculation step of obtaining a pixel value of the target pixel. 17. At least two-dimensional DC for image data
Image processing for decoding coded data including two-dimensional DCT coefficients obtained by T (Discrete Cosine Transform) transformation
A recording medium in which a program to be executed by a computer is recorded, wherein a class classification step of classifying the pixels forming the image data into any one of a plurality of classes, Of the tap coefficients for each class obtained by learning, an acquisition step of acquiring one corresponding to the class of the pixel of interest, which is the pixel of interest, and a predetermined prediction of the tap coefficient acquired in the acquisition step. The prediction tap used for the calculation is the two-dimensional D
By performing a predetermined prediction calculation using a CT coefficient, a prediction tap generation step that is generated from information obtained from the two-dimensional DCT coefficient, a tap coefficient of the class of the pixel of interest, and the prediction tap, A recording medium in which a program including a prediction calculation step of obtaining a pixel value of the target pixel is recorded. 18. Image data at least two-dimensional DC
An image processing device for learning tap coefficients used for decoding coded data including two-dimensional DCT coefficients obtained by T (Discrete Cosine Transform) transformation, wherein teacher data, which is image data to be a teacher of learning, is At least two-dimensional DCT conversion is performed for encoding, and encoded data including a two-dimensional DCT coefficient is output as student data to be the student of the learning. A class classifying unit for classifying the class into any of the classes, and the tap coefficient for predicting the target teacher data which is a pixel of interest among the pixels forming the teacher data Prediction taps used together with the two-dimensional DCT coefficient that constitutes the student data, and information obtained from the two-dimensional DCT coefficient A prediction tap generation means for generating the prediction coefficient and a prediction coefficient of the prediction value of the teacher data of interest obtained by performing a predetermined prediction calculation using the prediction tap and the tap coefficient; An image processing apparatus, comprising: a learning unit that performs learning required for each class. 19. The encoded data is the two-dimensional DCT-transformed version of the teacher data in predetermined block units.
A two-dimensional DCT coefficient of the target block that is the block including the target teacher data, and the two-dimensional DCT coefficient of a peripheral block that is a peripheral block of the target block. The image processing apparatus according to claim 18, wherein the prediction tap is generated using peripheral information which is obtained information. 20. The image processing apparatus according to claim 19, wherein the peripheral block is a block spatially or temporally adjacent to the block of interest. 21. The peripheral information is a pixel value obtained from the two-dimensional DCT coefficient.
9. The image processing device according to item 9. 22. The image processing apparatus according to claim 21, wherein the peripheral information is a pixel value of a pixel of a block adjacent to the target block. 23. The image processing apparatus according to claim 21, wherein the peripheral information is a pixel value of a block adjacent pixel which is a pixel adjacent to the target block in a block adjacent to the target block. . 24. The peripheral information includes pixel values of two pixels on the left and right of the same row as the teacher data of interest in the block adjacent pixel, and two pixels above and below the same column of the teacher data of interest. The image processing device according to claim 23, wherein the image processing device is a pixel value. 25. The peripheral information includes a pixel value closer to the target teacher data out of two pixels on the left and right of the same row as the target teacher data in the block adjacent pixel, and the target teacher data. 24. The image processing apparatus according to claim 23, which has a pixel value closer to the target teacher data, of the two pixels above and below the same column. 26. The image processing apparatus according to claim 19, wherein the peripheral information is a one-dimensional DCT coefficient obtained from the two-dimensional DCT coefficient. 27. The image processing apparatus according to claim 26, wherein the peripheral information is the one-dimensional DCT coefficient adjacent to the target block in a block adjacent to the target block. 28. The peripheral information is a pixel value of one or more pixels in a frame before or after the frame of the block of interest, or a one-dimensional DCT coefficient of the one or more pixels. The image processing device according to item 19. 29. The peripheral information is a pixel value of one or more pixels including a pixel at the same position as the teacher data of interest in a frame before or after a frame of the block of interest, or of one or more pixels of the pixel value. The image processing apparatus according to claim 28, wherein the image processing apparatus is a one-dimensional DCT coefficient. 30. The encoded data includes a motion vector indicating a motion of the image data, and the peripheral information is from a position corresponding to the teacher data of interest in a frame before or after a frame of the block of interest. 29. The image processing according to claim 28, which is a pixel value of one or more pixels including a pixel located at a position corresponding to the motion vector, or a one-dimensional DCT coefficient of the one or more pixels. apparatus. 31. At least two-dimensional DC for image data
An image processing method for learning a tap coefficient used for decoding encoded data including a two-dimensional DCT coefficient obtained by T (Discrete Cosine Transform) transformation, in which teacher data that is image data to be a teacher of learning is A student data generating step of outputting coded data including at least two-dimensional DCT conversion and including two-dimensional DCT coefficients as student data to be a student of the learning; a plurality of pixels forming the teacher data; A class classification step of performing a class classification for classifying into one of the classes, and the tap coefficient for predicting the target teacher data which is a pixel of interest among the pixels forming the teacher data A prediction tap to be used with is obtained from the two-dimensional DCT coefficient that constitutes the student data and the two-dimensional DCT coefficient. A prediction tap generation step of generating from the information, and the tap that statistically minimizes the prediction error of the prediction value of the teacher data of interest obtained by performing a predetermined prediction calculation using the prediction tap and the tap coefficient. A learning step of performing learning for obtaining a coefficient for each of the classes. 32. At least two-dimensional DC for image data
T (Discrete Cosine Transform) is a program that causes a computer to perform image processing for learning tap coefficients used to decode coded data including two-dimensional DCT coefficients obtained by transforming, and is image data to be a teacher of learning. Student data generation step of encoding the teacher data that is at least two-dimensional DCT-transformed, and outputting the encoded data including the two-dimensional DCT coefficient as student data to be a student of the learning; and configuring the teacher data. A class classification step of classifying the pixel to be classified into one of a plurality of classes, and predicting the target teacher data that is the pixel of interest among the pixels forming the teacher data Predictive taps used together with the tap coefficient to perform the two-dimensional DCT coefficient that constitutes the student data. , A prediction tap generation step of generating from information obtained from the two-dimensional DCT coefficient, and a prediction error of a prediction value of the teacher data of interest obtained by performing a predetermined prediction calculation using the prediction tap and the tap coefficient, A learning step of performing learning for obtaining, for each class, the tap coefficient that is statistically minimized. 33. At least two-dimensional DC for image data
A recording medium in which a program for causing a computer to perform image processing for learning tap coefficients used for decoding encoded data including two-dimensional DCT coefficients obtained by T (Discrete Cosine Transform) transformation is recorded. Student data generation step of encoding teacher data, which is image data to be a teacher of learning, by performing at least two-dimensional DCT conversion, and outputting coded data including two-dimensional DCT coefficients as student data to be a student of the learning A class classification step of classifying the pixels forming the teacher data into one of a plurality of classes, and a pixel of interest among the pixels forming the teacher data. Configure the student data with a prediction tap that is used with the tap coefficient to predict certain attention teacher data Note: a two-dimensional DCT coefficient, a prediction tap generation step of generating information obtained from the two-dimensional DCT coefficient, and prediction of the teacher data of interest obtained by performing a predetermined prediction calculation using the prediction tap and the tap coefficient. A recording medium having a program recorded thereon, which comprises a learning step of learning for each class the tap coefficient that statistically minimizes a value prediction error.