JP2003348599A - Image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently encode an interlaced image or a specific progressive image including field DCT (discrete cosine transform) type macroblocks by reducing spatially redundant image information included in an image signal with intra-image prediction processing in an image encoding method for applying DCT processing to the image signal for each of blocks constituting a marcoblock. <P>SOLUTION: Prediction values 111 of a quantized DCT coefficient of a coded block x(i) are generated by using a block r0(i) located adjacently to the up-left side of the coded block x(i), a block r1(i) located adjacently to the up side of the coded block x(i), and a block r2(i) located adjacently to the left side of the coded block x(i) as reference blocks regardless of the DCT type of the coded block. <P>COPYRIGHT: (C)2004,JPO

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method, and more particularly to a method for improving the coding efficiency of an image signal by performing an adaptive intra prediction process on the frequency component of an interlaced image signal. Things.

[0002]

2. Description of the Related Art In predictive coding for compressing image data corresponding to a moving image by using its redundancy, intra-screen prediction for predicting image data using image data in a frame to be encoded. There is coding and inter-picture prediction coding for predicting image data using image data of a frame other than the frame to be coded.

Specifically, in the above-mentioned intra prediction coding, a predicted value of image data of a frame to be coded is generated from image data in the frame, and a difference between the image data of the frame to be coded and the predicted value is calculated. This is a method of compressing image data by removing or reducing spatially redundant information contained in a large amount of image data as an original property of the image by encoding the value.

On the other hand, in the above-mentioned inter-picture predictive coding, a predicted value of image data of a frame to be encoded is generated from image data of another frame, and a difference value between the image data of the frame to be encoded and the predicted value is encoded. This is a method of compressing an image by removing or reducing temporally redundant information that is included in image data in a large amount when the motion of the image is small.

In recent image coding, discrete cosine transform (DCT) has been widely used, and a typical image coding method, MPEG (Moving Picture Expert Group).
In the p) method, an image space (frame) formed by a digital image signal is divided into a plurality of rectangular areas (blocks) which are units of DCT processing, and a DCT is performed for each block with respect to an image signal corresponding to each block. Processing is performed.

[0006] The intra-picture prediction method for the DCT coefficient, that is, the image data in the DCT domain (frequency domain) employed in the MPEG system is described in ISO / IEC JTC1 / SC29 / WG11 MPEG97 / N1642 which is a document related to MPEG4.
"Intra DC and AC Prediction for I-VOP and P-VOP" in MPEG-4 Video VerificationModel Version 7.0 (hereinafter referred to as MPEG-4 VM7.0)
Section.

According to the description of this document, the DCT coefficient of the DCT coefficient corresponding to the encoded block to be encoded is
The component and the AC component are predicted using DCT coefficients corresponding to three adjacent blocks located adjacent to the upper left, above, and left of the block to be coded in the image space.

FIG. 15 shows an intra-screen DC as described in the above-mentioned document, which is adopted in a conventional image coding method.
It is a figure for explaining a T coefficient prediction method. FIG. 15 shows four blocks (also referred to as DCT blocks) R0 to R2, each including 8 × 8 pixels, which are units of DCT processing.
Are shown, and each block is located adjacent to each other on an image space (spatial region) formed by the image signal.

Here, a block X is a block to be coded to be coded, and blocks R0, R1,
R2 is a coded block that has already been coded and is located adjacent to the upper left, upper, and left sides of the coded block on the space area.

In the conventional intra-screen DCT coefficient prediction method, when generating a predicted value of the DCT coefficient of the block X, the DCT coefficient of the block R1 or the block R2 is referred to.
Specifically, when referring to the DCT coefficients of the encoded block R1, the DC component in the upper left corner and the AC component in the uppermost row in the encoded block R1 are added to these components and the encoded block X. Is used as a predicted value of the DCT coefficient located at the same position. When referring to the DCT coefficients of the coded block R2, the DC component in the upper left corner and the AC component in the leftmost column of the coded block R2 are obtained by combining these components with the coded block X. It is used as a predicted value of the DCT coefficient located at the same position.

Further, the DCT coefficients of any of the encoded blocks are converted to the DCT coefficients of the encoded block X.
The determination as to whether to refer to the coefficient prediction value is made using the DC components of the encoded blocks R0, R1, and R2.

That is, if the absolute value of the difference between the DC components between the blocks R0 and R2 is smaller than the absolute value of the difference between the DC components between the blocks R0 and R1, the blocks arranged in the vertical direction Since the correlation between the DCT coefficients is strong, the DCT coefficient of the block R1 is referred to when the predicted value of the DCT coefficient of the block X to be coded is generated.
On the other hand, the absolute value of the difference between the DC components between the block R0 and the block R1 is equal to the D value between the block R0 and the block R2.
When the absolute value of the difference between the C components is smaller than the absolute value of the difference between the C components, the correlation between the DCT coefficients between the blocks arranged in the horizontal direction is strong.
The DCT coefficient of the block R2 is referred to.

However, the above document (MPEG-4 VM)
7.0) "Adaptive Frame / F
As described in the section “field DCT”, the DCT processing (frequency conversion processing) used for encoding an interlaced image includes a frame DCT processing and a field DCT processing.
There are two types of DCT processing, T processing. These D
The CT processing has a different processing unit. The frame DCT processing converts image data in frame units, and the field DCT processing converts image data in field units. In the MPEG system, frame DCT processing and field DCT processing are adaptively switched for each so-called macro block composed of four blocks.

Here, switching between frame DCT processing and field DCT processing for a macroblock is shown in FIG.
In the field DCT processing, DCT processing is performed on image data of each block in a macroblock in which scanning lines are rearranged, as shown in FIG. Become.

Specifically, in the case of the frame DCT processing, image data of each block in a macroblock in which even-numbered and odd-numbered scanning lines are alternately arranged is directly used as a D-value.
In the case of the CT processing and the field DCT processing, by reordering the scanning lines, the macro block is composed of a first field block composed of only even-numbered scanning lines and a macroblock composed of only odd-numbered scanning lines. After the image data is composed of blocks of two fields, DCT processing is performed on image data of each block in such a macroblock.

As described above, in the coding process of an interlaced image signal, macroblocks subjected to frame DCT processing and macroblocks subjected to field DCT processing are mixed as macroblocks located in the image space.

If the correlation between the pixel values in the first field and the second field is higher than the correlation between the pixel values in the first field and the second field, frame DCT processing is performed. Field D if
In a method in which the CT processing is performed, the frame DCT processing and the field DCT processing are switched.

Accordingly, even in the case of adjacent macro blocks or adjacent blocks in the image space (that is, sub-blocks constituting the macro block), the type of DCT processing may be different. The correlation of DCT coefficients between blocks or between adjacent blocks is lower than when the DCT processing type of a macroblock or block is the same.

In some cases, the field to which these belong differs between adjacent blocks. In such a case, the correlation between the DCT coefficients is lower between adjacent blocks than when the fields to which the blocks belong are the same. It will be.

[0020]

However, in a macroblock that has been subjected to field DCT processing (a macroblock of the field DCT type), the block of the first field and the block of the second field are mixed, so that the block to be coded is It is difficult to specify the coded block to be referred to when generating the predicted value of the DCT coefficient of the above, so that the conventional intra-screen prediction processing is performed on the field DCT type macroblock as described above. Simply cannot be applied. As a result, in the coding process of an interlaced image or the coding process of a specific progressive image in which field DCT type macroblocks coexist, the intra-frame prediction process cannot be applied, and the spatial There is a problem that redundant image information cannot be sufficiently reduced and efficient encoding processing cannot be performed.

The present invention has been made in order to solve the above-described problems. The present invention is also applicable to an encoding process of an interlaced image or an encoding process of a specific progressive image in which macroblocks of different DCT types are mixed. An object of the present invention is to provide an image processing method capable of sufficiently reducing spatially redundant image information included in an image signal and performing highly efficient encoding processing.

[0022]

According to an image processing method according to the present invention (claim 1), an image signal forming an image space composed of a plurality of pixels is converted into a plurality of rectangular macroblocks for dividing the image space. The encoding process of the image signal corresponding to the sub-block constituting the macro block,
An image encoding method performed for each sub-block, wherein, for an image signal of a predetermined macro block, an image of a first field constituting the image signal is located above the macro block, and the image signal is The horizontal pixel column is rearranged so that the image of the second field to be formed is located below the macroblock, and the macroblocks that have been subjected to the rearrangement processing or the macroblocks that have not been subjected to the rearrangement processing Is converted into a frequency component by performing frequency conversion for each of the four subblocks located at the upper left, upper right, lower left, and lower right constituting the macroblock, and the encoded sub The predicted value of the frequency component of the block is calculated from among the coded sub-blocks that have already been coded, near the upper side, near the left side, and Generated based on at least one of the encoded sub-blocks located in the upper vicinity, encodes the difference value between the frequency component of the encoded sub-block and its predicted value, and encodes the frequency component of the encoded sub-block. When generating the prediction value, regardless of whether the encoded sub-block is a sub-block on which the image signal is rearranged, a code located near the upper side of the encoded sub-block. Which of the frequency components of the coded sub-block and the coded sub-block located in the vicinity of the left side of the coded sub-block is to be referred to is determined in the vicinity of the upper side, the vicinity of the left side, and the vicinity of the upper left of the coded block. It is determined based on the DC component of the frequency component of the coded sub-block located.

According to a second aspect of the present invention, in the image processing method according to the first aspect, the encoded sub-block located near the upper side of the encoded sub-block is located above the encoded sub-block. Using an upper coded sub-block located adjacently, as a coded sub-block located near the left side of the coded sub-block,
Using a left-side coded sub-block located adjacent to the left side of the coded sub-block, as a coded sub-block located near the upper left of the coded sub-block, The absolute value of the difference between the DC component of the frequency component of the upper encoded sub-block and the DC component of the frequency component of the upper left encoded sub-block using the upper left encoded sub-block located adjacent to the side Is smaller than the absolute value of the difference between the DC component of the frequency component of the left encoded sub-block and the DC component of the frequency component of the upper left encoded sub-block, the frequency component of the left encoded sub-block is Generate a predicted value of the frequency component of the encoded sub-block by referring to
On the other hand, the absolute value of the difference between the DC component of the frequency component of the left encoded sub-block and the DC component of the frequency component of the upper left encoded sub-block is the DC component of the frequency component of the upper encoded sub-block. And when the absolute value of the difference between the DC component of the frequency component of the left upper coded sub-block is smaller than the predicted value of the frequency component of the coded sub-block with reference to the frequency component of the upper coded sub-block. Is generated.

According to a third aspect of the present invention, in the image processing method according to the first or second aspect, the predicted value of the frequency component of the encoded sub-block is generated based on the quantized frequency component. It is characterized by the following.

[0025]

Embodiments of the present invention will be described below with reference to the drawings. Embodiment 1 FIG. The image processing apparatus (image coding apparatus) according to the first embodiment of the present invention has already coded the adaptive intra-screen DCT coefficient prediction method, that is, the DCT type signal (frequency transform type signal) of the block to be coded. It is characterized in that intra-picture predictive coding of an image signal is performed using a method of generating a predicted value of a DCT coefficient of a block to be coded from a DCT coefficient of a block. Here, the DCT type signal indicates a signal indicating whether the block to be coded is subjected to frame DCT processing or field DCT processing.

FIG. 1 is a block diagram showing a configuration of an image coding apparatus according to the first embodiment. In the figure, 10
Reference numeral 00 denotes an image coding apparatus according to the first embodiment, which corresponds to an input digital image signal (input image signal) 110a to each of a plurality of blocks that divide an image space (frame) formed by the digital image signal 110a. The image signal corresponding to each block is encoded for each block.

That is, the image encoding apparatus 1000
The input image signal 110a is divided into blocks corresponding to the respective blocks for each frame or field as a unit of processing for frequency conversion, and the image signal 101 and the frequency conversion (DC
A blocker 100 that outputs a DCT type signal 102 indicating a processing unit of (T processing). The block generator 100 receives the input image signal 110a, and when the correlation of the pixel values between the fields is higher than that in the frame, the blocker 100 pre-divides the 16 × 16 pixels so that the field DCT processing is performed. The scanning lines are rearranged in units of macroblocks, and an image signal is output for each block of 8 × 8 pixels constituting the rearranged macroblocks.

When the correlation between pixel values between fields is smaller than that in a frame, the blocking unit 100 does not rearrange the scanning lines in units of macroblocks as described above. The input image signal is output for each block.

The image coding apparatus 1000 applies a discrete cosine transform (DCT) to the block image signal (hereinafter also referred to as a block image signal) 101.
Processing) to convert the blocked image signal into a frequency component (DCT coefficient) 104, and a DCT coefficient 104. The DCT coefficient 104 is quantized to obtain a quantization value (DCT coefficient quantization value) corresponding to each block. ) 106, an intra-screen prediction processing unit 110 that generates a predicted value 111 corresponding to a block to be coded by an intra-screen prediction process based on the DCT type signal 102, and the DCT coefficient quantization. An adder 107 that subtracts the predicted value 111 from the value 106 and outputs a DCT coefficient difference value 108. The DCT coefficient difference value 108
The data is variable-length coded by the device 109 and output as a bit stream (image coded signal) 110b.

Here, the intra-screen prediction processing unit 110
An adder 112 for adding the DCT coefficient difference value 108 and the intra prediction value 111; a block memory 115 for storing an output of the adder 112 as a DCT coefficient quantization value 116 of an encoded block; 102
And a DCT coefficient predictor 113 that generates a predicted value 111 of the quantized value of the DCT coefficient of the coded block from the quantized value of the DCT coefficient 114 of the coded block by an adaptive intra-screen DCT coefficient prediction method. Have been.

Next, the operation will be described. First, the overall operation in the encoding process using the adaptive DCT prediction process will be described. When a digital image signal (input image signal) 110 a is input to the present image encoding device 1000, the input image signal 110
a is divided into blocks corresponding to the respective blocks for each frame or field as a processing unit of the frequency conversion, and the blocked image signal 101,
Further, a DCT type signal 102 indicating a processing unit of the frequency conversion (DCT processing) is output.

At this time, if the correlation between the pixel values between the fields is higher than that in the frame, the blocking unit 100 sets a macro composed of 16 × 16 pixels in advance so that the field DCT processing is performed. A scanning line rearrangement process is performed on the image signal in units of blocks, and the image signal subjected to the scan line rearrangement process is output for each block of 8 × 8 pixels constituting the macro block. Is done.

When the correlation between the pixel values between the fields is smaller than that in the frame, the blocking unit 100 does not perform the above-described processing for rearranging the scanning lines in units of macroblocks. , The input image signal is output for each block.

The image signal 101 of the coded block to be coded is subjected to a discrete cosine transform (DCT process) by a DCT unit 103 to obtain a frequency component (DCT coefficient) corresponding to the coded block. The DCT coefficient 104 is further transformed into a quantizer 105
, And is output as a quantization value (DCT coefficient quantization value) 106 for the block to be coded.

Further, when the DCT coefficient quantized value 106 of the above-mentioned block to be encoded is supplied to the adder 107, the difference between this quantized value 106 and its predicted value 111 is obtained and output as a DCT coefficient difference value 108. Is done. This DCT
The coefficient difference value 108 is subjected to variable-length coding by a VLC unit 109 and a bit stream (coded image signal) 110
Output as b.

The D output from the adder 107
The CT coefficient difference value 108 is supplied to an intra-screen prediction processing unit 110, where a prediction value for the DCT coefficient quantization value 106 is generated.

That is, in the intra-screen prediction processing section 110, the DCT coefficient difference value 108 and the intra-screen prediction value 111 are added by the adder 112, and the added value is added to the DCT coefficient quantization value 116 of the encoded block. Is stored in the block memory 115. Then, in accordance with the DCT type signal 102, the DCT coefficient predictor 113 converts the DCT coefficient quantization value 114 of the coded block from the DCT coefficient quantization value 114 of the coded block by the adaptive intra-screen DCT coefficient prediction method. A predicted value 111 is generated.

Next, the adaptive intra-screen DCT coefficient prediction method in the above-mentioned encoding process will be described in detail. In the adaptive intra-screen DCT coefficient prediction method according to the first embodiment, a block to be referred to when generating a predicted value of a DCT coefficient corresponding to an encoded block is changed according to the DCT type of the encoded block. Things. In the first embodiment, the DCT region is defined as follows.

That is, the DCT domain (frequency domain) is
An image signal forming an image space (spatial region) is defined as a region formed by frequency components obtained by DCT processing (frequency conversion), and a DCT region is arranged according to the arrangement of macroblocks in the image signal space region (image space). Each macro block is assumed to be arranged in (frequency domain).

In the first embodiment, as shown in FIG. 3, when a macroblock is subjected to frame DCT processing, DCT processing is performed on the image signal of each block without rearranging the scanning lines in the macroblock. DCT coefficients corresponding to the upper left, upper right, lower left, and lower right blocks of the macroblock in the spatial domain are respectively DCT coefficients.
Block position (0) in the macroblock on the area,
If the macroblocks are arranged in the blocks of (1), (2), and (3), and the macroblocks are subjected to the field DCT processing, the image signal of each block is rearranged after rearranging the scanning lines in the macroblocks in the spatial domain. Are subjected to DCT processing, and the DCT data of each block of the first field left, the first field right, the second field left, and the second field right are respectively assigned to block positions (0), (1) in a macroblock in the DCT area. , (2), and (3).

Next, the DCT corresponding to the block to be encoded
Coefficient (data of encoded block in DCT domain)
Is predicted with reference to the DCT coefficient of the coded block, and at this time, an adaptive intra-screen DCT coefficient prediction method for switching the coded block to be referred to according to the DCT type of the block to be coded will be described in detail. .

First, the block to be encoded is a frame DCT.
In the prediction in the case of being processed (hereinafter, referred to as frame prediction), as shown in FIG. 4A, the block located at the upper left of the coded block x (i) is referred to as a reference block r0 (i), A block located above and adjacent to the coded block x (i) is referred to as a reference block r1 (i), and a block located to the left of the coded block x (i) is referred to as a reference block r2 (i). . FIG. 4 (a)
Are the reference blocks r0 (i) and r1 (i) when the coded block x (i) shown in FIG.
And r2 (i) are the blocks spatially closest to the coded block x (i), and typically the DCT coefficients of these reference blocks are the DCT coefficients of the coded block x (i).
It is considered that the correlation with the coefficient is high.

On the other hand, if the block to be coded is the field DC
In the prediction in the case where the T processing is performed (hereinafter, referred to as field prediction), as shown in FIG. 4B, a position adjacent to the left of the block located on two blocks of the encoded block x (i) is set. A reference block r0 (i),
A block located two blocks above the block to be coded x (i) is referred to as a reference block r1 (i), a block to be coded x
A block located on the left of (i) is referred to as a reference block r2.
(I). Reference blocks r0 (i), r1 (i) and r2 (i) when the encoded block x (i) shown in FIG. 4B is subjected to the field DCT processing.
Are the spatially closest blocks belonging to the same field as the block to be coded x (i). Usually, the DCT coefficients of these reference blocks have a high correlation with the DCT coefficients of the block to be coded x (i). Conceivable.

Next, the processing procedure of the adaptive intra-screen DCT coefficient prediction method used in the first embodiment will be described with reference to FIGS. FIG. 5 is a flowchart illustrating a processing procedure of the adaptive in-screen DCT coefficient prediction method according to the present embodiment.

In step 51, the DCT type of the encoded block x (i) is determined, and the subsequent processing differs depending on the result of this determination. That is, when the coded block x (i) has been subjected to the frame DCT processing, in step S52, the reference block r0 for the coded block x (i) shown in FIG.
A predicted value of the DCT coefficient of the block to be coded x (i) is generated by frame prediction referring to (i), r1 (i) and r2 (i).

On the other hand, if the coded block x (i) has been subjected to the field DCT processing, the reference block r0 (i) for the coded block x (i) shown in FIG. , R1 (i) and r2
(I) A predicted value of the DCT coefficient of the encoded block x (i) is generated by the referred field prediction.

As described above, the block to be coded x
By switching the reference block used for prediction according to the DCT type of (i), the DCT coefficient of a block having a high correlation of the DCT coefficient with the block to be coded x (i) can be used for prediction, As a result, efficient prediction can be performed.

Next, the processing procedure of the frame prediction method in step S52 shown in FIG. 5 will be described with reference to the flowchart in FIG. In FIG. 6, r0 (i), r1 (i),
r2 (i) and x (i) indicate the reference block and the coded block in FIG. 4A, respectively. In the processing procedure of the frame prediction method in FIG. 6, a reference block that has been subjected to frame DCT processing, that is, a reference block of the same DCT type as the block to be coded x (i) is preferentially used for prediction.

First, in step S611a, the reference block r2 on the left of the block to be coded x (i)
The DCT type of (i) is determined. The subsequent processing differs depending on the result of this determination. Next, step S612a
And S613a, the coded block x (i)
Of the reference block r1 (i) located adjacent to and above
The CT type is determined. The subsequent processing differs depending on the result of this determination. Thus, the reference block r1
According to the DCT types of (i) and r2 (i), FIG.
The following four processes (A1)
To (A4).

(A1) If both the reference blocks r1 (i) and r2 (i) have been subjected to the frame DCT processing, in step S614a, the reference block r0
(I) With reference to the DCT coefficients of r1 (i) and r2 (i), a predicted value of the DCT coefficient of the block to be coded x (i) is generated by a “predetermined method 1” described later.

(A2) When the reference block r1 (i) is subjected to the field DCT and the reference block r2 (i) is subjected to the frame DCT processing, the DCT of the block R2 shown in FIG. In a manner similar to the generation of the predicted value of the DCT coefficient of the block X from the coefficient, in step S615a, the reference block r2
A block to be coded x (i) using the DCT coefficients of (i)
To generate a predicted value of the DCT coefficient of.

(A3) When the reference block r1 (i) is subjected to the frame DCT processing and the reference block r2 (i) is subjected to the field DCT processing, the conventional method is used, that is, the block R1 shown in FIG. In the same way as generating the predicted value of the DCT coefficient of the block X from the DCT coefficient, in step S616a, the reference block r
The block to be coded x using the DCT coefficient of 1 (i)
A predicted value of the DCT coefficient of (i) is generated.

(A4) If both the reference blocks r1 (i) and r2 (i) have been subjected to the field DCT processing, in step S617a, the reference frame r0
(I) With reference to the DCT coefficients of r1 (i) and r2 (i), a predicted value of the block to be coded x (i) is generated by a “predetermined method 2” described later.

In step S617a, a predetermined value such as 0 may be used as a prediction value without referring to DCT coefficients of reference blocks r0 (i), r1 (i) and r2 (i). Good. Step S613
a and step S617a are omitted, and when the reference block r2 (i) is not subjected to the frame DCT processing,
Always in step S616a, reference block r1
A block to be coded x (i) using the DCT coefficients of (i)
May be generated.

Next, the processing procedure in the field prediction method in step 53 shown in FIG. 5 will be described with reference to the flowchart in FIG. The processing procedure of the field prediction method shown in FIG. 7 is the same as the processing procedure of the frame prediction method shown in FIG.

However, in the description of the field prediction method shown in FIG. 7, reference blocks r0 (i), r1
(I), r2 (i), and encoded block x (i)
Indicate the reference block and the block to be coded shown in FIG. 4B, respectively.

That is, in the processing procedure of the field prediction method shown in FIG. 7, the same D as that of the block to be coded x (i) is used.
A CT type reference block (a reference block on which field DCT processing has been performed) is preferentially used for prediction.

First, in step S711b, the reference block r2 adjacent to the left of the encoded block x (i)
The DCT type of (i) is determined. The subsequent processing differs depending on the result of this determination. Next, step S712b
And S713b, the coded block x (i)
The DCT type of the reference block r1 (i) located on the upper side is determined. The subsequent processing differs depending on the result of this determination. In this way, the field prediction process shown in FIG. 7 is performed by the following four processes (B1) to (B) depending on the DCT type of the reference blocks r1 (i) and r2 (i).
4).

(B1) If both the reference blocks r1 (i) and r2 (i) have been subjected to the field DCT processing, in step S714b, the reference block r0
(I) With reference to the DCT coefficients of r1 (i) and r2 (i), a predicted value of the DCT coefficient of the block to be coded x (i) is generated by a “predetermined method 1” described later.

(B2) When the reference block r1 (i) has been subjected to the frame DCT processing and the reference block r2 (i) has been subjected to the field DCT processing, the conventional method is used, that is, the block R2 shown in FIG. In the same way as generating the predicted value of the DCT coefficient of the block X from the DCT coefficient, in step S715b, the reference block r
2 (i) using the DCT coefficients
A predicted value of the DCT coefficient of (i) is generated.

(B3) When the reference block r1 (i) is subjected to the field DCT processing and the reference block r2 (i) is subjected to the frame DCT processing, the conventional method is used, that is, the block R1 shown in FIG. In the same manner as generating the predicted value of the DCT coefficient of the block X from the DCT coefficient, in step S716b, the reference block r
The block to be coded x using the DCT coefficient of 1 (i)
A predicted value of the DCT coefficient of (i) is generated.

(B4) If both the reference blocks r1 (i) and r2 (i) have been subjected to the frame DCT processing, in step S717b, the reference frame r0
(I) With reference to the DCT coefficients of r1 (i) and r2 (i), a predicted value of the block to be coded x (i) is generated by a “predetermined method 2” described later.

In step S717b, the reference blocks r0 (i), r1 (i), and r2 (i)
, A predetermined value such as 0 may be used as the predicted value. Also, steps S713b and S717b are omitted, and the reference block r2 (i)
Has not been subjected to the field DCT processing, always in step S716b, the D of the reference block r1 (i)
A predicted value of the DCT coefficient of the block to be encoded x (i) may be generated using the CT coefficient.

As described above, as in the prediction method shown in the processing procedure of FIGS. 6 and 7, the DCT type block between the block to be coded and the block of the same DCT type,
Efficient prediction can be performed by giving priority to a reference block having a high correlation of T coefficients and using the reference block for prediction of an encoded block.

Next, step S61 shown in FIG.
The processing procedure of the predicted value generation method based on “predetermined method 1” of 4a or “predetermined method 2” of step S617a will be described with reference to the flowchart shown in FIG.

In the method of FIG. 8, in order to perform the same processing as the conventional DCT coefficient prediction method, in the processing of steps S821a to S829a, the DCT coefficient of each block corresponding to the four blocks shown in FIG. Is assumed to be a virtual memory space (virtual buffer) for storing data, and a conventional D is used for each block on the virtual buffer.
Apply the CT coefficient prediction method.

In FIG. 8, R0, R1 and R2 represent reference blocks on the virtual buffer, and DC0, DC1
And DC2 represent the DC components of the DCT coefficients of the reference blocks R0, R1, and R2 on the virtual buffer, respectively. In the description of the prediction value generation method in FIG.
0 (i), r1 (i), r2 (i) and x (i) are shown in FIG.
The reference block and the block to be coded having the positional relationship shown in FIG. In the processing shown in FIG.
In steps S821a, S822a, and S823a, the DCT coefficient of the reference block R0 is generated. The DCT coefficient of the reference block R0 is determined based on the DCT type determination result of the reference block r0 (i) in step S821a. Is different.

That is, when the reference block r0 (i) has been subjected to the field DCT processing, step S822 is performed.
a, a DCT coefficient is generated from a block near the reference block r0 (i) by a predetermined method,
The CT coefficients are stored in the virtual buffer as DCT coefficients of the reference block R0. On the other hand, reference block r0 (i)
Have been subjected to the frame DCT processing, step S8
At 23a, the DCT coefficient of the reference block r0 (i) is stored in the virtual buffer as the DCT coefficient of the reference block R0. In the same manner as above, step S824a,
In steps 825a and 826a, DCT coefficients of the reference block R1 are generated, and steps S827a and 828a are performed.
And in step S829a, the reference block R2
Are generated and stored in the virtual buffer.

The subsequent processing is the same as the conventional DCT coefficient prediction method. In step S830a, the absolute value (| DC0-DC1 |) of the difference between the DC components of the DCT coefficients of the reference blocks R0 and R1 and the reference block R0 And R
2 is the absolute value of the difference between the DC components of the DCT coefficient (| DC0−D
C2 |) are compared. The absolute value of the difference between the DC components of the DCT coefficients of the reference blocks R0 and R2 (| DC0−
DC2 |) is smaller than the absolute value of the difference between the DC components of the DCT coefficients of the reference blocks R0 and R1 (| DC0-DC1 |), the coded block using the DCT coefficients of the reference block R1 in step S832a. x
A predicted value of the DCT coefficient of (i) is generated. Otherwise, in step S831a, the reference block R2
Of the block to be coded x (i) using the DCT coefficients of
A predicted value of the T coefficient is generated.

In the "predetermined method 1" of step S814a of FIG. 6, since it is known that the DCT type of the reference blocks r1 (i) and r2 (i) is a frame, the step S824a of FIG. , Step S
827a, step S825a, and step S828
The processing by a can be omitted.

Next, the processing procedure in “predetermined method 1” in step S714b or “predetermined method 2” in step S717b shown in FIG. 7 will be described with reference to the flowchart shown in FIG.

The processing procedure shown in the flowchart of FIG.
In the processing procedure shown in the flowchart of FIG. 8, a process of determining whether each block is subjected to frame DCT is performed.
This is replaced with a process of determining whether or not each block is subjected to the field DCT.
This is the same as the processing shown in FIG. In FIG. 9, reference blocks r0 (i), r1 (i), r2 (i), and x (i) represent a reference block and a coded block having the positional relationship shown in FIG. 4B, respectively. ing.

In the processing shown in FIG. 9, first, in step S9
21b, Step S922b, and Step S923b
In step S921b, the DCT coefficient of the reference block R0 is generated.
The subsequent process of generating the DCT coefficient of the reference block R0 differs depending on the result of the DCT type determination of 0 (i).

That is, if the reference block r0 (i) has been subjected to the frame DCT processing, step S922b
, A DCT coefficient is generated from a block near the reference block r0 (i) by a predetermined method, and the generated DCT coefficient is stored in the virtual buffer as a DCT coefficient of the reference block R0. On the other hand, the reference block r0
If (i) has been subjected to the field DCT processing, in step S923b, the D of the reference block r0 (i)
The CT coefficients are stored in the virtual buffer as DCT coefficients of the reference block R0. Step S9 as described above.
At 24b, 925b and 926b, the DCT coefficient of the reference block R1 is generated, and at step S927b,
In step 928b and step S929b, DCT coefficients of the reference block R2 are generated and stored in the virtual buffer.

The subsequent processing is the same as the conventional DCT coefficient prediction method. In step S930b, the absolute value (| DC0-DC1 |) of the difference between the DC components of the DCT coefficients of the reference blocks R0 and R1 and the reference block R0 And R
2 is the absolute value of the difference between the DC components of the DCT coefficient (| DC0−D
C2 |) are compared. The absolute value of the difference between the DC components of the DCT coefficients of the reference blocks R0 and R2 (| DC0−
DC2 |) is smaller than the absolute value of the difference between the DC components of the DCT coefficients of the reference blocks R0 and R1 (| DC0-DC1 |), the coded block using the DCT coefficients of the reference block R1 in step S932b. x
A predicted value of the DCT coefficient of (i) is generated. Otherwise, in step S931b, the reference block R2
Of the block to be coded x (i) using the DCT coefficients of
A predicted value of the T coefficient is generated.

In the "predetermined method 1" of step S714b of FIG. 7, since it is known that the DCT type of the reference blocks r1 (i) and r2 (i) is a field, the step S924b of FIG. , 927
b, the processing in step S925b and step S928b can be omitted.

Thus, the DC of the block to be coded is
In the prediction process of the T coefficient, when it is necessary to refer to the DCT coefficient of a block of a DCT type different from the block to be coded, instead of directly referencing the DCT coefficient of a DCT type different from the block to be coded, a reference block is used. From the block in the vicinity of the same DC
Efficient prediction can be performed by generating a DCT coefficient that is close to the characteristics of the T-type DCT coefficient and predicting the DCT coefficient of the encoded block with reference to the generated DCT coefficient.

FIG. 10 shows steps S822a and S822 in FIG.
25a, 828a, steps S922b, S92 in FIG.
FIG. 5B is a conceptual diagram illustrating a method of generating DCT coefficients from a block near a reference block r in 5b and S928b.

In the description of FIGS. 10 and 11, r
Indicates one of a reference block r0 (i), a reference block r1 (i), and a reference block r2 (i) in the frequency domain, and R indicates a reference block R0,
One of the reference block R1 and the reference block R2 is shown.

Steps S822a, 825a, and 8 in FIG.
28a, and steps S922b, S925b,
In the process of generating the DCT coefficient from the block near the reference block r in S928b, as shown in FIG. 10, the DCT coefficient is generated using the predetermined function from the DCT coefficients of the two blocks near the reference block r, The generated DCT coefficient is used as the DCT of the reference block R on the virtual buffer.
It is a coefficient.

FIG. 11 shows a procedure for generating DCT coefficients of the reference block R on the virtual buffer. Here, D
Different processing is performed depending on the position of the reference block r in the macroblock on the CT area.

For example, at the block position (0) or (2) in the macro block in the DCT area,
If the reference block r is located on the left side of the macroblock in the T region, in step S1142L, the D of the block located in the block positions (0) and (2) of the macroblock in the DCT region including the reference block r is determined.
From the CT coefficients, as shown in FIG. 10, a DCT coefficient of the reference block R on the virtual buffer is generated using a predetermined function. If the reference block r is located at the block position (1) or (3) of the macroblock in the DCT area, that is, at the right position of the macroblock in the DCT area, in step S1142R, the reference block r
As shown in FIG. 10, the DCT coefficient of the reference block R on the virtual buffer is calculated from the DCT coefficients of the blocks located at the block positions (1) and (3) of the macro block in the DCT region including Generate

As the predetermined function used here, a function that sets the average or weighted average of the DCT coefficients of two blocks located near the reference block r on the frequency domain as the DCT coefficient of the reference block R on the virtual buffer is used. Such,
Any function can be used as long as it can uniquely calculate the value of the DCT coefficient of the reference block R on the virtual buffer from the DCT coefficients of the two blocks near the reference block r.

Then, the DCT coefficient generated by referring to the two blocks in the frequency domain is used as the DCT coefficient of the reference block R on the virtual buffer in step S1143.

In this way, the DC of two blocks having the same area information as the reference block r in the spatial area is obtained.
By generating the DCT coefficient of the reference block R on the virtual region used for prediction from the T coefficient, the DCT coefficient close to the frequency characteristic of the DCT coefficient of the same DCT type as the block to be coded is obtained.
A T coefficient can be generated.

As described above, in the adaptive intra-screen DCT coefficient prediction method used in the first embodiment, the reference block used for prediction is switched according to the DCT type of the coded block, and the same DCT as the coded block is switched. When the DCT coefficient of the reference block of the type is preferentially used for prediction, and when the reference block is a DCT type different from the block to be coded, the DCT type of the reference block is determined based on the DCT coefficients of blocks near the reference block. DC of the reference block when it is the same as the DCT type of the block to be coded
Since a DCT coefficient of a block having a frequency characteristic close to that of the T coefficient is generated and used for prediction, intra-screen prediction of an interlaced image signal or a special progressive image can be efficiently performed in a DCT region (frequency component).

As a result, according to the first embodiment, in the encoding processing of the MPEG4 system for an interlaced image or a special progressive image in which macroblocks of different DCT types are mixed as macroblocks to be processed. It is possible to efficiently perform compression encoding of an image signal by removing or reducing spatially redundant image information by improving the efficiency of predicting the DCT coefficient of the block to be encoded using information in the screen. Become.

In the adaptive intra-screen DCT coefficient prediction method used in the first embodiment, the DCT coefficient of reference block r2 (i) shown in FIGS. 4A and 4B is used to encode block x (i). When the DCT type of the reference block r2 (i) is different from the DCT type of the coded block x (i) when using as the predicted value of the DCT coefficient of the It may be used as a predicted value of the DCT coefficient of the encoded block x (i). In some cases, the reference block r1 (i)
Also, only the DC component of the DCT coefficient may be used as the predicted value of the DCT coefficient of the encoded block x (i).

In the first embodiment, in steps S923b, S926, and S929b, reference blocks r (specifically, reference blocks r0 (i), r0
1 (i), r2 (i)), a reference block R (specifically, reference blocks R0, R1, R1) on a virtual buffer used for predicting a DCT coefficient of a block to be encoded.
Although the DCT coefficient of 2) is generated, the step S9 is performed.
In steps 23b, S926, and S929b, similarly to steps S922b, S925, and S928b, the reference block r (specifically, the reference blocks r0 (i), r1 (i), and r2 (i)) is used in the spatial domain. A DCT coefficient may be generated from the DCT coefficients of two blocks located in the vicinity using a predetermined function, and the generated DCT coefficient may be used as the DCT coefficient of the reference block R on the virtual buffer.

Further, in this case, in the field prediction process shown in FIG. 7, the reference block r2 (i) adjacent to the left of the block to be coded x (i) and the block to be coded x
Reference block r1 located adjacent to and above (i)
According to the DCT type of (i), the above four processes B
One of the processes B1 'to B4' is performed by performing the processes B1, B3 'and B4' shown below.
May be replaced by

In the process B3 ', the coded block adjacent to the upper side of the block to be coded (ie, FIG.
(Encoded block located between encoded block x (i) and encoded block r1 (i) shown in (b))
Is used as a reference block, and the DCT coefficient of this reference block is used as a predicted value of the DCT coefficient of the block to be coded.

Regarding the processing B1 'and the processing B4', in steps S922b and 923b, the encoded block located between the reference block r0 (i) and the reference block r0 (i) and the reference block r2 (i) is determined. Blocks and
A DCT coefficient of the reference block R0 on the virtual buffer is generated by performing weighted averaging with a weighting ratio of 0 to 1, and in steps S925b and 926b, the reference block r1 (i)
And the reference block r1 (i) and the encoded block x
The coded block located between (i) is 0: 1
Weighted average with the weighting ratio of, to generate the DCT coefficient of the reference block R1 on the virtual buffer, and step S9
In 28b and 929b, the reference block r2 (i) and the encoded block located between the reference block r2 (i) and the reference block r0 (i) are weighted and averaged at a weighting ratio of 1: 0. , Generate the DCT coefficient of the reference block R1 on the virtual buffer. In the processes B1 'and B4', each block r0 (i)
To r2 (i) correspond to the encoded block x (i) in FIG.
It is assumed that it is located at the position shown in FIG.

In other words, the processing B1 'and the processing B
4 'is the weighted average between the reference block r0 (i) and the adjacent coded block below,
The weighting ratio of one of the two blocks closer to the block to be coded x (i) is set as 1, and the reference block r1
A weighted average between (i) and a coded block located adjacent to the lower side is set as a ratio of one of the two blocks closer to the block to be coded x (i) being 1, and
Further, the weighted average between the reference block r2 (i) and the coded block positioned adjacent to the upper side of the reference block r2 (i) is defined as a ratio of one of the two blocks closer to the coded block x (i) being 1. Is what you do.

In this case, the arithmetic processing for the weighted averaging is simplified, and the encoding process is performed by referring to the frequency component of the encoded sub-block spatially closest to the encoded sub-block. Since a predicted value of the frequency component of the sub-block is generated, the prediction efficiency of the entire encoding process for an interlaced or specific progressive image is improved by an adaptive in-screen DCT coefficient prediction method using a simple operation process. be able to.

Embodiment 2 The image processing device (image decoding device) according to the second embodiment decodes an image coded signal using the adaptive intra-screen DCT coefficient prediction method used in the image coding device described in the first embodiment. It is characterized in that

FIG. 2 is a block diagram of an image decoding apparatus according to the second embodiment.
Or show the equivalent. This image decoding device 2000
An image encoding signal (bit stream) 110b obtained by encoding an image signal by the image encoding apparatus 1000 according to the first embodiment is received, and decoding using an adaptive intra-screen DCT coefficient prediction method is performed on the signal. Processing is performed.

That is, the image decoding apparatus 2000
Receives the bit stream 110b output from the image encoding apparatus 1000, performs variable length decoding on the bit stream 110b by analyzing the data, and obtains a DCT coefficient difference value 108 (DCT coefficient quantization value of the encoded block) corresponding to the decoded block. Variable length decoder (VLD unit) 203 for restoring the decoded value 107 and a difference value between the intra prediction value 111 thereof, and an intra prediction processing unit 210 for generating the intra prediction value 111 for the decoded block. , The intra prediction value 111 and the DCT
An adder 112 for adding the coefficient difference value 108 and restoring the DCT coefficient quantization value 106 for the block to be decoded
And

Here, the intra-screen prediction processing section 210
A block memory 11 for storing an output 106 of the adder 112 as a DCT coefficient quantization value of a decoded block.
5 in accordance with the DCT type signal 102 from the image encoding apparatus 1000 and the DCT coefficient prediction method for adaptively changing the intra-screen DCT coefficient from the DCT coefficient quantization value 114 of the decoded block stored in the block memory 115. The DCT coefficient predictor 113 generates a predicted value 111 for the DCT coefficient quantized value of the decoded block.

The image decoding apparatus 2000 performs an inverse quantization process on the output 106 of the adder 112 to restore the DCT coefficient 104 for the block to be decoded. An inverse DCT unit 209 that performs an inverse DCT process on the output of the inverse quantizer 207 to restore the image signal 101 for the block to be decoded.
A deblocker 200 for restoring an image signal 110a having a scanning line structure based on the DCT type signal 102 from 0
And

Next, the operation will be described. When the encoded image signal 110b from the image encoding device 1000 is input to the image decoding device 2000, the image encoded signal 1
10b is subjected to variable length decoding by data analysis in the VLD unit 203, and is output as a DCT coefficient difference value 108 for the block to be decoded.

The DCT coefficient difference value 108 for the block to be decoded is added to the predicted value 111 by the adder 112 to restore the quantized DCT coefficient 106 for the block to be decoded.

At this time, the DCT coefficient quantization value 106 for the above-mentioned block to be decoded is
10 and is stored in the block memory 115 as the DCT coefficient quantization value of the decoded block. Further, the DCT coefficient predictor 113 reads out the DCT coefficient quantization value 114 corresponding to the decoded block from the block memory 115, and here, based on the DCT type signal 102 from the image coding apparatus 1000. With reference to the DCT coefficient quantization value 114 from the block memory 115, the adaptive DCT coefficient prediction processing for generating a prediction value for the DCT coefficient difference value 108 of the next decoded block to be processed next to the above-mentioned decoded block is performed. , Is performed in the same manner as the predicted value generation processing in the intra-screen prediction processing unit 110 of the image encoding device 1000.

Further, the DCT coefficient quantization value 106 is
The inverse quantizer 207 transforms the block to the DCT coefficient 104 for the block to be decoded by an inverse quantization process. The DCT coefficient 104 is converted into an image for the block to be decoded by an inverse DCT unit 209 by an inverse discrete cosine transform. It is converted into a signal 101.

When the image signal 101 for the block to be decoded is supplied to the inverse blocker 200, the inverse blocker 200 scans the scanning line based on the DCT type signal 102 from the image encoding device 1000. The image signal 110a having the structure is reproduced.

As described above, in the second embodiment, since the image coded signal is decoded using the adaptive intra-screen DCT coefficient prediction method, the image signal corresponding to the interlaced image or the special progressive image is obtained. Adaptive in-screen DC
An image coded signal (bit stream) obtained by performing intra prediction coding using the T coefficient prediction processing can be efficiently and correctly decoded by the intra prediction processing in the DCT domain.

Embodiment 3 Next, Embodiment 3 of the present invention
An image processing apparatus (image coding apparatus) according to the first embodiment will be described. The image coding apparatus according to the third embodiment uses the frame prediction performed in step S52 of the prediction value generation procedure shown in FIG.
The process of generating the predicted value of the CT coefficient is performed according to the processing procedure shown in FIG.
The process of generating the predicted value of the DCT coefficient of the block to be coded by the field prediction described above is performed by the processing procedure shown in FIG.

The adaptive intra-screen DCT coefficient prediction processing by the image encoding apparatus according to the third embodiment is the same as the adaptive intra-screen DCT coefficient prediction processing according to the first embodiment except for the frame prediction method and the field prediction method. Therefore, only the frame prediction method and the field prediction method according to the third embodiment will be described with reference to FIGS.

FIG. 12 is a flowchart showing a processing procedure in the third embodiment for realizing the frame prediction method in step S52 shown in FIG. In FIG.
r0 (i), r1 (i), r2 (i) and x (i)
Represent the reference block and the block to be coded shown in FIG.

In the third embodiment, first, at step S
At 1221a, the DCT type of the reference block r2 (i) is determined. The subsequent processing differs depending on the result of this determination.

That is, when the reference block r2 (i) has been subjected to the frame DCT processing, the DCT coefficient of the coded block x (i) is determined using the prediction value generation method shown in FIG. Generate predicted values. 8 is the same as that described in the first embodiment, and a description thereof will not be repeated.

In the “predetermined method” of step S1222a in FIG. 12, since it is known that the DCT type of r2 (i) is a frame DCT, FIG.
Steps S827a and S828a can be omitted.

On the other hand, when the reference block r2 (i) has been subjected to the field DCT processing, encoding is performed in the same manner as in the conventional method shown in FIG. 15, that is, as shown in FIG. 15, from the DCT coefficients of the reference block R1. In the same manner as generating the predicted value of the DCT coefficient of the block X, the reference block r
The block to be coded x using the DCT coefficient of 1 (i)
A predicted value of the DCT coefficient of (i) is generated.

FIG. 13 is a flowchart showing a processing procedure in the third embodiment for realizing the field prediction method in step S53 shown in FIG. The processing shown in FIG. 13 is the same as the processing of FIG. 12 except that the frame and the field are exchanged. Since the outline of the processing is the same, the description thereof is omitted here. However, in the description of the field prediction method in FIG.
0 (i), r1 (i), r2 (i) and x (i) are
Each of them represents the reference block and the block to be coded in FIG.

As described above, in the adaptive intra-screen DCT coefficient prediction method in the image encoding apparatus according to the third embodiment, the frame prediction method in step S52 and the field prediction method in step S53 in FIG. By simplifying compared to the first mode, it is possible to simplify and speed up the intra prediction process in the DCT domain at the time of encoding.

Embodiment 4 Next, Embodiment 4 of the present invention
An image processing device (image decoding device) will be described. The image decoding apparatus according to the fourth embodiment uses the frame prediction in step S52 of the prediction value generation procedure shown in FIG.
The process of generating the predicted value of the CT coefficient is performed according to the processing procedure shown in FIG.
The process of generating the predicted value of the DCT coefficient of the block to be coded by the field prediction in the above is performed by the processing procedure shown in FIG.

In the image decoding apparatus according to the fourth embodiment having such a configuration, when performing the adaptive intra-screen DCT coefficient prediction method, the frame prediction method in step S52 and the field prediction method in step S53 in FIG. Therefore, the present embodiment is simplified as compared with the second embodiment, thereby simplifying and speeding up the intra prediction process in the DCT domain at the time of decoding.

Embodiment 5 FIG. An image processing apparatus (image encoding apparatus) according to the fifth embodiment of the present invention includes a DCT type signal of an encoded block (that is, whether the encoded block has been subjected to frame DCT processing or not.
A method of generating a predicted value of the DCT coefficient of the coded block from the DCT coefficient of the coded block having a predetermined positional relationship with the coded block regardless of whether the T processing has been performed. Is used to perform intra prediction coding of an image signal. Where DCT
The type signal indicates a signal indicating whether the coded block is subjected to frame DCT processing or field DCT processing. The block is 16 × 16
It is assumed that four sub-blocks each composed of 8 × 8 pixels that constitute a macro block composed of pixels are shown. These four sub-blocks are upper left (block position (0) in FIG. 3), upper right (block position (1) in FIG. 3), lower left (block position (2) in FIG. 3), lower right ( It is located at the block position (3) in FIG.

FIG. 17 is a block diagram showing a configuration of an image coding apparatus according to the fifth embodiment. In the figure, 3
Reference numeral 000 denotes an image encoding apparatus according to Embodiment 5, which converts an input digital image signal (input image signal) 110a into
An image space (frame) formed by this is divided so as to correspond to each of a plurality of macroblocks to be divided, and an image signal corresponding to a block constituting each macroblock is encoded for each of the blocks. I have.

That is, the image encoding device 3000
In the same manner as in the image encoding apparatus 1000 according to the first embodiment, the input image signal 110a is divided into blocks corresponding to the respective blocks for each frame or field serving as a unit of frequency conversion processing, and And a DCT type signal 102 indicating a processing unit of the frequency conversion (DCT processing). The blocker 100 receives the input image signal 110a, and if the correlation between pixel values between fields is higher than that in the frame, the blocker 100 performs one-step processing so that field DCT processing is performed.
The arrangement is such that the scanning lines are rearranged in units of macroblocks of 6 × 16 pixels, and an image signal is output for each block of 8 × 8 pixels constituting the macroblock in which the rearrangement of the scanning lines has been performed. ing. When the correlation between the pixel values between the fields is smaller than that in the frame, the blocking unit 100 does not rearrange the scanning lines in units of macroblocks as described above, and Is output for each of the above blocks.

More specifically, in the blocker 100, the reordering process of the scanning lines is performed in such a manner that the image of the first field formed by the image signals corresponding to the odd-numbered horizontal pixel columns (horizontal scanning lines) is obtained. An image of a second field formed by an image signal corresponding to an even-numbered horizontal pixel column (horizontal scanning line), which is located above the macroblock, that is, at block positions (0) and (1), is This is performed so as to be located below the block, that is, at block positions (2) and (3).

The block generator 100 divides an image signal corresponding to a macroblock and outputs the divided image signal corresponding to the blocks at the block positions (0) to (3).

The image coding apparatus 3000 performs the discrete cosine transform on the image signal 101 corresponding to the block to be coded, as in the image coding apparatus 1000 of the first embodiment. (DCT processing) and an output 104 of the DCT 103
105, and the intra prediction processing unit 3 for generating the predicted value 111 corresponding to the above-mentioned block to be coded.
10, and an adder 107 that subtracts the predicted value 111 from the output (DCT coefficient quantized value) 106 of the quantizer 105 and outputs a DCT coefficient difference value 108,
The DCT coefficient difference value 108 is subjected to variable-length encoding by the VLC unit 109, and is output as a bit stream (encoded image signal) 110b.

Further, the intra prediction processing section 310 includes an adder 112 for adding the DCT coefficient difference value 108 and the intra prediction value 111, and outputs the output of the adder 112 to the DCT coefficient quantum of the encoded block. Block memory 115 for storing as a coded value 116,
The DCT coefficient predictor 313 generates the prediction value 111 of the coefficient quantization value from the DCT coefficient quantization value 114 of the coded block adjacent to the block to be coded in the image space.

In the fifth embodiment, as shown in FIG. 19, the DCT coefficient predictor 313 is adjacent to the upper left of the coded block x (i) regardless of the DCT type of the coded block. The block r0 (i) that is located and the block r that is located adjacent to the upper side of the block to be coded x (i)
1 (i) and a block r2 (i) located on the left of the coded block x (i) are used as reference blocks, and the predicted value 111 of the DCT coefficient quantization value of the coded block x (i) is used as a reference block. Is generated.

It should be noted that the DCT unit 103, the quantizer 105, the adders 107 and 112, the VL
The C unit 109 and the block memory 115 have the same configuration as that of the first embodiment.

Next, the operation will be described. First, the overall operation of the image coding apparatus according to the fifth embodiment will be briefly described. Digital image signal (input image signal) 11
When 0a is input to the present image encoding device 3000, the input image signal 110a is divided into blocks by a blocker 100 so as to correspond to each of the blocks for each frame or field as a unit of frequency conversion processing. At the same time, the blocked image signal 101 and the DCT type signal 102 indicating the processing unit of the frequency conversion (DCT processing) are output.

At this time, if the correlation between the pixel values between the fields is higher than that in the frame, the blocking unit 100 sets a macro composed of 16 × 16 pixels in advance so that the field DCT processing is performed. A scanning line rearrangement process is performed on the image signal in units of blocks, and the image signal subjected to the scan line rearrangement process is output for each block of 8 × 8 pixels constituting the macro block. Is done. In this case, in the macroblock on which the scanning line rearrangement processing has been performed, the image of the first field formed by the image signal corresponding to the odd-numbered horizontal pixel column (horizontal scanning line) is located above the macroblock, that is, An image of a second field formed by an image signal corresponding to an even-numbered horizontal pixel column (horizontal scanning line) located at block positions (0) and (1) is located below the macroblock, that is, the block. They will be located at positions (2) and (3).

When the correlation between pixel values between fields is smaller than that in a frame, the blocking unit 100 performs the above-described processing for rearranging scanning lines in units of macroblocks. Instead, the input image signal is output for each block.

The image signal 101 of the coded block to be coded is subjected to a discrete cosine transform (DCT process) by the DCT unit 103 to obtain a frequency component (DCT coefficient) corresponding to the coded block. The DCT coefficient 104 is further transformed into a quantizer 105
, And is output as a quantization value (DCT coefficient quantization value) 106 for the block to be coded.

Further, when the DCT coefficient quantized value 106 of the above-mentioned block to be coded is supplied to the adder 107, the difference between this quantized value 106 and its predicted value 111 is calculated and output as a DCT coefficient difference value 108. Is done. This DCT
The coefficient difference value 108 is subjected to variable-length coding by a VLC unit 109 and a bit stream (coded image signal) 110
Output as b.

The D output from the adder 107
The CT coefficient difference value 108 is supplied to an intra-screen prediction processing unit 310, where a prediction value for the DCT coefficient quantization value 106 is generated.

That is, in the intra-screen prediction processing section 310, the DCT coefficient difference value 108 and the intra-screen prediction value 111 are added by the adder 112, and the added value is added to the DCT coefficient quantization value 116 of the coded block. Is stored in the block memory 115. Then, the DCT coefficient predictor 313 generates a predicted value 111 of the quantized DCT coefficient of the block to be encoded from the quantized DCT coefficient 114 of the encoded block.

Next, the intra-screen DC in the above-mentioned encoding process
The T coefficient prediction method will be described in detail. The intra-screen DCT coefficient prediction method according to the fifth embodiment differs from the first embodiment in generating a predicted value of a DCT coefficient corresponding to an encoded block, regardless of the DCT type of the encoded block. Instead, it always refers to a coded block having a predetermined positional relationship with the coded block.

Also in the fifth embodiment, the first embodiment
Similarly to the DCT domain (frequency domain), an image signal forming an image space (spatial domain) is subjected to DCT processing (frequency conversion).
It is assumed that each macroblock is arranged in the DCT domain (frequency domain) as in the arrangement of macroblocks in the spatial domain (image space).

Also, in the fifth embodiment, similarly to the first embodiment, when a macroblock is subjected to frame DCT processing as shown in FIG. 3, each block is not rearranged in the macroblock without rearranging the scanning lines. DC to image signal
DCT corresponding to each of the upper left, upper right, lower left, and lower right blocks in the macroblock in the spatial domain after T processing is performed.
Coefficients are arranged in blocks at block positions (0), (1), (2), and (3) in a macroblock in the DCT domain, respectively. On the other hand, when the macroblock is subjected to field DCT processing, the spatial domain is used. After rearranging the scanning lines in the upper macro block, DCT processing is performed on the image signal of each block, and the DCT coefficients of the first field left, first field right, second field left, and second field right blocks are calculated. , Each of the block positions (0), (1), (2),
It is assumed to be arranged in the block (3).

Next, the DCT corresponding to the encoded block
Coefficient (data of encoded block in DCT domain)
Will be described in detail with reference to the DCT coefficient of the coded block in accordance with the fifth embodiment.

First, the block to be encoded is a frame DCT.
Regardless of whether it is processed or field DCT processed, as shown in FIG.
A block located at the upper left of (i) is referred to as a reference block r0.
(I), a block positioned adjacent to the upper side of the block to be coded x (i) is a reference block r1 (i), and a block positioned to the left of the block to be coded x (i) is a reference block r2 (i). ).

Then, as shown in the flowchart of FIG. 20, similar to the conventional DCT coefficient prediction method, step S2 is performed.
At 130a, reference blocks r0 (i) and r1
The absolute value of the difference between the DC components of the DCT coefficient of (i) (| DC0
−DC1 |) and reference blocks r0 (i) and r2
The absolute value of the difference between the DC components of the DCT coefficient of (i) (| DC0
−DC2 |) is compared. Reference block r0
The absolute value (| DC0−DC2 |) of the difference between the DC components of the DCT coefficients of (i) and r2 (i) is equal to the reference block r0.
If the absolute value (| DC0−DC1 |) of the difference between the DC components of the DCT coefficients of (i) and r1 (i) is smaller than the absolute value (| DC0−DC1 |), the encoding is performed using the DCT coefficient of the reference block r1 (i) in step S2132a. DCT of block x (i)
A predicted value of the coefficient is generated. Otherwise, in step S2131a, the DC of the reference block r2 (i) is
A predicted value of the DCT coefficient of the encoded block x (i) is generated using the T coefficient.

As described above, in the fifth embodiment, in the process of predicting the DCT coefficient of the block to be coded, the coded block having a predetermined positional relationship with respect to the block to be coded is used as the reference block. Of the correlation of DCT coefficients between reference blocks arranged vertically adjacent in the vicinity of the coding block and the magnitude of correlation of DCT coefficients between reference blocks arranged horizontally in the vicinity of the block to be coded. A direction in which the DCT coefficient has a strong correlation is obtained, a reference block located in a direction in which the DCT coefficient has a strong correlation with the coded block is selected, and a DCT coefficient of the coded block is predicted from the DCT coefficient of the selected reference block. Since the value is obtained, the intra prediction for an interlaced image signal or a special progressive image is efficiently performed in the DCT domain (frequency component). It can be performed by a single processing procedure.

As a result, according to the fifth embodiment, in the encoding processing of the MPEG4 system for an interlaced image or a special progressive image in which macroblocks of different DCT types are mixed as macroblocks to be processed. It is possible to efficiently and easily perform compression coding of image signals by reducing spatially redundant image information by improving the efficiency of predicting the DCT coefficient of the block to be coded using information in the screen. Becomes

In the intra-screen DCT coefficient prediction method used in the fifth embodiment, the DCT coefficient of reference block r1 (i) or reference block r2 (i) in FIG. When used as the predicted value of the coefficient, only the DC component of the DCT coefficient of the reference block r1 (i) or the reference block r2 (i) is
It may be used as a predicted value of the DCT coefficient of the block to be coded x (i).

Embodiment 6 FIG. The image processing apparatus (image decoding apparatus) according to the sixth embodiment performs decoding of an image coded signal using the in-screen DCT coefficient prediction method used in the image coding apparatus described in the fifth embodiment. It is characterized by performing.

FIG. 18 is a block diagram of an image decoding apparatus according to the sixth embodiment. The same reference numerals as in FIG. 17 denote the same parts or corresponding parts. This image decoding device 4000
Is the image encoding device 3000 according to the fifth embodiment.
Receives an image coded signal (bit stream) 110b obtained by encoding an image signal, and performs a decoding process using an intra-screen DCT coefficient prediction method.

That is, the image decoding device 4000
Receives the bit stream 110b output from the image encoding device 3000, performs variable-length decoding on the bit stream 110b by analyzing the data, and obtains a DCT coefficient difference value 108 (DCT coefficient quantization value of the encoded block) corresponding to the decoded block. Variable length decoder (VLD unit) 203 for restoring the decoded value 107 and the difference value between the intra prediction value 111), an intra prediction processing unit 410 for generating the intra prediction value 111 for the decoded block, , The intra prediction value 111 and the DCT
The adder 112 adds the coefficient difference value 108 and restores the DCT coefficient quantization value 116 for the block to be decoded.
And

Here, the intra-screen prediction processing section 410
A block memory 11 for storing an output 116 of the adder 112 as a DCT coefficient quantization value of a decoded block.
5 and the DCT coefficient of the decoded block stored in the block memory 115 from the DCT coefficient quantization value 114 of the decoded block by the intra-screen DCT coefficient prediction method in the image encoding device 3000 of the fifth embodiment. And a DCT coefficient predictor 313 that generates a predicted value 111 for the quantized value.

The image decoding apparatus 4000 also performs an inverse quantization process on the output 116 of the adder 112 to restore the DCT coefficient 104 for the block to be decoded. An inverse DCT unit 209 that performs an inverse DCT process on the output of the inverse quantizer 207 and restores the image signal 101 for the block to be decoded;
A deblocker 200 for restoring an image signal 110a having a scanning line structure based on the DCT type signal 102 from 0
And

In the deblocker 200, the image signals of the blocks belonging to the same macroblock in the image space are combined in accordance with the position of the block in the macroblock, and the image signal corresponding to the macroblock is obtained. And an image of a first field formed by image signals corresponding to odd-numbered horizontal pixel columns is formed above the macroblock and formed by image signals corresponding to even-numbered horizontal pixel columns. For the image signal of the macroblock on which the horizontal pixel column rearrangement process has been performed at the time of encoding so that the image of the second field is located below the macroblock, the first field and the second field are used. The horizontal pixel column is subjected to reverse sorting so that an image of a frame composed of two fields is formed. With respect to the image signal of the macroblock that has not been subjected to the rearrangement process, the image signal of the image space including a plurality of macroblocks is generated without performing the reverse rearrangement process of the horizontal pixel row. ing.

Next, the operation will be described. When the encoded image signal 110b from the image encoding device 3000 is input to the image decoding device 4000, the encoded image signal 1
10b is subjected to variable-length decoding by data analysis in the VLD unit 203, and is output as a DCT coefficient difference value 108 for the block to be decoded.

The DCT coefficient difference value 108 for the block to be decoded is added to the predicted value 111 by the adder 112 to restore the quantized DCT coefficient 116 for the block to be decoded.

At this time, the DCT coefficient quantization value 116 for the above-mentioned block to be decoded is
10 and is stored in the block memory 115 as the DCT coefficient quantization value of the decoded block. Further, the DCT coefficient predictor 313 reads out the DCT coefficient quantization value 114 corresponding to the decoded block from the block memory 115. Here, the DCT coefficient quantization value 114 from the block memory 115 is referred to. hand,
The DCT coefficient prediction processing for generating a prediction value for the DCT coefficient difference value 108 of the next decoding block processed next to the above-described decoding target block is performed by the intra-screen prediction processing unit 110 of the image encoding device 3000. Is performed in the same manner as described above.

Further, the DCT coefficient quantization value 116 is
The inverse quantizer 207 transforms the block to the DCT coefficient 104 for the block to be decoded by an inverse quantization process. The DCT coefficient 104 is converted into an image for the block to be decoded by an inverse DCT unit 209 by an inverse discrete cosine transform. It is converted into a signal 101.

When the image signal 101 for the block to be decoded is supplied to the inverse blocker 200, the inverse blocker 200 scans the scanning line based on the DCT type signal 102 from the image encoder 3000. The image signal 110a having the structure is reproduced.

As described above, in the sixth embodiment, the in-screen D
Since the image coded signal is decoded by using the CT coefficient prediction method, the image signal corresponding to the interlaced image or the special progressive image is obtained by intra-frame prediction encoding using the intra-frame DCT coefficient prediction process. The encoded image signal (bit stream) can be efficiently and correctly decoded by a simple intra-screen prediction process in the DCT domain.

Furthermore, an encoding or decoding program for realizing image processing by the image encoding device or the image decoding device shown in each of the above embodiments is recorded on a data storage medium such as a flexible disk. By doing so, the processing described in each of the above embodiments can be easily performed in an independent computer system.

FIG. 14 illustrates a case where the image encoding process or the image decoding process of the first to sixth embodiments is performed by a computer system using a flexible disk storing the encoding or decoding program. FIG.

FIG. 14A shows the appearance, cross-sectional structure, and flexible disk body of the flexible disk as viewed from the front, and FIG. 14B shows an example of the physical format of the flexible disk body.

The flexible disk FD has a structure in which the flexible disk body D is accommodated in a flexible disk case FC. The surface of the flexible disk body D is concentrically arranged from the outer periphery toward the inner periphery. A plurality of tracks Tr are formed, and each track Tr is divided into 16 sectors Se in the angular direction.
Therefore, in the flexible disk FD in which the program is stored, the flexible disk body D has data as the program recorded in an area (sector) Se allocated thereon.

FIG. 14C shows a configuration for recording the program on the flexible disk FD and performing image processing using the program stored on the flexible disk FD.

The above program is stored on a flexible disk F
In the case of recording on D, data as the above program is written from the computer system Cs to the flexible disk FD via the flexible disk drive FDD. When the arbitrary shape encoding device or the arbitrary shape decoding device is constructed in the computer system Cs by the program recorded on the flexible disk FD, the program is read from the flexible disk FD by the flexible disk drive FD, and Load to Cs.

In the above description, a flexible disk is used as a data storage medium. However, even when an optical disk is used, encoding or decoding by software can be performed in the same manner as in the case of the flexible disk. it can. Further, the recording medium is not limited to the above-described optical disk or flexible disk, but may be an IC card, a ROM cassette, or the like as long as it can record a program. Similarly, an encoding process or a decoding process by software can be performed.

[0161]

As described above, according to the image processing method of the present invention, an image signal forming an image space consisting of a plurality of pixels is divided into a plurality of rectangular macroblocks for dividing the image space. An image encoding method for encoding an image signal corresponding to a sub-block constituting the macro block for each sub-block, wherein the image signal is formed for an image signal of a predetermined macro block. A horizontal pixel column rearrangement process so that the image of the first field is located above the macroblock and the image of the second field constituting the image signal is located below the macroblock. The image signal of the macroblock subjected to the rearrangement processing or the macroblock not subjected to the rearrangement processing is converted to the upper left, upper right, lower left, The frequency conversion is performed for each of the four sub-blocks located below and converted into frequency components, and the predicted value of the frequency component of the to-be-encoded sub-block to be subjected to the encoding process is calculated by the encoding process for which the encoding process has been completed. Of the coded sub-blocks, based on at least one of the coded sub-blocks located near the upper side, near the left side, and near the upper left of the coded sub-block. When encoding the difference value of the prediction value and generating the prediction value of the frequency component of the encoded sub-block, the encoded sub-block is a sub-block on which the image signal is rearranged. Irrespective of whether or not the coded sub-block is located near the upper side of the coded sub-block and the coded sub-block located near the left of the coded sub-block. Which frequency component of the sub-block is to be referred to is determined based on the DC component of the frequency component of the coded sub-block located near the upper side, near the left side, and near the upper left of the block to be coded. Field DCT
In the processed macro block, the positions of the block of the first field and the block of the second field are defined, and when generating the predicted value of the DCT coefficient of the encoded block, Blocks can be specified. This makes it possible to apply the conventional intra-picture prediction processing to the field DCT type macroblock as described above, and to convert an interlaced image or a specific progressive image in which field DCT type macroblocks are mixed into an image signal. Can be sufficiently reduced by the intra-picture prediction processing to efficiently encode the image.

Further, the sub-blocks near the to-be-encoded sub-block, which refer to the frequency components, are converted into the DC components of the frequency components of the encoded sub-blocks located near the upper side, near the left side, and near the upper left side of the to-be-coded block. Since the determination is made based on the components, the encoded sub-block having a high correlation between the DCT coefficient and the encoded sub-block can be easily specified.

According to the present invention, in the above-described image processing method, the encoded sub-block located near the upper side of the encoded sub-block may be an upper code located adjacent to the upper side of the encoded sub-block. Using a coded sub-block, as a coded sub-block located near the left side of the coded sub-block, using a left coded sub-block positioned adjacent to the left side of the coded sub-block, As an encoded sub-block located near the upper left of the encoded sub-block, an upper left encoded sub-block located adjacent to the upper left of the encoded sub-block is used. The absolute value of the difference between the DC component of the frequency component of the left encoded sub-block and the DC component of the frequency component of the upper left encoded sub-block is When the absolute value of the difference between the DC component of the lock frequency component and the DC component of the frequency component of the upper left encoded sub-block is smaller than the frequency component of the left encoded sub-block, the encoded sub-block is referred to. And the absolute value of the difference between the DC component of the frequency component of the left encoded sub-block and the DC component of the frequency component of the upper left encoded sub-block is calculated by the upper code When the absolute value of the difference between the DC component of the frequency component of the encoded sub-block and the DC component of the frequency component of the upper left encoded sub-block is smaller than the absolute value of the frequency component of the upper encoded sub-block, Generating a predicted value of the frequency component of the encoded sub-block. Reduction will be frequency components of the spatially close encoded sub blocks are referenced to the sub-blocks, it is possible to improve the prediction efficiency.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image processing device (image encoding device) according to a first embodiment of the present invention.

FIG. 2 is a block diagram illustrating a configuration of an image processing device (image decoding device) according to a second embodiment of the present invention.

FIG. 3 is a diagram for explaining an intra-picture prediction encoding process performed by the image encoding device according to the first embodiment, and shows an arrangement of DCT blocks in a macro block in a DCT area.

FIG. 4 is a diagram for explaining an intra-picture prediction encoding process performed by the image encoding device according to the first embodiment, and shows a positional relationship between a reference block and an encoded block in frame prediction and field prediction. .

FIG. 5 is a flowchart showing adaptive intra-screen DCT coefficient prediction processing by the image encoding device of the first embodiment and the image decoding device of the second embodiment.

FIG. 6 is a flowchart illustrating a frame prediction method in a prediction process performed by the image encoding device according to the first embodiment and the image decoding device according to the second embodiment.

FIG. 7 is a flowchart illustrating a field prediction method in a prediction process performed by the image encoding device according to the first embodiment and the image decoding device according to the second embodiment.

FIG. 8 is a flowchart showing an example of a prediction value generation method in the frame prediction.

FIG. 9 is a diagram showing a sequence of a predicted value generation method in the field prediction by a flowchart.

FIG. 10 is a diagram for explaining an example of a method of generating a DCT coefficient of a virtual buffer in prediction processing performed by the image encoding device according to the first embodiment and the image decoding device according to the second embodiment.

FIG. 11 is a flowchart illustrating an example of a method of generating a DCT coefficient of a virtual buffer in a prediction process performed by the image encoding device according to the first embodiment and the image decoding device according to the second embodiment.

FIG. 12 is a flowchart illustrating a frame prediction method in an image encoding device according to Embodiment 3 of the present invention and an image decoding device according to Embodiment 4;

FIG. 13 is a flowchart showing a field prediction method in the image encoding device according to the third embodiment and the image decoding device according to the fourth embodiment.

FIGS. 14 (a) and (b) are data storage media storing a program for performing, by a computer system, an intra prediction coding process and an intra prediction decoding process according to each embodiment of the present invention. FIG. 14 for explaining FIG.
(C) is a diagram showing the computer system.

FIG. 15 is a diagram for describing a method of predicting DCT coefficients in a screen using a conventional image processing apparatus.

FIG. 16 illustrates a frame / field DCT switching operation.
FIG. 9 is a schematic diagram for explaining a scan line replacement process.

FIG. 17 is a block diagram for describing an image encoding device according to a fifth embodiment of the present invention.

FIG. 18 is a block diagram illustrating an image decoding device according to a sixth embodiment of the present invention.

FIG. 19 is a diagram for describing reference blocks used in the prediction processing according to the fifth embodiment.

FIG. 20 is a diagram illustrating an example of a flowchart of a predicted value generation method according to the fifth embodiment.

[Explanation of symbols]

100 block generator 101 Image signal 102 DCT type signal 103 DCT unit 104 DCT coefficient 105 Quantizer 106 DCT coefficient quantization value 108 DCT coefficient difference value 109 variable length encoder 110, 210, 310, 410 In-screen prediction processing unit 110a Input image signal 110b Image encoded signal (bit stream) 111 DCT coefficient predicted value 113,313 DCT coefficient predictor 115 block memory 116 DCT coefficient quantization value 200 Deblocker 203 variable length decoder 207 inverse quantizer 209 Inverted DCT unit 1000,3000 image coding device (image processing device) 2000,4000 Image decoding device (image processing device) Cs computer system D Flexible disk body FC flexible disk case FD flexible disk FDD flexible disk drive

Claims (3)

[Claims] 1. An image signal forming an image space consisting of a plurality of pixels is divided into a plurality of rectangular macroblocks for partitioning the image space, and an image signal corresponding to a subblock constituting the macroblock is divided. An image processing method for performing an encoding process for each sub-block, wherein, for an image signal of a predetermined macro block, an image of a first field constituting the image signal is located above the macro block, and The horizontal pixel column is rearranged so that the image of the second field constituting the image signal is located below the macroblock, and the rearranged macroblock or the rearranged process is performed. The image signal of the macro block not subjected to the frequency conversion is performed for each of four sub blocks located at the upper left, upper right, lower left, and lower right constituting the macro block. Into a frequency component, and the predicted value of the frequency component of the coded sub-block to be coded is calculated from the coded sub-block of the coded sub-block which has already been coded. Generating based on at least one of the encoded sub-blocks located in the neighborhood, the neighborhood on the left side, and the neighborhood on the upper left, encoding the difference value between the frequency component of the encoded sub-block and its predicted value; When generating the predicted value of the frequency component of the sub-block, regardless of whether the coded sub-block is a sub-block subjected to image signal rearrangement processing, Which of the frequency components of the coded sub-block located near the upper side and the coded sub-block located near the left side of the coded sub-block is to be referred to Image processing method according to claim 該被 upper vicinity of the coding blocks is determined based on the DC component of the frequency components of the left neighboring, and encoded sub blocks positioned at the upper left vicinity, it. 2. The image processing method according to claim 1, wherein the encoded sub-block located near the upper side of the encoded sub-block is an upper code located adjacent to and above the encoded sub-block. Using a coded sub-block, as a coded sub-block located near the left of the coded sub-block, using a left coded sub-block positioned adjacent to the left of the coded sub-block, As the coded sub-block located near the upper left of the coded sub-block, an upper left coded sub-block located adjacent to the upper left of the coded sub-block is used. The absolute value of the difference between the DC component of the frequency component of the left encoded sub-block and the DC component of the frequency component of the upper left encoded sub-block is If the absolute value of the difference between the DC component of the frequency component of the block and the DC component of the frequency component of the upper left encoded sub-block is smaller than the absolute value of the A predicted value of the frequency component of the block is generated. On the other hand, the absolute value of the difference between the DC component of the frequency component of the left encoded sub-block and the DC component of the frequency component of the upper left encoded sub-block is If the absolute value of the difference between the DC component of the frequency component of the encoded sub-block and the DC component of the frequency component of the upper left encoded sub-block is smaller than the absolute value of the difference, the frequency component of the upper encoded sub-block is referred to. An image processing method, comprising: generating a predicted value of a frequency component of an encoded sub-block. 3. The image processing method according to claim 1, wherein a predicted value of a frequency component of the encoded sub-block is generated based on a quantized frequency component. Processing method.