JP2002064826A - Apparatus and method for converting resolution of compressed image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To rapidly convert an image compression coded in directions of a space axis and a time base into an image compression coded with different resolution. SOLUTION: A variable length decoding means 101 decodes a compressed image and outputs a picture type and motion vector information/orthogonal transformation image of each frame. A dequantization means 102 dequantizes the decoded orthogonal transformation image. A motion compensating I frame generating means 103 transforms a P, B frames into an I frame in an orthogonal transformation space. A resolution converting means 104 resolution converts the orthogonal transformation image of the I frame in the orthogonal transformation space. A motion compensating PB frame generating means 105 transforms the I frame after the resolution conversion into the P, B frames in the orthogonal transformation space. A quantizing means 106 quantizes the orthogonal transformation image. A variable length encoding means 107 outputs the compressed image after the resolution conversion.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and method for transforming an image compressed and coded in the spatial and temporal directions into a compressed and coded image having a different resolution in an orthogonal transform space.

[0002]

2. Description of the Related Art In recent years, with the spread of digital broadcasting and the Internet, transcoding techniques for changing the resolution of compressed images have become important. In this transcoding technique, for example, a plurality of program images received by digital broadcasting are reduced and synthesized, and T
Program list display on V, format conversion to convert video of a certain compression format to compression format for video conference by changing resolution, and change resolution of compressed image according to network bandwidth and computer processing capacity Thus, it can be used for a gateway mechanism or the like that suppresses the bit rate.

[0003] The coding method of the compressed image is CCITT.
(Comite Consultatif International Telegraphique
Expert Group) for video conferencing. H.261 coding method and ISO (International Standards Organizatio)
n) MPEG (Moving Picture Experts Group) coding method and the like. H. H.261 and MPEG coding methods include inter-frame prediction and DCT (Discrete Cosine Transcription).
(sform: discrete cosine transform), quantization, and variable-length code, and employs intra-frame coding for performing compression based on spatial correlation and inter-frame coding for performing compression based on temporal correlation.

In intra-frame encoding, first, a pixel block (8 × 8 pixel block) constituting a frame is orthogonally transformed by using DCT which is one of orthogonal transforms, and is transformed into a DCT block representing a frequency distribution. Assuming that the luminance value of the pixel block has the distribution shown in FIG.
The DCT block subjected to the CT conversion is as shown in FIG. 21B, and generally has a characteristic that energy is collected toward a lower frequency. Next, fine quantization is performed on the calculated DC component of the DCT block and low-frequency component coefficients close thereto, and coarse quantization is performed on the high-frequency component. This allows
The DCT block can be quantized without any visual degradation. Then, as shown in FIG.
The coefficients of the T block are sequentially scanned in a zigzag scanning manner from a low frequency region, and run length coding and variable length coding are performed. Run-length coding converts the coefficients of a DCT block into a set of (the number of zero coefficients, the value of a non-zero coefficient). Together with the zigzag scanning method, the DCT block compresses high frequency information having many zero coefficients. it can. Variable-length coding was converted by run-length coding (the number of zero coefficients,
Variable length bits are sequentially generated in a Huffman code table prepared in advance according to the appearance probability of a set of non-zero coefficients.

On the other hand, inter-frame prediction coding predicts a current frame from a past or future frame, as shown in FIG. 23. For each block in the current frame, a past or future frame is predicted. Then, compression in the time axis direction is performed by obtaining a motion vector and difference information.
A frame predicted and coded from a past frame is an inter-frame forward prediction coded frame (P frame), and a frame predicted and coded from two past and future frames is a bi-directional predictive coded frame (P frame). B picture), and the respective picture types are called P picture and B picture. On the other hand, what is encoded in a frame without performing predictive encoding is an intra-coded frame (I
Frame), and its picture type is called an I picture.

In the motion vector detection method, for each macroblock (16 × 16 pixel block) in the current image, a macroblock with the smallest average error is searched from the past and future images while shifting it by half or one pixel at a time. For this reason, although the accuracy is high, an enormous amount of calculation is required, which causes a reduction in processing speed. As a result of the motion vector detection, a macroblock having a minimum average error larger than a certain threshold is coded as a macroblock (I macroblock) for intra-frame coding.

[0007] MPEG and H.264. In order to change the resolution of a compressed image using intra-frame prediction coding and inter-frame prediction coding such as H.261, generally the following i) to iii)
For this reason, a method is employed in which a compressed image is temporarily expanded, the number of pixels is changed at an uncompressed level, and the image is recompressed. i) Since it is not possible to extract only the information necessary for changing the number of pixels from the bit stream after variable length encoding, it is necessary to temporarily perform variable length decoding on the compressed image. ii) When the resolution is changed as shown in FIG.
Because the pixel configuration of the pixel block changes before and after the change,
It is difficult to change the resolution at the quantization or DCT level for processing in block units. iii) As shown in FIG. 24 (b), the motion vector is held in macroblock units, and when the number of pixels is changed,
It is necessary to newly calculate the motion vector after the change and the difference image, and the motion vector is a half pixel or pixel unit vector, not a block unit. Is difficult to change.

However, in this resolution conversion method, in addition to the resolution conversion processing, motion compensation, motion vector detection,
Huge image processing such as DCT and inverse DCT is required, and high-speed processing cannot be performed.

As a device for converting the resolution at a high speed, for example, there is an image coded data conversion device disclosed in Japanese Patent Application Laid-Open No. Hei 10-271494. As shown in FIG. 25, this apparatus comprises a variable length decoding unit, an inverse quantization unit, and an inverse DCT.
And a transcoder 22 having a vector correction means, a picture determination means, an image sampling means and an image memory, and an encoder 23 having a DCT means, a quantization means and a variable length coding means.

[0010] The variable length decoding means 211 decodes the compressed image and separates it into picture type data 24, motion vector data 25 and pixel data 26 in the frequency domain of the decoded image. The inverse quantization means 212 inversely quantizes the pixel data 26 in the frequency domain, sends the inversely quantized pixel data 27 in the frequency domain to the inverse DCT means 213, and
3 converts the pixel data 27 in the frequency domain into the pixel data 28 in the spatial domain and stores it in the image memory 224.

The picture determining means 222 determines the picture type data 24 and outputs activation instruction signals 31 and 32 if the picture type is to be used. The image sampling means 223 thins out the pixel data 29 read from the image memory 224 in response to the start instruction signal 31 and outputs corrected image data 30. The vector correction means 221 corrects the motion vector data according to the activation instruction signal 32 and outputs correction vector data 33. The corrected image data 30 is converted into pixel data 34 in the frequency domain by the DCT unit 231,
It is quantized at 32. The quantized pixel data 35 and the correction vector data 33 in the frequency domain are variable-length coded by a variable-length coding unit 233. It is to be noted that the picture type used by the picture judging means is determined, for example, from MPEG2 of image size 720 × 480 to image size 3
H. 52 × 240. When generating a compressed image of H.263, H.263 is used. In H.263, a nonexistent B picture is used as a dropped frame.

[0012]

However, in the prior art, when changing the resolution, DCT and inverse DC are still required.
There is a problem that the image processing of T is required, and high-speed processing cannot be performed. SUMMARY OF THE INVENTION It is an object of the present invention to perform resolution conversion in an orthogonal transform space and reduce the amount of DCT and inverse DCT calculations.

Further, when performing the motion compensation processing, a problem arises that a prediction memory is required before and after resolution conversion. An object of the present invention is to reduce a prediction memory required at the time of resolution conversion.

In addition, when performing motion compensation processing in the orthogonal transform space, there is a problem that an orthogonal transform block at an arbitrary position must be extracted from a plurality of orthogonal transform blocks. An object of the present invention is to extract an orthogonal transform block at an arbitrary position in an orthogonal transform space.

Further, there is a problem that a plurality of extraction matrices for extracting orthogonal transformation blocks at an arbitrary position are required. An object of the present invention is to reduce the number of necessary extraction matrices to reduce the memory capacity.

Further, there is a problem that a large amount of calculation is required to extract an orthogonal transform block at an arbitrary position in the orthogonal transform space. An object of the present invention is to provide a plurality of orthogonal transformation block extraction methods to reduce the amount of calculation.

In addition, there is a problem that a large amount of calculation is required to extract an orthogonal transform block at an arbitrary position in the orthogonal transform space. SUMMARY OF THE INVENTION It is an object of the present invention to reduce the amount of calculation by providing a pre-process for extracting an orthogonal transform block.

In addition, when performing motion compensation processing for decoding, there is a problem that motion compensation cannot be performed at high speed because the motion vector is in half-pixel units. SUMMARY OF THE INVENTION It is an object of the present invention to correct a motion vector so as to reduce the amount of calculation for decoding motion compensation processing.

Further, depending on the method of calculating the motion vector after the resolution conversion, there is a problem that it takes time to extract the orthogonal transformation block at the designated position in the motion compensation processing for encoding. SUMMARY OF THE INVENTION It is an object of the present invention to change the method of determining a motion vector after resolution conversion, and to reduce the amount of calculation of the motion compensation processing for encoding.

Further, depending on the method of calculating the motion vector after resolution conversion, there is a problem that it takes time to extract the orthogonal transformation block at the designated position. SUMMARY OF THE INVENTION It is an object of the present invention to correct a motion vector after resolution conversion and to reduce the amount of calculation in the motion compensation processing for encoding.

[0021]

SUMMARY OF THE INVENTION The present invention relates to a resolution conversion apparatus for changing the number of pixels of a compressed image which has been compression-encoded in the spatial and temporal directions. Variable length decoding means for outputting a picture type, motion vector information, and an orthogonally transformed image, inverse quantization means for inversely quantizing the orthogonally transformed image decoded by the variable length decoding means, and information generated by the inverse quantization means. A motion compensation I frame generating means for transforming P and B frames into I frames in an orthogonal transform space using an orthogonal transform image and a picture type and motion vector information generated by the variable length decoding means; Resolution conversion means for converting the resolution of the I-frame generated by the generation means in an orthogonal transform space, and outputting the I-frame after the resolution conversion; Using picture type and the motion vector information generated by said variable length decoding means and the orthogonal transformation image of I frame after the resolution conversion more generated picture type after resolution conversion and motion vector information and P, B
A motion compensation PB frame generating means for generating an orthogonal transform image of a frame in an orthogonal transform space, a quantization means for quantizing the resolution-converted orthogonal transform image generated by the motion compensation PB frame generating means, Variable-length coding means for performing variable-length coding on the orthogonally transformed image quantized by the conversion means and the picture type and the motion vector information after resolution conversion generated by the motion compensation PB frame generation means. As a result, the amount of operation of DCT and inverse DCT at the time of resolution conversion can be reduced.

The motion compensated PB frame generating means is a motion vector candidate after resolution conversion based on the motion vector information obtained by performing variable length decoding on the compressed image in accordance with the change rate of the number of vertical and horizontal pixels. Motion vector candidate extracting means for extracting a motion vector; resolution converted motion vector determining means for determining a motion vector after resolution conversion from the motion vector candidates extracted by the motion vector candidate extracting means; Block extraction means for extracting one frame of the orthogonal transformation block of the reference destination using the motion vector information determined by the means and the past or future orthogonal transformation image after the resolution conversion, and 1 block extracted by the block extraction means. A PB frame for generating P and B frames from the orthogonal transform block for the frame and the orthogonal transformed image after resolution conversion. Those comprising a chromatography beam generating means.

A resolution conversion apparatus for changing the number of pixels of a compressed image which has been compression-encoded in the space axis and time axis directions, wherein the resolution conversion apparatus decodes the compressed image in a variable length and orthogonalizes the picture type and motion vector information of each frame. Variable length decoding means for outputting a transformed image, inverse quantization means for inversely quantizing the orthogonally transformed image decoded by the variable length decoding means, orthogonal transformed image generated by the inverse quantization means, and the variable length decoding means A motion compensation I frame generating means for transforming P and B frames into I frames in an orthogonal transform space using the picture type and motion vector information generated by Using the converted image and the picture type and motion vector information generated by the variable length decoding means, the picture type after resolution conversion is used. A PB frame generation unit for resolution conversion that generates, in an orthogonal conversion space, an orthogonally converted image of P and B frames before resolution conversion and motion vector information, and an orthogonally converted image generated by the PB frame generation unit for resolution conversion. Resolution conversion means for performing resolution conversion in an orthogonal transformation space, quantization means for quantizing the orthogonally transformed image after resolution conversion generated by the resolution conversion means, orthogonal transformation image quantized by the quantization means and the resolution conversion Variable-length coding means for performing variable-length coding on the picture type and motion vector information after resolution conversion generated by the PB frame generating means for use. As a result, it is possible to reduce the number of prediction memories required for resolution conversion.

Further, the resolution conversion PB frame generation means uses the motion vector information obtained by performing variable length decoding on the compressed image to select a motion vector candidate after resolution conversion in accordance with a change rate of the number of vertical and horizontal pixels. Motion vector candidate extracting means for extracting the following motion vector, resolution converted motion vector determining means for determining a motion vector after resolution conversion from the motion vector candidates extracted by the motion vector candidate extracting means, and the resolution converted motion vector A motion vector mapping unit that maps the motion vector after resolution conversion determined by the determination unit to a motion vector before resolution conversion, motion vector information generated by the motion vector mapping unit, and a past or future orthogonal transform after resolution conversion Using the image, the orthogonal transformation block of the reference destination is extracted for one frame. A block extracting means for, P from the orthogonal transformation images after orthogonal transformation blocks and resolution conversion for one frame extracted by the block extracting means, in which and a PB frame generating means for generating a B-frame.

Further, the motion-compensated I-frame generating means uses the motion vector information obtained by performing variable-length decoding on the compressed image and the past or future orthogonally transformed image after the inverse quantization to obtain a reference destination. A block extracting means for extracting one orthogonal transform block for one frame, an I frame for generating an I frame from the orthogonal transform block for one frame extracted by the block extracting means and the current orthogonal transform image after the inverse quantization.
And a frame generating means.

Further, the block extracting means divides a motion vector of the block into a motion vector (block vector) for each block and a motion vector (pixel vector) in the block when processing each block constituting the frame. Motion vector analysis means, four block extraction means for extracting four orthogonal transformation blocks from an orthogonal transformation image in a memory using the block vector generated by the motion vector analysis means, and the motion vector analysis means. Block generating means for generating an orthogonal transform block at a target position using the pixel vector and the four orthogonal transform blocks extracted by the four block extracting means.

The block generation means generates an orthogonal transformation block by limiting a low frequency area used by the orthogonal transformation image in accordance with the change rate of the number of pixels in the vertical and horizontal directions.

The block generation means divides and holds a generation matrix necessary for generating an orthogonal transformation block at an arbitrary position.

The block generation means includes a pixel vector analysis means for analyzing whether the value of the pixel vector is even or odd, and an operation method selection means for selecting a block generation means to be used based on the result of analysis by the pixel vector analysis means. It is provided with.

The block generating means generates an orthogonal transform block at a target position after calculating the sum / difference between the four orthogonal transform blocks.

The motion vector analysis means corrects the pixel vector in pixel units.

The resolution-converted motion vector determining means takes the average of the motion vector candidates as a resolution-converted motion vector.

The motion vector after resolution conversion is determined by the motion vector determination means as a motion vector after resolution conversion, from which a block is easily extracted from motion vector candidates.

The resolution-converted motion vector determining means corrects the determined resolution-converted motion vector into a motion vector from which blocks can be extracted at high speed.

[0035]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below with reference to FIGS.

(First Embodiment) FIG. 1 shows a second embodiment of the present invention.
1 shows a schematic configuration of one embodiment. Resolution converter 1
Are variable length decoding means 101, inverse quantization means 102, motion compensation I frame generation means 103, resolution conversion means 10
4. Motion compensation PB frame generation means 105, quantization means 106, variable length coding means 107, information holding memory 10
8. It has prediction memories 150 and 160.

The variable-length decoding means 101 performs variable-length decoding on the input compressed image 120, and outputs the picture type and motion vector information 121 of each frame and the orthogonally transformed image
2 is output. The picture type is composed of an I picture that is an intra-coded image, a P picture that is an inter-frame forward predictive encoded image, and a B picture that is an inter-frame bidirectional predictive encoded image. The orthogonally transformed image 122 output from the variable length decoding means 101 is inversely quantized by the inverse quantization means 102, and the picture type and the motion vector information
1 is held in the information holding memory 108.

If the picture type 121 from the image information holding memory 108 is a P picture or a B picture, the motion compensation I frame generating means 103
2, the dequantized orthogonally transformed image 123, the contents of the past frame memory PREV 151 and the future frame memory BEF 152 of the prediction memory 150, the picture type and the motion vector information 1 from the image information holding memory 108.
21, the P and B frames are transformed into I frames in the orthogonal transform space. When the picture type is an I picture, the frame is not converted and is stored in the prediction memory 150. The orthogonally-transformed image 124 of the I frame that has been converted or input from the inverse quantization means 102 is sent to the resolution conversion means 104.

FIG. 2 shows a motion compensation I frame generating means 10.
3 shows a schematic configuration. The motion vector correction unit 801
The block extraction unit 802 corrects the half-pixel accuracy motion vector information 121 to pixel-precision motion vector information. The block extraction unit 802 uses the motion vector information 810 corrected by the motion vector correction unit 801 and the orthogonal transformation image in the prediction memory 150. Is used to extract the orthogonal transformation block of the reference destination for one frame. I frame generating means 803
Generates an I frame from the orthogonal transform block 811 for one frame extracted by the block extracting unit 802 and the orthogonally transformed image 123 after inverse quantization, and
4 outputs an I-frame orthogonal transform image 124.

The motion compensation method includes half-pel motion compensation for calculating a motion vector in half-pixel units and full-pel motion compensation for calculating a motion vector in pixel units. As shown in FIG. 3, the former half-pel motion compensation is a rounded average of two pixel values when the prediction reference position is between two pixels, and four when the prediction reference position is between four pixels.
A value obtained by taking a rounded average of pixel values is used as a reference image. Half-pel motion compensation not only improves prediction accuracy but also has a function of applying a low-pass filter to a reference image to loosen the image slowly. In a normal MPEG, a motion vector is expressed in half-pel units. When the prediction reference position of the motion vector is between two pixels or four pixels, a block extraction unit 802 that extracts an orthogonal transformation block in pixel units
Cannot extract an orthogonal transformation block in half-pixel units. In the resolution conversion apparatus, the motion vector whose prediction reference position is between two pixels or four pixels is corrected by correcting the prediction reference position on a pixel-by-pixel basis, thereby reducing the operation of taking a rounded average between pixels, It is possible to quickly extract the orthogonal transformation block at the predicted position. The function of applying a low-pass filter to the reference image is performed by the block extracting means 8.
This is realized by setting all high-frequency regions of the orthogonal transformation block extracted at 02 to 0.

FIG. 4 is a flow chart showing the flow of processing for generating an I frame after half-pel motion compensation in the orthogonal transform space by the resolution converting apparatus. The operation will be described below. Step 901: Determine whether the picture type of the frame is an I-frame. If the frame is an I frame, the process ends. If the frame is a PB frame, the process proceeds to step 902. Step 902: By the motion vector correction means 801
From the motion vector information, the motion vector (m #
x, m # y). Step 903: By the motion vector correction means 801
A half-pixel unit motion vector is corrected to a pixel unit motion vector. Step 904: It is determined whether or not processing for one frame has been performed. Step 905: The block extracting means 802 extracts one frame of the orthogonal transformation block of the reference destination using the corrected motion vector information and the orthogonal transformation image in the prediction memory. Step 906: The I frame generation unit 803 generates an I frame from the extracted orthogonal transformation block of the reference destination and the orthogonally transformed image after the inverse quantization, and ends the processing.

Returning to FIG. 1, the resolution conversion means 104
The I frame 124 generated by the motion compensation I frame generating means 103 is subjected to resolution conversion in an orthogonal transform space,
The I frame 125 after the resolution conversion is output.

For the resolution conversion in the orthogonal transform space, for example,
A high-speed conversion method as described in JP-A-8-180194 is used. Then, the motion-compensated PB frame generating means 105 outputs the I frame 1 after the resolution conversion.
From 25, a PB frame is generated in the orthogonal transform space. As described above, all the processes related to the resolution conversion are performed in the orthogonal transform space, so that the operations of the inverse DCT and the DCT become unnecessary. When the motion vector (horizontal direction, vertical direction) is a motion vector indicating a block such as (0, 0) or (8, 0), or when the macroblock in the P or B frame is an I macroblock In addition, the processing at the time of motion compensation becomes almost zero, and further high-speed calculation becomes possible.

The motion compensation PB frame generating means 105
I after resolution conversion generated by the resolution conversion means 104
Frame orthogonal transform image 125 and variable length decoding means 101
Type and motion vector information 1 generated by
21, the picture type and motion vector information 127 after the resolution conversion and the orthogonally transformed image 1 of the P and B frames
26 is generated in the orthogonal transform space.

FIG. 5 shows a motion compensation PB frame generating means 1.
05 shows a schematic configuration. Motion vector candidate extraction means 10
01 extracts a motion vector that is a candidate for a motion vector after resolution conversion from the motion vector information 121 according to the change rate of the number of pixels in the vertical and horizontal directions. The motion vector after resolution conversion is determined from the motion vector candidate 1010 extracted by the extraction means 1001.
The block extracting unit 1003 calculates the motion vector information 1 determined by the resolution-converted motion vector determining unit 1002.
011 and the orthogonally transformed image in the prediction memory 160,
The orthogonal transform block of the reference destination is extracted for one frame. The orthogonal transformation block 1012 for one frame extracted by the block extraction unit 1003 and the orthogonally transformed image 126 after resolution conversion are sent to the PB frame generation unit 1004, and the orthogonally transformed image 126 after motion compensation is output.

FIG. 6 shows the operations of the motion vector candidate extracting means 1001 and the resolution-converted motion vector determining means 1002 when the image is reduced by に in each of the vertical and horizontal directions. First, the motion vector candidate extraction unit 1001 selects a motion vector candidate after reduction from motion vectors before reduction. In the selection method, a motion vector of a pre-reduction macroblock having correlation with image information of the post-reduction macroblock is set as a candidate. Next, the motion vector of the motion vector candidate is reduced by に in each of the vertical and horizontal directions. Then, the resolution-converted motion vector determination unit 1002 determines a reduced motion vector from the motion vector candidates. The decision method is
A method of using the average vector of all the motion vector candidates as the reduced motion vector, a method of using the first candidate motion vector as the reduced motion vector, and the like can be given. In the determination method of the resolution conversion device, in order to reduce the amount of motion compensation calculation after resolution conversion, a motion vector candidate that can speed up the orthogonal transformation block extraction processing is set as the resolution-converted motion vector. For example, (0,0), (8,0),
Priority is given to a motion vector in block units in each of the vertical and horizontal directions such as (0, 16) and a motion vector in block units only in the vertical or horizontal direction such as (0, 4), (3, 0), (2, 8). To obtain a motion vector after resolution conversion. The motion vector of the block unit in each of the vertical and horizontal directions has a calculation amount of the orthogonal transformation block extraction processing of 0, and the motion vector of the block unit only in the vertical direction or the horizontal direction has the calculation amount of the orthogonal transformation block extraction processing of 1 which is the normal calculation amount. Since it can be reduced to / 2, the calculation amount of the motion compensation processing after resolution conversion can be reduced. Also, the motion vector determining means 1002 after resolution conversion
It is also possible to correct the motion vector after the resolution conversion determined in the above to one that can speed up the orthogonal transformation block extraction processing. For example, the average vector of all the motion vector candidates is calculated as a motion vector of a block unit in each of the vertical and horizontal directions such as (0, 0), (8, 0), (0, 16), (0, 4),
Only the vertical or horizontal direction such as (3, 0) and (2, 8) is corrected to a motion vector in block units. This makes it possible to generate a motion vector that takes into account information of all motion vector candidates as a motion vector from which an orthogonal transform block can be extracted at a higher speed.

Returning to FIG. 1, the orthogonally transformed image 126 after the resolution conversion generated by the motion compensation PB frame generation means 105 is quantized by the quantization means 106 and
The picture data and the motion vector information 127 after the resolution conversion generated by the B frame generating means 105 are sent to the variable length coding means 107 to be subjected to variable length coding.

FIG. 7 is a flowchart showing the flow of processing in units of frames of the resolution conversion apparatus of FIG. 1, and its operation will be described below. Step 201: The variable-length decoding means 101 performs variable-length decoding on the compressed image 11, and extracts a frame picture type, motion vector information, and an orthogonal transform image. Step 202: The inverse quantization means 102 inversely quantizes the orthogonally transformed image. Step 203: The picture type of the frame is determined by the motion compensation I frame generating means 103. In the case of an I frame, the process proceeds to step 205, in the case of a P frame, to step 204, and in the case of a B frame, to step 207. Step 204: The motion compensation I frame generating means 103 uses the orthogonal transform image of the PREV memory 151 in the prediction memory 150, the orthogonal transform image 123 dequantized in step 202, and the motion vector information 121 extracted in step 201. To perform motion compensation to generate an I-frame orthogonal transform image 124. Step 205: The motion-compensated I-frame generating means 103 stores the orthogonally transformed image in the BEF memory 152 in the prediction memory 150 in the PREV memory 151. Step 206: The I-frame orthogonal transform image or the I-frame orthogonal transform image of the I-frame generated in step 204 is stored in the BEF memory 152 in the prediction memory 150 by the motion compensation I-frame generating means 103. Step 207: The PREV memories 151 and B in the prediction memory 150 by the motion compensation I frame generation means 103
Motion compensation is performed using the orthogonally transformed image in the EF memory 152, the orthogonally transformed image 123 dequantized in step 202, and the motion vector information 121 extracted in step 201, to generate an I-frame orthogonally transformed image. Step 208: The resolution conversion unit 104 performs resolution conversion of the orthogonally-transformed image 124 of the I frame from the motion compensation I frame generation unit 103. Step 209: Motion compensation PB frame generating means 105
It is determined whether the picture type of the frame is an I frame. In the case of an I frame, P
In the case of a frame, the process proceeds to step 210, and in the case of a B frame, the process proceeds to step 213. Step 210: motion compensation PB frame generating means 105
, The motion vector information 1 extracted in step 201
21, motion vector information 127 after resolution conversion is generated, and the orthogonally transformed image of the PREV memory 161 in the prediction memory 160 and the orthogonally transformed image 125 generated in step 208 are generated.
, An orthogonally transformed image of the P frame after resolution conversion is generated. Step 211: motion compensation PB frame generating means 105
Thus, the orthogonally transformed image in the BEF memory 162 in the prediction memory 160 is stored in the PREV memory 161. Step 212: motion compensation PB frame generating means 105
As a result, the I-frame orthogonally transformed image or the I-frame orthogonally transformed image generated in step 208 is stored in the BEF memory 162 in the prediction memory 160. Step 213: Motion compensation PB frame generating means 105
, The motion vector information 1 extracted in step 201
, The motion vector information 127 after resolution conversion is generated from the PREV memory 161 and the BE
A B frame after resolution conversion is generated from the orthogonally transformed image in the F memory 162 and the orthogonally transformed image 125 generated in step 208. Step 214: the motion compensation P
The orthogonal transformation image 126 from the B frame generation unit 105 is quantized. Step 215: The variable-length coding unit 107 performs variable-length coding on the quantized orthogonally-transformed image 126 and the resolution-converted motion vector information 127, generates a resolution-converted compressed image 129, and ends the processing. .

As described above, the resolution conversion apparatus 1 performs the motion compensation at the time of decoding, then performs the resolution conversion, and performs the motion compensation at the time of encoding in the orthogonal transform space, thereby obtaining the resolution. DCT during conversion, inverse D
Since the amount of calculation of CT can be reduced, the practical effect is large. Although the orthogonal transform is described using DCT, other orthogonal transform such as wavelet transform may be used.
Similar effects can be obtained.

When performing motion compensation processing for decoding, all the motion vectors are corrected to pixel-based motion vectors, and if necessary, the high-frequency region of the orthogonal transform block to be extracted is set to 0, so that decoding can be performed. Since the calculation amount of the motion compensation processing can be reduced, the practical effect is large.

When a motion vector after resolution conversion is determined, by selecting a motion vector candidate that can extract an orthogonal transform block at high speed, the amount of calculation in the encoding motion compensation processing can be reduced. Its practical effect is great.

Further, by correcting the motion vector after the resolution conversion, the speed of extraction of the orthogonal transform block can be further increased, and the practical effect is large.

(Second Embodiment) FIG. 8 shows a schematic configuration of a second embodiment of the present invention. Resolution converter 2
Is obtained by reducing the prediction memory. The resolution conversion apparatus 1 includes a variable length decoding unit 301, an inverse quantization unit 302, a motion compensation I frame generation unit 303, a resolution conversion PB frame generation unit 304, a resolution conversion unit 305, a quantization unit 306, and a variable length encoding unit. 307, an information holding memory 308, and a prediction memory 350. 8, a variable length decoding unit 301, an inverse quantization unit 302, a motion compensation I
The operations of the frame generation unit 303, the quantization unit 306, and the variable length encoding unit 307 are the same as those of the variable length decoding unit 101 in FIG.
Inverse quantization means 102, motion compensation I frame generation means 10
3, the same as the quantization means 106 and the variable length coding means 107.

The variable-length decoding means 301 performs variable-length decoding on the input compressed image 320, and outputs the picture type and motion vector information 321 of each frame and the orthogonally-transformed image 32.
2 is output. The orthogonal transform image 322 output from the variable length decoding unit 301 is inversely quantized by the inverse quantization unit 302, and the picture type and the motion vector information 321 are stored in the information storage memory 308.

When the picture type 321 from the image information holding memory 308 is a P picture or a B picture, the motion compensation I frame generating means 303
2, the dequantized orthogonally transformed image 323, the contents of the past frame memory PREV 351 and the future frame memory BEF 352 of the prediction memory 350, the picture type and the motion vector information 3 from the image information holding memory 308.
21, the P and B frames are transformed into I frames in the orthogonal transform space. When the picture type is an I picture, the frame is not converted and is stored in the prediction memory 350. The orthogonally transformed image 324 of the I frame that has been transformed or input from the inverse quantization means 302 is sent to the resolution conversion means 304.

Resolution conversion PB frame generating means 304
Are the orthogonally-transformed image 324 of the I frame generated by the motion-compensated I-frame generating unit 303 and the picture type and motion vector information 321 generated by the variable-length decoding unit.
Is used to generate a picture type and motion vector information 326 after resolution conversion and an orthogonally transformed image 325 of P and B frames before resolution conversion in an orthogonally transformed space.

A method of performing motion compensation after resolution conversion before changing the resolution will be described with reference to FIG. In the case where the motion vector information of each macro block before resolution conversion is as shown in FIG. 9A, when the resolution is changed to ２, the resolution conversion PB frame generation unit 304 The motion vector information after resolution conversion as shown in FIG. 9B is calculated from the motion vector information 321 of FIG.
As shown in (c), mapping is performed on a macroblock before resolution conversion. Then, the motion compensation I frame generating means 3
From the orthogonally-transformed image 324 of the I frame generated in step 03 and the orthogonally-transformed image in the prediction memory, P and B frames before resolution conversion are regenerated using the mapped motion vector information. The regenerated P and B frames before resolution conversion have a similar relationship with the P and B frames after resolution conversion with respect to the motion vector information and the difference information. The calculated resolution-converted motion vector information is stored in the information holding memory 30.
8 is stored.

FIG. 10 shows a schematic configuration of the resolution conversion PB frame generating means 304. The motion vector candidate extracting unit 1101 extracts a motion vector as a motion vector candidate after resolution conversion from the motion vector information 321 as a motion vector according to a change rate of the number of pixels in the vertical and horizontal directions. Determining means 1102
Is used to determine a motion vector after resolution conversion from the motion vector candidates 1110 extracted by the motion vector candidate extraction unit 1101. The motion vector mapping unit 1103 includes a motion vector determination unit 11 after resolution conversion.
02 after the resolution conversion determined by the
1 is mapped to a motion vector before resolution conversion, and the block extracting unit 1104 uses the motion vector information 1112 generated by the motion vector The conversion block is extracted for one frame. The orthogonal transform block 1113 for one frame and the I-frame orthogonal transform image 324 extracted by the block extracting unit 1104 are sent to the PB frame generating unit 1105, and the orthogonal transform image 325 of the P and B frames before the resolution conversion is subjected to the orthogonal transform. Generate in space.

The resolution conversion means 305 outputs a P for resolution conversion.
The resolution conversion is performed on the orthogonally transformed image of the I frame and the orthogonally transformed image 325 of the difference between the P and B frames from the B frame generation unit 304. The orthogonally transformed image 327 having undergone the resolution conversion is quantized by the quantization means 306, and the quantized image 328 is converted into a variable length code together with the picture type and motion vector information 326 after the resolution conversion generated by the resolution conversion PB frame generating means. It is sent to the means 307 and is subjected to variable length coding.

FIG. 11 is a flowchart showing the flow of processing of the motion compensation I frame generation means 303, resolution conversion PB frame generation means 304, and resolution conversion means 305 of this resolution conversion apparatus in units of frames. Will be described. Step 401: The picture type of the frame is determined by the motion compensation I frame generating means 303. The process proceeds to step 404 for an I frame, to step 402 for a P frame, and to step 406 for a B frame. Step 402: The orthogonal compensation image of the PREV memory 351 in the prediction memory 350, the orthogonal transformation image 323 inversely quantized by the inverse quantization unit 302, and the motion extracted by the variable length decoding unit 301 by the motion compensation I frame generation unit 303. Motion compensation is performed using the vector information 321 to generate an I-frame orthogonally transformed image. Step 403: PB frame generating means 3 for resolution conversion
In step 04, a motion vector for resolution conversion is calculated using the mapping method shown in FIG. 9, and a P frame is regenerated. Step 404: PB frame generating means 3 for resolution conversion
04, the BEF memory 352 in the prediction memory 350
Is stored in the PREV memory 351. Step 405: The resolution-converted PB frame generating means 304 converts the I-frame orthogonally transformed image or the I-frame orthogonally transformed image generated in step 402 into the BEF memory 35 in the prediction memory 350.
Save to 1. Step 406: The PREV memories 351 and B in the prediction memory 350 by the motion compensation I frame generation means 303
Orthogonal transform image of EF memory 352 and inverse quantization means 302
Using the orthogonally transformed image 323 inversely quantized in the above and the motion vector information 321 extracted by the variable length decoding means 301,
The motion compensation is performed to generate an I-frame orthogonal transform image. Step 407: Resolution conversion PB frame generating means 3
In step 04, a motion vector for resolution conversion is calculated using the mapping method shown in FIG. 9, and a B frame is regenerated. Step 408: The resolution conversion means 305 converts the resolution of the orthogonally converted image 325 of the I, P, B frame before the resolution conversion, and ends the processing.

As described above, in the orthogonal transform space, the resolution converter 2 first performs motion compensation at the time of decoding, then performs motion compensation for resolution conversion, and performs resolution conversion. Since the required prediction memory at the time of resolution conversion can be reduced, the practical effect is large.

(Third Embodiment) When performing motion compensation in an orthogonal transform space, it is necessary to extract an orthogonal transform block at an arbitrary position. However, in the third embodiment, an orthogonal transform block at an arbitrary position is required. The block extraction process is performed at high speed. FIG. 12 shows the motion-compensated I-frame generating means 10 of FIG.
3, the motion compensation PB frame generation means 105, the motion compensation I frame generation means 303 of FIG.
It is a schematic configuration of block extracting means 802, 1003, and 1104 used in the frame generating means 305. The block extracting means includes a motion vector analyzing means 501, four block extracting means 502, a block generating means 503, and a constant memory 550.

The motion vector analysis means 501 divides the motion vector information into a motion vector (block vector) in blocks and a motion vector (pixel vector) in a block. Using the block vector generated by the means 501, four orthogonal transform blocks are extracted from an orthogonal transform image. Also, the block generation means 503
Is to generate an orthogonal transformation block at a target position using the pixel vector generated by the motion vector analysis unit 501 and the four orthogonal transformation blocks extracted by the four block extraction unit 502.

As shown in FIG. 13, the motion vector analyzing means 501 determines the positions of 2 × 2 (four) orthogonal transform blocks including the orthogonal transform block to which each orthogonal transform block is referred. The motion vector 510 is divided into a block vector 511 and a pixel vector 512.
The block vector 511 represents a vector to the upper left orthogonal transform block among the 2 × 2 orthogonal transform blocks, and the pixel vector 512 is a reference destination of the 2 × 2 orthogonal transform block viewed from the upper left orthogonal transform block. Represents a vector to the orthogonal transform block. Therefore, “motion vector = block vector + pixel vector” holds. The four block extraction unit 502 extracts 2 × 2 orthogonal transformation blocks 514 including the block of the reference destination from the orthogonal transformation image 513 input using the block vector 511 generated by the motion vector analysis unit 501. The block generation unit 503 includes four block extraction units 5
2 and 2 × 2 orthogonal transformation blocks 514 extracted in step S 02 and the pixel vector 51 generated by the motion vector analysis unit 501.
2, the orthogonal transformation block 515 to be referred to is extracted.
The constant memory 550 outputs necessary arithmetic expressions and constants 516 to the block generation means 503.

A method of extracting an orthogonal transform block at an arbitrary position from 2 × 2 orthogonal transform blocks will be described with reference to FIG. The description will be made using DCT as the orthogonal transform. First, 2 × 2 pixel blocks 531 are calculated from 2 × 2 8 × 8 DCT blocks 530. 2x2 8
The matrix representing the x8 DCT block is W, the matrix of 2x2 8x8 pixel blocks is X, and the transformation matrix of the inverse DCT 8x8 is IT8.
Assuming that 8, the equation (1) holds. Here, A ^T represents the transposed matrix of A.

[0066]

(Equation 1)

Next, 2 × 2 8 × 8 pixel blocks (1
6x16 pixel block) 8x of the area specified from 531
The eight pixel block 532 is extracted. Assuming that the matrix of the 8 × 8 pixel block is Y, the vertical extraction matrix is Sv, and the horizontal extraction matrix is Sh, Equation (2) holds. The Sv and Sh extraction matrices are, for example, 2 × 2 8 × 8 pixel blocks (16 × 16
3 pixels vertically from the upper left of the pixel block) and 2 pixels horizontally
When extracting an 8x8 pixel block at a pixel position,
As shown in FIG. 15, Sv is a matrix of 8 rows and 16 columns, and Sh is a matrix of 16 rows and 8 columns.

[0068]

(Equation 2)

Then, the extracted 8 × 8 pixel block 53
The 2 × 8 × 8 DCT block 533 is calculated. 8x8
The DCT block matrix is Z, and the DCT 8x8 transformation matrix is T
If 88 (equivalent to IT88 ^T ), the equation (3) is established.

[0070]

(Equation 3)

When the expressions (1), (2) and (3) are summarized,
If the equation (4) is satisfied and a DCT of the vertical extraction matrix Sv and the horizontal extraction matrix Sh is prepared in advance, 2 × 2 8 × 8 DCT blocks (Z) to be extracted are obtained.
It can be generated directly from the T block (W).

[0072]

(Equation 4)

As shown in FIG. 16, Sv is represented by Sv # l, Sv # r, Sh
Is divided into Sh # t and Sh # b, Expression (4) can be expressed by Expression (5).

[0074]

(Equation 5)

As shown in FIG. 17, S [I] (I = 0 to 8)
When a matrix is defined, Sv # l and Sh # b are all S [I] (I =
0 to 8), Sv # r and Sh # t are all S [I]
(I = 0 to 8). From equation (6), the transpose of S [I] (I = 0-8) is given by D
The result of the CT is equivalent to the transposed matrix of the result of DCT of the matrix of S [I] (I = 0 to 8). Therefore, S [I]
Only the matrix obtained by DCT of the nine matrices (I = 0 to 8) is stored in the constant memory 550, and the corresponding matrix is extracted from the constant memory 550 according to the position where the block is extracted. For the Sh # t part,
What is necessary is to perform the processing of the expression (5).

[0076]

(Equation 6)

When the compressed image is reduced, the calculation of the expression (5) is performed to calculate 2 × 2 orthogonal transform blocks (W) and the orthogonal transform block (Z) to be extracted according to the change rate of the number of pixels in the vertical and horizontal directions. High-speed processing can be performed by limiting the low-frequency region to be used. For example, in the case of performing 縮小 reduction in each of the vertical and horizontal directions, 4 × 8 × 8 areas of 2 × 2 orthogonal transform blocks (W) and orthogonal transform blocks (Z) to be extracted are used.
Assuming that only the x4 low-frequency region is valid and the other regions are set to 0, the calculation of Expression (5) starts from eight 8 × 8 matrix calculations.
This can be reduced to eight 4 × 4 matrix operations.

FIG. 18 shows that the block extracting means of FIG.
5 is a flowchart showing a flow of a process of extracting a block at an arbitrary position in the orthogonal transform space, and the operation will be described below. Step 601: By the motion vector analysis means 501,
The motion vector (m # x, m # y) of each block is extracted from the motion vector information 510. Step 602: By the motion vector analysis means 501,
It is determined whether the x direction (m # x) of the motion vector is 0 or more. 0
If so, go to step 603; if negative, go to step 6
Go to 05. Step 603 and Step 605: The x direction (b # x) of the block vector 511 is calculated from the x direction (m # x) of the motion vector by the motion vector analysis unit 501. Step 604 and Step 606: The motion vector analysis means 501 calculates the x direction (p # x) of the pixel vector 512 from the x direction (m # x) of the motion vector. Step 607: By the motion vector analysis means 501,
It is determined whether the y direction (m # y) of the motion vector is 0 or more. 0
If so, go to step 608; if negative, go to step 6
Proceed to 10. Step 608 and Step 610: The motion vector analysis unit 501 calculates the y direction (b_y) of the block vector 511 from the y direction (m # y) of the motion vector. Step 609 and Step 611: The motion vector analysis unit 501 calculates the y direction (p_y) of the pixel vector 512 from the y direction (m # y) of the motion vector. Step 612: 2 × 2 orthogonal transform blocks including the orthogonal transform block to be extracted are extracted by the four block extracting means 502 using the calculated block vector (b # x, b # y) 511. Step 613: S [p # x], S [p # x], using the x direction (p # x) of the pixel vector calculated by the block generation unit 503.
The DCT matrix of [8-p # x] is extracted from the constant memory 550. Step 614: Using the y direction (p # y) of the pixel vector calculated by the block generation unit 503, S [p # y], S [p # y]
The DCT matrix of [8-p # y] is extracted from the constant memory 550. Step 615: The block generation means 503 transposes the extracted DCT matrices of S [p # x] and S [8-p # y]. Step 616: By the block generation means 503,
The block extraction operation of the equation (5) is performed. Step 617: It is determined whether the processing for one frame has been performed. If the process has been performed to the end, the process ends.

Next, a description will be given of a method for performing the process of extracting the orthogonal transformation block at a high speed. S used in equation (5)
[I] (I = 0 to 8) matrix generated by DCT (S
d) can be all represented by equation (7) (A0-
A37: constant).

[0080]

(Equation 7)

Using the expression (7), the right side of the expression (5) is expanded, and the processing of “αβ + αγ → α (β + γ)” is performed.
Calculation result of sum / difference of two orthogonal transform block coefficients and A
An arithmetic expression composed of constants 0 to A37 (hereinafter, block extraction arithmetic expression). The matrix of S [I] (I = 0 to 8) is DC
Since all the nine matrices generated by T are as shown in equation (7), "S [I] (I =
A0 of 9 matrices generated by performing DCT on matrices 0 to 8)
It is only necessary to store only the “constant of A37”, “the arithmetic expression of the sum / difference of 2 × 2 orthogonal transform block coefficients”, and “the block extraction arithmetic expression”. In the orthogonal transformation block extraction process in this case, sum / difference of 2 × 2 orthogonal transformation block coefficients is first performed by using the constant memory 550, and then,
The constants of A0 to A37 are determined according to the position where the block is extracted, and finally the processing of block extraction is performed. Since the number of times of multiplication can be reduced as compared with the matrix operation of equation (5), further high-speed processing is possible. Becomes

Further, by changing the method of extracting the orthogonal transformation block in accordance with the value of the block vector 511 in the block generation means 503, it is possible to reduce the total amount of calculation. FIG. 19 shows the configuration of the orthogonal transformation block generation means 503 for that purpose. The orthogonal transformation block generation unit 503 in FIG. 19 includes a pixel vector analysis unit 701 that analyzes whether the value of the pixel vector 512 is even or odd,
From the result analyzed by the pixel vector analysis means 702,
Block generation means to be used (even / even block generation means 7
04, even / odd block generating means 705, odd / even block generating means 706, and odd / odd block generating means 707).

The pixel vector analysis means 701 analyzes whether the values of p # x and p # y of the pixel vector (p # x, p # y) are even or odd, and sends an even / even The block generation unit 704, the even / odd block generation unit 705, the odd / even block generation unit 706, and the odd / odd block generation unit 707 are used to inform which block is to be extracted.

The block extracting method of the even / even block generating means 704 will be described with reference to FIG. The description will be made using DCT as the orthogonal transform. In FIG. 20, first, the inverse DCT 4 × 4 is applied to the 4 × 4 low frequency region 711 of the 2 × 2 8 × 8 DCT blocks 710, and the 8 × 8
A pixel block 712 is generated. The generated 8x8 pixel block 712 is composed of 2x2 8x8 pixel blocks (16x16 pixel blocks) generated by performing inverse DCT 8x8 on 2x2 8x8 DCT blocks, and halving them in each of the vertical and horizontal directions.
The 8x8 block to be extracted, which corresponds to the reduced one, is 4
x4 blocks. Next, the pixel vector (p # x, p
#y), the values of p # x and p # y are set to 1/2, and the 4x4 pixel block 713 located at the position of p # x in the horizontal direction and p # y in the vertical direction from the upper left of the generated 8x8 pixel block is extracted. Extract. And
DCT4x4 is performed on the extracted 4x4 pixel block 713,
The 4 × 4 DCT block 714 is mapped to the 4 × 4 low frequency region 716 of the 8 × 8 DCT block 715. The operation of FIG. 20 is similar to the expression (5) in that four 8 × 8D
An 8 × 8 DCT block at a specified position can be directly extracted from the CT block, and the amount of calculation is eight times a 4 × 4 matrix operation, thereby enabling high-speed processing. In addition, the block extraction method of the even / odd block generation unit 705 and the odd / even block generation unit 706 also takes into account a model in which only the vertical direction is reduced by 、 and the horizontal direction is reduced by ２. As in the case of the block generation unit 704, high-speed processing can be performed.

As described above, the block extracting means shown in FIG. 12 first divides a motion vector into a block vector and a pixel vector, and then uses the block vector to extract 2 × 2
By selecting orthogonal transform blocks, and using a constant matrix prepared in advance, an orthogonal transform block at an arbitrary position in the orthogonal transform space is extracted by extracting an orthogonal transform block at a position indicated by the pixel vector. Since it can be extracted, its practical effect is great.

The vertical extraction matrix Sv and the horizontal extraction matrix Sh are divided into Sv # l, Sv # r, Sh # t, Sh # b, and Sv #
By using r and Sh # t as transposed matrices, the number of extraction matrices prepared in advance can be reduced to nine S [I] (I = 0 to 8).
Its practical effect is great.

Further, the "operation formula for sum / difference of orthogonal transform block coefficients" and "block extraction operation formula" common to block extraction at an arbitrary position and "S [I] (I = 0 to 8) are prepared in advance, and the sum / difference of a plurality of orthogonal transform block coefficients is calculated before performing the operation of extracting the orthogonal transform block at an arbitrary position. Since the amount of calculation for orthogonal transformation block extraction can be reduced, the practical effect is large.

Further, by analyzing whether the value of the pixel vector is even or odd, and by changing the method of extracting the orthogonal transformation block according to the analysis result, the amount of calculation of the orthogonal transformation block extraction can be reduced. The effect is great. Although the description has been made using the DCT as the orthogonal transform, a similar effect can be obtained by using another orthogonal transform such as a wavelet transform.

[0089]

As described above, according to the present invention, an advantageous effect that the amount of calculation of DCT and inverse DCT at the time of resolution conversion can be reduced can be obtained. Further, an advantageous effect that a memory for motion compensation necessary for resolution conversion can be reduced can be obtained.
When performing motion compensation processing in an orthogonal transform space, first, a motion vector is divided into a block vector and a pixel vector,
Next, 2 × 2 orthogonal transform blocks are selected using the block vector, and the orthogonal transform block at the position where the pixel vector is located is extracted using a constant matrix prepared in advance to obtain the orthogonal transform space. Has an advantageous effect that an orthogonal transformation block at an arbitrary position can be extracted. Further, by dividing an extraction matrix for extracting an orthogonal transformation block at an arbitrary position and using a transposed matrix, an advantageous effect that the memory capacity of the extraction matrix can be reduced can be obtained. Also, if the pixel vector value is an even number
By analyzing whether the number is an odd number and changing the method of extracting the orthogonal transformation block according to the analysis result, an advantageous effect that the amount of calculation for block extraction can be reduced can be obtained. Also,
By performing a sum / difference calculation of a plurality of orthogonal transform block coefficients before performing an operation to extract an orthogonal transform block at an arbitrary position from the 2 × 2 orthogonal transform blocks, it is possible to reduce a calculation amount of block extraction.

When performing motion compensation processing for decoding, all the motion vectors are corrected to pixel-based motion vectors, and if necessary, the high-frequency region of the orthogonal transform block to be extracted is set to 0, so that decoding can be performed. An advantageous effect that the amount of calculation of the motion compensation processing can be reduced is obtained.
In addition, when determining a motion vector after resolution conversion, by selecting a motion vector candidate that can extract an orthogonal transform block at high speed, there is an advantageous effect that the calculation amount of the motion compensation processing for encoding can be reduced. can get. Furthermore, by correcting the motion vector after the resolution conversion so that the orthogonal transform block can be extracted at a high speed, an advantageous effect that the amount of calculation of the motion compensation processing for encoding can be reduced can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a schematic configuration of a resolution conversion apparatus according to a first embodiment.

FIG. 2 is a block diagram showing a schematic configuration of a motion compensation I frame generation unit of the resolution conversion device of FIG. 1;

FIG. 3 illustrates half-pel motion compensation.

FIG. 4 is a flowchart showing the operation of the motion compensation I frame generating means of FIG. 2;

FIG. 5 is a block diagram showing a schematic configuration of a motion compensation PB frame generation unit of the resolution conversion device of FIG. 1;

FIG. 6 is a view for explaining a motion vector determination method after resolution conversion.

FIG. 7 is a flowchart showing the operation of the resolution conversion apparatus of FIG. 1;

FIG. 8 is a block diagram illustrating a schematic configuration of a resolution conversion device according to a second embodiment.

FIG. 9 is a view for explaining a method of regenerating a motion vector before resolution conversion.

10 is a block diagram illustrating a schematic configuration of a resolution conversion PB frame generation unit of the resolution conversion device in FIG. 8;

FIG. 11 is a flowchart showing the operation of the resolution conversion apparatus in FIG. 8;

FIG. 12 is a block diagram showing a schematic configuration of a block extracting unit.

FIG. 13 is a view for explaining a method of dividing a motion vector into a block vector and a pixel vector.

FIG. 14 is a view for explaining a method of extracting an orthogonal transformation block at an arbitrary position.

FIG. 15 is a view for explaining vertical and horizontal block extraction matrices;

FIG. 16 is a view for explaining a method of dividing a vertical and horizontal block extraction matrix into two matrices;

FIG. 17 is a view for explaining vertical and horizontal block extraction matrix values;

FIG. 18 is a flowchart showing the operation of the block extracting means of FIG.

FIG. 19 is a block diagram showing a schematic configuration of an orthogonal transformation block generation unit in FIG. 12;

FIG. 20 is a view for explaining a method of extracting an orthogonal transformation block at an even-numbered position.

FIG. 21 is a diagram illustrating frequency characteristics after DCT.

FIG. 22 is a view for explaining run-length encoding and variable-length encoding.

FIG. 23 is a view for explaining inter-frame predictive coding;

FIG. 24 is a diagram illustrating a problem in changing the resolution.

FIG. 25 is a block diagram showing a configuration of a conventional moving image encoding device.

[Explanation of symbols]

 101, 301: variable length decoding means 102, 302 ... inverse quantization means 103, 303 ... motion compensation I frame generation means 104, 305 ... resolution conversion means 105 ... motion compensation PB frame generation Means 106, 306: Quantization means 107, 307: Variable length code means 108, 308: Information holding memories 150, 160, 350 ... Prediction memory 304: Resolution conversion PB frame generation means

Claims (28)

[Claims] 1. A resolution conversion apparatus for changing the number of pixels of a compressed image which has been compression-encoded in a spatial axis and a time axis direction, the variable-length decoding of the compressed image, and orthogonalizing the picture type and motion vector information of each frame. A variable-length decoding unit that outputs a transformed image; an inverse quantization unit that inversely quantizes the orthogonally transformed image decoded by the variable length decoding unit; an orthogonally transformed image generated by the inverse quantization unit; and the variable length decoding. Using the picture type and motion vector information generated by the means, the P and B frames are converted into I frames,
A motion-compensated I-frame generating means for performing conversion in an orthogonal transform space; and an I-frame generated by the motion-compensated I-frame generating means, in which the resolution is converted in an orthogonal transform space.
Resolution conversion means for outputting a frame; and a resolution-converted picture using the orthogonally-converted image of the I-frame after the resolution conversion generated by the resolution conversion means and the picture type and motion vector information generated by the variable-length decoding means. Type, motion vector information and P,
A motion compensation PB frame generating means for generating an orthogonally transformed image of a B frame in an orthogonally transformed space; a quantizing means for quantizing the resolution-converted orthogonally transformed image generated by the motion compensated PB frame generating means; A resolution conversion device for a compressed image, comprising: a variable-length coding unit for performing variable-length coding on the orthogonally transformed image quantized by the quantization unit and the picture type and the motion vector information after resolution conversion generated by the motion compensation PB frame generation unit. . 2. The resolution conversion apparatus according to claim 1, wherein the motion compensation PB frame generation unit calculates a motion compensation PB frame from the motion vector information obtained by performing variable length decoding on the compressed image.
A motion vector candidate extracting unit that extracts a motion vector that is a candidate for a motion vector after resolution conversion in accordance with a change rate of the number of pixels in the vertical and horizontal directions, and a motion vector candidate extracted by the motion vector candidate extracting unit after resolution conversion. Using the resolution-converted motion vector determining means for determining the motion vector, the motion vector information determined by the resolution-converted motion vector determining means, and the past or future orthogonally transformed image after the resolution conversion, 1 conversion block
A resolution of a compressed image, comprising: a block extraction unit for extracting frames, and a PB frame generation unit for generating P and B frames from an orthogonal transformation block for one frame extracted by the block extraction unit and an orthogonally transformed image after resolution conversion. Conversion device. 3. A resolution conversion device for changing the number of pixels of a compressed image compressed and encoded in a spatial axis and a time axis direction, the variable length decoding of the compressed image being performed, the picture type of each frame and the motion vector information being orthogonalized. Variable length decoding means for outputting a transformed image, inverse quantization means for inversely quantizing the orthogonally transformed image decoded by the variable length decoding means, orthogonal transformed image generated by the inverse quantization means, and the variable length decoding means Using the picture type and motion vector information generated by
A motion-compensated I-frame generating unit for converting in an orthogonal transform space; an I-frame orthogonally-transformed image generated by the motion-compensated I-frame generating unit; and a picture type and motion vector information generated by the variable-length decoding unit. ,
A resolution conversion PB frame generation unit that generates a picture type and motion vector information after resolution conversion and an orthogonally converted image of P and B frames before resolution conversion in an orthogonal conversion space; Resolution conversion means for performing resolution conversion on the generated orthogonally transformed image in an orthogonal transformation space; quantizing means for quantizing the orthogonally transformed image after resolution conversion generated by the resolution conversion means; and quadrature quantized by the quantization means. A resolution converter for a compressed image, comprising: a converted image; and a variable-length code unit for performing variable-length coding on the picture type and motion vector information after resolution conversion generated by the resolution conversion PB frame generation unit. 4. The resolution conversion apparatus according to claim 3, wherein the resolution conversion PB frame generation unit calculates a motion vector information obtained by performing variable length decoding on the compressed image.
A motion vector candidate extracting unit that extracts a motion vector that is a candidate for a motion vector after resolution conversion in accordance with a change rate of the number of pixels in the vertical and horizontal directions, and a motion vector candidate extracted by the motion vector candidate extracting unit after resolution conversion. Motion vector determining means for determining a motion vector after resolution conversion; motion vector mapping means for mapping a motion vector after resolution conversion determined by the motion vector determining means after resolution conversion to a motion vector before resolution conversion; Using the motion vector information generated by the mapping means and the past or future orthogonally transformed image after the resolution conversion, a block extracting means for extracting one frame of the orthogonal transformation block of the reference destination, and the block extracting means One frame orthogonal transform block and orthogonality after resolution conversion A resolution converter for a compressed image, comprising: a PB frame generation unit that generates P and B frames from the converted image. 5. The resolution conversion device according to claim 1, wherein the motion compensation I frame generating means includes: a motion vector information obtained by performing variable length decoding of a compressed image; Block extracting means for extracting one frame of the orthogonal transformation block of the reference using the past or future orthogonal transformation image after quantization, and the orthogonal transformation block for one frame extracted by the block extracting means, A resolution conversion device for a compressed image, comprising: an I-frame generation unit configured to generate an I-frame from the current orthogonally-transformed image after quantization. 6. The resolution conversion apparatus according to claim 2, wherein said block extracting means calculates a motion vector of the block when processing each block constituting a frame. A motion vector analysis unit for dividing a motion vector (block vector) in units of blocks and a motion vector (pixel vector) in a block; And an orthogonal transform block at a target position using a pixel vector generated by the motion vector analyzing means and four orthogonal transform blocks extracted by the four block extractor. A resolution conversion device for a compressed image, comprising: 7. The resolution conversion device according to claim 6, wherein the block generation means is configured to change the number of pixels in the vertical and horizontal directions according to a change rate.
A resolution conversion apparatus for a compressed image, which generates an orthogonal transformation block by limiting a low frequency region used by the orthogonal transformation image. 8. The resolution conversion apparatus according to claim 6, wherein the block generation means divides and holds a generation matrix necessary for generating an orthogonal transformation block at an arbitrary position. Resolution converter. 9. The resolution conversion device according to claim 6, wherein the block generation means analyzes the pixel vector analysis means for analyzing whether the value of the pixel vector is an even number or an odd number. And a calculation method selecting means for selecting a block generating means to be used. 10. The resolution conversion apparatus according to claim 6, wherein the block generating means calculates a sum and a sum between four orthogonal transformation blocks.
A resolution conversion device for a compressed image, which generates an orthogonal transformation block at a target position after calculating the difference. 11. The resolution conversion device according to claim 6, wherein the motion vector analysis means corrects the pixel vector in pixel units. 12. The resolution conversion apparatus according to claim 2, wherein the resolution-converted motion vector determining means takes a total average of the motion vector candidates as a resolution-converted motion vector. A resolution converter for compressed images. 13. The resolution conversion apparatus according to claim 2, wherein the resolution-converted motion vector determination means sets a motion vector after resolution conversion to one from which a block is easily extracted from motion vector candidates. A resolution converter for a compressed image. 14. The resolution conversion apparatus according to claim 2, wherein the resolution-converted motion vector determining means corrects the determined resolution-converted motion vector to a motion vector that can extract blocks at high speed. A resolution converter for compressed images. 15. A resolution conversion method for changing the number of pixels of a compressed image which has been compression-encoded in the spatial axis and the time axis direction, comprising: decoding the compressed image with variable length; A variable length decoding step of outputting a transformed image; an inverse quantization step of inversely quantizing the orthogonal transformed image decoded by the variable length decoding step; an orthogonal transformed image generated by the inverse quantization step; and the variable length decoding A motion-compensated I-frame generating step of transforming the P and B frames into an I-frame in an orthogonal transform space using the picture type and the motion vector information generated by the steps; and an I-frame generated by the motion-compensated I-frame generating step.
A resolution conversion step of converting the resolution of the frame in the orthogonal conversion space and outputting an I-frame after the resolution conversion; and an orthogonally-transformed image of the resolution-converted I-frame generated in the resolution conversion step and the variable-length decoding step. A motion compensation PB frame generation step of generating a picture type and motion vector information after resolution conversion and orthogonal transformation images of P and B frames in an orthogonal transformation space using the picture type and the motion vector information; A quantization step of quantizing the resolution-converted orthogonally-transformed image generated by the frame generation step; an orthogonally-transformed image quantized by the quantization step; and a resolution-converted picture type generated by the motion-compensated PB frame generation step. Variable length code for variable length coding of motion vector information Resolution converting method of compressing an image and a step. 16. The resolution conversion method according to claim 15, wherein said motion-compensated PB frame generating step comprises:
A motion vector candidate extraction step of extracting a motion vector that is a candidate for a motion vector after resolution conversion in accordance with a change rate of the number of pixels in the vertical and horizontal directions, and a motion vector candidate extracted by the motion vector candidate extraction step after resolution conversion. Using the resolution-converted motion vector determination step for determining the motion vector, the motion vector information determined in the resolution-converted motion vector determination step, and the past or future orthogonally transformed image after the resolution conversion, A block extracting step of extracting a transformed block for one frame, and P, P from the orthogonal transformed block for one frame extracted in the block extracting step and the orthogonal transformed image after resolution conversion.
A PB frame generating step of generating a B frame. 17. A resolution conversion method for changing the number of pixels of a compressed image which has been compression-encoded in a spatial axis and a time axis direction, wherein the compressed image is variable-length decoded, and a picture type and motion vector information of each frame are orthogonalized. A variable length decoding step of outputting a transformed image; an inverse quantization step of inversely quantizing the orthogonal transformed image decoded by the variable length decoding step; an orthogonal transformed image generated by the inverse quantization step; and the variable length decoding step Using the picture type and the motion vector information generated by the above, a motion compensation I frame generation step of transforming the P and B frames into an I frame in an orthogonal transform space; and an I frame generated by the motion compensation I frame generation step.
Using the orthogonally transformed image of the frame and the picture type and motion vector information generated by the variable length decoding step, the picture type and motion vector information after resolution conversion and the orthogonally transformed image of the P and B frames before resolution conversion Generating a resolution-converted PB frame in the orthogonal transform space, generating a resolution-converted PB frame in the orthogonal transform space in the orthogonal transform space, and generating the resolution-converted PB frame in the orthogonal transform space. A quantization step of quantizing the orthogonally transformed image after the resolution conversion, and the orthogonally transformed image quantized by the quantization step and the picture type and motion vector information after the resolution conversion generated in the step of generating the resolution conversion PB frame. To
And a variable-length code step of performing variable-length coding. 18. The resolution conversion method according to claim 17, wherein the step of generating a resolution conversion PB frame includes:
A motion vector candidate extraction step of extracting a motion vector that is a candidate for a motion vector after resolution conversion in accordance with a change rate of the number of pixels in the vertical and horizontal directions, and a motion vector candidate extracted by the motion vector candidate extraction step after resolution conversion. A step of determining a motion vector after resolution conversion for determining a motion vector; a step of mapping a motion vector after resolution conversion determined by the step of determining motion vector after resolution conversion to a motion vector before resolution conversion; Using the motion vector information generated by the mapping step and the past or future orthogonally transformed image after the resolution conversion, a block extraction step of extracting one frame of the orthogonal transformation block of the reference destination, and the block extraction step 1 frame orthogonal From the transform block and the orthogonally transformed image after resolution conversion, P,
A PB frame generating step of generating a B frame. 19. The resolution conversion method according to claim 15, wherein the motion-compensated I-frame generating step includes: a motion vector information obtained by performing variable length decoding on a compressed image; A block extracting step of extracting a reference orthogonal transformation block for one frame by using the quantized past or future orthogonal transformation image, and an orthogonal transformation block for one frame extracted in the block extracting step and the inverse. Generating an I frame from the current orthogonally transformed image after quantization. 20. Any one of claims 16, 18, and 19
The resolution conversion method according to any one of the preceding claims, wherein the block extracting step includes, when performing processing of each block constituting a frame, converting a motion vector of the block into a motion vector (block vector) in units of blocks and a motion vector (block vector) in the block. A motion vector analysis step of dividing the motion vector into pixel vectors), a four block extraction step of extracting four orthogonal transformation blocks from the in-memory orthogonal transformation image using the block vector generated in the motion vector analysis step, A resolution conversion method for a compressed image, comprising: a block generation step of generating an orthogonal transformation block at a target position using the pixel vector generated in the analysis step and the four orthogonal transformation blocks extracted in the four block extraction step. 21. The resolution conversion method according to claim 20, wherein the block generation step generates an orthogonal transformation block by limiting a low-frequency region used in the orthogonal transformation image according to a change rate of the number of vertical and horizontal pixels. Resolution conversion method for compressed images. 22. The resolution conversion method according to claim 20, wherein the block generation step divides and holds a generation matrix necessary for generating an orthogonal transformation block at an arbitrary position. Resolution conversion method. 23. The resolution conversion method according to claim 20, wherein the block generation step includes a step of analyzing whether the value of the pixel vector is even or odd, and a step of analyzing the pixel vector. And a calculation method selection step of selecting a block generation step to be used. 24. The resolution conversion method according to claim 20, wherein the block generation step generates an orthogonal transformation block at a target position after calculating a sum / difference between the four orthogonal transformation blocks. A resolution conversion method for a compressed image. 25. The resolution conversion method according to claim 20, wherein the motion vector analysis step corrects a pixel vector in pixel units. 26. The resolution conversion method according to claim 16 or 18, wherein the resolution-converted motion vector determining step takes a total average of motion vector candidates as a motion vector after resolution conversion. Resolution conversion method for compressed images. 27. The resolution conversion method according to claim 16 or 18, wherein the step of determining a motion vector after resolution conversion sets a motion vector after resolution conversion to a block which can be easily extracted from motion vector candidates. A resolution conversion method for a compressed image. 28. The resolution conversion method according to claim 16, wherein the resolution-converted motion vector determining step corrects the determined resolution-converted motion vector into a motion vector that allows high-speed block extraction. Resolution conversion method for compressed images.