JP2002344973A - Method for converting size of image coding data, transmission method for image coding data and image coding data size converter - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a coded data image size converter that is configured to simply obtain excellent image quality for the conversion of a size of image data coded by using motion compensation prediction and orthogonal transform. SOLUTION: An image data converter 14 obtains macro block image data converted from supplied image coded data to have a prescribed image size and a motion vector converter 15 obtains a conversion motion vector with respect to the macro block image data by multiplying a prescribed conversion ratio with a motion vector with respect to the received image coding data. When the conversion ratio is smaller than 1, an integrated motion vector is obtained depending on the dispersion of a plurality of conversion motion vectors required for conversion of image data and by using the integrated motion vector and the obtained macro block image data, the image coded data subjected to size conversion is obtained.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a method for decoding coded data of a digital video using motion compensated prediction.
The present invention relates to a size conversion method of image encoded data capable of changing an image size of an output image, an image encoded data transmission method, and an image encoded data size conversion device. The present invention relates to a data conversion process capable of conversion.

[0002]

2. Description of the Related Art MPEG (moving) is a technique for encoding moving picture signals such as television signals with high efficiency.
picture experts group) The standard is digital broadcasting, DV
D (Digital versatile Disc), digital tape recorders, and widely used as signals transmitted in communication networks.

[0003] The MPEG was introduced in 1988 by the ISO / IE.
C (International Organization for Standardization
/ International Electrotechnical Commission) J
TC1 / SC2 (Joint Technical Committee 1 / Sub
committee 2 It is an organization that considers the video coding standards established by the International Standards Organization / International Electrotechnical Commission Technical Committee 1 / Specialized Subcommittee 2).

[0004] The SC2 is currently a moving picture as SC29,
In addition, activities for establishing standards relating to encoding of audio signals and the like have been continued, and international standards established by MPEG people are commonly referred to as MPEG.

[0005] The first established MPEG1 (MPEG Phase 1) is a coding standard for moving picture signals accompanied by audio signals and intended for storage media recorded at a transmission rate of about 1.5 Mbps. JPEG (Joint Photographic Coding Experts Grou
p) and ISDN (Integrated services digital netwo)
H.rk) for the purpose of video compression for video conferences and videophone low transfer rates. 261 (CCITT SGXV, current IT
This is a coding standard that uses the basic technology of UT SG15).

[0006] Thus, MPEG1 was introduced in August 1993.
The disc was enacted in January as ISO / IEC 11172, and many discs encoded and recorded according to the MPEG1 standard have been commercialized.

[0007] MPEG2 (MPEG Phase 2), which was subsequently enacted, was established in January 1994 with the purpose of being a general-purpose standard so that it can support various applications such as communication and broadcasting.
It was enacted in January as ISO / IEC 13818 and “H.262”.

[0008] The encoding by MPEG1 and MPEG2 is composed of a plurality of encoding techniques. These techniques convert a "frame" image constituting a moving image into a block of 16x16 pixels called a "macroblock". For each macroblock unit, a motion amount called a “motion vector” is obtained between a reference image and a coded image separated by a predetermined number of frames in the future or the past in time, and the motion amount is calculated based on the motion amount. A motion compensated prediction technique for encoding a coded picture from a reference picture, and a DCT (Discrete Cosine Transform) which is one of orthogonal transform techniques for an error signal of the motion compensated prediction or the coded picture itself. : Discrete Cosine Transform) to convert image information into frequency information amount and obtain only visually significant information from the converted frequency domain information. It is defined based on two element technologies of image encoding, a "transform encoding" technology for performing compression encoding.

There are three modes in the motion compensation prediction in the case of prediction from both the past, the future, and the past and the future, and the three modes are 16 pixels × 1.
Switching is possible for each MB (macroblock) consisting of data of 6 pixels.

The prediction direction is determined by the picture type given to the frame of the input image, that is, I (Intra-code).
d), P (Predictive-coded), and B (Bidirectional
ly predictive-coded) are defined.

[0011] The I picture is a picture to be coded without performing motion prediction, while the P picture has two modes for prediction from the past and coding without performing prediction. Has four MCs (Motion Comp.) That perform intra-frame prediction without performing prediction from the future, prediction from the past, prediction from both the past and future directions, and prediction.
ensation) mode.

The motion compensation performed using these future or past images is performed by performing pattern matching on a motion area for each MB to obtain a motion vector with half-pel (1/2 of the distance between pixels) accuracy. The image signal supplied is encoded based on the image obtained by moving the future or past reference image position in the vector direction in accordance with the motion vector amount.

There are a horizontal direction and a vertical direction in the direction of the motion vector obtained in such a manner, and the vector information is transmitted together with the MC mode as additional information of the MB.

[0014] Of the picture data thus produced, three types of pictures, I, P and B, are I pictures.
The pictures are arranged and transmitted in a predetermined order starting from the picture,
A set of pictures (frame images) from the I picture to a picture before the next I picture is represented by GOP (Group O).
f Picture), and in encoding performed on ordinary storage media, etc., a GOP
Is configured.

[0015] The I picture and the P and B pictures coded as motion compensated pictures are DCT (discrete).
te cosine transform), that is, an integral transform using a cosine function as an integral kernel is performed as an orthogonal transform that discretely transforms into a finite space.

In the orthogonal transformation, the MB is converted into an 8-pixel × 8-pixel D
Although the image data is divided into CT blocks and two-dimensional DCT is performed, frequency components of image data are generally high in low frequency components and low in high frequency components. Therefore, image data is subjected to DCT and represented by transform coefficients concentrated in low frequency components. be able to.

Then, the DCT-processed image data (DC
The T coefficient is quantized by a quantizer. In other words, the DCT coefficient is calculated and calculated by a predetermined quantized value by the quantizer, and the quantized value is 2 × 8 pixels × 8 pixels.
The dimensional frequency is obtained as a quantized value weighted by visual characteristics, and the quantized value is used as a scalar multiplied by a predetermined quantization scale.

The quantized value is multiplied by an inverse quantized value obtained at the time of decoding the encoded image data, so that the characteristic of the quantized value given at the time of encoding is canceled at the time of decoding. I have.

Next, the configuration of an MPEG encoder that performs encoding and decoding by such a method will be described. FIG.
0 shows the configuration of the MPEG encoder and outlines its operation.

The MPEG encoder 50 has an input terminal 5
1, adder 52, DCT unit 53, quantizer 54, VLC
Unit 55, buffer 56, code amount controller 57, inverse quantizer 61, inverse DCT unit 62, adder 63, image memory 64,
And a motion compensation predictor 65.

First, the moving picture signal supplied to the input terminal 51 is supplied to the motion compensation predictor 65 and the adder 52. In the adder, the signal supplied from the motion compensation predictor 65 is inverted and added. The signal obtained by the addition is DC
It is supplied to the T unit 53.

In the DCT unit 53, the supplied image signal is subjected to the above-described discrete cosine transform, and the obtained DTS
The CT transform coefficient is supplied to a quantizer 54, where the quantized data is quantized based on the predetermined quantized value, and the quantized data is quantized by an inverse quantizer 61 and VLC (variable length).
h coding) unit 55.

In the VLC unit 55, the supplied quantized data is variable-length coded, but a direct current (DC) component obtained by performing a DCT transform among the quantized values is a DP component.
CM (differential pulse code modulation) modulation is performed.

Further, an alternating current (AC) component is obtained while performing a zigzag scan in the order of data of a high frequency component from data of a low frequency component, and the obtained data has a zero run length and an effective run length. The Huffman coding is performed such that the coefficient value is regarded as one event, and codes having shorter code lengths are assigned in ascending order of occurrence probability.

The data subjected to the Huffman encoding, which is the variable length encoding, is temporarily stored in a buffer 56, and the temporarily stored data is supplied as an encoded data output at a predetermined transfer rate.

The generated code amount for each macro block of the supplied data is supplied to a code amount controller 57 and compared with a preset target code amount, and the generated code amount obtained by the comparison is obtained. Is supplied to the quantizer 54, and the quantizer 54 obtains encoded data at a predetermined transfer rate by changing the value of the quantization scale based on the code amount of the difference. The amount of code is controlled.

On the other hand, the image data quantized by the quantizer 54 is supplied to an inverse quantizer 61 where the image data is inversely quantized.
The inversely quantized data is supplied to an inverse DCT unit 62, where the inverse DCT is performed, and the inverse DCT data is supplied to an adder 63.

In the adder 63, the signal is added to the reference image supplied from the motion compensation predictor 65, and a signal obtained by the addition is supplied to an image memory 64 and temporarily stored therein. The temporarily stored image data is stored in the motion compensation predictor 6.
5 is used as a reference decoded image for calculating the difference image, thereby making the MPEG encoder 5
From 0, it can be obtained as encoded data with motion compensation.

The coded data obtained in this way is M
It is supplied to a PEG decoder and decoded. FIG.
The configuration of the PEG decoder will be described, and an outline of the operation will be described.

An MPEG decoder 70 shown in FIG. 1 includes an encoded data input terminal 71, a buffer 72, a VLD unit 73,
It comprises an inverse quantizer 74, an inverse DCT unit 75, an adder 76, an image memory 77, and a motion compensation predictor 78.

First, the encoded data supplied to the input terminal 71 is temporarily stored in a buffer 72, and the temporarily stored encoded data is stored in a VLD (variable length de
coding) unit 73.

In the VLD unit 73, the data encoded by the VLC unit 55 is subjected to variable-length decoding, and the data relating to the direct current (DC) component and the alternating current (AC) component is obtained.

Of the obtained data, the AC component data is quantized arranged in an 8 × 8 matrix in the same zigzag scan from low to high frequency components as performed by the MPEG encoder 50. Data is obtained, and the obtained quantized data is supplied to the inverse quantizer 74.

In the inverse quantizer 74, the inverse quantization is performed by the above-described quantization matrix, and the data obtained by the inverse quantization is supplied to the inverse DCT unit 75, where the inverse DCT operation is performed and the image is processed. Data is obtained as decoded data.

The obtained image data is temporarily stored in an image memory 77, the temporarily stored image data is supplied to a motion compensation predictor 78, and the supplied image data is used to calculate a difference image in the motion compensation prediction. Used as a reference decoded image for performing the decoding.

As described above, the image data constituting the moving picture is encoded and transmitted or recorded by the MPEG encoder 50, and the encoded data received or reproduced is decoded by the MPEG decoder 70. This method is obtained as moving image information, and such a method is used in both MPEG1 and MPEG2.

The encoding rate handled by MPEG1 is 1.
The coding rate of a standard TV signal handled by MPEG2 is 15 Mbps, and the coding rate of an HDTV signal is higher.

On the other hand, a high-efficiency low-rate coding method having a low coding rate is a coding method for a communication line having a low transmission rate, in which many functions for handling multimedia signals are added. The MPEG4 standard was established as a method.

The MPEG4 was initially considered to be used in a low-rate communication network such as the Internet, and the coding system was discussed. MPEG4
Image size handled by MPEG2 is 720 in the horizontal direction.
An image size that is half or one-fourth that of the pixel resolution is handled.

As the data having different image sizes, an image having a normal resolution or an image having a small size is used in accordance with the data size that can be handled by each communication path. In order to obtain the above image data software as an encoded image by MPEG4, data conversion for converting a normal size image into a small image size and using it is necessary.

[0041]

By the way, generally, the image size conversion of coded data is performed by decoding coded data to obtain image data, and downconverting the obtained image data. Then, after obtaining data of a desired image size, the data is compression-encoded again and obtained as coded data having a converted image size.

However, such a method of decoding and re-encoding an image to obtain coded data of which the image size has been converted requires a large image memory at the time of re-encoding, and also requires decoding and re-encoding. A large amount of data processing steps are required during signal processing.

In particular, the processing step for obtaining a motion vector at the time of encoding requires a large arithmetic processing step, for example, the calculation of the absolute value of the difference for block matching requires addition processing for the number of pixels.

Such image size conversion includes decoding,
In addition, a large apparatus is required for re-encoding, which is economically unfavorable, and a new distortion component is added to the image signal at the time of re-encoding, which is also unfavorable in terms of image quality.

Therefore, it is economically preferable to generate encoded data in which the number of pixels is changed without decoding the encoded image data, and it is desirable to obtain a new encoded signal while maintaining good reproduction image quality. It is possible and preferable.

A method for converting an image size without decoding image data from an encoded signal is disclosed in Japanese Patent Application Laid-Open No. H10-336672, entitled "Encoding method conversion apparatus and its motion vector detection method". Is disclosed.

The publication discloses a method of obtaining encoded image data as data encoded by another encoding method, and relates to a signal processing amount without deteriorating a motion vector detection accuracy for motion prediction compensation. As a way to reduce
When decoding the encoded image data, the motion vector is temporarily stored, a candidate vector is selected using the temporarily stored motion vector, and the image size ratio is converted using the selected candidate vector. A method is disclosed.

However, in the method of performing the image size conversion, when selecting a new motion vector from a plurality of candidate vectors, a plurality of candidate vectors are used for each new image data after the size conversion. A motion compensation prediction is performed to obtain a difference image between the predicted image and the predicted image, and a motion vector in which the sum of absolute difference values between the obtained difference image and the pixel value is minimized is obtained. Therefore, although the quality of the converted image signal can be kept good, the number of steps for signal operation processing is only a large number.

Therefore, according to the present invention, when performing image size conversion of data encoded using motion compensation prediction,
The converted motion vector is calculated by multiplying the motion vector by the converted image size ratio. When the image size ratio is larger than 1, the obtained motion vector value is used for all macroblocks, When the size is small, the motion vector is obtained by using the statistics of a plurality of motion vectors corresponding to the image before the conversion corresponding to the size of the macroblock after the conversion. An object of the present invention is to provide a data transmission method and a configuration of an encoded data image size conversion device.

[0050]

The present invention comprises the following means 1) to 6) to solve the above-mentioned problems. That is,

1) A motion vector is obtained for each unit image having a predetermined number of pixels as a unit, motion compensation is performed, and image data of each motion-compensated unit image is orthogonally transformed to obtain a first image. To obtain first encoded data obtained by encoding image data of an image size and obtain the obtained first encoded data as second encoded data encoded by image data of a second image size. A method of converting the size of encoded image data to perform the image size conversion of (a), wherein a first step of obtaining orthogonally transformed image data and a motion vector for each unit image from the first encoded data and temporarily storing them ( 14a, 14b, 15a) and the image conversion ratio, which is the ratio of the second image size to the first image size, is greater than 1; A plurality of conversion unit images are generated from the stored one unit image based on the image conversion ratio, and respective motion vectors of the plurality of conversion unit images are used for motion compensation of the one unit image. The motion vector temporarily stored in the first step is obtained as a first conversion motion vector obtained by multiplying the image conversion ratio, and when the image conversion ratio is smaller than 1, the motion vector is temporarily stored in the first step. Generating one conversion unit image from the plurality of unit images based on the image conversion ratio, and multiplying a motion vector of each of the plurality of unit images by the image conversion ratio to generate a plurality of second conversion motion vectors. A second step (14c, 14d, 15b, 1b) of generating and obtaining one integrated motion vector based on the obtained and obtained plurality of second converted motion vectors. c), and at least a third step (17) of performing variable-length encoding on one of the first transformed motion vector and the integrated motion vector and the data of the transformed unit image. A size conversion method for encoded image data, characterized in that:

2) When the correlation between the plurality of motion vectors related to the plurality of unit images is low, the integrated motion vector in the second step is to use the motion vector that gives the lowest activity of the motion compensation error data. A method for converting the size of encoded image data according to item 1).

3) When the correlation between the plurality of motion vectors relating to the plurality of unit images is low, the integrated motion vector in the second step is a motion vector having a motion amount of 0. ).

4) When the plurality of conversion motion vectors related to the plurality of unit images include a highly correlated motion vector, the integrated motion vector in the second step is based on the highly correlated motion vector. The method for converting the size of encoded image data according to 1), wherein an integrated motion vector is generated and obtained.

5) A motion vector is obtained for each unit image in units of a predetermined number of pixels to perform motion compensation, and the image data of each motion-compensated unit image is orthogonally transformed to obtain a first image. To obtain first encoded data obtained by encoding image data of an image size and obtain the obtained first encoded data as second encoded data encoded by image data of a second image size. A coded data transmission method for transmitting coded data obtained by performing image size conversion of (a), wherein image data and a motion vector, which are orthogonally transformed for each unit image, are obtained from the first coded data and temporarily stored. The first step of storing (14
a, 14b, 15a) and the image conversion ratio, which is the ratio of the second image size to the first image size, is greater than 1 than the one unit image temporarily stored in the first step. A plurality of conversion unit images are generated based on the image conversion ratio, and respective motion vectors of the plurality of conversion unit images are used for motion compensation of the one unit image and are temporarily stored in the first step. A first value obtained by multiplying the obtained motion vector by the image conversion ratio
When the image conversion ratio is smaller than 1, one conversion unit image is generated from the plurality of unit images temporarily stored in the first step based on the image conversion ratio. Multiplying each motion vector of the plurality of unit images by the image conversion ratio to obtain a plurality of second converted motion vectors, and forming one integrated motion vector based on the obtained plurality of second converted motion vectors. The second step (14c, 14d,
15b, 15c), a third step (17) of performing variable-length encoding on one of the first transformed motion vector and the integrated motion vector, and the data of the transformed unit image, and a third step thereof. A fourth step (32) of generating the coded data obtained in the step as data arranged in a packet of a predetermined size.

6) A motion vector is obtained for each unit image having a predetermined number of pixels as a unit, motion compensation is performed, and each image data of the motion-compensated unit image is orthogonally transformed to obtain a first image. To obtain first encoded data obtained by encoding image data of an image size and obtain the obtained first encoded data as second encoded data encoded by image data of a second image size. A temporary storage means (14a, 14a,) for obtaining orthogonally transformed image data and a motion vector for each unit image from the first coded data and temporarily storing the same. 14b, 15a) and the image conversion ratio, which is the ratio of the second image size to the first image size, is greater than 1; A plurality of conversion unit images are generated from the obtained one unit image based on the image conversion ratio, and respective motion vectors of the plurality of conversion unit images are used for motion compensation of the one unit image. A first converted motion vector obtained by multiplying the motion vector temporarily stored in the temporary storage unit by the image conversion ratio is obtained. When the image conversion ratio is smaller than 1, a plurality of temporarily stored motion vectors are temporarily stored in the temporary storage unit. One conversion unit image is generated from the unit images based on the image conversion ratio, and a plurality of second conversion motion vectors are obtained by multiplying each of the motion vectors of the plurality of unit images by the image conversion ratio. 1 based on the obtained plurality of second transformed motion vectors.
Data conversion means (14c, 14d, 15b, 15c) for obtaining and generating one integrated motion vector, and any one of the first converted motion vector and the integrated motion vector, and data of the converted unit image. And a variable-length coding unit (17) for performing variable-length coding.

[0057]

BEST MODE FOR CARRYING OUT THE INVENTION Preferred embodiments of a method for converting image data of encoded data, a method for transmitting encoded image data, and an apparatus for converting encoded image size according to the present invention will be described below. FIG. 1 shows the configuration of a coded data image size conversion apparatus equipped with the coded data image size conversion method, which will be described below with reference to the drawings.

An encoded data image size conversion device 10 shown in FIG.
VLD (variable length decoding) unit 13, image data converter 14, motion vector converter 15, header parameter converter 16, re-VLC (variable length coding)
It comprises a device 17, a code amount controller 18, and a buffer 19.

Next, the operation of the coded data image size converter 10 configured as described above will be described. First,
MPEG (moving picture e) supplied to the input terminal 11
(xperts group) The coded data is supplied to the buffer 12, temporarily stored, and read out by the VLD unit 13 as needed.

The VLD unit 13 decodes the encoded data VLC according to a predetermined format, and among the data obtained by decoding, the image data is sent to the image data converter 14 and the motion vector is sent to the motion vector converter. Fifteen
And the header information signal is transmitted to the header parameter converter 1
6.

Among these converters, the header parameter converter 16 converts the encoded data image size converter 10
Of the header information performed based on the image size conversion performed in
It is supplied to the LC unit 17.

The motion vector converter 15 converts the motion vector associated with the image size conversion by a method described later, and the image data converter 14 converts the image size by a predetermined method described later. The motion vector and the image data obtained by the conversion are supplied to the VLC unit 17 again.

In the re-VLC unit 17, each of the supplied converted data is subjected to variable-length coding according to a method defined by the MPEG standard, and the variable-length coded data is stored in a buffer 19. Supplied and temporarily stored.

In the buffer 19, the code amount information of the temporarily stored encoded data is supplied to the code amount controller 18, and the code amount controller 18 compares the code amount of the supplied encoded data with the target code (not shown). The target code amount set by the amount setting means is compared, and the code amount increase / decrease information obtained by the comparison is supplied to the image data converter 14.

The image data converter 14 converts the image data based on the supplied code amount increase / decrease information, for example, by changing a scale value such as a quantization scale. The code amount of the coded data is adjusted, and the size of the coded data supplied from the buffer 19 is controlled so as to be a predetermined target code amount.

As described above, the image size of the coded data supplied to the coded data image size converter 10 is a coded data output as a coded data output which is a predetermined code amount converted into a predetermined image size. This is supplied from the image size conversion device 10.

The image size conversion of the encoded image data thus performed will be further described. The conversion of the image size is, for example, 640 pixels in the horizontal direction × 4 in the vertical direction.
A large screen of 80 pixels is divided by 1 /
In the case of converting into a screen of 320 pixels × 240 pixels which is 2, four data of a macroblock of 16 pixels × 16 pixels (hereinafter sometimes abbreviated as MB), that is, data of 32 pixels × 32 pixels is converted to 16 pixels × What is necessary is just to convert it into data of 16 pixels.

When the motion vectors of the four MBs are at the same distance in the same direction, the image data converter 1
4, the image data is converted, and the same four motion vectors may be one motion vector in which the vertical and horizontal motion amounts are halved.

However, an image signal to be coded and transmitted often moves in a complicated manner in the screen, and adjacent MBs usually have different vector values. The image size conversion method differs depending on the difference between the vector values.

The conversion of the image size with respect to the difference between the motion vectors for each MB will be further described with reference to the drawings.
FIG. 2 shows that a motion vector in a large image has four MBs.
Is exemplified.

In the figure, 16 pixels × 16 pixels in a large screen of 640 pixels in the horizontal direction × 480 pixels in the vertical direction.
The motion vectors of each of the four MBs of pixels are 2a, 2b
2c and 2d.

Although these motion vectors are directed to the upper right direction, they are different from each other, and the identity of the motion vectors pointing in such different directions is determined by the correlation value, for example, from the average value of the motion vectors. The difference value can be expressed by using a variation, a variance, or a standard deviation value.

Here, a method of converting an image size related to a correlation value of a motion vector of each MB constituting a screen and a case where the conversion increases and decreases the image size will be described in detail.

First, a case where the image size is converted from a small image to a large image will be described. FIG.
Next, a motion vector in a small image is illustrated for one MB.

In the figure, 16 pixels × 16 pixels in a small screen of 320 pixels in the horizontal direction × 240 pixels in the vertical direction.
One MB of a pixel and a motion vector related to the MB are 3
Indicated as a.

When converting from a small image size as shown in this figure to an image size in which each of the horizontal and vertical directions is twice as shown in FIG.
Image data composed of two MBs are converted into image data of two MBs each in the vertical and horizontal directions, and the motion vector relating to those MBs is obtained by doubling the motion vector 3a in each of the vertical and horizontal directions. What is necessary is just to make the motion vector obtained by multiplication into each motion vector of 2a, 2b, 2c, and 2d.

When the image size is converted to a large image size, encoding is performed by generating a plurality of new motion vectors in this way and generating a plurality of MB data based on one MB of image data. The image size of the data can be converted.

Next, a case where the image size is converted to a small image size will be described. When converting to the small image size, the image data of a plurality of MBs is converted into the image data of one MB to obtain data of a small image size. Vectors need to be obtained, and the method of synthesizing the motion vectors differs depending on the correlation between a plurality of motion vectors.

First, in the case where the correlation between a plurality of motion vectors to be synthesized is low, from the large image size as shown in FIG. 2, the horizontal and vertical directions shown in FIG. When converting into a certain image size, the correlation between the plurality of motion vectors 2a, 2b, 2c, and 2d is low, so that the motion vectors cannot be integrated.

When the correlation of such motion vectors is low, the activity is the lowest among the motion compensation error data using the respective motion vectors, that is, the value of the motion compensation error image generated at the time of decoding is the smallest. Is preferably selected as a candidate, and a value obtained by multiplying the selected motion vector by a size conversion ratio (1/2 in this case) is preferably used.

As a motion vector that simply gives low compensation error data, there is a method in which image data of a corresponding macro block has a high frequency and a large level DCT coefficient value. It is preferable to select the error data by calculating the error data related to the motion vector.

Therefore, there is a method of giving a motion vector as zero in order to reduce the processing amount for selecting such a motion vector. That is, when the respective vector values are different from each other by more than a predetermined number of pixels, even if any of the four different motion vectors is used as the representative vector, This is because it is often said that a compensation error approximately equal to the error amount when the value is set to zero is obtained.

The case where the correlation between a plurality of motion vectors to be synthesized is low has been described above. Next, the case where the image size is reduced and the correlation between the plurality of motion vectors is large will be described.

FIG. 4 shows an example of the distribution of motion vectors when a group of motion vectors having a high correlation is partially included. In the example shown here, nine MBs are converted into one MB of image data, that is, each of the horizontal and vertical image sizes is reduced to 1/3.
Is converted to.

As described above, the image data of many MBs is
In the case where the data is converted into MB data, a plurality of motion vectors may partially have a large correlation, and in the example shown here, the motion vectors of 9 MBs indicated as 4a to 4i Of these, the motion vectors of the six MBs except for 4a, 4b, and 4d have almost the same vector value.

When there are many common vectors as described above, the motion vector value which has the highest frequency as the motion vector integrating the entire block is adopted as the mode value.

In order to statistically determine the mode value of the motion vector, the most frequent value is determined as the integrated motion vector based on the distribution of the motion vector value, and the determined vector value is determined as the integrated motion vector. Value.

However, the motion vector values of 4c, 4e, 4f, 4g, 4h and 4i, which are the same vector values, are slightly different values, but the difference is, for example, about 0.5 pixel. If the error level is a predetermined error level, the error is ignored.

In this manner, a group of motion vectors having a high degree of correlation is selected, and the mode or average value of the motion vectors can be used as an integrated motion vector value. The motion vector value is obtained by multiplying the motion vector value by a factor of, for example, 1/3 times in accordance with the conversion ratio of the image size.
5 to be supplied to the re-VLC unit 17.

The case where the image size is reduced and the correlation between a plurality of motion vectors is partially large has been described above. Next, the case where the correlation between motion vectors is entirely large will be described.

FIG. 5 shows an example of the distribution of motion vectors when the motion vectors have high correlation. In the example shown here, each of the horizontal and vertical is converted to 1/3 image data by converting nine MBs into one MB of image data as in FIG.
Is converted to.

As described above, when the correlation between a plurality of motion vectors is large as a whole, each motion vector value is multiplied by the degree of correlation (variance value) of the plurality of motion vectors to obtain a sum, and the sum is calculated. Is divided by the number of motion vectors to obtain an integrated motion vector value, and a motion vector value obtained by multiplying the obtained integrated vector value by a size conversion ratio of an image size of, for example, 1/3 is a converted motion vector value. Used as

As described above, when the image size is converted to an image size having a ratio larger than 1, and when the image size is converted to an image size having a ratio smaller than 1, there is no correlation between a plurality of motion vector values. The method of converting each motion vector has been described in the case where a group of motion vectors having a large correlation exists and in the case where there is a large correlation in the entire motion vector.

Next, the configuration of a coded data image size conversion apparatus equipped with such a motion vector conversion method,
The operation of the device will be described in detail. FIG. 6 shows the configuration of the encoded data image size conversion device.

The coded data image size conversion device 10a shown in FIG.
2, a VLD unit 13, an AC / DC coefficient memory 14a,
An image data converter 14 including a parameter memory 14b, an image size converter 14c, and a requantizer 14d;
A motion vector converter 15 including a motion vector memory 15a, a size converter 15b, and a motion vector statistical processor 15c; a header parameter converter 16;
And a code amount controller 18 and a buffer 19.

Next, the operation of the coded data image size conversion apparatus 10a configured as described above will be described. First, data that has been subjected to image size conversion and that has been encoded based on, for example, the MPEG standard is supplied to the buffer 12 via the input terminal 1 and is temporarily stored therein.

The encoded data temporarily stored in the buffer 12 is read in response to a read request from the VLD unit 13, and the read encoded data is supplied to the VLD unit 13.

In the VLD unit 13, the DCT obtained by subjecting the image data to discrete cosine transform from the coded data which has been subjected to the variable length decoding by the MPEG2 encoder and transmitted.
The direct current (DC) component and the alternating current (AC) component of the coefficient also indicate the coding mode, quantization scale, and which block information in the macroblock of the coded and transmitted image data for each macroblock. CBP (coded blo
ck pattern), motion vector information, and the like.

Among the obtained pieces of information related to encoding, the information related to the DC component and the AC component of the DCT coefficient are stored in the AC / DC coefficient memory 14a in the encoding mode,
Parameter information such as a quantization scale and CBP is supplied to a parameter memory 14b, and information related to a motion vector is supplied to a motion vector memory 15a. Parameter value information described in a header such as an MPEG sequence header and a picture header is The information is supplied to a header parameter value converter 16 having a built-in memory (not shown), and the supplied information is temporarily stored in each memory.

As described above, the information temporarily stored in each memory is read out as necessary and subjected to image size conversion processing. Next, the image size conversion processing will be described.

First, image size conversion of image data will be described. Image data temporarily stored in the AC / DC coefficient memory 14a is supplied to an image size converter 14c.
In this case, the supplied AC component data and DC component data are arranged as an 8 × 8 matrix in a predetermined order in the direction from low frequency to high frequency.

The image data components arranged in the matrix are converted into DCT coefficient values such that only DCT-converted coefficient values relating to the frequency component positions of predetermined image data are obtained in accordance with the image conversion ratio. .

That is, the method of obtaining the DCT coefficient value in the converted image size differs depending on the size conversion ratio of the image. 1/2
The DCT coefficient when set as double is obtained by halving the pixel size for each of horizontal and vertical.
The low-frequency side 3 × 3 frequency component region is requantized by the requantizer 14.
d.

Note that this size conversion is described in JP-A-2000-2
The technical content disclosed in Japanese Patent No. 17111 “Digital Video Decoding Apparatus and Program Recording Medium Capable of Converting Image Size” can be applied.

Thus, the image size converter 14c
Is supplied with a parameter of a coded block pattern (CBP) for each macro block stored in the parameter memory 14b, and generates AC component data and DC component data of DCT coefficients validated based on the CBP parameter. Then, the image size is converted.

The image data subjected to the image size conversion processing is supplied to a requantizer 14d. The requantizer 14d stores quantization scale information for each macro block temporarily stored in a parameter memory 14b. Coding mode information is provided.

The coding mode information includes a mode of motion compensation prediction, that is, the prediction mode includes forward prediction, backward prediction,
There are bidirectional prediction, independent mode, and the like, and the prediction mode information is used without being changed.

Further, the quantization scale information supplied together with the encoding mode information is supplied to the code amount controller 1 so that the requantized data after the image size conversion has a desired bit rate.
Requantization is performed by the requantizer 14d based on the quantization scale information supplied from 8.

The requantized data requantized by the requantizer 14d is rearranged in a predetermined order defined by MPEG, and the rearranged requantized data is re-quantized by the re-VLC unit. 17 is supplied.

In the re-VLC unit 17, the motion vector stored in the motion vector memory 15a is
5b, the size of the motion vector is converted based on the ratio of the image size, and the size-converted motion vector is supplied to the motion vector statistical processor 15c, and converted into MB data after image size conversion based on the processing method described above. The calculated motion vector value is also supplied.

In the re-VLC unit 17, information obtained by converting parameter information such as an MPEG sequence header and a picture header by the header parameter converter 16 based on the image size conversion processing is generated as header information. The generated header information is also supplied.

That is, the header information generated by the conversion is
When the encoded data is encoded data based on the MPEG1 standard, the horizontal_size # value and vertical_siz # value described in the picture header of the sequence header of the encoded data are the converted image size, Also bi
t # rate # value, vbv # buffer # size # value, and vbv # delay
The header information according to the above is changed to a value based on the coded data after conversion, and is supplied to the VLC unit 17 again.

In the re-VLC unit 17, the re-quantizer 14
d, the data supplied from the motion vector statistical processor 15c and the header parameter converter 16 are re-VLC-converted, and the coded data generated by the processing is supplied to the buffer 19.

The generation of the transform coded data performed in this way is performed only with respect to the image size conversion.
When the format of encoded data supplied as an output signal is converted into encoded data based on a different MPEG format, signal processing for that is required.

For example, the image size of encoded data generated based on the MPEG2 format is
In the case where the data is supplied as a bit stream signal of encoded data based on G4, the signal processing in the re-VLC is performed as VLC signal processing conforming to the MPEG4 standard.

The coded data obtained by the VLC signal processing is supplied to the buffer 19, where bit rate information of the coded data is obtained based on the code amount supplied at a predetermined time interval. , And the obtained bit rate information is supplied to the requantizer 14d.

The requantizer 14d compares the preset bit rate with the bit rate after image size conversion. For example, when the bit rate after conversion is higher than the set bit rate, the quantization scale is changed. By changing the value to a large value and performing the re-VLC process, the code amount of the converted bit stream is reduced.

On the contrary, when the bit rate of the converted bit stream is smaller than the predetermined set bit rate, the quantization scale in the requantizer 14d is changed to a small value, and the re-VLC-converted bit stream is changed. Feedback code amount control is performed so that the code amount of the stream becomes large, so that encoded data after size conversion at a desired bit rate is obtained.

As described above, the configuration of the encoded data image size conversion apparatus to which the encoded data before the size conversion is supplied and which supplies encoded data of a predetermined image size as an output signal.
The operation was described.

According to the coded data image size conversion apparatus, when converting the image size of the image data coded by using the motion compensation prediction, a macroblock whose image size has been converted is generated and the macroblock is converted. The motion vector after the image size conversion of the block is obtained by multiplying the motion vector corresponding to the image before the conversion corresponding to the size of the macroblock after the conversion by the conversion size ratio, and the image size is converted into a large value. Sometimes, the obtained motion vector value is used for all the macroblocks, and when the image size is converted to a small value, a plurality of motion vectors of the image before the conversion corresponding to the size of the macroblock after the conversion are used. A motion vector is obtained after the conversion using the statistics, and the obtained motion vector and the converted macro block are used. It transformed coding data of the image size is to be obtained Te.

In the data size conversion apparatus, when the correlation between a plurality of converted motion vectors used for converting an image size into a small value is low, the motion compensation error data of each motion vector is used. Conversion processing is performed using a motion vector obtained as the lowest activity.

When the correlation between a plurality of motion vectors used in the conversion process is low when the image size is reduced in the data size conversion device, the conversion process is performed by setting the motion vector to zero and the correlation between the motion vectors is reduced. Is high, by selecting a group of motion vectors having a high correlation from a plurality of motion vectors, or by obtaining a motion vector that gives the mode of the motion vectors, or by averaging the motion vectors Thus, the encoded data after the size conversion is generated.

Further, in the data size conversion device,
The converted motion vector when the image size is reduced is obtained by calculating a weight value and a plurality of motion vector values based on the degree of correlation between the plurality of motion vectors, and performing size conversion using the obtained motion vector values. Coded data is generated.

As described above, according to the configuration of the coded data image size conversion apparatus described in these examples, when a new motion vector is selected from a plurality of candidate vectors,
For the new image data after size conversion, the motion vector conversion can be calculated only with the statistics of the motion vector value of the related part, so the processing of detecting the motion vector from the decoded image data itself, which has been performed conventionally. , And a difference image between a predicted image and a predicted image generated based on motion compensated prediction by using individual vectors of a plurality of candidate vectors, and further, a sum of absolute difference values between pixel values of the difference images and It is not necessary to perform complicated signal processing such as finding a motion vector that minimizes the motion vector, and it is possible to keep the deterioration of the detection accuracy of the motion vector small and to reduce the scale of hardware constituting the apparatus. An image size conversion device that performs size conversion can be configured.

The code amount control before and after image size conversion shown in these examples can be applied to either fixed bit data size control or variable bit data size control. .

In the embodiment described above, the encoding method of the bit stream to be converted is MPEG1 or MPEG.
2, the coding method is not limited to these, and the present invention is also applicable to a bit stream coded by the same motion compensation prediction.

[0127] In the case of MPEG1 and MPEG2, macroblocks having a size of 16 pixels x 16 pixels are used for encoding. However, in the case of an encoding method in which encoding is performed using other pixel sizes, the following is used. Even if there is, a unit image is defined in units of a predetermined number of pixels, a motion vector is obtained for each unit pixel, motion compensation is performed, and each image data for each unit image subjected to the motion compensation is subjected to data conversion. In the case where the energy distribution of the image data is concentrated, the above-described image size conversion of the encoded data can be applied.

Next, a case will be described in which the image size conversion is applied to a bit stream rate conversion in a communication channel transmitted in a packetized data structure. FIG. 7 shows a configuration of an image coded data transmission apparatus that receives coded data that is packetized and transmitted, converts the coded data into data having a different packet size, and transmits the data.

The image coded data transmission device 30 shown in FIG.
Is a coded data input terminal 11, a header remover 31, a buffer 12, a VLD unit 13, an image data converter 14, a motion vector converter 15, a header parameter converter 16,
It comprises a re-VLC unit 17, a code amount controller 18, a buffer 19, and a packetizer 32.

[0130] The coded image data transmitting apparatus 30 receives a bit stream constituted as data of the first packet size, obtains coded data, and obtains the coded data as shown in FIG. The image size is converted by the same method as that of the coded data image size conversion apparatus shown in FIG.
Is generated and supplied as a bit stream of the second packet size.

FIG. 8 shows the operation of the header remover. In the figure, the header removal operation is started in S41,
42, the header supplied to the data supplied in the first packet size is removed, and encoded data is obtained in S43.
The operation of supplying the encoded data obtained by 44 to the buffer 12 is continued.

The coded data temporarily stored in the buffer 12 in this way is subjected to image size conversion as described above, and the coded data subjected to the image size conversion is temporarily stored in the buffer 19, where necessary. And supplied to the packetizer 32.

FIG. 9 shows the operation of the packetizer. The operation of the packetizer 32 is started in S51, and in S52
Is packetized according to the packet size of the second bit stream, a header signal is added to the first part of the packet in S53, and a bit stream is generated.
The bit stream thus generated at 54 is supplied to a network (not shown), and all such a series of operations are performed, and the end of S55.

The image coded data transmission apparatus 30 operated as described above converts the bit rate of the reception and transmission of a desired image signal between networks having different transmission capacities by converting the image size. It is to be generated and transmitted as encoded data.

The conversion of the image size is performed by, for example, xDSL.
A video bit stream coded by MPEG2 supplied from a line is converted into coded data of an MPEG4 format whose image size is, for example, 1/3 in length and width, and is transmitted to a friend terminal connected to ISDN. Used in some cases.

As described above, the image coded data transmission apparatus 30 can reduce the image size between a plurality of networks having different transmission bit rates, which are planned to be actively introduced in the future, using a wired or wireless system. By performing the conversion, a bit stream having a bit rate adapted to each network can be generated and supplied.

[0137]

According to the first aspect of the present invention, the coded data that has been subjected to motion compensation prediction and orthogonal transform for each predetermined number of pixels and compression-coded according to the first image size is converted to the second image size. When transforming into image encoded data encoded by, an orthogonal transformation coefficient and a transformation motion vector corresponding to the transformation ratio of the image size are obtained, and an image is formed based on the obtained orthogonal transformation coefficient and the transformation motion vector. Since the converted motion vector can be obtained by simple signal processing when obtaining the coded data whose size has been converted, there is an effect that a size conversion method of image coded data which can be performed with a small number of signal processing steps can be provided.

According to the second aspect of the present invention, when the correlation between the plurality of converted motion vectors related to a plurality of unit image data is low, the motion vector giving the activity of the lowest motion compensation error data is used. Since the conversion of the image size is performed in this manner, in addition to the effect of the first aspect, there is an effect that a method of converting the size of encoded image data with higher image quality can be provided.

According to the third aspect of the present invention, especially when the correlation between the converted motion vectors is low, the motion vector whose motion amount is 0 is used.
In addition to the effects described above, there is an effect that it is possible to provide a method for converting the size of encoded image data, which is easier to convert.

According to the fourth aspect of the present invention, particularly when a plurality of conversion motion vectors related to a plurality of unit image data include a vector having a high correlation, integration is performed based on the motion vector having a high correlation. Since the image size is converted so as to generate a motion vector, in addition to the effect of the first aspect, there is an effect that a method for converting the size of encoded image data with higher image quality can be provided.

According to the fifth aspect of the present invention, encoded data compressed and encoded according to the first image size is subjected to motion compensation prediction and orthogonal transform for each predetermined number of pixels supplied from the network. When converting into image encoded data encoded according to the second image size, an orthogonal transformation coefficient and a transformation motion vector corresponding to the transformation ratio of the image size are obtained, and the obtained orthogonal transformation coefficient and transformation motion are obtained. To convert the size of the encoded image data based on the vector, the converted motion vector is converted to a motion vector with little deterioration in detection accuracy by simple signal processing, and the size of the encoded image data is converted to obtain a network with a different transmission rate. Thus, there is an effect that a method of transmitting encoded image data that can be transmitted to a computer can be provided.

According to the sixth aspect of the present invention, encoded data that has been subjected to motion-compensated prediction and orthogonal transformation for each predetermined number of pixels and that has been compression-encoded by the first image size is encoded by the second image size. When transforming into encoded image encoded data, an orthogonal transform coefficient and a transform motion vector corresponding to the image size transformation ratio are obtained, and image encoding is performed based on the obtained orthogonal transform coefficient and the transform motion vector. Since the data size conversion is performed, the converted motion vector can be obtained as a motion vector with little deterioration in detection accuracy by simple signal processing, so that there is an effect that a configuration of an image encoded data size conversion device that can be easily realized can be provided. .

[Brief description of the drawings]

FIG. 1 is a schematic block diagram of an encoded data image size conversion device according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating a motion vector in a large image according to an embodiment of the present invention for four macroblocks.

FIG. 3 is a diagram showing a motion vector in a small image according to an embodiment of the present invention for one macroblock.

FIG. 4 is a diagram showing a distribution of motion vectors having a highly correlated motion vector group according to the embodiment of the present invention.

FIG. 5 is a diagram illustrating a distribution of a motion vector having a highly correlated motion vector value according to the embodiment of the present invention.

FIG. 6 is a diagram showing a configuration of an encoded data image size conversion device according to an embodiment of the present invention.

FIG. 7 is a diagram illustrating a schematic configuration of an image coded data transmission device according to an embodiment of the present invention.

FIG. 8 is a flowchart illustrating an operation of the header remover according to the embodiment of the present invention.

FIG. 9 is a flowchart illustrating an operation of the packetizer according to the embodiment of the present invention.

FIG. 10 is a diagram showing a configuration of a conventional MPEG encoder.

FIG. 11 is a diagram showing a configuration of a conventional MPEG decoder.

[Explanation of symbols]

 10, 10a Encoded data image size converter 11 Input terminal 12 Buffer 13 VLD unit 14 Image data converter 14a AC / DC coefficient memory 14b Parameter memory 14c Image size converter 14d Requantizer 15 Motion vector converter 15a Motion vector Memory 15b Size converter 15c Motion vector statistical processor 16 Header parameter converter 17 Re-VLC unit 18 Code amount controller 19 Buffer 30 Image coded data transmission device 31 Header remover 32 Packetizer 50 MPEG encoder 51 Input terminal 52 adder 53 DCT unit 54 quantizer 55 VLC unit 56 buffer 57 code amount controller 61 inverse quantizer 62 inverse DCT unit 63 adder 64 image memory 65 motion compensation predictor 70 MPEG decoder 71 encoded data input Child 72 buffer 73 VLD 74 inverse quantizer 75 inverse DCT unit 76 adder 77 the image memory 78 the motion compensation predictor

Claims (6)

[Claims] A motion vector is obtained for each unit image having a predetermined number of pixels as a unit, motion compensation is performed, and the image data of each of the motion-compensated unit images is orthogonally transformed to obtain a first image. To obtain first encoded data obtained by encoding image data of an image size and obtain the obtained first encoded data as second encoded data encoded by image data of a second image size. A size conversion method of image encoded data for performing image size conversion of the above, wherein orthogonally transformed image data and a motion vector for each unit image are obtained from the first encoded data and temporarily stored.
And if the image conversion ratio, which is the ratio of the second image size to the first image size, is greater than 1, the first
Generating a plurality of conversion unit images based on the image conversion ratio from one unit image temporarily stored in the step, and calculating a motion vector of each of the plurality of conversion unit images by the motion compensation of the one unit image. To obtain a first converted motion vector obtained by multiplying the motion vector temporarily stored in the first step by the image conversion ratio, and when the image conversion ratio is smaller than 1, the first step Generating one conversion unit image from the plurality of unit images temporarily stored based on the image conversion ratio, and multiplying each motion vector of the plurality of unit images by the image conversion ratio to generate a plurality of second unit images. A second step of obtaining a converted motion vector and generating and obtaining one integrated motion vector based on the obtained plurality of second converted motion vectors; A third step of performing variable-length encoding on any one of a first transformed motion vector and the integrated motion vector and the data of the transformed unit image. How to convert the size of coded data. 2. The integrated motion vector in the second step, wherein when the correlation between the plurality of motion vectors related to the plurality of unit images is low, the motion vector giving the lowest activity of the motion compensation error data is used. 2. The method according to claim 1, wherein the size of the encoded image data is converted. 3. The integrated motion vector in the second step, wherein a motion vector having a motion amount of 0 is used when the correlation between the plurality of motion vectors related to the plurality of unit images is low. Item 2. The method for converting the size of encoded image data according to Item 1. 4. The integrated motion vector in the second step, wherein the plurality of converted motion vectors related to the plurality of unit images include a highly correlated motion vector.
2. The method according to claim 1, wherein an integrated motion vector is generated based on the highly correlated motion vector. 5. A motion compensation is performed by obtaining a motion vector for each unit image having a predetermined number of pixels as a unit, and the first image data is obtained by orthogonally transforming the image data of each of the motion compensated unit images. Obtaining first encoded data obtained by encoding image data having an image size, and obtaining the obtained first encoded data as second encoded data encoded by image data having a second image size; A coded data transmission method for transmitting coded data obtained by performing image size conversion of (a), wherein orthogonally transformed image data and a motion vector for each unit image are obtained from the first coded data to temporarily 1st to remember
And if the image conversion ratio, which is the ratio of the second image size to the first image size, is greater than 1, the first
Generating a plurality of conversion unit images from one unit image temporarily stored based on the image conversion ratio, and calculating a motion vector of each of the plurality of conversion unit images by the motion compensation of the one unit image. To obtain a first converted motion vector obtained by multiplying the motion vector temporarily stored in the first step by the image conversion ratio, and when the image conversion ratio is smaller than 1, the first step Generating one conversion unit image from the plurality of unit images temporarily stored based on the image conversion ratio, and multiplying each motion vector of the plurality of unit images by the image conversion ratio to generate a plurality of second unit images. A second step of obtaining a converted motion vector and generating and obtaining one integrated motion vector based on the obtained plurality of second converted motion vectors; A third step of performing variable-length encoding on one of the first transformed motion vector and the integrated motion vector and the data of the transformed unit image, and encoding the encoded data obtained by the third step. A fourth step of generating as data arranged in a packet of a predetermined size, and a method of transmitting encoded image data. 6. A motion vector is calculated for each unit image having a predetermined number of pixels as a unit, motion compensation is performed, and each image data of the motion compensated unit image is orthogonally transformed to obtain a first image. To obtain first encoded data obtained by encoding image data of an image size and obtain the obtained first encoded data as second encoded data encoded by image data of a second image size. An image-encoded data size conversion device that performs image size conversion of: a temporary storage unit that obtains and temporarily stores image data and a motion vector that are orthogonally transformed for each unit image from the first encoded data; When the image conversion ratio, which is the ratio of the second image size to the first image size, is greater than one, a plurality of unit images temporarily stored by the temporary storage unit are stored. A plurality of conversion unit images are generated based on the image conversion ratio, and the respective motion vectors of the plurality of conversion unit images are used for motion compensation of the one unit image and temporarily stored in the temporary storage unit. A first transformed motion vector obtained by multiplying the vector by the image transformation ratio;
When the image conversion ratio is smaller than 1, one conversion unit image is generated from the plurality of unit images temporarily stored by the temporary storage unit based on the image conversion ratio, and each of the plurality of unit images is generated. Data conversion means for obtaining a plurality of second converted motion vectors by multiplying the motion vector by the image conversion ratio, and generating one integrated motion vector based on the obtained plurality of second converted motion vectors; A variable-length coding unit configured to perform variable-length coding on any one of the first converted motion vector and the integrated motion vector and the data of the converted unit image. Coded data size conversion device.