JP2003153271A - Moving picture encoding sequence conversion apparatus and method, and its program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving picture encoding sequence conversion apparatus and method, and its program that can reduce a processing quantity at re- encoding without deteriorating an encoding efficiency by predicting a motion vector at motion compensation prediction in the case of re-encoding a moving picture encoding sequence depending on applications. SOLUTION: A moving picture decoding section 100 receives a pixel coefficient code 1001 and a motion vector code 1002 to generate a decoded image 1004, calculates complexity of a coefficient distribution (encoding complexity information 1006) from the pixel coefficient code 1001 and outputs a motion vector 1003 used in decoding. A motion vector prediction section 108 references the encoding complexity information 1006 to generate a prediction vector 1005 in re-encoding. A moving picture encoding section 110 references the motion vector 1005 to re-encode a decoded image 1008 down-sampled by a resolution conversion section 109.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture code sequence conversion apparatus, a moving picture code sequence conversion method, and a program thereof, and more particularly, to re-encoding a moving picture code sequence according to the intended use and re-encoding. The present invention relates to a moving image code sequence conversion device, a moving image code sequence conversion method, and a program thereof that perform motion vector prediction to reduce the processing amount of.

[0002]

2. Description of the Related Art When a moving picture code string encoded with high quality is used in a worse transmission environment, it is necessary to convert the code string according to a moving picture coding method represented by MPEG according to the application. There is. Further, in order to convert the encoding method itself, a means of once decoding the code string into an image and then re-encoding it is used. At this time, when both coding methods involve motion-compensated prediction, the processing amount is estimated by estimating the motion vector used for motion-compensated prediction during re-encoding from the motion vector included in the original code string. The method of reduction is taken. Further, when converting a wideband code string to a narrowband, a method of converting the resolution of an image is usually used. It is necessary to consider resolution conversion also in motion vector estimation during re-encoding.

FIG. 9 shows a moving picture code string conversion that involves resolution conversion at the time of re-encoding in the moving picture code string converting apparatus and method (hereinafter referred to as conventional example 1) disclosed in Japanese Patent Laid-Open No. 2000-59789. It is the figure which changed the structure of the apparatus to perform according to this invention. Hereinafter, the moving image coding conversion method will be described with reference to FIG.

After the pixel coefficient code 1001 and the motion vector code 1002 of the code string are decoded, the coefficient code is inverse-quantized and inverse-transform coded. The motion compensation unit 105 also performs motion compensation processing on the reference image stored in the frame memory 106 according to the decoded motion vector 1003. The result of the inverse transform coding and the result of the motion compensation are added by the adder 107 to obtain the decoded image 1
004 is obtained. The decoded image 1004 is the resolution conversion unit 1
09 downsampled.

With the re-encoding, the motion vector predictor 10
8 is the motion vector 1003 and the resolution conversion rate 100.
According to 7, the prediction vector 1005 is calculated. The motion search unit 111 calculates a motion vector 1011 by referring to the decoded image 1008 whose resolution has been converted, the reference image stored in the frame memory 112, and the prediction vector 1005. The motion compensation unit 113 performs motion compensation using the motion vector 1011 and the reference image stored in the frame memory 112 to generate a predicted image 1012. The subtractor 114 takes the difference between the decoded image 1008 whose resolution has been converted and the predicted image 1012, and generates a difference image. The difference image obtained by the subtractor (differential device) 114 is transformed and quantized, and then variable-length coded to become a pixel coefficient code 1009.

The motion vector 1011 is also variable length coded and output as a motion vector code 1010.
The quantized difference image is again inversely quantized, then inversely transformed, added to the predicted image 1012, and then stored in the frame memory 112.

FIG. 2 is a conceptual diagram for explaining the relationship between blocks and candidate vectors in resolution conversion. In FIG. 2, F1 represents a frame of the original stream, and F2 represents a re-encoded frame which has been reduced by half in both the horizontal and vertical directions. An area on the original stream corresponding to the block b2 at the time of re-encoding is an area b1 composed of four 2 × 2 blocks.
Becomes The motion vector prediction unit integrates four corresponding motion vectors to calculate a prediction vector. As a method of integration, prediction efficiency is calculated for each candidate vector and the most efficient vector is selected.

Further, the stream transmission device and method disclosed in Japanese Patent Laid-Open No. 2000-232646, and the providing medium (hereinafter referred to as Conventional Example 2) are re-encoded using motion vectors calculated in the past. As a result, a stream transmission device that suppresses image deterioration due to re-encoding with a small amount of data has been described.

Further, the encoding device and method disclosed in Japanese Patent Laid-Open No. 2000-232647 and the providing medium (hereinafter referred to as Conventional Example 3) perform re-encoding using the motion vector calculated in the past. As a result, an encoding device that suppresses image deterioration due to re-encoding with a small amount of data has been described.

Further, in the moving picture re-encoding device disclosed in Japanese Patent Laid-Open No. 2000-244929 (hereinafter, Conventional Example 4), the base band information is obtained by decoding the encoded moving picture data. Special playback processing such as fast-forward, slow, pause, etc. is performed on the moving image signal, processing such as sub-picture or GUI overlay is performed, and information on various coding parameters extracted from the moving image coded data at the time of decoding, special The optimum encoding parameters were decided based on information such as reproduction and overlay processing, and it was possible to realize high-quality re-encoding at low cost.

Further, a coding conversion device disclosed in Japanese Patent Laid-Open No. 2000-354242 (hereinafter, conventional example 5).
Describes a coding conversion device that has coding systems with different coding efficiencies and switches the coding systems according to the transmission system when re-encoding a decoded moving image signal.

Further, in the encoding method conversion device and the motion vector detecting method thereof (hereinafter referred to as Conventional Example 6) disclosed in Japanese Patent Laid-Open No. 10-336672, the encoded image data is decoded to obtain a predetermined code. When re-encoding by the encoding method, the motion vector at the time of re-encoding is selected from the motion vectors at the time of decoding and the re-encoding process is performed.

Further, in the moving picture code string conversion apparatus and method (hereinafter referred to as Conventional Example 7) disclosed in Japanese Patent Laid-Open No. 11-275592, an incoming code string is decoded and taken out from the code string. We used motion vectors to reconstruct motion vectors of different motion compensation systems, and re-encoded using new motion vectors.

[0014]

DISCLOSURE OF THE INVENTION In the conventional example 1,
For vector integration, it was necessary to perform motion compensation processing for each candidate vector and compare prediction error powers. This processing can be realized with a small amount of calculation as compared to the case where the moving image coding unit independently performs the motion search without performing vector prediction, but there is a problem that the amount of calculation that cannot be ignored is still required. For a device that cannot allocate a sufficient amount of calculation for re-encoding processing, a method that requires a small amount of calculation and does not impair the prediction efficiency is also required for vector integration.

Further, when the encoding method at the time of re-encoding employs sub-block motion compensation in which the block to be encoded is subdivided into several sub-blocks and motion compensation processing is performed for each block, further encoding is performed. Since it is expected that the conversion efficiency can be improved, it is desirable to further include the correspondence of the motion vector prediction to the sub-block motion compensation and the appropriate switching determination between the normal motion-compensated prediction. In this case as well, since it is applied to a device that cannot allocate a sufficient amount of calculation for re-encoding processing, a method that has a low amount of calculation and does not impair the prediction efficiency is required.

Also in the conventional examples 2 to 7, the image characteristics (texture, etc.) of each block to be coded.
In consideration of the above, re-encoding is not performed, and it is difficult to realize a low operation rate and high prediction efficiency in re-encoding.

The present invention has been made in view of the above problems, and takes into consideration the image characteristics (parameters related to transform coefficients) of each block to be coded, and reduces the motion vector at the time of re-coding. An object of the present invention is to provide a moving image code string conversion device, a moving image code string conversion method, and a program for calculating the amount of calculation with high prediction efficiency.

[0018]

In order to achieve the above object, the invention according to the first aspect is such that the decoded image obtained by decoding the first moving image code string is resolution-converted and then newly encoded. A moving picture code string conversion device for generating a second moving picture code string, which is used for the complexity of coefficient codes included in the first moving picture code string and for motion compensation prediction in the first moving picture code string. And a second motion vector based on the first motion vector
Second used for motion-compensated prediction in a moving image code sequence
Is generated.

Further, the invention according to claim 2 is a moving image in which the decoded image obtained by decoding the first moving image code sequence is resolution-converted and then newly encoded to generate the second moving image code sequence. An image code sequence conversion apparatus, which decodes a first moving image code sequence using motion compensation prediction, and from the first moving image code sequence,
Coding complexity information indicating the complexity of coefficient code for each block, which is a coding unit, and the first used for motion compensation prediction.
Motion vector detecting means for detecting the motion vector of the motion vector, resolution conversion means for performing resolution conversion on the decoded image obtained by the motion image decoding means in accordance with a predetermined resolution conversion rate, and moving image decoding means. The detected first motion vector and coding complexity information, the resolution conversion rate,
Motion vector predicting means for generating a second motion vector used for motion compensated prediction in the second moving image code sequence, and referring to the second motion vector, using motion compensated prediction, And a moving picture coding means for generating a moving picture code sequence of No. 2.

According to the invention described in claim 3, in the moving picture code string conversion apparatus according to claim 2, the moving picture decoding means uses the block in the first moving picture code string as coding complexity information. It is characterized in that the code amount of each coefficient code is output.

According to the invention described in claim 4, in the moving picture code string conversion apparatus according to claim 2, the moving picture decoding means uses the block in the first moving picture code string as coding complexity information. It is characterized by outputting the number of non-zero coefficients among the transform coefficients obtained by decoding the coefficient code for each.

According to a fifth aspect of the present invention, in the moving image code string conversion apparatus according to the second aspect, the moving image decoding means uses the block in the first moving image code string as the encoding complexity information. It is characterized in that the sum of absolute values of the transform coefficients obtained by decoding the coefficient code for each is output.

Further, according to the invention of claim 6, in the moving picture code string converting apparatus according to claim 2, the motion vector predicting means sets the area before resolution conversion corresponding to each block at the time of re-encoding. A plurality of blocks included therein are selected, a motion vector corresponding to a block having the largest coding complexity indicated by the coding complexity information among the plurality of blocks is scale-converted according to the resolution conversion rate, and a second motion is performed. It is characterized by being a vector.

Further, the invention according to claim 7 is a moving image in which the decoded image obtained by decoding the first moving image code sequence is resolution-converted and then newly encoded to generate the second moving image code sequence. An image code sequence conversion apparatus, which decodes a first moving image code sequence using motion compensation prediction, and from the first moving image code sequence,
Coding complexity information indicating the complexity of coefficient code for each block, which is a coding unit, and the first used for motion compensation prediction.
Motion vector detecting means for detecting the motion vector of the motion vector, resolution conversion means for performing resolution conversion on the decoded image obtained by the motion image decoding means in accordance with a predetermined resolution conversion rate, and moving image decoding means. A coding mode determination unit that determines a coding mode indicating a type of motion compensation prediction for each block in the second moving image code string, with reference to the detected first motion vector and coding complexity information, A second motion vector used for motion-compensated prediction in the second moving image code string with reference to the first motion vector and coding complexity information detected by the moving image decoding means, and the resolution conversion rate. And a motion vector predicting means for generating a second motion image coding sequence using motion compensation prediction with reference to the second motion vector and the coding mode. , Characterized by having a.

According to the invention described in claim 8, in the moving picture code string converting apparatus according to claim 7, the moving picture decoding means uses the block in the first moving picture code string as coding complexity information. It is characterized in that the code amount of each coefficient code is output.

According to the invention described in claim 9, in the moving picture code string converting apparatus according to claim 7, the moving picture decoding means uses the block in the first moving picture code string as coding complexity information. It is characterized by outputting the number of non-zero coefficients among the transform coefficients obtained by decoding the coefficient code for each.

According to the invention described in claim 10, in the moving picture code string conversion apparatus according to claim 7, the moving picture decoding means uses the block in the first moving picture code string as coding complexity information. It is characterized in that the sum of absolute values of the transform coefficients obtained by decoding the coefficient code for each is output.

According to the eleventh aspect of the present invention, in the moving picture code string conversion apparatus according to the seventh aspect, the coding mode determining means has a normal mode for performing motion compensation prediction for the entire block and a block. Is divided into a plurality of sub-blocks, and one of the sub-block modes in which motion compensation prediction is performed separately for the plurality of sub-blocks is selected, and the moving picture coding means determines the coding mode. Motion compensation prediction is performed based on the coding mode selected by the means, and the motion vector prediction means outputs one motion vector in the normal mode, and outputs the motion vectors of a plurality of sub blocks in the sub block mode. It is characterized by doing.

According to the twelfth aspect of the present invention, in the moving picture code string conversion apparatus according to the eleventh aspect, the motion vector predicting means corresponds to each block at the time of re-encoding in the normal mode, and has a resolution. Selects multiple blocks included in the area before conversion, and scales the motion vector corresponding to the block with the highest coding complexity included in the coding complexity information among the multiple blocks according to the resolution conversion rate. , And the second motion vector.

According to the thirteenth aspect of the present invention, in the moving picture code sequence conversion apparatus according to the eleventh aspect, the motion vector predicting means corresponds to each block at the time of re-encoding in the sub-block mode, It is characterized in that a plurality of blocks included in a region before resolution conversion are selected, the motion vector of the plurality of blocks is scale-converted according to the resolution conversion rate, and then the motion vector of the sub-block is allocated at the time of re-encoding.

According to a fourteenth aspect of the present invention, in the moving image code sequence conversion apparatus according to the eleventh aspect, the encoding mode determining means corresponds to the block at the time of re-encoding and the frame before resolution conversion. If the average of the coding complexity information of each block included in the area is larger than the threshold value Ta, the sub-block mode is selected for the block at the time of re-coding.

According to a fifteenth aspect of the present invention, in the moving picture code string converting apparatus according to the eleventh aspect, the encoding mode determining means corresponds to the block at the time of re-encoding and the frame before resolution conversion. When the variance of the coding complexity information of each block included in the area is larger than the threshold value Tb, the sub-block mode is selected for the block at the time of re-coding.

According to the sixteenth aspect of the present invention, in the moving image code sequence conversion apparatus according to the eleventh aspect, the encoding mode determining means corresponds to the block at the time of re-encoding and the frame before resolution conversion. If the variance of the motion vector of each block included in the area is larger than the threshold Tc1 and is equal to or smaller than the threshold Tc2, the sub-block mode is selected for the block at the time of re-encoding.

According to a seventeenth aspect of the present invention, in the moving picture code string conversion apparatus according to the eleventh aspect, the encoding mode determining means corresponds to the block at the time of re-encoding,
When the variance of the motion vector of each block included in the area within the frame before resolution conversion is less than or equal to the threshold value Tc1 or greater than the threshold value Tc2, the normal mode is selected for the block at the time of re-encoding. And

In the eighteenth aspect of the present invention, a moving image is generated in which a decoded image obtained by decoding the first moving image code sequence is resolution-converted and then newly encoded to generate a second moving image code sequence. An image code sequence conversion method, wherein the degree of complexity of coefficient codes included in the first moving image code sequence and a first motion vector used for motion compensation prediction in the first moving image code sequence,
The second motion vector used for the motion compensation prediction in the second moving image code sequence is generated based on

The invention described in claim 19 is a moving image in which a decoded image obtained by decoding the first moving image code sequence is resolution-converted and then newly encoded to generate a second moving image code sequence. An image code sequence conversion method, which decodes a first moving image code sequence using motion compensation prediction, and represents the complexity of coefficient code for each block, which is a coding unit, from the first moving image code sequence. A moving picture decoding step of detecting coding complexity information and a first motion vector used for motion compensation prediction;
A resolution conversion step of performing resolution conversion on a decoded image obtained by the moving image decoding step according to a predetermined resolution conversion rate, and a first motion vector and coding complexity information detected by the moving image decoding step. And a resolution conversion rate, and a motion vector prediction step of generating a second motion vector used for motion compensation prediction in the second moving image code sequence, and a motion vector prediction step with reference to the second motion vector. And a moving picture coding step of generating a second moving picture code sequence using the compensation prediction.

According to the twentieth aspect of the present invention, in the moving picture code string conversion method according to the nineteenth aspect, the moving picture decoding step uses the block in the first moving picture code string as coding complexity information. It is characterized in that the code amount of each coefficient code is output.

According to the twenty-first aspect of the present invention, in the moving picture code string conversion method according to the nineteenth aspect, the moving picture decoding step uses the block in the first moving picture code string as coding complexity information. It is characterized by outputting the number of non-zero coefficients among the transform coefficients obtained by decoding the coefficient code for each.

[0039] According to the invention of claim 22, in the moving picture code string conversion method according to claim 19, the moving picture decoding step uses the block in the first moving picture code string as coding complexity information. It is characterized in that the sum of absolute values of the transform coefficients obtained by decoding the coefficient code for each is output.

According to the twenty-third aspect of the present invention, in the moving picture code string conversion method according to the nineteenth aspect, the motion vector prediction step is performed on the area before resolution conversion corresponding to each block at the time of re-encoding. Select a plurality of blocks included, the motion vector corresponding to the block with the largest coding complexity indicated by the coding complexity information among the plurality of blocks, scale conversion according to the resolution conversion rate,
It is characterized in that it is the second motion vector.

According to the invention described in claim 24, in the moving picture code string conversion method according to claim 19, the first motion vector and the coding complexity information detected in the moving picture decoding step are referred to. Then, the motion compensation prediction is performed with reference to the coding mode determination step of determining the coding mode indicating the type of motion compensation prediction for each block in the second moving image code sequence and the second motion vector and the coding mode. And a second moving picture coding step of generating a second moving picture coded sequence using.

According to a twenty-fifth aspect of the present invention, in the moving picture code string conversion method according to the twenty-fourth aspect, the coding mode determining step includes a normal mode in which motion compensation prediction is performed on the entire block, and a block. Is divided into a plurality of sub-blocks, and one of the sub-block modes in which motion compensation prediction is separately performed for the plurality of sub-blocks is selected, and the second moving image coding step is performed by Based on the coding mode selected in the coding mode determination step, motion compensation prediction is performed, and the motion vector prediction step is
It is characterized in that one motion vector is output in the normal mode, and motion vectors of a plurality of sub-blocks are output in the sub-block mode.

According to the twenty-sixth aspect of the present invention, in the moving picture code string conversion method of the twenty-fifth aspect, the motion vector prediction step corresponds to each block at the time of re-encoding in the sub-block mode, It is characterized in that a plurality of blocks included in a region before resolution conversion are selected, the motion vector of the plurality of blocks is scale-converted according to the resolution conversion rate, and then the motion vector of the sub-block is allocated at the time of re-encoding.

According to the twenty-seventh aspect of the present invention, in the moving image code sequence conversion method according to the twenty-fifth aspect, the encoding mode determination step corresponds to the block at the time of re-encoding and the frame before resolution conversion is performed. If the average of the coding complexity information of each block included in the area is larger than the threshold value Ta, the sub-block mode is selected for the block at the time of re-coding.

According to the invention described in Item 28, in the moving image code string conversion method according to Item 25, the coding mode determining step corresponds to the block at the time of re-encoding, and the frame before resolution conversion is performed. When the variance of the coding complexity information of each block included in the area is larger than the threshold value Tb, the sub-block mode is selected for the block at the time of re-coding.

According to the twenty-ninth aspect of the present invention, in the moving image code string conversion method according to the twenty-fifth aspect, the encoding mode determination step corresponds to the block at the time of re-encoding, and the frame before resolution conversion is performed. If the variance of the motion vector of each block included in the area is larger than the threshold Tc1 and is equal to or smaller than the threshold Tc2, the sub-block mode is selected for the block at the time of re-encoding.

According to the invention of claim 30, in the moving picture code string converting method of claim 25, the coding mode determining step corresponds to the block at the time of re-encoding,
When the variance of the motion vector of each block included in the area within the frame before resolution conversion is less than or equal to the threshold value Tc1 or greater than the threshold value Tc2, the normal mode is selected for the block at the time of re-encoding. And

According to a thirty-first aspect of the present invention, a program for generating a second moving image code string by newly encoding after performing resolution conversion on a decoded image obtained by decoding the first moving image code string. The second moving image is based on the complexity of the coefficient code included in the first moving image code sequence and the first motion vector used for motion compensation prediction in the first moving image code sequence. It is characterized by causing a computer to execute a process of generating a second motion vector used for motion compensation prediction in an image code sequence.

Further, the invention of claim 32 is a program for generating a second moving image code sequence by newly encoding after performing resolution conversion on a decoded image obtained by decoding the first moving image code sequence. That is, the first moving image code sequence using motion compensation prediction is decoded, and the encoding complexity information indicating the complexity of the coefficient code for each block, which is the encoding unit, is decoded from the first moving image code sequence. And a first motion vector used for motion-compensated prediction, and resolution conversion is performed on a decoded image obtained by the moving image decoding process and the moving image decoding process according to a predetermined resolution conversion rate. The resolution conversion process, the first motion vector and encoding complexity information detected by the moving image decoding process, and the resolution conversion rate are referred to and used for motion compensation prediction in the second moving image code string. Second movement A computer performs a motion vector prediction process that generates a tor and a first motion image coding process that refers to a second motion vector and uses motion compensation prediction to generate a second motion image code string. It is characterized by

According to the thirty-third aspect of the present invention, in the program of the thirty-second aspect, the moving image decoding process uses the coefficient code of each block in the first moving image code string as the encoding complexity information. It is characterized by outputting the code amount.

According to a thirty-fourth aspect of the invention, in the program of the thirty-second aspect, the moving picture decoding process uses a coefficient code for each block in the first moving picture code string as coding complexity information. It is characterized by outputting the number of non-zero coefficients among the decoded transform coefficients.

According to the thirty-fifth aspect of the invention, in the program of the thirty-second aspect, the moving picture decoding process uses a coefficient code for each block in the first moving picture code string as the coding complexity information. It is characterized in that the sum of absolute values of the decoded transform coefficients is output.

According to the thirty-sixth aspect of the present invention, in the program of the thirty-second aspect, the motion vector prediction process includes a plurality of blocks included in an area before resolution conversion corresponding to each block at the time of re-encoding. It is possible to select the motion vector corresponding to the block having the largest coding complexity indicated by the coding complexity information among the plurality of blocks and perform the scale conversion according to the resolution conversion rate to be the second motion vector. Characterize.

According to the thirty-seventh aspect of the invention, in the program of the thirty-second aspect, the second motion vector and the coding complexity information detected by the moving image decoding process are referred to and the second motion vector is referred to. By referring to a coding mode determination process that determines a coding mode that represents a type of motion compensation prediction for each block in a moving image code sequence and a second motion vector and a coding mode, motion compensation prediction is used to And a second moving image encoding process for generating two moving image code strings.

According to a thirty-eighth aspect of the present invention, in the program of the thirty-seventh aspect, the coding mode determination processing includes a normal mode in which motion compensation prediction is performed on the entire block, and the block includes a plurality of sub-blocks. Sub-block mode in which motion compensation prediction is separately performed for a plurality of sub-blocks, and one of the coding modes is selected, and the second moving image coding process is performed by the coding mode determination process. Motion-compensated prediction is performed based on the selected coding mode. Motion vector prediction processing outputs one motion vector in normal mode, and outputs motion vectors of multiple sub-blocks in sub-block mode. Is characterized by.

According to a thirty-ninth aspect of the invention, in the program of the thirty-eighth aspect, the motion vector prediction process corresponds to each block at the time of re-encoding in the sub-block mode, and the region before resolution conversion is performed. Is selected, the motion vector of the plurality of blocks is scale-converted according to the resolution conversion ratio, and then the motion vector of the sub-block is re-encoded.

According to the invention described in Item 40, in the program according to Item 38, the encoding mode determination process corresponds to the block at the time of re-encoding and is included in the area in the frame before resolution conversion. When the average of the coding complexity information of each block is larger than the threshold value Ta, the sub-block mode is selected for the block at the time of re-coding.

According to the invention described in Item 41, in the program according to Item 38, the encoding mode determination process corresponds to a block at the time of re-encoding and is included in an area in a frame before resolution conversion. When the variance of the coding complexity information of each block to be reproduced is larger than the threshold value Tb, the sub-block mode is selected for the block at the time of re-encoding.

According to the 42nd aspect of the invention, in the program according to the 38th aspect, the encoding mode determination processing corresponds to a block at the time of re-encoding and is included in an area in a frame before resolution conversion. When the variance of the motion vector of each block to be reproduced is larger than the threshold Tc1 and smaller than or equal to the threshold Tc2, the sub-block mode is selected for the block at the time of re-encoding.

According to the invention described in Item 43, in the program according to Item 38, the encoding mode determination process corresponds to the block at the time of re-encoding, and the encoding mode determination process is performed in an area in the frame before resolution conversion. The feature is that the normal mode is selected for the block at the time of re-encoding when the variance of the motion vector of each included block is less than or equal to the threshold Tc1 or greater than the threshold Tc2.

Further, according to the present invention, a moving picture decoding unit that decodes the first moving picture code sequence, a resolution conversion unit that performs resolution conversion of the decoded image according to a predetermined resolution conversion rate, and a moving picture decoding unit. Refer to the motion vector used for the motion compensation prediction obtained in the section, the resolution conversion rate, and the coding complexity information for each block, which is the coding unit in the first moving image code string, obtained in the moving image decoding unit. And a motion vector prediction unit that generates a prediction vector,
It is characterized by comprising a moving picture coding unit which performs re-coding by referring to a prediction vector. As the coding complexity information, the code amount for each block, the number of non-zero coefficients among the transform coding coefficients, or the sum of the absolute values of the transform coding coefficients is used, and the motion vector prediction unit uses each block at the time of recoding. The one having the largest coding complexity information is selected from a plurality of blocks included in the area before resolution conversion corresponding to, and the corresponding motion vector is scale-converted according to the resolution conversion rate to be a prediction vector.

Further, according to the present invention, in addition to the above, by referring to the motion vector used for the moving image decoding and the coding complexity information for each block, the motion compensation prediction for each block at the time of re-coding is performed. It is characterized by having a coding mode determination unit that determines a coding mode indicating a type. The moving picture coding unit has a normal mode in which motion compensation prediction is performed on the entire block that is a coding unit, and a subblock mode in which the block is divided into a plurality of subblocks and motion compensation prediction is performed for each subblock. To switch. The coding mode determination unit selects one of the two modes, and the motion vector prediction unit outputs one motion vector in the normal mode, and outputs the motion vector of each sub block in the sub block mode. The coding mode is determined based on coding complexity information or motion vector distribution.

[0063]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS (First Embodiment) FIG. 1 is a block diagram of a moving picture code string conversion apparatus according to a first embodiment of the present invention. FIG. 2 is a conceptual diagram illustrating the relationship between blocks and candidate vectors in resolution conversion. FIG. 3 is a diagram showing the concept of the relationship between the coding complexity and the motion compensation prediction. Hereinafter, the first embodiment of the present invention will be described with reference to FIGS. 1 to 3.

The moving picture code string converting apparatus is provided with a moving picture decoding unit 1.
00, the motion vector prediction unit 108, and the resolution conversion unit 1
09 and a moving picture decoding unit 110.

The moving picture decoding unit 100 includes a coefficient code interpretation unit 1
01, a motion vector code interpretation unit 102, an inverse quantization unit 103, an inverse transform encoding unit 104, and a motion compensation unit 105.
And a frame memory 106 and an adder 107. The moving image decoding unit 100 receives the pixel coefficient code 1001 and the motion vector code 1002 in the code string.

The coefficient code interpretation unit 101 decodes the input pixel coefficient code 1001 into transform coefficients. Inverse quantizer 1
03 inversely quantizes the pixel coefficient code 1001 decoded into the transform coefficient (for example, DCT coefficient) by the coefficient code interpretation unit 101. The inverse transform encoding unit 104 includes an inverse quantization unit 10
The pixel coefficient code 1001 inversely quantized by 3 is inversely transformed and coded.

The frame memory 106 stores a reference screen used when motion compensation is performed. The motion vector code interpretation unit 102 receives the input motion vector code 10
02 is decoded and a motion vector 1003 is generated. The motion compensation unit 105 performs motion compensation processing on the reference image stored in the frame memory 106 according to the generated motion vector 1003. The adder 107 adds the result of the inverse transform coding by the inverse transform coding unit 104 and the result of the motion compensation by the motion compensating unit 105 to obtain the decoded image 1
004 is generated.

The decoded image 1004 has a resolution conversion unit 109.
Downsamples the decoded image 1004 according to a predetermined resolution conversion rate 1007.

The coefficient code interpretation unit 101 outputs the coding complexity information 1006 indicating the complexity of the coefficient code for each block as a coding unit. Encoding complexity information 1006
As the (coding complexity), the code amount required for coefficient coding for each block, or the number of non-zero coefficients among each transform coefficient or the sum of absolute values is used. Motion vector predictor 10
8 is a motion vector 1003 and coding complexity information 100.
6, the motion vector prediction value (prediction vector 1005) in each block at the time of re-encoding is generated.

Motion vector prediction will be described with reference to FIG. In FIG. 2, a frame F1 represents an image frame in the original stream, and a frame F2 represents a re-encoded frame which has been reduced by half in both the horizontal and vertical directions. The block b2 at the time of re-encoding is 2 on the original stream.
This corresponds to a region b1 composed of four blocks of × 2. The motion vector prediction unit 108 integrates four motion vectors corresponding to each block to calculate a prediction vector 1005. The coding complexity information 1006 is used for vector integration. The four motion vectors on the area b1 correspond to the motion vector 1003, and the motion vector on the block b2 corresponds to the prediction vector 1005.

The relationship between the coding complexity information and the motion compensation prediction will be described with reference to FIG. Frames F01, F1
1 is a different image frame. Frame F0
0 is a reference frame corresponding to the frame F01,
The frame F10 is a reference frame corresponding to the frame F11. Encoding target block b0 in F01
1 and the encoding target block b11 in F11
Have different textures from each other, and the texture of the encoding target block b01 is the encoding target block b.
More complex than 11 textures. Area b00, b1
0 is a region to be referred to when motion compensation prediction is optimally performed. The area b00 ′ is the area b00
Is a region slightly shifted from the region b10 ′, and a region b10 ′ is a region slightly shifted from the region b10.

When the motion compensation prediction is not optimal in the actual encoding and the region b01 is predictively encoded by referring to the region b00 ', the prediction error in pixel units becomes E01. Similarly, when the motion-compensated prediction is not optimal in the actual coding and the region b11 is predictively coded with reference to b10 ', the prediction error in pixel units becomes E11.

In general, when the prediction errors in a plurality of regions are compared, a region having a complicated texture has a larger prediction error when the motion vector deviates from the other regions.
Here, since the texture of the region b01 is more complicated than that of the region b11, the prediction error E01 is equal to the prediction error E11.
Get bigger.

Since there is a high correlation between the texture complexity and the coding complexity, it is considered that the higher the coding complexity of a block, the more the prediction efficiency decreases as the vector at the time of motion compensation deviates from the optimum value. . The motion vector prediction unit 108 selects the largest candidate of the coding complexity information 1006 (coding complexity) from the plurality of blocks included in the region b1, and sets the corresponding motion vector to the resolution conversion rate 1007. According to the above, the prediction vector 1005 is obtained.

The moving picture coding unit 110 uses the prediction vector 1
The decoded image 1008 after resolution conversion is encoded using 005 to generate a pixel coefficient code 1009 and a motion vector code 1010.

The moving picture coding unit 110 includes a motion search unit 11
1, a frame memory 112, a motion compensation unit 113,
The subtracter (differential device) 114, the transform coding unit 115, the quantization unit 116, the variable length coding units 117 and 118, the dequantization unit 119, the inverse transform coding unit 120, and the adder 1
21 and.

The motion search unit 111 detects the decoded image 1008,
The motion vector 1011 is referred to by referring to the reference image and the prediction vector 1005 stored in the frame memory 112.
To calculate.

The frame memory 112 stores a reference screen used when motion compensation is performed. Motion compensation unit 1
13 generates a predicted image 1012 in which motion compensation is performed on the reference image from the motion vector 1011. Subtractor 114
Generates a difference image 1013 between the decoded image 1008 and the predicted image 1012. The transform coding unit 115 performs transform coding of the difference image 1013. The quantization unit 116 receives the difference image 101 that has been transform-coded by the transform-encoding unit 115.
Quantization of 3 is performed. The variable length coding unit 117 performs variable length coding on the difference image 1013 quantized by the quantizing unit 116 to generate a pixel coefficient code 1009.

The variable length coding unit 118 uses the motion vector 1
011 is variable-length coded and output as a motion vector code 1010.

The inverse quantizer 119 inversely quantizes the quantized difference image 1014. The inverse transform coding unit 120
The inverse quantization unit 119 inversely transforms the inversely quantized difference image. The adder 121 adds the prediction image 1012 to the difference image inversely transformed by the inverse transform encoding unit 120.
The frame memory 112 is an image generated by the adder 121, which is the inversely transformed difference image and the predicted image 101.
The sum of 2 is stored as the reference image.

FIG. 4 and FIG. 5 are flow charts showing the flow of the operation by the moving picture code string conversion apparatus in the first embodiment of the present invention. Hereinafter, the operation of the moving image code string conversion apparatus according to the present embodiment will be described with reference to FIG. 1 and with reference to FIGS. 4 and 5.

First, the coefficient code interpretation unit 101 decodes the input pixel coefficient code 1001 into transform coefficients (step S101). Next, the inverse quantization unit 103 inversely quantizes the decoded transform coefficient (step S102). The inverse transform encoding unit 104 inverse transform encodes the inversely quantized transform coefficient (step S103).

Further, the motion vector code interpretation unit 102
The input motion vector code 1002 is decoded to generate a motion vector 1003 (step S104). The motion compensation unit 105 performs motion compensation on the reference image stored in the frame memory 106 based on the generated motion vector 1003 (step S105).

The adder 107 adds the result of the inverse transform coding and the result of the motion compensation to generate a decoded image 1004 (step S106).

The resolution conversion unit 109 converts the resolution of the generated decoded image 1004 based on the preset resolution conversion rate 1007 (step S107).

The coefficient code interpretation unit 101 also generates coding complexity information 1006 based on the input pixel coefficient code 1001 (step S108).

The motion vector prediction unit 108 generates a prediction vector 1005 based on the motion vector 1003, the coding complexity information 1006, and the resolution conversion rate 1007 (step S109).

The motion search unit 111 uses the frame memory 11
2, the reference image, the prediction vector 1005,
And the motion vector 1011 is generated based on the decoded image 1008 after the resolution conversion (step S110).

The motion compensating section 113 calculates the motion vector 101.
1, the motion compensation of the reference image stored in the frame memory 112 is performed, and the predicted image 1012 is generated (step S111).

The subtractor 114 takes the difference between the decoded image 1008 after the resolution conversion and the predicted image 1012 to generate a difference image 1013 (step S112). The transform coding unit 115 performs transform coding of the generated difference image 1013. The quantization unit 116 uses the transform-encoded difference image 1
Quantization of 013 is performed (step S113).

Next, transitioning to FIG. 5, the variable length coding unit 1
Reference numeral 17 denotes variable-length coding of the difference image 1013 that has been transform-coded and quantized, and the difference image 1013 that has been variable-length coded after being transform-coded and quantized is converted into the pixel coefficient code 100.
It is output as 9 (step S114).

Further, the variable length coding unit 118 performs variable length coding on the motion vector 1011 generated by the motion search unit 111, and performs the variable length coding on the motion vector 1011.
Is output as the motion vector code 1010 (step S115).

Transform-coded / quantized difference image 101
3 is inversely quantized by the inverse quantization unit 119 and then inversely transformed and coded by the inverse transform coding unit 120 (step S116).

The adder 121 generates the sum of the difference image inversely transformed by the inverse transform encoder 120 and the prediction image 1012 produced by the motion compensator 113, and the produced sum is used as a reference image in the frame memory 112. (Step S117). As described above, the moving image code string conversion device is
The operation ends.

As described above, according to the present embodiment, the motion vector corresponding to the block having the largest coding complexity value among the plurality of blocks in the predetermined region before resolution conversion is scaled. , Prediction vector 1
005. Therefore, in the present embodiment, it is possible to perform re-encoding processing with high prediction efficiency and low calculation amount.

(Second Embodiment) FIG. 6 shows a second embodiment of the present invention.
It is a block diagram of the moving image code sequence conversion apparatus in the embodiment of. FIG. 7 is a diagram showing the concept of the relationship between the variance of the coding complexity and the sub-block motion compensation prediction in the second embodiment of the present invention. FIG. 8 is a diagram showing an example of motion vector allocation in sub-block motion compensation and normal motion compensation by the moving picture code string conversion apparatus according to the second embodiment of the present invention. Hereinafter, the second embodiment of the present invention will be described with reference to FIGS. 6 to 8.

Unless otherwise specified, the structure of the moving picture code string converting apparatus in this embodiment will be described below as being the same as the structure in the first embodiment of the present invention.

As shown in FIG. 6, the moving picture code string converting apparatus according to the present embodiment has a higher coding mode than the moving picture code string converting apparatus according to the first embodiment shown in FIG. It is characterized by further including a determining unit 122.

The moving picture coding unit 110 in this embodiment.
In addition to the block-based motion-compensated prediction that is normally performed (normal mode), a sub-block mode that performs motion-compensated prediction for each of a plurality of sub-blocks in a block is used.
For example, when the standard MPEG-4 image encoding method defined by ISO / IEC is used as the image encoding method performed by the moving image code sequence conversion apparatus according to the present embodiment, the moving image encoding unit 110 has a size of 16 Ordinary motion-compensated prediction performed on × 16 blocks, or size 8 × 8
Sub-block motion-compensated prediction performed on the four blocks is selectively performed.

The coding mode determining unit 122 refers to the motion vector 1003 and the coding complexity information 1006, and the coding mode 1015 representing the motion compensation prediction mode.
To generate.

The determination of the coding mode will be described in more detail with reference to FIG. As described with reference to FIG. 3 in the first embodiment, since the prediction efficiency when the motion vector is not optimal decreases as the block having a higher coding complexity, the sub-block mode in which the region is subdivided and the motion compensation prediction is performed. It is considered that the gain by using is large.

The encoding mode determining unit 122 has the sum or average of the encoding complexity information 1006 of each of a plurality of blocks included in the area b1 before resolution conversion corresponding to the block b2 at the time of re-encoding. The threshold value Ta, which is a predetermined value, is compared (the threshold value may be different between the case of sum and the case of average). Encoding complexity information 100
When the sum or average of 6 is smaller than the threshold value Ta, the coding mode determination unit 122 determines the motion compensation prediction of the block b2 as the normal mode. Also, the encoding complexity information 10
When the sum or average of 06 is larger than the threshold value Ta, the coding mode determination unit 122 determines the motion compensation prediction of the block b2 as the sub-block mode.

As another determination index, the coding complexity information 100 for a plurality of blocks included in the area b1 is used.
A variance of 6 is used. Hereinafter, the relationship between the variance of the coding complexity information and the motion compensation prediction will be described with reference to FIG. 7.

As shown in FIG. 7, the image frame F2
The object C2 inside is moving in the direction of the vector V1, whereas the background area (the area excluding the object C2 in the frame F2) is moving in the direction of the vector V2. The block b2 is a sub-block sb21, sb22, sb.
23 and sb24.

When coding the block b2 on the boundary of the object C2, the right sub-block (sub-blocks sb23, sb24) having a large common area with the object C2,
An area on the left side (sub-block s) completely included in the background area
Prediction efficiency is higher if motion compensation is separately performed for b21 and sb22). Coding efficiency is greatly improved for blocks having different coefficients in subblocks by performing motion compensation prediction for each region having a different coefficient distribution.

The coding mode determining unit 122 determines that the block b2 is set when the variance of the coding complexity information 1006 in the area b1 is larger than a threshold value Tb which is a predetermined value.
The motion compensation prediction of is set to the sub-block mode. In addition, the coding mode determination unit 122 sets the motion compensation prediction of the block b2 to the normal mode when the variance of the coding complexity information 1006 in the region b1 is smaller than the threshold Tb.

As another determination index, motion vectors 1003 corresponding to a plurality of blocks included in the area b1.
Dispersions may be used. When the variance of the motion vector 1003 corresponding to the plurality of blocks included in the region b1 is larger than the first threshold value Tc1 which is a predetermined value,
The coding mode determination unit 122 sets the motion-compensated prediction of the block b2 to the sub-block mode.

In general, in the sub-block mode, motion compensation is performed on a subdivided area, so that the prediction efficiency is improved as compared with normal motion compensation, leading to an improvement in image quality. On the other hand, since the number of motion vectors to be coded increases, if the code amount assigned to the motion vector significantly increases, the coding efficiency rather decreases. When the variance of the motion vector 1003 corresponding to a plurality of blocks included in the region b1 is larger than the second threshold value Tc2 which is a predetermined value, the coding mode determination unit 122 performs vector coding. Since it is considered that the overall coding efficiency is lowered, the motion-compensated prediction of the block b2 is performed in a normal block unit.

As described above, the motion vector 100
When the variance of 3 is larger than the first threshold Tc1 and is equal to or smaller than the second threshold Tc2, the sub-block mode is selected as the encoding mode. If the variance of the motion vector 1003 is less than or equal to the first threshold Tc1 or greater than the second threshold Tc2, the normal mode is selected as the encoding mode.

When the coding mode refers to a normal motion compensation prediction (normal mode), the motion vector predicting unit 123 calculates coding complexity information 1006 from the candidate vector as in the vector prediction in the first embodiment of the present invention. To generate a prediction vector 1005.

When the coding mode refers to the sub-block mode, the motion vector 1003 is appropriately assigned as the motion vector of the sub-block, and the resolution conversion rate 10
The vector scaled according to 07 is the prediction vector 100
Generate as 5.

FIG. 8 shows normal motion-compensated prediction (normal mode) and motion-compensated prediction by four sub-blocks (sub-block mode) in an image downsampled to half in both the horizontal and vertical directions in this embodiment. 3 is a diagram showing vector allocation in FIG.

The frame F1 is an image frame in the original stream. The area b1 indicates a predetermined area on the frame F1 and is made up of a plurality of (2 × 2, four in this embodiment) blocks. The motion vector is calculated for each of the plurality of blocks included in the region b1. Also,
The frame F2 is a frame re-encoded in the normal mode, and the block b2 is a block on the frame F2 corresponding to the area b1 on the frame F1. Further, the frame F2 ′ is a frame re-encoded in the sub-block mode, and the block b2 ′ is a block on the frame F2 ′ corresponding to the area b1. The four motion vectors on the area b1 correspond to the motion vector 1003, and the motion vector on the block b2 or the block b2 ′ corresponds to the prediction vector 1005.

When motion-compensated prediction is performed for each sub-block for the block b2 'in the resolution F2' of the image F1 (sub-block mode), motion vectors corresponding to a plurality of blocks included in the region b1 are calculated. Are each scaled and assigned to sub-blocks. Further, when the normal motion compensation prediction is performed on the block b2 (normal mode), it is appropriate from the motion vectors corresponding to the plurality of blocks included in the region b1 by the same method as that shown in the first embodiment. After selecting a proper vector, scaling is performed according to the resolution conversion rate.

The motion search unit 124 uses the coding mode 101.
5, the motion vector 1011 is generated by referring to the prediction vector 1005. The motion compensation unit 125 also performs motion compensation prediction based on the coding mode 1015 and the motion vector 1011.

As described above, in this embodiment, the sum or average of the coding complexity information 1006, the variance of the coding complexity information 1006, and the motion vector 1 are used as the criterion for selecting the coding mode.
A dispersion of 003 was used. As the criterion, any one of the above items may be used,
The combination (including all of) may be used.

Although one block in this embodiment is composed of four sub-blocks, the number of sub-blocks constituting the block may have other values.

According to this embodiment, when the region to be re-encoded is composed of a plurality of blocks and the sum or average of the encoding complexity information 1011 in the plurality of blocks is larger than the threshold value Ta, the re-encoding is performed. Frame of time F2
Motion-compensated prediction is performed for each sub-block in. Therefore, in the present embodiment, it is possible to improve the coding efficiency in the re-coding process.

Further, according to the present embodiment, when the area to be re-encoded is composed of a plurality of blocks and the variance of the encoding complexity information 1011 in the plurality of blocks is larger than the threshold Tb, the re-encoding is performed. Motion-compensated prediction is performed for each sub-block in the frame F2 at that time. Therefore,
In this embodiment, it is possible to improve the coding efficiency in the re-coding process.

Further, according to the present embodiment, when the region to be re-encoded is composed of a plurality of blocks and the variance of the motion vector in the plurality of blocks is larger than the threshold value Tc1 and not more than the threshold value Tc2, the re-encoding is performed. Motion-compensated prediction is performed for each sub-block in the frame F2 at the time of conversion.
Therefore, in the present embodiment, it is possible to improve the coding efficiency in the re-coding process.

Also, the moving picture code string conversion apparatus decodes the first moving picture code string using the motion compensation prediction, and the coefficient code for each block, which is a coding unit, is complicated from the first moving picture code string. Of the encoded complexity information indicating the degree and the first motion vector used for the motion compensation prediction, the process of performing resolution conversion on the decoded image according to a predetermined resolution conversion rate, and the detected process. A process of generating a second motion vector used for motion compensation prediction in a second moving image code string by referring to the first motion vector, coding complexity information, and resolution conversion rate; and a second motion vector And a process of generating a second moving image code sequence using motion compensation prediction, and a block in the second moving image code sequence by referring to the detected first motion vector and coding complexity information. Movement of each A process of determining a coding mode representing the type of compensation prediction and a process of referring to the second motion vector and the coding mode to generate a second moving image code string using motion compensation prediction are performed. . The above-mentioned processing is executed by a computer program included in the moving picture code string conversion apparatus. However, the above-mentioned program is recorded in a recording medium such as an optical disk or a magnetic disk and may be loaded from the above-mentioned recording medium. Alternatively, it may be loaded from an external device connected via a predetermined network.

[0122]

According to the present invention, vector integration at the time of re-encoding is performed by comparing the complexity of each block obtained from coefficient codes. Therefore, the integration process used in the present invention requires almost no calculation amount, and by considering the coefficient distribution of the block, the motion compensation process can be performed without significantly impairing the prediction efficiency.

Further, according to the present invention, in the conversion to the coding method having the sub-block motion compensation as one of the functions, the complexity and the motion vector distribution for each sub-block obtained from the coefficient code of the original stream are calculated. It is used to switch appropriately to the normal motion compensation prediction. Therefore, the coding efficiency can be further improved without increasing the calculation amount.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a moving picture code string conversion apparatus in a first embodiment of the present invention.

[Fig. 2] Fig. 2 is a diagram illustrating a relationship between an original stream associated with resolution conversion and a motion vector upon re-encoding.

FIG. 3 is a conceptual diagram illustrating a relationship between coding complexity and motion compensation prediction according to the first embodiment of the present invention.

FIG. 4 is a flowchart showing a flow of operations performed by the moving picture code string conversion apparatus in the first embodiment of the present invention.

FIG. 5 is a flowchart showing a flow of operations performed by the moving image code string conversion apparatus in the first embodiment of the present invention.

FIG. 6 is a block diagram showing a configuration of a moving picture code string conversion device according to a second embodiment of the present invention.

[Fig. 7] Fig. 7 is a conceptual diagram illustrating a relationship between variance of coding complexity and sub-block motion compensation prediction according to the second embodiment of the present invention.

[Fig. 8] Fig. 8 is a diagram illustrating an example of motion vector allocation in sub-block motion compensation and normal motion compensation according to the second embodiment of the present invention.

FIG. 9 is a block diagram showing a configuration of a moving image code string conversion apparatus according to a conventional technique.

[Explanation of symbols]

100 Video decoding unit 101 coefficient code interpretation unit 102 motion vector code interpretation unit 103, 119 inverse quantizer 104, 120 inverse transform coding unit 105, 113, 125 Motion compensation unit 106, 112 frame memory 107, 121 adder 108, 123 Motion vector predictor 109 resolution converter 110 Video Coding Unit 111, 124 motion search unit 114 subtractor 115 transform coding unit 116 quantizer 117, 118 Variable length coding unit 122 Encoding mode determination unit 1001, 1009 Pixel coefficient code 1002, 1010 motion vector code 1003, 1011 motion vector 1004, 1008 Decoded image 1005 prediction vector 1006 encoding complexity information 1007 Resolution conversion rate 1012 prediction image 1013 Difference image 1014 coefficient data 1015 encoding mode F1 and F2 frames b1 area b2 coded block sb21 to sb24 sub blocks Ta, Tb, Tc1, Tc2 threshold

   ─────────────────────────────────────────────────── ─── Continued front page    F term (reference) 5C059 KK11 KK41 MA00 MA23 MC11                       MC38 ME01 NN01 NN21 NN28                       PP04 RC16 TA61 TB08 TC04                       TC10 TC18 TD12 UA02 UA05                       UA33                 5J064 AA02 BA09 BB03 BC01 BC02                       BC08 BC16 BD01

Claims (43)

[Claims] 1. A moving picture code string conversion apparatus for generating a second moving picture code string by newly encoding after performing resolution conversion on a decoded image obtained by decoding a first moving picture code string. The second motion vector based on the complexity of the coefficient code included in the first motion picture code sequence and the first motion vector used for motion compensation prediction in the first motion picture code sequence.
Second used for motion-compensated prediction in a moving image code sequence
A moving picture code string conversion apparatus, characterized in that it generates a motion vector of 2. A moving picture code string conversion apparatus for generating a second moving picture code string by newly encoding the decoded image obtained by decoding the first moving picture code string after resolution conversion. , Decoding the first moving image code sequence using motion compensation prediction, and encoding complexity information indicating the degree of complexity of a coefficient code for each block, which is an encoding unit, from the first moving image code sequence, , A first motion vector used for motion compensated prediction,
A moving image decoding means for detecting the resolution, a resolution converting means for performing resolution conversion on the decoded image obtained by the moving image decoding means in accordance with a predetermined resolution conversion rate, and a moving image decoding means for detecting Motion vector prediction that refers to the first motion vector and coding complexity information, and the resolution conversion rate to generate a second motion vector used for motion compensation prediction in the second moving image code string And a moving picture coding means for generating the second moving picture code string by using the motion compensation prediction with reference to the second motion vector. Converter. 3. The moving picture decoding means outputs, as the coding complexity information, a code amount of a coefficient code for each block in the first moving picture code string. Video code string converter. 4. The moving picture decoding means outputs, as the coding complexity information, the number of non-zero coefficients among the transform coefficients obtained by decoding the coefficient code for each block in the first moving picture code string. The moving picture code string conversion apparatus according to claim 2, wherein 5. The moving image decoding means outputs, as the encoding complexity information, an absolute value sum of transform coefficients obtained by decoding the coefficient code of each block in the first moving image code string. The moving picture code string conversion apparatus according to claim 2. 6. The motion vector predicting means selects a plurality of blocks included in a region before resolution conversion corresponding to each of the blocks at the time of re-encoding, and indicates the plurality of blocks by the encoding complexity information. The motion vector corresponding to the block with the largest coding complexity
The moving picture code sequence conversion apparatus according to claim 2, wherein scale conversion is performed according to the resolution conversion rate to obtain the second motion vector. 7. A moving picture code string conversion device for generating a second moving picture code string by newly encoding the decoded image obtained by decoding the first moving picture code string after resolution conversion. , Decoding the first moving image code sequence using motion compensation prediction, and encoding complexity information indicating the degree of complexity of a coefficient code for each block, which is an encoding unit, from the first moving image code sequence, , A first motion vector used for motion compensated prediction,
And a resolution conversion unit that performs resolution conversion on the decoded image obtained by the moving image decoding unit according to a predetermined resolution conversion rate, and a resolution conversion unit that detects the moving image decoding unit. A coding mode determining unit that determines a coding mode indicating a type of motion compensation prediction for each block in the second moving image code string, with reference to a first motion vector and coding complexity information; By referring to the first motion vector and coding complexity information detected by the moving picture decoding means, and the resolution conversion rate, the second motion vector predictor used in the motion compensation prediction in the second moving picture code string is referred to. The motion vector predicting means for generating a motion vector, the second motion vector and the encoding mode are referred to, and the second motion picture code sequence is generated by using the motion compensation prediction. And a moving picture coding means for generating the moving picture coding sequence. 8. The moving image decoding means outputs, as the encoding complexity information, a code amount of a coefficient code for each block in the first moving image code string. Video code string converter. 9. The moving picture decoding means outputs, as the coding complexity information, the number of non-zero coefficients among the transform coefficients obtained by decoding the coefficient code for each block in the first moving picture code string. The moving picture code string conversion apparatus according to claim 7, wherein: 10. The moving image decoding means outputs, as the encoding complexity information, an absolute value sum of transform coefficients obtained by decoding the coefficient code for each block in the first moving image code string. The moving picture code string conversion apparatus according to claim 7. 11. The encoding mode determination means divides the block into a normal mode in which the motion compensation prediction is performed on the entire block and a plurality of sub-blocks, and separately performs the motion compensation on the plurality of sub-blocks. A sub-block mode for performing prediction, or any one of coding modes is selected, and the moving picture coding means is based on the coding mode selected by the coding mode determining means, and the motion compensation prediction is performed. 8. The motion vector predicting means outputs one motion vector in the normal mode, and outputs the motion vectors of the plurality of sub blocks in the sub block mode, respectively. Video code sequence converter. 12. The motion vector predicting means selects a plurality of blocks included in a region before resolution conversion corresponding to each block at the time of re-encoding in the normal mode, and performs the encoding among the plurality of blocks. The motion vector corresponding to the block having the largest coding complexity included in the complexity information is scale-converted according to the resolution conversion rate to be the second motion vector. Video code string converter. 13. The motion vector predicting means selects a plurality of blocks included in a region before resolution conversion corresponding to each block at the time of re-encoding in the sub-block mode, and a motion vector possessed by the plurality of blocks. 12. The moving picture code sequence conversion apparatus according to claim 11, wherein after performing scale conversion according to the resolution conversion rate, is assigned as a motion vector of the sub-block at the time of re-encoding. 14. The encoding mode determination means corresponds to a block at the time of re-encoding, and an average of the encoding complexity information of each block included in an area in a frame before resolution conversion is larger than a threshold value Ta. In this case, the moving picture code string conversion apparatus according to claim 11, wherein the sub-block mode is selected for the block at the time of re-encoding. 15. The encoding mode determining means corresponds to a block at the time of re-encoding, and a variance of the encoding complexity information of each block included in an area in a frame before resolution conversion is larger than a threshold Tb. In this case, the moving picture code string conversion apparatus according to claim 11, wherein the sub-block mode is selected for the block at the time of re-encoding. 16. The encoding mode determining means corresponds to a block at the time of re-encoding, and a motion vector of each block included in an area in a frame before resolution conversion has a variance larger than a threshold value Tc1 and a threshold value Tc2. 12. The moving picture code sequence conversion apparatus according to claim 11, wherein the sub-block mode is selected for the block at the time of re-encoding in the following cases. 17. The encoding mode determination means corresponds to a block at the time of re-encoding, and a variance of motion vectors of each block included in an area in a frame before resolution conversion is equal to or less than a threshold value Tc1, or The moving picture code sequence conversion apparatus according to claim 11, wherein the normal mode is selected for the block at the time of re-encoding when the value is larger than the threshold value Tc2. 18. A decoded image obtained by decoding the first moving image code sequence is resolution-converted and then newly encoded to obtain a second moving image.
A moving picture code string conversion method for generating a moving picture code string, the method being used for the complexity of coefficient codes included in the first moving picture code string and motion compensated prediction in the first moving picture code string. And a second motion vector based on the second motion vector
Second used for motion-compensated prediction in a moving image code sequence
A method for converting a moving image code string, which comprises generating a motion vector of 19. A decoded image obtained by decoding a first moving image code sequence is resolution-converted and then newly encoded to obtain a second image.
Is a moving picture code string conversion method for generating a moving picture code string, wherein the first moving picture code string is decoded using motion compensation prediction, and the first moving picture code string is converted into a coding unit. Encoding complexity information indicating the complexity of coefficient code for each block, and a first motion vector used for motion compensation prediction,
And a resolution conversion step of performing resolution conversion on the decoded image obtained by the moving picture decoding step according to a predetermined resolution conversion rate, and a moving picture decoding step detected by the moving picture decoding step. Motion vector prediction that refers to the first motion vector and coding complexity information, and the resolution conversion rate to generate a second motion vector used for motion compensation prediction in the second moving image code string And a first moving picture coding step of generating the second moving picture code sequence by using the motion compensated prediction with reference to the second motion vector. Image code string conversion method. 20. The moving image decoding step outputs, as the encoding complexity information, a code amount of a coefficient code for each block in the first moving image code string. Video code string conversion method. 21. The moving image decoding step outputs, as the encoding complexity information, the number of non-zero coefficients among the transform coefficients obtained by decoding the coefficient code for each block in the first moving image code string. 20. The moving picture code string converting method according to claim 19, wherein 22. The moving image decoding step outputs, as the encoding complexity information, an absolute value sum of transform coefficients obtained by decoding the coefficient code for each block in the first moving image code string. 20. The moving picture code string conversion method according to claim 19. 23. The motion vector predicting step selects a plurality of blocks included in an area before resolution conversion corresponding to each of the blocks at the time of re-encoding, and indicates by the encoding complexity information among the plurality of blocks. The motion vector corresponding to the block with the largest coding complexity
20. The moving image code sequence conversion method according to claim 19, wherein scale conversion is performed according to the resolution conversion rate to obtain the second motion vector. 24. A type of motion-compensated prediction for each block in the second moving image code string is represented by referring to the first motion vector and coding complexity information detected in the moving image decoding step. A coding mode determining step of deciding a coding mode; and a second moving image code sequence generation step using the motion compensation prediction with reference to the second motion vector and the coding mode. 20. The moving picture code string conversion method according to claim 19, further comprising: 25. The encoding mode determining step divides the block into a normal mode in which the motion compensation prediction is performed on the entire block and a plurality of sub blocks, and the motion compensation is performed separately for the plurality of sub blocks. A sub-block mode for performing prediction and one of the coding modes is selected, and the second moving image coding step is based on the coding mode selected by the coding mode determination step, A motion-compensated prediction is performed, and the motion vector prediction step outputs one motion vector in the normal mode and outputs motion vectors of the plurality of sub-blocks in the sub-block mode. 24. The moving image code string conversion method described in 24. 26. The motion vector predicting step selects, in the sub-block mode, a plurality of blocks corresponding to each block at the time of re-encoding and included in an area before resolution conversion, and a motion vector of the plurality of blocks is selected. 26. The moving image code sequence conversion method according to claim 25, wherein after performing scale conversion according to the resolution conversion rate, is assigned as a motion vector of the sub-block at the time of re-encoding. 27. The encoding mode determining step corresponds to a block at the time of re-encoding, and an average of the encoding complexity information of each block included in an area in a frame before resolution conversion is larger than a threshold Ta. 26. In the case, the sub-block mode is selected for the block at the time of re-encoding, the moving picture code string conversion method according to claim 25. 28. The encoding mode determining step corresponds to a block at the time of re-encoding, and a variance of the encoding complexity information of each block included in an area in a frame before resolution conversion is larger than a threshold Tb. 26. In the case, the sub-block mode is selected for the block at the time of re-encoding, the moving picture code string conversion method according to claim 25. 29. The encoding mode determining step corresponds to a block at the time of re-encoding, and a motion vector of each block included in an area in a frame before resolution conversion has a variance larger than a threshold Tc1 and a threshold Tc2. 26. The moving picture code string conversion method according to claim 25, wherein the sub-block mode is selected for the block at the time of re-encoding when the following cases are true. 30. The encoding mode determining step corresponds to a block at the time of re-encoding, and a motion vector variance of each block included in an area in a frame before resolution conversion is equal to or less than a threshold value Tc1, or 26. The moving picture code string conversion method according to claim 25, wherein the normal mode is selected for a block at the time of re-encoding when the value is larger than a threshold value Tc2. 31. A decoded image obtained by decoding a first moving image code sequence is resolution-converted and then newly encoded to obtain a second moving image.
And a complexity of coefficient codes included in the first moving image code sequence, and a first motion used for motion compensation prediction in the first moving image code sequence. The second based on the vector and
Second used for motion-compensated prediction in a moving image code sequence
A program for causing a computer to execute the process of generating the motion vector of. 32. The decoded image obtained by decoding the first moving image code sequence is resolution-converted and then newly encoded to generate the second moving image.
Which is a program for generating a moving picture code string of the following, decoding the first moving picture code string using motion compensated prediction, and coefficient for each block which is a coding unit from the first moving picture code string. Coding complexity information indicating the complexity of a code, a first motion vector used for motion compensation prediction,
And a resolution conversion process for performing resolution conversion on the decoded image obtained by the moving image decoding process according to a predetermined resolution conversion rate, and a moving image decoding process detected by the moving image decoding process. Motion vector prediction that refers to the first motion vector and coding complexity information, and the resolution conversion rate to generate a second motion vector used for motion compensation prediction in the second moving image code string A program for causing a computer to execute a process, a first moving image coding process of referring to the second motion vector, and using the motion compensated prediction to generate the second moving image code sequence. . 33. The moving image decoding process outputs the code amount of a coefficient code for each block in the first moving image code string as the encoding complexity information. Program of. 34. The moving image decoding process outputs, as the encoding complexity information, the number of non-zero coefficients among the transform coefficients obtained by decoding the coefficient code for each block in the first moving image code string. 33. The program according to claim 32, wherein 35. The moving image decoding process outputs, as the encoding complexity information, an absolute value sum of transform coefficients obtained by decoding the coefficient code of each block in the first moving image code string. The program according to claim 32. 36. The motion vector prediction process selects a plurality of blocks included in an area before resolution conversion corresponding to each of the blocks at the time of re-encoding, and indicates the plurality of blocks by the encoding complexity information. The motion vector corresponding to the block with the largest coding complexity
33. The program according to claim 32, wherein scale conversion is performed in accordance with the resolution conversion rate to obtain the second motion vector. 37. A type of motion-compensated prediction for each block in the second moving image code string is represented by referring to a first motion vector and coding complexity information detected by the moving image decoding process. A coding mode determination process for determining a coding mode, and a second motion vector coding sequence using the motion compensation prediction with reference to the second motion vector and the coding mode. 33. The program according to claim 32, for causing a computer to execute the moving image encoding process of. 38. The encoding mode determination process divides the block into a normal mode in which the motion compensation prediction is performed on the entire block and a plurality of sub blocks, and the motion compensation is performed separately for the plurality of sub blocks. A sub-block mode for performing prediction, and one of the coding modes is selected, and the second moving image coding process is based on the coding mode selected by the coding mode determination process, and A motion-compensated prediction is performed, and the motion vector prediction process outputs one motion vector in the normal mode and outputs motion vectors of the plurality of sub-blocks in the sub-block mode. 37. The program according to 37. 39. In the motion vector prediction process, in the sub-block mode, a plurality of blocks corresponding to each block at the time of re-encoding and included in an area before resolution conversion are selected, and motion vectors of the plurality of blocks are selected. 39. The program according to claim 38, wherein after being scale-converted in accordance with the resolution conversion rate, it is allocated as a motion vector of the sub-block at the time of re-encoding. 40. The encoding mode determination process corresponds to a block at the time of re-encoding, and an average of the encoding complexity information of each block included in a region in a frame before resolution conversion is larger than a threshold value Ta. 39. The program according to claim 38, wherein in the case, the sub-block mode is selected for the block at the time of re-encoding. 41. The encoding mode determination process corresponds to a block at the time of re-encoding, and a variance of the encoding complexity information of each block included in an area in a frame before resolution conversion is larger than a threshold Tb. 39. The program according to claim 38, wherein in the case, the sub-block mode is selected for the block at the time of re-encoding. 42. The encoding mode determination process corresponds to a block at the time of re-encoding, and a motion vector of each block included in an area in a frame before resolution conversion has a variance larger than a threshold value Tc1 and a threshold value Tc2. 39. The program according to claim 38, wherein the sub-block mode is selected for the block at the time of re-encoding if: 43. The encoding mode determination process corresponds to a block at the time of re-encoding, and a variance of motion vectors of each block included in a region in a frame before resolution conversion is equal to or less than a threshold value Tc1, or 39. The program according to claim 38, wherein the normal mode is selected for the block at the time of re-encoding when it is larger than the threshold value Tc2.