JP4164903B2 - Video code string conversion apparatus and method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to code string conversion in high-efficiency coding for converting image information into a digital signal with a smaller code amount in order to efficiently transmit, store, and display an image. In particular, the present invention relates to a case where the coding method is inter-picture predictive coding in which motion compensation is performed in units of blocks, and conversion into images having different numbers of pixels.
[0002]
[Prior art]
<Conversion of video code sequence>
It is necessary to convert a code string encoded by moving image high-efficiency encoding represented by MPEG or the like to a different data rate or convert a variable transfer rate to a fixed transfer rate. In this case, in principle, the image is completely decoded and re-encoded at a different rate. However, if the basic encoding process is the same, a part of the information can be used as it is.
Specifically, the motion vector (MV) information is used as it is in re-encoding, and the motion vector detection that requires many operations can be omitted. In addition, since the motion compensation inter-picture prediction process does not change, deterioration due to re-encoding is only a difference in quantization and can be minimized. Such processing is described in 1993 Image Encoding Symposium Proceedings 1-6 “Examination of encoding control in image re-encoding” and H.Sun, W. Kwok and J. Zdepski, “Architectures for MPEG Compressed Bitstream Scaling” (IEEE Transaction on Circuit and System for Video Technology, Vol. 6, No. 2, April 1996).
[0003]
<Conventional Video Code Sequence Converter>
FIG. 5 shows an example of the configuration of a conventional moving image code string converter.
The code sequence of the motion compensated inter-picture predictive encoding that comes from the code sequence input terminal 1 is returned to a fixed length code by the variable length decoder 2 in which the code sequence of the prediction residual and the MV code sequence are returned. A DCT (discrete cosine transform) coefficient obtained as a fixed length code becomes a coefficient value in the inverse quantizer 3 and is given to the inverse DCT 4. The inverse DCT 4 converts 8 × 8 coefficients into a reproduction prediction residual signal and supplies it to the adder 5. The adder 5 adds a prediction signal, which will be described later, to the reproduction prediction residual signal to form a reproduction image. On the other hand, the MV information output from the variable length decoder 2 is given to the motion compensation predictors 7 and 14.
[0004]
The reproduced image signal obtained in this way is supplied to the image memory 8 and the prediction subtracter 9. The motion compensation predictor 7 performs motion compensation on the image stored in the image memory 8 based on the MV, and forms a prediction signal. The obtained prediction signal is given to the adder 5. Next, the re-encoding system will be described. The reproduced image signal obtained from the adder 5 is subtracted from the prediction signal given from the motion compensation predictor 14 in the subtracter 9 and is given to the DCT 10 as a prediction residual.
[0005]
The DCT 10 performs DCT conversion processing and supplies the obtained coefficient to the quantizer 11. The quantizer 11 quantizes the coefficient with a predetermined step width, and supplies the coefficient that has become a fixed-length code to the variable-length encoder 12 and the inverse quantizer 18. The quantization step width is different from the quantization step width of the inverse quantizer 3 corresponding to the transfer rate change. The variable-length encoder 12 compresses the fixed-length prediction residual with the variable-length code, further variably encodes the MV, and outputs the resulting code string from the code string output terminal 13.
[0006]
On the other hand, the inverse quantizer 18 and the inverse DCT 17 perform the inverse processing of the DCT 10 and the quantizer 11 to reproduce the inter-picture prediction residual. The obtained inter-reproduced image prediction residual is added to the inter-image prediction signal by the adder 16 to form a regenerated image, which is given to the image memory 15. The reproduced image stored in the image memory 15 is given to the motion compensation predictor 14. The motion compensated predictor 14 creates an inter-picture prediction signal according to the MV given from the variable length decoder 2, and provides it to the subtracter 9 and the adder 16.
Here, since the motion compensation inter-picture prediction process is the same in the decoding unit and the encoding unit, it seems that the adder 5 and the subtracter 9 are canceled out and only the intra-image processing is performed. Furthermore, since DCT4 is a reversible conversion process for inverse DCT11, it seems to cancel and only requantization is performed.
[0007]
However, the reproduced image stored in the decoding-system image memory 8 and the reproduced image stored in the re-encoding-system image memory 15 are images having different quantization errors because of different quantization processes. The prediction signal will be slightly different. Therefore, if the inter-image prediction process is omitted, a large error does not occur in one prediction process, but errors accumulate in the cyclic prediction process, resulting in a large shift. That is, if the prediction process or the like is omitted, the image quality is greatly deteriorated.
[0008]
[Problems to be solved by the invention]
The conventional moving image code string conversion apparatus is compatible only with images having the same number of pixels before and after conversion. In the case of the same number of pixels, the motion vector can be used as it is. However, if the number of pixels before and after conversion is different, the boundary of the motion compensation block changes, so that the motion vector cannot be used as it is. In conversion involving conversion of the number of pixels, it is necessary to perform motion vector detection again, and the amount of processing and image quality degradation have been problems.
The present invention has been made paying attention to the above points. In addition to decoding an incoming code string, a motion vector corresponding to an image having a different number of pixels is reconstructed using a motion vector extracted from the code string. Another object of the present invention is to provide a moving image code string conversion device that re-encodes a decoded image in which the number of pixels is converted using a new motion vector, thereby reducing processing amount and image quality degradation.
[0009]
[Means for Solving the Problems]
The present invention relates to the conversion of a motion image code sequence predicted between motion compensated images, receives a first motion image code sequence, separates a first motion vector from the code sequence, and outputs the first motion image code sequence. Is subjected to inter-picture predictive decoding using the first motion vector corresponding to the first motion compensation block to obtain a first decoded image signal, the number of pixels is converted, and the number of pixels having a different number of pixels is converted. 2 decoded image signals are obtained. On the other hand, the second motion vector for re-encoding the second decoded image with the second motion compensation block is the first motion compensation existing at the center position of the second motion compensation block on the actual image. obtained by specifying the block, scales in accordance with the first motion vector for the first motion compensation block to the conversion ratio of the number of pixels in the pixel number conversion means, therewith, the second The decoded image signal is inter-picture prediction encoded by the second motion compensation block to obtain a code sequence, and is multiplexed with the information of the second motion vector to obtain a second video code sequence conversion apparatus And its method.
[0011]
(Work)
In the present invention, all or a part of the incoming code sequence is decoded, and a MV corresponding to a motion compensation block of an image having a different number of pixels is reconstructed using a motion vector (MV) extracted from the code sequence. The decoded image whose number of pixels is converted is re-encoded using the MV.
An image to be re-encoded can optimize the balance between resolution and quantization error with respect to the re-encoding transfer rate by converting the number of pixels. Since the MV obtained by the reconstruction corresponds to the motion compensation block in the image in which the number of pixels is converted, the motion compensation error in the re-encoding is minimized. The MV reconstruction is a slight process compared to the case where motion vector detection is performed. Since the motion vector is similar to the incoming one, the change in the inter-picture prediction residual is also minimal, and the image quality deterioration is reduced.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
<Execution video code string converter>
An embodiment of the moving picture code string converter according to the present invention will be described below.
FIG. 1 shows the configuration, and the same reference numerals are given to the same components as those of the conventional example of FIG. In FIG. 1, an MV reconstructor 6 and a pixel number converter 19 are added as compared with FIG. The internal configuration of the MV reconstructor 6 is shown in FIGS. What is different from the conventional example in the embodiment is the number of pixels of the re-encoded image and the motion vector (MV) associated therewith. The processing operation of each processing part such as DCT and quantization is basically the same as the conventional example.
[0013]
The decoding system will be described. The code coming from the code string input terminal 1 is converted into a fixed-length code by the variable length decoder 2 from the code string of the prediction residual and the first MV code string corresponding to the motion compensation block of the code string. The DCT coefficient obtained as a fixed length code is converted into a coefficient value by the inverse quantizer 3 and is given to the inverse DCT 4. The inverse DCT 4 converts 8 × 8 coefficients into a reproduction prediction residual signal and supplies it to the adder 5. The adder 5 adds a prediction signal obtained from the motion compensation predictor 7 described below to the reproduction prediction residual signal, and a reproduction image is obtained. On the other hand, the first MV taken out from the variable length decoder 2 is supplied to the motion compensation predictor 7 and the MV reconstructor 6.
[0014]
The reproduced image signal obtained in this way is supplied to the image memory 8 and the pixel number converter 19. The pixel number converter 19 resamples the reproduced image in the horizontal direction to obtain an image in which the number of pixels is converted. Details of the resample processing will be described later.
The resampled reproduced image signal is supplied to the prediction subtracter 9 and the MV reconstructor 6. The MV reconstructor 6 uses the first MV obtained by the variable length decoder 2 to reconstruct a second MV corresponding to an image having a different number of pixels necessary for re-encoding. The MV reconstruction method will be described later.
[0015]
Next, the re-encoding system will be described. The reproduced image signal whose number of pixels has been converted by the number of pixels converter 19 is subtracted from the prediction signal given from the motion compensated predictor 14 in the subtracter 9 and given to the DCT 10 as a prediction residual.
The DCT 10 performs a DCT conversion process, and gives the obtained coefficient to the quantizer 11. The quantizer 11 quantizes the coefficient with a predetermined step width, and supplies the coefficient that has become a fixed-length code to the variable-length encoder 12 and the inverse quantizer 18.
The variable length encoder 12 converts the prediction residual into a code compressed with the variable length code, the second MV is also variable length encoded, and a code string formed by multiplexing both is output from the code output terminal 13.
[0016]
On the other hand, the inverse quantizer 18 and the inverse DCT 17 perform the inverse processing of the DCT 10 and the quantizer 11 to reproduce the inter-picture prediction residual. The obtained inter-reproduced image prediction residual is added to the inter-image prediction signal by the adder 16 to form a regenerated image, which is given to the image memory 15. The reproduced image stored in the image memory 15 is given to the motion compensation predictor 14. The motion-compensated inter-picture predictor 14 generates an inter-picture prediction signal according to the second MV given from the MV reconstructor 6 and provides it to the subtracter 9 and the adder 16 to perform inter-picture prediction coding.
The processing content of each part of re-encoding is basically the same as that of the conventional example of FIG. 5, but specific parameters and the like differ depending on the number of pixels.
[0017]
<Pixel number converter>
The pixel number converter 19 resamples the reproduced image in the normal horizontal direction. Specifically, a reproduction image that is normally 720 pixels is converted by 3/4 to 540 pixels, 2/3 to 480 pixels, 1/2 to 360 pixels, and the like. The state of the pixel at that time (only one line) is shown in FIG. The resampling process is performed by creating pixels at different sample positions while switching tap coefficients with a digital filter and extracting only necessary pixels.
Since the motion compensation block is 16 × 16 pixels, the number of encoded pixels needs to be a multiple thereof. For 540 pixels, 4 pixel dummy pixels are added, 544 pixels, 360 pixels are deleted, 8 pixels are deleted, and 352 pixels are obtained. To do.
[0018]
Such conversion of the number of pixels is usually performed with a significant change in the transfer rate. In other words, if an attempt is made to convert a code string of about 8 Mbps with 720 pixels to 5 Mbps or less with 720 pixels, motion compensation prediction information such as MV remains unchanged, and only prediction residual information is reduced. Image quality deteriorates.
If the number of encoding target pixels is reduced, the resolution is lowered, but the motion compensation prediction information is also reduced, so that the information can be appropriately balanced. The extent to which the number of pixels is reduced depends on the transfer rate of re-encoding, and if it is desired to lower the rate, the number of pixels is also reduced.
Note that the conversion of the number of pixels in the vertical direction has a problem with an interlace signal, but it is possible with a progressive image. Further, since the image quality is not improved even if the re-encoding transfer rate is increased from the original transfer rate, it is difficult to consider increasing the number of pixels.
[0019]
Another example is a case where a total of 16 pixels at the right end and the left end of 720 pixels are deleted to make 704 pixels. The Rec.601 format used as a digital standard image has a horizontal effective pixel count of 720, but in reality there are only about 711 pixels in relation to the analog image signal standard, and there are blank samples on the left and right. To do. Therefore, blank samples are excluded and only 704 pixels are encoded. In this case, resampling is not performed and only the pixels are deleted. Further, although the resolution is not lowered, the effect of reducing the number of pixels is slight.
[0020]
<MV reconstructor>
The MV needs to be compatible with the motion compensation block that uses it. In this embodiment, the number of pixels is different between the image of the incoming code sequence and the image that is re-encoded and output, and the size of the motion compensation block and the block boundary position on the actual image change.
This state is shown in FIG. 4, and a dotted line indicates a block (first block) of 720 pixels before conversion. When the number of pixels is reduced by pixel number conversion, the block (second block) becomes larger on the actual image even with the same 16 × 16 pixels. Accordingly, the boundary position of the block also changes. When 720 is converted to 704, the block size does not change, but the boundary position changes.
Further, since the MV value is indicated by the horizontal and vertical pixel movement amounts, even if the first MV and the second MV move in the same manner on the actual image, the pixel movement amount changes with the conversion of the number of pixels. Need to be converted.
[0021]
The configuration of the MV reconstructor 6 is as shown in FIG.
In FIG. 2, the incoming first MV is stored in the MV buffer 21, and the first block included in the target second block is specified from the positional relationship between the first block and the second block, and the first MV is MV It is given to the searcher 22.
The MV searcher 22 multiplies the MV by the vertical component value conversion ratio of the number of pixels in the vertical direction, and multiplies the horizontal component value by the conversion ratio of the number of pixels in the horizontal direction. Specifically, if the horizontal 720 is converted to 480, the vertical component value is left as it is, and the horizontal component value is multiplied by 2/3. The first MV converted in this way is re-searched as a second MV candidate using the reproduced image with the converted number of pixels, and the MV with the smallest error is output. At that time, a peripheral value of the first MV obtained by multiplying the MV candidate by the conversion ratio may be added.
[0022]
The MV reconstructor may be capable of simple processing as shown in FIG.
In FIG. 3, the incoming first MV is stored in the MV buffer 21, and the first MV is given to the MV selector 23.
The MV selector 23 selects the first MV of the first block at the center of the target second block from the positional relationship between the first block and the second block. This is multiplied by the conversion ratio of the number of pixels in the same manner as described above to obtain the second MV. In this case, since no search is performed, a reproduced image is not necessary.
[0023]
【The invention's effect】
In the present invention, all or part of the incoming code string is decoded, and a motion vector corresponding to a motion compensation block of an image having a different number of pixels is reconstructed using the motion vector extracted from the code string. The decoded image in which the number of pixels is converted is re-encoded using the motion vector.
By re-encoding the image with the converted number of pixels, the balance between resolution and quantization error can be optimized for the re-encoding transfer rate, and the quality of the reconstructed image of the converted code string can be improved. .
[0024]
MV reconstruction is a little processing compared to MV detection, and the scale of the apparatus can be greatly reduced as compared with the case of complete decoding and re-encoding. In addition, when MV detection is completely performed by re-encoding, an image used for MV detection is deteriorated in a reproduced image, so that it becomes easy to detect a motion different from the original motion. Since the re-search is based on MV, erroneous detection is unlikely to occur. Since the MV is similar to the incoming one, the change in the inter-picture prediction residual due to re-encoding is also minimal, and the image quality degradation is small.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a configuration of an embodiment of a moving image code string conversion apparatus according to the present invention.
FIG. 2 is a diagram illustrating a first embodiment of an MV reconstructor of a video code string converter according to the present invention.
FIG. 3 is a diagram illustrating a second embodiment of the MV reconstructor of the video code string converter according to the present invention.
FIG. 4 is a diagram showing a state of a motion compensation block of the present invention.
FIG. 5 is a diagram illustrating a configuration example of a conventional moving image code string conversion device.
FIG. 6 is a diagram illustrating a state of pixel number conversion according to the present invention.
[Explanation of symbols]
1 Code string input terminal 2 Variable length decoder (separation means)
3, 18 Quantizer 4, 17 DCT
5, 16 Adder 6 MV reorganizer 7, 14 dynamic compensation predictor 8, 15 Image memory 9 Subtractor 10 DCT
11 Quantizer 12 Variable Length Encoder (Multiplexing Unit)
13 Code string output terminal 21 MV buffer 22 MV searcher 23 MV converter

Claims (2)

In a moving image code string conversion apparatus that converts a code string of a moving image predicted between motion compensated images,
Separating means for receiving a first moving image code string and obtaining a first motion vector from the code string;
Decoding means for performing inter-picture prediction using the first motion vector corresponding to the first motion compensation block and decoding the first moving picture code string to obtain a first decoded picture signal;
A pixel number converting means for converting the number of pixels of the first decoded image signal to obtain a second decoded image signal having a different number of pixels;
A first motion compensation block in which a second motion vector for re-encoding the second decoded image signal with a second motion compensation block is present at the center position of the second motion compensation block on the actual image And a motion vector reconstruction unit obtained by scaling the first motion vector for the first motion compensation block according to a conversion ratio of the number of pixels in the pixel number conversion unit;
Re-encoding means for inter-predicting the second decoded image signal with the second motion compensation block using the second motion vector to obtain a code string;
Video code stream conversion apparatus characterized by having a multiplexing means for obtaining the second motion vector video code stream of information and the multiplexes the code string obtained by the re-encoding means in the second. In a moving image code sequence conversion method for converting a code sequence of a moving image predicted between motion compensated images,
Receiving a first video sequence and separating a first motion vector from the sequence;
The first moving image code string is subjected to inter-picture prediction decoding using the first motion vector corresponding to the first motion compensation block, to obtain a first decoded image signal,
Converting the number of pixels of the first decoded image signal to obtain a second decoded image signal having a different number of pixels;
A first motion compensation block in which a second motion vector for re-encoding the second decoded image signal with a second motion compensation block is present at the center position of the second motion compensation block on the actual image And the first motion vector for the first motion compensation block is determined according to the conversion ratio of the number of pixels of the first decoded image signal and the number of pixels of the second decoded image signal. Obtained by scaling
Using the second motion vector, the second decoded image signal is subjected to inter-picture prediction encoding with the second motion compensation block to obtain a code string,
Video code stream conversion method characterized by obtaining the second video code stream by multiplexing the information and the code sequence of said second motion vectors.

Images