KR100843080B1 - Video transcoding method and apparatus thereof - Google Patents

Abstract

The present invention relates to a method for rapidly selecting a suitable reference frame from several reference frames when transcoding an input video stream into another format having a different GOP structure.

A transcoder according to an embodiment of the present invention includes a reconstruction unit for reconstructing a transform coefficient and a video frame from an input video stream, and a first frame and the first frame referred to by the video frame based on the magnitude of the transform coefficient. And a selector for selecting one of the second frames at different positions from the one frame, and an encoder for encoding the reconstructed video frame with reference to the selected frame.

Transcoding, transcoder, reference frame, motion vector, block

Description

Video transcoding method and apparatus

1A is a diagram illustrating a GOP structure of an MPEG-2 video main profile.

1B is a diagram illustrating a GOP structure of an H.264 Baseline Profile.

2A and 2B illustrate the concept of multiple references supported by H.264.

3A and 3B illustrate a method of selecting a reference frame during transcoding.

4 is a block diagram showing the configuration of a transcoder according to an embodiment of the present invention.

FIG. 5 is a block diagram showing a configuration of a recovery unit included in the transcoder of FIG. 4. FIG.

FIG. 6 is a block diagram showing a configuration of an encoding unit included in the transcoder of FIG. 4. FIG.

(Symbol description of main part of drawing)

100: transcoder 110: restoring unit

111: entropy decoder 112: inverse quantization unit

113: inverse transform unit 114: inverse predictor

120: selector 130: encoder

131: prediction unit 132: transformation unit

133: quantization unit 134: entropy encoder

The present invention relates to a method for rapidly selecting an appropriate reference frame from several reference frames when transcoding an input video stream into another format having a different group of picture (GOP) structure.

As information and communication technology including the Internet is developed, not only text and voice but also video communication are increasing. Existing text-oriented communication methods are not enough to satisfy various needs of consumers. Accordingly, multimedia services that can accommodate various types of characteristics such as text, video, and music are increasing. The multimedia data has a huge amount and requires a large storage medium and a wide bandwidth in transmission. Therefore, in order to transmit multimedia data including text, video, and audio, it is essential to use a compression coding technique.

The basic principle of compressing data is to eliminate redundancy in the data. Spatial overlap such as repeating the same color or object in an image, temporal overlap such as when there is almost no change in adjacent frames in a video frame, or the same sound repeats repeatedly in audio, or human visual and perceptual ability. Data can be compressed by eliminating visual redundancy that takes into account frequency insensitivity. In a general video coding method, temporal redundancy of video data is removed by temporal filtering based on motion compensation, and spatial redundancy is removed by spatial transform.

The result of removing the redundancy of the video data is again loss coded according to a predetermined quantization step through a quantization process. The quantized result is finally losslessly coded through entropy coding.

However, the encoded video data may be transmitted and decoded to the final terminal device as it is, but may be transcoded in consideration of network conditions or performance of the final terminal device before being transmitted to the final terminal device. For example, if the encoded video data is not suitable for transmission over the current network, the transmitting server side changes the signal-to-noise ratio (SNR), frame rate, resolution or coding scheme (codec) of the video data. This process is called "transcoding".

Conventional methods of transcoding MPEG-2 coded video data in an H.264 method may be divided into a method of converting in a frequency domain and a method of converting in a pixel domain. The method of converting in the frequency domain is mainly used when the similarity between the input format and the output format of transcoding is large, and the method of converting in the pixel region is mainly used when the similarity is small. In particular, the method of converting in the pixel region recycles the motion vector estimated at the time of encoding.

However, when the GOP (Group of Pictures) structure is changed or the motion vector reference method is changed by transcoding, it is difficult to use the existing motion vector as it is. For this reason, if the motion vector is recalculated from the reconstructed images during transcoding, it will consume a lot of time and resources. In addition, if a reference to a far-away frame is referred to avoid recomputation, more residuals may occur than referring to a previous frame, resulting in an increase in bit rate and deterioration in image quality.

When transcoding is performed between video streams having different GOP structures (reference methods), it is very difficult to select which frame as a reference frame in order to obtain an appropriate trade-off between computational complexity, image quality, and bit rate. .

Accordingly, the present invention provides a method and apparatus for selecting an appropriate reference frame in consideration of transcoding speed and picture quality in a transcoding process having a different GOP structure (reference method) between an input and an output. It is.

Technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, a transcoder for converting an input video stream to generate an output video stream includes: a reconstruction unit for reconstructing a transform coefficient and a video frame from the input video stream; A selection unit selecting one of a first frame referred to by the video frame and a second frame at a position different from the first frame based on the magnitude of the transform coefficient; And an encoder which encodes the reconstructed video frame with reference to the selected frame.

According to an aspect of the present invention, there is provided a transcoding method comprising: reconstructing a transform coefficient and a video frame from an input video stream; Selecting one of a first frame referenced by the video frame and a second frame at a position different from the first frame based on the magnitude of the transform coefficient; And encoding the reconstructed video frame with reference to the selected frame.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1A illustrates a GOP structure of an MPEG-2 video main profile, and FIG. 1B illustrates a GOP structure of an H.264 baseline profile. As shown in FIG. 1, a B frame may refer to an I or P frame before and after, but may not refer to another B frame. However, a P frame may refer to an I frame or another P frame. Such references are generally made within a single GOP structure.

On the other hand, the H.264 baseline profile, as shown in Fig. 2, has a structure in which some frames refer to the immediately preceding frame. However, in general, H.264 can refer to any frame within a single GOP, and has a structure that allows multiple references.

2A and 2B illustrate the concept of multiple references supported by H.264. Referring to FIG. 2A, it can be seen that the current P frame 10 may refer to a plurality of frames 20 and 25 at the same time. This is possible because the unit for estimating the motion vector and generating the residual of the current frame is made in units of macroblocks (MBs), not units of frames.

2b shows that different macroblocks MB1 and MB2 of the current P frame 10 refer to regions ref1 and ref2 on different frames 20 and 25, respectively. As such, in H.264, selecting a suitable reference frame for each macroblock provides diversity and adaptability of video coding.

The transcoder needs to recalculate the motion vector of the input video in order to transcode the input video as shown in FIG. 1A into the output video as shown in FIG. 2B having different GOP structures. However, recalculating the motion vector to refer to the previous frame in the output video consumes a lot of computation time. On the other hand, if the far frame is referenced using the input video reference method as it is without recomputation, a large residual may be generated as compared to the previous frame, resulting in deterioration of the image quality (or bit rate increase). Therefore, when transcoding, it is necessary to find a certain trade-off between the amount of computation and the image quality (or bit rate).

3A and 3B illustrate a method of selecting a reference frame during transcoding, in which FIG. 3A is an input video structure before transcoding and FIG. 3B is an output video structure after transcoding. In FIG. 3A, the frame currently being processed is B ₂ , and a motion vector indicates an I frame. In the MPEG-2 structure, all forward reference vectors of frame B ₂ point to I frames. On the other hand, in the H.264 structure as shown in FIG. 3B, forward motion vectors mv1 and mv2 of MB1 and MB2 may indicate an I frame or a P1 frame. If the motion vector mv2 (I) pointing to the I frame does not make the residual much larger than the motion vector mv2 (P ₁ ) pointing to the P1 frame, it would be advantageous to select mv2 (I) for the computation speed. . On the other hand, if vice versa, it would be advantageous to choose mv2 (P ₁ ).

According to the present invention, in the case of transcoding in which the GOP structure is changed, when the output video standard supports multiple references such as H.264, the input video and the previous frame are designated as a new reference frame in selecting a reference frame. It is intended to provide a way to determine one of the things. By following the reference frame of the input video as it is, high-speed conversion is possible by reusing existing motion vectors, and selecting a new reference frame requires a large amount of computation, but obtains excellent image quality. Therefore, it is possible to perform transcoding in consideration of transcoding speed and image quality through appropriate trade-off between the two.

4 is a block diagram showing the configuration of a transcoder 100 according to an embodiment of the present invention. Transcoder 100 converts the input video stream to produce an output video stream. To this end, the transcoder 100 may include a reconstructor 110, a selector 120, and an encoder 130.

The reconstructor 110 reconstructs the transform coefficient and the video frame from the input video stream. The selector 120 selects one of a first frame referred to by the video frame and a second frame at a position different from the first frame based on the magnitude of the transform coefficient. The encoder 130 encodes the reconstructed video frame with reference to the selected frame.

5 is a block diagram showing the configuration of the restoration unit 110. The reconstruction unit 110 may include an entropy decoder 111, an inverse quantization unit 112, an inverse transformer 113, and an inverse predictor 114.

The entropy decoder 111 losslessly decodes the input video stream using algorithms such as variable length decoding and arithmetic decoding to restore quantization coefficients and motion vectors.

The inverse quantizer 112 inverse quantizes the reconstructed quantization coefficients. This inverse quantization process corresponds to the inverse of the quantization process performed in the video encoder. A transform coefficient of the inverse quantization result may be obtained. The transform coefficient is provided to the selector 120.

The inverse transform unit 113 inversely transforms the transform coefficient by using an inverse spatial transform technique such as an inverse DCT transform or an inverse wavelet transform.

The inverse predictor 114 generates a prediction frame by motion compensating a reference frame with respect to the current frame by using the motion vector reconstructed by the entropy decoder 111, and generates the predicted frame by the inverse transform unit 113. It adds the inverse transformed result to produce a reconstructed frame.

Again, referring to FIG. 4, the selector 120 may use the first frame used as a reference frame in the input video stream as it is, using the transform coefficient provided from the reconstructor 110, or another second frame. Choose whether to use. The frame selected by this is used by the encoder 130 as a reference frame. For this selection, the selector 120 calculates a predetermined threshold value from the transform coefficient, and uses the threshold value as the criterion of determination.

In the present invention, a method of using a fixed threshold value in one frame and a variable method in which the threshold value is adaptively changed even in a frame to be suitable for a real-time application are taken as an example.

Fixed Threshold  How to use

In this embodiment, the threshold TH _g is fixed within a single frame. The threshold TH _g may be determined in various ways, but may be calculated as shown in Equation 1 as an example.

In Equation 1, N is the number of blocks of one frame, and C _m (i, j) is a transform coefficient at the position of coordinate (i, j) in the m-th block. In addition, V _ctl is a control parameter (default value = 1.0) that can adjust the size of the threshold. The block may have a DCT block size that is a unit of DCT transformation or a macroblock size that is a unit of motion estimation.

When the index of the current block is k, the selection criterion of the reference frame is shown in Equation 2 below.

if (S | C _k (i, j) | <TH _g )

Then, select Ref _orig as the reference frame.

else, Ref ₀ is selected as the reference frame.

In Equation 2, S | C _k (i, j) | means the sum of the absolute values of the transform coefficients included in the current block, and Ref _orig denotes the first frame used as a reference frame of the current block in the input video stream. Ref ₀ means a second frame at a position different from the first frame. Preferably, the second frame is the frame immediately before the frame to which the current block belongs (the current frame).

Equation 2 means that for a block having an energy larger than the average, a frame closer to the current frame is selected as the reference frame. This means that blocks with less than average energy will use the motion vectors in the input video stream, while larger blocks will obtain new motion vectors using the relatively adjacent frames as reference frames. In this way, a suitable trade-off can be found between image quality and transcoding speed.

However, as shown in Equation 1, a method for obtaining a threshold value considering a block that has not yet been processed may require a lot of calculation. Therefore, when the index of the block to be processed is k, an embodiment considering only the currently processed block as shown in Equation 3 below may be considered in calculating the threshold value TH _g .

The size of the block that is the unit for selecting the reference frame and the macroblock to which the actual motion vector is allocated may be different. In this case, merging or decomposition of the motion vector may be necessary.

variable Threshold  How to use

In a real-time application of transcoder, whether the frames can be processed until the time limit is an important issue. In the real-time transcoding situation, it is necessary to variably adjust the threshold value using the currently available calculation time as one factor. That is, the variable threshold TH _l may be calculated by multiplying the fixed threshold TH _g by the variable factor RTfactor as shown in Equation 4 below.

Equation 4 improves the transcoding speed by increasing the threshold TH _l when it is likely to exceed the time limit for processing the current frame, and decreases the threshold TH _l when sufficient time remains. This means that the image quality can be improved.

The RTfactor can be determined in various ways, but considering the factor to be considered is the index of the block currently being processed, the remaining time of the time limit, etc., it can be determined as in Equation 5 below.

In Equation 5, k is an index number (0 ≦ k <N) of a block currently being processed, and N is the total number of blocks forming a frame. In addition, T _due means the time at which the current frame should be converted, T _cur means the current time, and frame rate means the number of frames per second during video playback. The framerate is a constant but multiplied to normalize (T _due -T _cur ). Thus, the numerator and denominator of equation (5) both have values between 0 and 1. Equation 5 means that as the number of remaining blocks to be processed in the current frame increases, the RT factor increases, so that the transcoding speed increases, and as the processing time increases, the RT factor decreases, thereby lowering the transcoding speed and improving image quality.

For the same purpose as in Equation 5, the RTfactor may be defined as in Equation 6 below.

The selector 120 compares the sum of the fixed or variable threshold described above with the absolute value of the transform coefficients included in the current block, and selects whether to use the motion vector and the reference frame (first frame) of the input video stream as it is. It is selected whether to calculate a motion vector with reference to a new frame (second frame). This selection is made for each block and provided to the encoder 130 as reference frame information.

Since the method of approximating the forward motion vector is already known when the motion vector is the reverse motion vector, if the forward motion vector cannot be obtained, the forward motion vector is approximated to obtain the forward motion vector, and then the existing motion vector and the reference frame are obtained. Can be used instead. For example, if a macroblock of a B frame refers to a block of this trailing P frame, the macroblock that covers the largest area of the macroblocks of the P frame overlapping this block is selected, and this macro The motion vector for the previous row I frame of the block can be obtained. In this case, the motion vector for the I frame that can be used in the B frame may be calculated as the sum of the motion vector for the P frame block and the motion vector for the I frame of the macroblock most overlapping among the P frame blocks.

6 is a block diagram showing the configuration of the encoder 130. The encoder 130 may include a predictor 131, a transformer 132, a quantizer 133, and an entropy encoder 134.

The prediction unit 131 obtains a motion vector by using one of the first frame and the second frame as a reference frame for each block of the current frame using the reference frame information. The first frame refers to a frame used as a reference frame of the current frame among the frames reconstructed by the reconstructor 110, and the second frame refers to a frame at a different time position from the first frame.

In this case, when any block of the current frame uses the first frame as the reference frame, the prediction unit 131 allocates the motion vector of the existing input video stream to the current block as it is. In addition, when the block uses the second frame as a reference frame, the motion vector is estimated with reference to the second frame and the estimated motion vector is assigned to the current block.

The prediction unit 131 generates a prediction frame by motion compensating the corresponding reference frame (the first frame or the second frame) using the motion vector assigned to the blocks belonging to the current frame, and the prediction frame in the current frame. Create a residual by subtracting

The transform unit 132 performs a spatial transform on the generated residual. As the spatial transformation method, a discrete cosine transform (DCT), a wavelet transform, or the like may be used. As a result of the spatial transform, a transform coefficient is obtained. When the DCT is used as the spatial transform method, the DCT coefficient is obtained, and when the wavelet transform is used, the wavelet coefficient is obtained.

The quantization unit 133 quantizes the transform coefficients obtained by the spatial transform unit 132 to generate quantization coefficients. Quantization refers to an operation of dividing the transform coefficients represented by arbitrary real values into discrete values. Such quantization methods include scalar quantization and vector quantization. Among them, a simple scalar quantization method is performed by dividing transform coefficients by corresponding values in a quantization table and rounding them to integer positions.

The entropy encoder 134 losslessly encodes the quantization coefficients and the motion vectors provided by the predictor 131 to generate an output video stream. As such a lossless coding method, arithmetic coding, variable length coding, or the like may be used.

Up to now, each of the components of FIGS. 4 to 6 is software such as a task, a class, a subroutine, a process, an object, an execution thread, a program, or a field-programmable gate array (FPGA) that is performed in a predetermined area on a memory. Or hardware such as an application-specific integrated circuit (ASIC), or a combination of the software and hardware. The components may be included in a computer readable storage medium or a part of the components may be distributed over a plurality of computers.

Although embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features thereof. I can understand that. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

According to the present invention, in transcoding an input video stream into another format having a different GOP structure, by selecting an optimal reference frame, a relatively high picture quality or a low bit rate can be realized within a limited computation power. There is this.

Claims (18)

A transcoder that converts an input video stream to produce an output video stream, A reconstruction unit for reconstructing transform coefficients and video frames from the input video stream; A selection unit selecting one of a first frame referred to by the video frame and a second frame at a position different from the first frame based on the magnitude of the transform coefficient; And And an encoder configured to encode the reconstructed video frame with reference to the selected frame. The method of claim 1, wherein the second frame A transcoder, which is a frame located immediately before the video frame in encoding order. The method of claim 1, The input video stream is an MPEG standard video stream and the output video stream is an H.264 standard video stream. The method of claim 1, wherein the selection unit If the sum of the absolute values of the transform coefficients for a specific block does not exceed a predetermined threshold, the first frame is selected as a reference frame for the specific block, And selecting the second frame as a reference frame for the particular block if the sum of the absolute values of the transform coefficients for the particular block exceeds a predetermined threshold. The method of claim 4, wherein the threshold is A transcoder, which is the sum of the absolute values of the transform coefficients belonging to a single frame divided by the number of blocks. The method of claim 4, wherein the threshold is And a sum of absolute values of transform coefficients belonging to the currently processed block among transform coefficients belonging to a single frame divided by the number of the currently processed blocks. The method of claim 4, wherein the threshold is The sum of absolute values of transform coefficients belonging to a single frame divided by the number of blocks is multiplied by a predetermined variable coefficient, and the variable coefficient is determined by the number of residual blocks to be processed in the single frame and the remaining time to be processed. Transcoder. The method of claim 7, wherein the variable coefficient is A transcoder calculated by dividing the number of residual blocks to be processed by the number of blocks belonging to the single frame by the product of the remaining time times the frame rate. The method of claim 1, wherein the encoder And if the selected frame is a first frame, uses the motion vector as it is, and if the selected frame is a second frame, estimating the motion vector with reference to the second frame. A transcoding method for converting an input video stream to produce an output video stream. Recovering transform coefficients and video frames from the input video stream; Selecting one of a first frame referenced by the video frame and a second frame at a position different from the first frame based on the magnitude of the transform coefficient; And And encoding the reconstructed video frame with reference to the selected frame. The method of claim 10, wherein the second frame And a frame located immediately before the video frame in encoding order. The method of claim 10, Wherein the input video stream is an MPEG standard video stream and the output video stream is an H.264 standard video stream. The method of claim 10, wherein the selecting step Selecting the first frame as a reference frame for the particular block if the sum of the absolute values of the transform coefficients for the particular block does not exceed a predetermined threshold; And Selecting the second frame as a reference frame for the particular block if the sum of the absolute values of the transform coefficients for the particular block exceeds a predetermined threshold. The method of claim 13, wherein the threshold is Wherein the sum of the absolute values of the transform coefficients belonging to a single frame is divided by the number of blocks. The method of claim 13, wherein the threshold is And a sum of absolute values of transform coefficients belonging to the currently processed block among transform coefficients belonging to a single frame divided by the number of the currently processed blocks. The method of claim 13, wherein the threshold is The sum of absolute values of transform coefficients belonging to a single frame divided by the number of blocks is multiplied by a predetermined variable coefficient, and the variable coefficient is determined by the number of residual blocks to be processed in the single frame and the remaining time to be processed. Transcoding method. The method of claim 16, wherein the variable coefficient Calculated by dividing the number of residual blocks to be processed by the number of blocks belonging to the single frame by the product of the remaining time times the frame rate. The method of claim 10, wherein the encoding is performed. If the selected frame is the first frame, using the motion vector as it is, and if the selected frame is the second frame, estimating the motion vector with reference to the second frame, Transcoding method.

Images