JP2008503919A - Method and apparatus for optimizing video coding - Google Patents

Abstract

  An encoder for encoding video signal data corresponding to a plurality of pictures and a method corresponding thereto are provided. The encoder performs video analysis of the video signal data using a plurality of overlap analysis windows regarding at least a part of the plurality of pictures corresponding to the video signal data, and encodes the video signal data encoding parameters based on the result of the video analysis. A duplicate window analysis unit (310) that performs the adaptation of

Description

(Cross-reference of related applications)
This application claims the benefit of US Provisional Patent Application No. 60/581280, filed Jun. 18, 2004.

  The present invention relates generally to video encoders and video decoders, and more particularly to a method and apparatus for optimizing video coding.

  Multi-pass video coding schemes include MPEG-2 and JVT / H. It is used in many video coding architectures such as H.264 / MPEG AVC to increase coding efficiency. The underlying concept of these schemes is that the entire sequence is tested several times while performing analysis and collecting statistics that can be used in future iterations to improve coding performance. It is to become.

  The two-pass encoding scheme is already used in several encoding systems such as MICROSOFT (registered trademark) WINDOWS MEDIA (registered trademark) and REALVIDEO (registered trademark) encoder. According to this coding scheme, the encoder first performs an initial coding pass on the entire sequence with some initial default settings and collects statistics on the coding efficiency of each picture in the sequence. . After this process is complete, the entire sequence is once again processed and encoded while simultaneously taking into account previously generated statistics. This can significantly improve the coding efficiency and can also satisfy some specified coding constraints or requirements, for example meeting a given bit rate constraint of the encoded stream. This is because the encoder knows more about the characteristics of the video sequence or picture, and therefore the encoder can better select parameters such as the quantizer and dead zone processing used for encoding. . Statistics that can be collected during this first coding pass and that can be used for this purpose include bits per picture, spatial activity (ie average normalized macroblock variance and Average), temporal activity (ie, motion vector / motion vector variance), distortion (eg, root mean square error (MSE)), and the like. Although using these methods can significantly improve the performance of the encoding, these methods tend to be very complex and can only be used offline (the entire sequence is encoded first, After that, the second pass is performed.) Therefore, it is not suitable for a real-time encoder. These methods do not necessarily take into account all possible statistics that can be inferred from the initial encoding step.

(Summary of Invention)
The present invention relates to a method and apparatus for optimizing video coding, which addresses the shortcomings and problems of the prior art and other prior art described above.

  According to an aspect of the present invention, an encoder is provided that encodes video signal data corresponding to a plurality of pictures. The encoder performs video analysis of the video signal data using a plurality of overlap analysis windows regarding at least a part of the plurality of pictures corresponding to the video signal data, and encodes the video signal data encoding parameters based on the result of the video analysis. Includes a duplicate window analysis unit that performs

  According to another aspect of the invention, a method for encoding video signal data corresponding to a plurality of pictures is provided. The method includes performing video analysis of the video signal data using a plurality of overlap analysis windows for at least some of the plurality of pictures corresponding to the video signal data, and encoding the video signal data based on the results of the video analysis. Adapting the optimization parameters.

  The above and other aspects, features and advantages of the present invention will become apparent upon reading the following detailed description of exemplary embodiments in conjunction with the accompanying drawings.

  The invention can be better understood with reference to the following exemplary drawings.

  The present invention relates to a method and apparatus for optimizing video coding. An advantage of the present invention is that it allows a video encoder to compress a video sequence with significantly improved subjective and objective quality at a given bit rate. This is achieved by a special processing of the video sequence that provides a simple analysis of the current picture compared to the subsequent N pictures to be encoded. The encoder then uses this analysis result to provide the encoding parameters (picture type / slice type, quantizer, thresholding parameters, Lagrange λ, etc.) used to encode the current picture. , But not limited to these) can be determined better. Unlike the prior art systems that do dual-pass or multi-pass coding of entire sequences to improve coding performance, the present invention is relatively simple and therefore has an impact on complexity. Is also relatively small. The principles of the present invention can also be used with other multi-pass coding strategies to further increase efficiency. Similarly, a general system (using previously encoded M pictures) can also be formed.

  In accordance with the principles of the present invention, only the portion of the entire sequence where picture windows overlap is first analyzed. Based on the statistical value thus generated, the encoding parameter of each picture is adjusted as appropriate. These coding parameters include picture type / slice type determination (I, P, B), frame / field determination, B picture interval, picture or macroblock quantization value (QP), coefficient thresholding, Lagrange parameters, chroma offset, weighted prediction, reference picture selection, multi-block sizing, entropy parameter initialization, intra mode determination, and deblocking filter parameters, but are not limited to these It is not done. The analysis method may require various complexity costs, but the analysis method can be used to perform picture / macroblock analysis. Analysis methods include full first pass coding, simple first pass motion estimation using spatial analysis, or simple spatio-temporal analysis metrics including but not limited to variance and image differences. . In addition, overlapping picture windows (and their overlapping pictures) can be made as small or large as necessary (more or less), so there are various tradeoffs between delay and performance. Can do.

  In this specification, the principle of the present invention will be described. Accordingly, those skilled in the art will appreciate that various configurations can be devised that implement the principles of the invention that are not explicitly described or illustrated herein, but are within the spirit and scope of the invention.

  All examples and conditions expressed herein are for educational purposes to help the reader understand the principles of the invention and the concepts that will help the inventor to advance the technology. And should not be construed as being limited to these specifically recited examples and conditions.

  Moreover, all statements herein reciting principles, aspects and embodiments of the invention and specific examples of the invention are intended to include both structural and functional equivalents thereof. In addition, these equivalents include both currently known equivalents and equivalents that will be developed in the future, i.e. any future developed elements that perform the same function regardless of their structure. Shall be included.

  Thus, for example, those skilled in the art will appreciate that the block diagrams presented herein represent conceptual diagrams of exemplary circuits that implement the principles of the invention. Similarly, any flowchart, flowchart, state transition diagram, pseudo-code, etc. may be substantially expressed in a computer-readable medium and executed by a computer or processor that may or may not be explicitly indicated. It should also be understood to represent a complex process.

  The functions of the various elements shown in the figures can be realized using dedicated hardware and hardware capable of executing software in conjunction with appropriate software. When implemented by a processor, these functions can be implemented by a single dedicated processor, by a single shared processor, or by multiple individual processors that can share part of them. You can also. Furthermore, the explicit use of the terms “processor” or “controller” should not be construed to refer only to hardware capable of executing software, but digital signal processor (DSP) hardware. , Implicitly including, but not limited to, read only memory (ROM), random access memory (RAM) and non-volatile storage for storing software.

  Conventional and / or custom hardware may also be included. Similarly, any switches shown in the drawings are conceptual only. The function of the switch can be implemented by program logic operation, dedicated logic or interaction between program control and dedicated logic, or manually, and the creator chooses a specific technology based on specific context. can do.

  In the claims of this specification, an arbitrary element expressed as a means for executing a specific function is, for example, (a) a combination of circuit elements that execute the function, (b) firmware, microcode, or the like. Any method of performing the function is included, including any form of software in combination with appropriate circuitry that executes the software to perform the function. The invention as defined in the claims lies in combining the functions performed by the various means recited in the manner required by the claims. Applicant thus regards any means which can carry out these functions as equivalent to those shown herein.

  In accordance with the principles of the present invention, unlike conventional methods that consider the entire video sequence or individual windows during each pass, each pass is performed on overlapping windows to adjoin predetermined features. A new multi-pass coding architecture is disclosed that allows reuse between windows. This architecture allows optimal encoding with much fewer steps, so video quality can be significantly improved with lower memory requirements and lower latency, even at lower costs and lower complexity. Advantages of multi-pass coding such as improvement can also be obtained. This feature is real-time considering that the encoder can determine the best parameters even during the first pass due to the similarity between adjacent windows, so that no further iterations are required for final encoding. This is particularly important when applied to encoding.

Referring to FIG. 1, the entire window-based two-pass coding architecture is indicated by reference numeral 100. The size of the processing / analysis window is W _p pieces worth of pictures, the size of the overlap allowed between two adjacent groups is W _o pieces worth of pictures. Processing the first window yields some initial statistics that can be used to determine a pre-set of coding characteristics for all frames in this window. More specifically, when using the 2-pass scheme, all frames that do not belong to the next window can be immediately encoded based on the generated parameters. However, this information can be used directly for processing / analysis of this next window. For example, these parameters can be used as an initial seed during the processing of this window, and the analysis can be improved given that there is a high temporal correlation in most sequences. More importantly, the coding parameters used for the initial frame of the window which belongs to the previous window by selecting the W _o, further refine / adjusted based on the newly generated statistics be able to. This basically allows faster convergence to the optimal solution when multiple iterations / passes are used, for example after processing the entire sequence or M adjacent windows. This temporal window can be as large or small as possible depending on the encoder's capabilities or requirements, and this scheme is repeated with different window sizes (which can be larger or smaller than W _o and W _p ). Obviously you can do that too.

  A number of different criteria can be used in the multipath pre-analysis step of the inventors. These criteria can be determined by the complexity constraints of the encoder architecture, and from simple spatio-temporal methods (including but not limited to edge detection, texture analysis metrics, and absolute image differences). Even complex strategies can be considered, including but not limited to discrete cosine transform (DCT) analysis, first pass intra coding, motion estimation / compensation, and full coding. Latency can also be adjusted by increasing or decreasing the analysis and / or overlapping windows.

  As an example of such a system, the following criteria can be calculated during this analysis:

For all picture k in the window W _p, calculates the following.

  (I) The average value MBmean (k, i, j) is calculated as follows for each macroblock at the position (i, j).

  (Ii) The root mean square value MBsqmean (k, i, j) is calculated as follows.

  (Iii) The variance value MBvariance (k, i, j) is calculated as follows.

MBvariance (k, i, j) = MBsqmean (k, i, j) − (MBmean (k, i, j)) ²

(Iv) The average macroblock average value AMM _k is calculated for the entire picture as follows.

(V) Calculate the average macroblock variance AMV _k as follows:

(Vi) Calculate the picture variance PV _k as follows:

ここで、ｃ［ｘ，ｙ］は位置（ｘ，ｙ）におけるピクセル値に対応し、ＰＭＢ_ＷおよびＰＭＢ_Ｈはそれぞれマクロブロックを単位とするピクチャの幅および高さであり、Ｂ_ＷおよびＢ_Ｈは現在のピクチャ中の各マクロブロックの幅および高さである（通常はＢ_Ｗ＝Ｂ_Ｈ＝１６）。

Here, c [x, y] corresponds to the pixel value at the position (x, y), PMB _W and PMB _H are the width and height of the picture in units of macroblocks, and B _W and B _H Is the width and height of each macroblock in the current picture (usually B _W = B _H = 16).

  Further, the following temporal characteristics for the picture m (for example, m = k + 1) are also calculated as follows.

(I) The average absolute picture difference MAPD _{k, m} is calculated as follows.

(II) The average absolute weighted picture difference MAWPD _{k, m} is calculated as follows.

(III) The average absolute offset / picture difference MAWPD _{k, m} is calculated as follows.

(IV) The root mean square picture error MSPE _{k, m} is calculated as follows.

(V) The absolute picture variance difference APVD _{k, m} is calculated as follows.

APVD _{k, m} = | PV _k -PV _m |

Other spatio-temporal properties that can be calculated are: absolute difference in histogram, histogram of absolute difference, χ ² metric between k and M, arbitrary (or multiple) edge operators (canny edge operator, Sobel edge, Field-based metrics for detecting interlaced properties of k edges, or sequences, including but not limited to operators, or pre-wit edge operators). Two other statistics that may be useful and can be inferred from the above are the distance from the closest past encoded intra picture of the current picture (last_idistance _k ), and the most of the current picture The distance (next_idistance _k ) from the coded intra picture in the near future. These are measured, for example, by picture number, coding order, or picture order count (poc). These statistics can be improved by considering a scene change / shot detector and / or a default GOP (group of pictures) structure. Temporal characteristics can be calculated using the original image or the reconstructed image (e.g. when the invention is applied to a multipath aspect), and these metrics are calculated using motion estimation / compensation. It can also be considered.

  Based on the above metrics, the encoder can decide to modify the parameters of a particular picture, macroblock or subblock related to the encoding process. These include quantized values (QP), coefficient dead zone processing / threshold processing, macroblock encoding and Lagrangian values for picture level determination between frames and fields, deblocking filter parameters, encoding and This includes parameters such as reference picture ordering, scene / shot (including but not limited to fade / dissolve / wipe / flash) detection, GOP structure and the like.

In an exemplary embodiment of the invention, the above parameters are considered to adapt the picture quantization value (QP) when encoding a picture k of slice type cur_slice_type _k as follows: In this embodiment, distance _{k, k + 1} is considered as the distance between two adjacent pictures in units of pictures.
if (next_idistance _k > 3 && cur_slice_type _k == I_Slice)
{
if (PV _k <1 && MAPD _{k, k + 1} <1 && last_idistance _k > 5 ^* distance _{k, k + 1} )
QP _k = QP _k −4
else if (MAPD _{k, k + 1} <3 && (k == 0 || last_idstance _k > 5 ^* distance _{k, k + 1} ))
QP _k = QP _k −3
else if (MAPD _{k, k + 1} <10)
QP _k = QP _k −2
else if (MAPD _{k, k + 1} <15)
QP _k = QP _k −1
}
else if (AMV _k > 10 && AMV _k <60)
{
if (PV _k <500 && next_idistance _k > 3 ^* distance _{k, k + 1} )
{
if (MAPD _{k, k + 1} <10 && AMV _k <35 && last_idistance _k > 2 ^* distance _{k, k + 1} )
QP _k = QP _k −2
else
QP _k = QP _k −1
}
else if (PV _k <1500 && next_idistance _k > 0)
{
if (MAPD _{k, k + 1} <25)
QP _k = QP _k −1
}
}
else if (MAPD _{k, k + 1} == 0 && next_idistance _k > 3 ^* distance _{k, k + 1} && last_idistance _k > 4 ^* distance _{k, k + 1} )
QP _k = QP _k −2
else (((MAPD _{k, k + 1} <2 && next_idistance _k > 3 ^* distance _{k, k + 1} && last_idistance _k > 2 ^* distance _{k, k + 1} )
|| last_idistance _k > 30) && next_idistance _k > 5)
{
if (MAPD _{k, k + 1} <1)
QP _k = QP _k −3
else if (MAPD _{k, k + 1} <4)
QP _k = QP _k −2
else if (MAPD _{k, k + 1} <10)
QP _k = QP _k −1
}

In the above embodiment, it is not considered whether the previous or near past picture has already updated its QP according to the above rules. Therefore, it is possible to update the QP value more than necessary, which may be undesirable in terms of rate-distortion (RD) performance. For this purpose, the parameter last_idistance _k is updated so that it is equal to the value of the last adjusted QP regardless of the picture type.

Similarly, macroblock / block variance, average and edge statistics can be used to determine local coding parameters. For example, in order to select the Lagrange λ of the macroblock at the position (i, j), the following rule may be considered.
if (cur_slice_type _k ! = B_Slice)
{
if (contains_edges (k, i, j))

ｅｌｓｅ ｉｆ（ｃｕｒ＿ｓｌｉｃｅ＿ｔｙｐｅ_ｋ＝＝Ｉ＿Ｓｌｉｃｅ）
｛
ｉｆ（ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＜１５｜｜ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＞６０）

else if (cur_slice_type _k == I_Slice)
{
if (MBvariance (k, i, j) <15 || MBvariance (k, i, j)> 60)

ｅｌｓｅ ｉｆ（ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＞＝１５＆＆ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＜＝４０）

else if (MBvariance (k, i, j)> = 15 && MBvariance (k, i, j) <= 40)

ｅｌｓｅ

else

｝
ｅｌｓｅ／／ｃｕｒ＿ｓｌｉｃｅ＿ｔｙｐｅ_ｋ＝＝Ｐ＿Ｓｌｉｃｅ
｛
ｉｆ（ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＜１５｜｜ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＞６０）

}
else / cur_slice_type _k == P_Slice
{
if (MBvariance (k, i, j) <15 || MBvariance (k, i, j)> 60)

ｅｌｓｅ ｉｆ（ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＞＝１５＆＆ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＜＝４０）

else if (MBvariance (k, i, j)> = 15 && MBvariance (k, i, j) <= 40)

ｅｌｓｅ

else

｝
｝
ｅｌｓｅ
｛
ｂｓｃａｌｅ＝ｍａｘ（２．００，ｍｉｎ（４．００，（ＱＰ／６．０）））；
ｉｆ（ｃｏｎｔａｉｎｓ＿ｅｄｇｅｓ（ｋ，ｉ，ｊ））

}
}
else
{
bscale = max (2.00, min (4.00, (QP / 6.0)));
if (contains_edges (k, i, j))

ｅｌｓｅ
｛
ｉｆ（ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＜１５｜｜ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＞６０）

else
{
if (MBvariance (k, i, j) <15 || MBvariance (k, i, j)> 60)

ｅｌｓｅ ｉｆ（ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＞＝１５＆＆ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＜＝４０）

else if (MBvariance (k, i, j)> = 15 && MBvariance (k, i, j) <= 40)

ｅｌｓｅ

else

｝
ｉｆ（ｎａｌ＿ｒｅｆｅｒｅｎｃｅ＿ｉｄｃ＝＝１）
λ＝０．８０×λ
｝

}
if (nal_reference_idc == 1)
λ = 0.80 × λ
}

Similar decisions can be made in selecting the quantization value or coefficient thresholding used for the remaining encodings. Specifically, H.C. The quantization of the coefficient W at H.264 is performed as follows.
Z = int ({| W | + f × (1 << q_bits)} >> qbits) · sgn (W)
Here, Z is the final quantized value, and q_bits is based on the quantizer QP of the current macroblock. The term of f × (1 << q_bits) functions as a term for rounding the quantization process, and is “optimal” when ½ × (1 << q_bits).

  Referring now to FIG. 2, the overall effect of dead zone processing during transformation and quantization is indicated by reference numeral 200. In FIG. 2, a section near zero is called a dead zone. The dead zone quantizer is characterized by two parameters, zero bin-width (2s-2f) and outbin width (s), as shown in FIG. Dead zone optimization by f is often used as an efficient way to obtain good rate-distortion performance. However, it is well known that by introducing a dead zone during this process (ie, reducing the term of f), it is usually possible to further reduce the bit rate while keeping the quality impact low. This is especially true for low resolution content that does not include high resolution material details (and film grain information). Although f = 1/2 may be used, if this value is used, the increase in the bit rate becomes large, and the performance may be lowered in terms of rate-distortion evaluation.

Considering that various frequencies are more important than others, one alternative approach would take this observation into account to improve performance. Instead of using a constant f value for all transform coefficients, various values are taken into account, especially in the matrix approach where each dead zone parameter is selected based on frequency location. Therefore, in this case, Z can be calculated as follows.
Z = int ({| W | + f (i, j) × (1 << q_bits)} >> qbits) · sgn (W)
Here, i and j correspond to the current column or row in the block transform coefficient. The array f can be determined by the type of slice or macroblock and the texture characteristics (distribution or edge information) of the current block. For example, if a block contains multiple edges or has low dispersion characteristics, artifacts from the dead zone processing process are more visible, so it is important not to capture them further. is there. On the other hand, blocks with high spatial activity can mask more artifacts and increase the dead zone without significantly affecting quality. The dead zone depends on whether the current block provides useful information for future blocks of the picture (i.e. whether any pixels in the current block are used to predict other pixels) (Depending on what is not done).

For example, when using a 4 × 4 transform, the following dead zone processing matrix can be used.
if (cur_slice_type _k == I_Slice)
{
if (MBvariance (k, i, j) <15 || MBvariance (k, i, j)> 60)

ｅｌｓｅ ｉｆ（ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＞＝１５＆＆ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＜＝４０｜｜ｃｏｎｔａｉｎｓ＿ｅｄｇｅｓ（ｋ，ｉ，ｊ））

else if (MBvariance (k, i, j)> = 15 && MBvariance (k, i, j) <= 40 || contains_edges (k, i, j))

ｅｌｓｅ

else

｝
ｅｌｓｅ ｉｆ（ｃｕｒ＿ｓｌｉｃｅ＿ｔｙｐｅ_ｋ＝＝Ｐ＿Ｓｌｉｃｅ）
｛
ｉｆ（ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＜１５｜｜ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＞６０）

}
else if (cur_slice_type _k == P_Slice)
{
if (MBvariance (k, i, j) <15 || MBvariance (k, i, j)> 60)

ｅｌｓｅ ｉｆ（ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＞＝１５＆＆ＭＢｖａｒｉａｎｃｅ（ｋ，ｉ，ｊ）＜＝４０｜｜ｃｏｎｔａｉｎｓ＿ｅｄｇｅｓ（ｋ，ｉ，ｊ））

else if (MBvariance (k, i, j)> = 15 && MBvariance (k, i, j) <= 40 || contains_edges (k, i, j))

ｅｌｓｅ

else

｝
ｅｌｓｅ／／Ｂ＿ｓｌｉｃｅｓ
｛

}
else / B_slices
{

｝

}

Under certain conditions, the encoder may be able to perform temporal analysis using future frames. In this case, temporal analysis can be performed by considering only previously coded pictures and assuming that future pictures have similar temporal characteristics. For example, when the current picture has a high similarity (for example, MAPD _{k, k−1} is small), it is assumed that the similarity (MAPD _{k, k + 1} ) with the next picture to be encoded is also small. Therefore, all the indices (k, k + 1) can be replaced with (k, k-1), and coding parameters can be adapted based on already available information.

  Referring now to FIG. 3, the entire video encoder is indicated by reference numeral 300. The input of the video encoder 300 is connected to the input of the pre-analysis block 310 by signal communication. The pre-analysis block 310 includes a plurality of frame delays 312 connected in signal communication with each other. The plurality of frame delays 312 are connected in series vertically and are also connected in parallel by a parallel signal path. This parallel signal path is also connected to the input of the temporal analyzer 315 by signal communication. The output of the last serially connected frame delay 312 farthest from the input of the encoder 300 is the input of the spatial analyzer 320, the inverting input of the first summing junction 325, the first of the motion compensator 375. An input and a first input of motion estimation / mode decision block 370 are connected in signal communication. The output of the first summing junction 325 is connected to the input of the converter 330 by signal communication. The output of the converter 330 is connected in signal communication to the first input of the quantizer 335. The output of the quantizer 335 is connected in signal communication to the first input of the variable length coder 340 and the input of the inverse quantizer 345. The output of the variable length coder 340 is an output that can be used outside the video encoder 300. The output of the inverse quantizer 345 is connected to the input of the inverse transformer 350 by signal communication. The output of the inverse converter 350 is connected in signal communication to the non-inverting first input of the second summing junction 355. The output of the second summing junction 355 is connected in signal communication to the first input of the loop filter 360. The output of the loop filter 360 is connected in signal communication to a first input of the picture reference storage device 365. The output of the picture reference store 365 is connected in signal communication to a second input of the motion estimation / mode determination block 370 and a second input of the motion compensator 375. A first output of motion estimation / mode decision block 370 is connected in signal communication to a second input of variable length coder 340. A second output of motion estimation / mode decision block 370 is connected in signal communication to a third input of motion compensator 375. The output of the motion compensator 375 is connected in signal communication to the non-inverting input of the first summing junction 325 and the non-inverting second input of the second summing junction 355. The first output of the spatial analyzer 320 is connected in signal communication to the second input of the quantizer 335. The second output of the spatial analyzer 320 is in signal communication with the second input of the loop filter 360, the third input of the motion estimation / mode decision block 370, and the non-inverting input of the first summing junction 325. Connected. A first output of temporal analyzer 315 is connected in signal communication to a second input of quantizer 335. A second output of temporal analyzer 315 is connected in signal communication to a fourth input of motion estimation / mode determination block 370. A third output of temporal analyzer 315 is connected in signal communication with a third input of loop filter 360 and a second input of picture reference store 365.

  The pictures are considered during the temporal analysis step. This temporal analysis step includes slice type determination, GOP structure, weighted parameters (by motion estimation / mode determination block 370), quantized values and dead zone processing (by quantizer 335), reference order and handling. Several parameters are determined, including (picture reference store 365), picture coding ordering, frame / field picture level adaptation decisions, and deblocking parameters (loop filter 360). Similarly, a spatial analysis is performed on each encoded frame. This spatial analysis is similarly quantized and dead zone processed (quantizer 335), Lagrange parameter and slice type determination (motion estimation / mode determination block 370), inter / intra mode determination, frame / field. Can affect picture level and macroblock level adaptation decisions, as well as deblocking (loop filter 360).

With reference now to FIG. 4, an exemplary process for encoding video signal data is indicated generally by the reference numeral 400. In this process, the same bitstream can be parsed or encoded multiple times while collecting and updating the necessary statistics for each iteration. These statistics are used in each subsequent pass to improve the encoding performance by adapting the encoder parameters given the video characteristics or user requirements. Specifically, k frames (ie, excluding unstored pictures) have L paths (also referred to herein as L “repeats” and “repeats”) and sizes (N, M ) Window. Here, N is the total number of frames in the window, and M is the number of frames that overlap between adjacent windows. The frame to be encoded is indexed using the variable frm and the current position in the window is indexed using the variable w _index .

The process includes a start block 405 that passes control to a function block 410. In function block 410, the sequence size is set to k, the number of iterations is set to L, the variable i is set to zero (0), and control is passed to function block 415. In function block 415, the window size is set to N, the overlap size is set to M, the variable frm is set to zero (0), and control is passed to function block 420. In function block 420, the variable w _index is set to zero (0) and control is passed to function block 425. Thus, it will be appreciated that the window parameters are initialized for each encoding pass. This also makes it possible to use different window sizes or to adapt different window sizes based on previous analysis steps (for example, if a scene change is detected, And N and M can be adjusted to include only complete scenes.)

The function block 425 performs temporal analysis of each window to be processed while considering all N frames in the window to generate temporal statistics (tstat _{i, frm... Frm + N−1} ). The pass or encoding step statistics are best adapted or refined using the current statistics. In function block 425, control is then passed to function block 430. In function block 430, spatial analysis is performed on the frame with index frm (w _index in the current window) until the condition of w _index <N−M is no longer satisfied, and control is passed to function block 435. Function block 435 encodes these frames based on the results of temporal and spatial analysis, generates / collects encoder statistics that may be needed if multiple passes are required, and controls Pass to block 440.

In function block 440, the values of the variables frm and w _index are incremented and control is passed to decision block 445. At decision block 445, it is determined whether the variable frm is less than k.

If the variable frm is less than k, control passes to a decision block 450 that determines whether w _index is less than (N−M). Otherwise, if the variable frm is greater than or equal to k, control passes to a decision block 455 that determines whether i is less than L.

If w _index is less than (N−M), control passes to function block 430. Otherwise, if w _index is equal to or greater than (N−M), control returns to function block 420.

  If i is greater than or equal to L, control returns to function block 415. Otherwise, if i is less than L, control passes to end block 460.

  In the following, some of the many further advantages / features of the present invention will be described by means of various exemplary embodiments of the present invention. For example, one advantage / feature is that an encoding apparatus and method that performs video analysis based on overlapping overlapping windows of content to be encoded and uses this information to adapt encoding parameters. Is to be provided. Another advantage / feature is the use of spatio-temporal analysis in video analysis. Yet another advantage / feature is to consider the precoding pass in video analysis. Yet another advantage / feature is that both spatio-temporal analysis and precoding passes are considered in video analysis. Another advantage / feature is that at least one of picture coding type, edge, mean and variance information is used for spatial analysis and adaptation of Lagrangian parameters, quantization and dead zone processing That is. Yet another advantage / feature is the use of absolute difference and absolute variance to perform quantization parameter adaptation. Yet another advantage / feature is that only the previously encoded pictures are considered in the performed video analysis. Furthermore, another advantage / feature is the use of the video analysis performed to determine slice type, GOP and picture coding structure and order, weighted parameters, quantized values and dead zone processing, Lagrangian parameters , Reference number, reference order and handling, frame / field picture and macroblock determination, deblocking parameters, inter block sizing, intra spatial prediction, and direct mode Determining at least one of the encoding parameters. Another advantage / feature is that video analysis can be performed in multiple iterations, taking into account previously generated statistics to adapt to encoding parameters or analysis statistics. Furthermore, another advantage / feature is that the window size and overlapping window regions can be adapted based on previously generated analysis statistics.

  These and other features and advantages of the present invention can be readily ascertained by those skilled in the art based on the teachings herein. It should be understood that the teachings of the present invention can be implemented in various forms of hardware, software, firmware, special purpose processors, or combinations thereof.

  More preferably, the teachings of the present invention are implemented as a combination of hardware and software. Furthermore, the software is preferably implemented as an application program tangibly implemented on the program storage device. The application program can be uploaded and executed on a machine having any suitable architecture. The machine is implemented on a computer platform having hardware such as one or more central processing units (“CPU”), random access memory (“RAM”) and input / output (“I / O”) interfaces. It is preferred that The computer platform may also include an operating system and microinstruction code. The various processes and functions described herein may be part of the microinstruction code or part of the application program or any combination thereof that can be executed by the CPU. In addition, various other peripheral devices may be connected to the computer platform such as an additional data storage device and a printing device.

  Furthermore, some of the system components and methods shown in the accompanying drawings are preferably implemented in software, and the actual connections between system components or process functional blocks will vary depending on the method of implementing the invention. Please understand that there is. Given the teachings herein, one of ordinary skill in the related art will be able to contemplate these and similar embodiments or configurations of the present invention.

  Although exemplary embodiments have been described herein with reference to the accompanying drawings, the present invention is not limited to these specific embodiments, and those skilled in the art will depart from the spirit or scope of the present invention. It should be understood that various changes and modifications can be made without doing so. All such changes and modifications are intended to be included within the scope of the present invention as set forth in the appended claims.

FIG. 2 is a block diagram illustrating an exemplary window-based two-pass encoding architecture according to the principles of the present invention. FIG. 6 is a plot of the effects of dead zone processing during conversion and quantization according to the principles of the present invention. 1 is a block diagram illustrating an encoder according to the principles of the present invention. 5 is a flow diagram illustrating an exemplary encoding process according to the principles of the present invention.

Claims (24)

An encoder that encodes video signal data corresponding to a plurality of pictures,
Performing video analysis of the video signal data using a plurality of overlap analysis windows for at least some of the plurality of pictures corresponding to the video signal data, and encoding the video signal data based on a result of the video analysis The encoder comprising an overlapping window analysis unit (310) that performs parameter adaptation.   The encoder of claim 1, wherein the overlapping window analysis unit (310) performs the video analysis of the video signal data using a spatiotemporal analysis.   The overlapping window analysis unit (310) may perform at least one of picture coding type information, edge information, average information and variance information on spatio-temporal analysis, adaptation of Lagrangian parameters and quantization parameters, and dead The encoder according to claim 2, used for at least one of zone processing.   The encoder according to claim 3, wherein the overlap window analysis unit (310) adapts the quantization parameter using absolute difference and variance.   The encoder of claim 1, wherein the overlap window analysis unit (310) performs the video analysis of the video signal data using a pre-encoding pass.   The encoder of claim 1, wherein the overlap window analysis unit (310) performs the video analysis of the video signal data using both a spatio-temporal analysis and a pre-encoding pass.   The overlapping window analysis unit (310) may perform at least one of picture coding type information, edge information, average information and variance information on spatio-temporal analysis, adaptation of Lagrangian parameters and quantization parameters, and dead The encoder according to claim 6, which is used for at least one of zone processing.   The encoder according to claim 7, wherein the overlap window analysis unit (310) performs quantization parameter adaptation using absolute difference and variance.   The video signal data includes a plurality of frames, each of the plurality of frames representing a corresponding picture, and the duplicate analysis unit performs the video analysis so as to consider only previously encoded pictures. The encoder according to item 1.   The coding parameters include slice type, picture and GOP coding structure and order, weighted parameters, quantized values and dead zone processing, Lagrange parameters, reference number, reference order and handling, frame / field The encoder of claim 1, comprising at least one of picture and macroblock parameters, deblocking parameters, inter block size, intra spatial prediction, and direct mode.   The overlapping window analysis unit (310) performs the video analysis in a plurality of iterations and adapts one of the encoding parameters and analysis statistics based on previously generated analysis statistics. The encoder according to 1.   Each of the overlapping windows has a window size of P pictures and an associated overlapping size, and the overlapping window analysis unit is configured to determine the window size and the window size based on previously generated analysis statistics. The encoder according to claim 1, wherein the overlap size is adapted. A method for encoding video signal data corresponding to a plurality of pictures, comprising:
Performing video analysis of the video signal data using a plurality of overlap analysis windows for at least some of the plurality of pictures corresponding to the video signal data (425, 430);
Adapting coding parameters of the video signal data based on the result of the video analysis (435).   The method of claim 13, wherein the performing step performs the video analysis of the video signal data using spatio-temporal analysis.   In the execution step and the adaptation step, respectively, at least one of picture coding type information, edge information, average information and variance information is applied to spatiotemporal analysis, adaptation of Lagrangian parameters and quantization parameters, and The method of claim 14 for use in at least one of dead zone processing.   The method of claim 15, wherein the quantization parameter adaptation is performed using absolute difference and variance.   The method of claim 13, wherein the performing step performs the video analysis of the video signal data using a pre-encoding pass.   14. The method of claim 13, wherein the performing step performs the video analysis of the video signal data using both a spatiotemporal analysis and a pre-encoding pass.   The executing step and the adapting step each of at least one of picture coding type information, edge information, average information and variance information, spatio-temporal analysis, adaptation of Lagrangian parameters and quantization parameters, and The method of claim 18 for use in at least one of dead zone processing.   The method of claim 19, wherein the quantization parameter adaptation is performed using absolute difference and variance.   The video signal data includes a plurality of frames, each of the plurality of frames representing a corresponding picture, and the performing step performs the video analysis to consider only previously encoded pictures. 14. The method according to 13.   The coding parameters include slice type, picture and GOP coding structure and order, weighted parameters, quantized values and dead zone processing, Lagrange parameters, reference number, reference order and handling, frame / field 14. The method of claim 13, comprising at least one of picture and macroblock parameters, deblocking parameters, inter block size, intra spatial prediction, and direct mode.   Performing the video analysis in a plurality of iterations in the execution step, and adapting one of the encoding parameters and the analysis statistics based on the previously generated analysis statistics in the adaptation step. Item 14. The method according to Item 13.   Each of the overlapping windows has a window size and an associated overlapping size, and the executing step adapts the window size and the overlapping size based on previously generated analysis statistics 14. The method of claim 13, comprising:

Images