CN1347620A - Method and architecture for converting MPE G-2 4:2:2-profile bitstreams into main-profile bitstreams - Google Patents

Abstract

A system for converting the color format of a digital video bitstream. The system accounts for the allowable formats of the pre- and-post-conversion bitstreams, including quantizer precision level, and whether luma and chroma data have separate quantization matrices, or share a common quantization matrix. In a particular implementation, an MPEG-2 4:2:2 P bitstream having a color format of 4:2:2 or 4:2:0 is converted to a MP bitstream having a color format of 4:2:0. Coding efficiencies are achieved by using the luma quantization matrix to re-quantize the chroma data, and re-using luma motion vectors for performing motion compensation of the chroma data. Further efficiencies can be achieved by representing a 4:2:2 reference picture in a 4:2:0 format for converting inter coded frames, and changing the position of a pixel downsizing filter and clip function. Adjustment of the quantization precision is provided as required. A transcoding function can also be achieved.

Description

Conversion MPEG-2 4:2:2 one profile bit stream is method and the framework of main outline bit stream

Background of invention

The invention provides a kind of system of converting digital video bit stream form, being particularly suitable for the MPEG-2 digital of digital video data is the main outline form from 4:2:2 profile format conversion.

Altogether use following a few abbreviation herein:

The 4:2:2P-4:2:2 profile;

CBP-encoding block pattern;

DCT-discrete cosine transform;

HDTV-high-definition television;

HL-high-grade (level);

IDCT-reverse DCT;

MB-macro block;

MC-motion compensation;

ML-main grade;

MP-main outline;

MV-motion vector;

QDC-quantification direct current;

SDTV-standard definition television;

VBV-screen buffer verifier;

VLC-variable length coder; And

VLD-variable-length decoder.

Many pictorial elements (also being the pixel of just knowing) definable goes out video image, and pixel is the least member of raster scanning line in the image, and it has relevant color space. For example: in the YCrCb color space, Y is the brightness composition, and Cr and Cb are the aberration compositions. There are many sampling forms to define, comprise 4:4:4,4:2:2 and 4:2:0, for example with regard to the 4:2:2 form, macro block has four 8 * 8Y pieces, two 8 * 8Cr pieces and two 8 * 8Cb pieces, with regard to this form, the sampling frequency of Y, Cr and Cb composition is respectively 13.5MHz, 6.75MHz and 6.75MHz.

With the 4:2:0 form, macro block has four 8 * 8Y pieces, 8 * 8Cr piece and 8 * 8Cb piece.

Moreover, developed now manyly for the digital video coding standard to coding video data, comprise defined Moving Picture Experts Group-2 in (especially) nineteen ninety-five ISO/IEC 13818-2 MPEG-2 video (ITU-RH.262).

MPEG-2 indicates the many groups of constrained parameters that use two-dimensionses to put in order, and wherein a dimension is called " profile " series, points out the coding characteristic supported, and other dimensions are called " grade ", points out the open ended compositions such as resolution, bit rate of image. Most important profile tier group is collectively referred to as " main outline on the main grade " or MP@ML, and MP@ML supports 4:2:0 color subsample ratio.

Except main outline, also have other profiles, for example: the sampling profile is similar to main outline but does not have the B image, and the scalable profile of SNR then adds the scalable ability of SNR to main outline, and high profile then adds 4:2:2 color format capability to the spatial scalable profile.

When the sampling tier definition was used for standard form placed in the middle (SIF) video, main grade just defined the video for CCIR601. In addition, two higher levels of high-definition television are that height-1440 grade and the every row that every row mostly is 1440 pixels most mostly is most the high-grade of 1920 pixels.

Further, developed recently the 4:2:2 profile (being also referred to as 4:2:2P) that can hold 4:2:2 and 4:2:0 color form. See also " the MPEG-2 4:2:2 profile on high-grade " of " the MPEG-24:2:2 profile on the main grade " that in January, 1996, ISO/IEC13818-2 the 2nd revised, ANSI/SMPTE 308M TV SMPTE standard in 1997 and " the 4:2:2 profile on high-grade " that in March, 1998, ISO/IEC13818-2 proposed draft the 5th was revised, all be incorporated herein by reference at this.

The 4:2:2 profile is specialized in the usefulness of professional video application, wherein requires easily to carry out the editor of compressed video and the multiple generation coding/decoding of video. The main application target of this profile is:

Store

The editor of visual effect and generation

Video-tape or disc recording for specialty (special-purpose quality)

The film studio post-production of high-quality video sequence

The storage of special-purpose quality video and effective transmission of distribution

4:2:2P can provide higher video quality, better colourity resolution and allow high-order in the bit rate (on main grade (ML), the highest 50Mbit/s) of main outline (namely being MP@ML). Especially 4:2:2P provides minute other quantization matrix of brightness and chroma data. In the application in film studio, for being easy to chroma key and other special efficacys, usually need very high-quality video and ITU-R 601 4:2:2 video formats. Because the demand that is easy to edit needs more frequently INTRA (interior) image, also caused high Coding Rate. 4:2:2P allows all I Image Codings, so can be fast recovers from error of transmission and can simplify editing application.

Can both provide the image of good merchantable brand matter more than the MP with same hue form even have the 4:2:2P of 4:2:0 color form, this is because 4:2:2P allows higher quantification accuracy.

4:2:2P further allows to keep high-quality required high bit rate when only using the I Image Coding, and 4:2:2P also allows to use P and B Image Coding type, the bit rate that it can further improve quality or reduce same quality.

Further, play in the chain at traditional TV, video will be through too much for coding/decoding, and local TV station can (for example passing through cable television network) can add homemade TV programme and advertisement therein before video is broadcast to the man of rating family. Moreover with simulated television, the generation of multiple encoding/decoding can cause significant image quality loss. In other words, 4:2:2P can keep high-quality after the generation of multiple encoding/decoding. Do not having in the situation of image operation control or Image Coding type change between each generation through too much generation, the quality of 4:2:2P encoder also remains with near constant quality after the first generation. Use manipulated image or change the Image Coding type to cause quality a bit deteriorated between each generation, however, the quality that obtains is acceptable for many application.

The telephotography canopy can produce " special-purpose quality " (contribution quality) video (normally ITJ-R601 source video or 4:2:2P ML) or 4:2:2P HL compressed video usually, then can encode and transmits this video as " distribution quality " (distribution quality) TV. The DTV of North America and HDTV have used MPEG-2 MP@ML and MP@HL as its video encoding standard, and MP@ML and MP@HL coding and transmission all are the distribution quality video of 4:2:0 form. The quantization matrix that MP provides brightness and chroma data to share. For example: the intensive data rate that comprises the HDTV system of compressed video, compressed audio, situation access and auxiliary data channel approximately is 18 to 20Mbit/s.

Therefore, a kind of efficient system need to be arranged is MP distribution quality bit stream with the special-purpose quality bit stream translation of 4:2:2P precommpression.

If reduce the complexity of coding, this system should allow to reuse motion vector, avoids changing to the MB type of coding, and only processes chrominance block with a kind of mc unit. This system should further provide can be in the remaining chrominance block of INTER (centre) coding MB is carried out ten gets the ability that one (decimate) filtering is processed.

If be presented in the 4:2:2P bit stream, native system should provide a kind of simple method to replace the chromaticity quantization matrix during format conversion.

The CBP of chrominance block should make with extra care in this system.

System also should provide the adjustment of MPEG DC coefficient accuracy variable i ntra_dc_precision, abides by the requirement that meets employed coding standard. For example, the maximum intra_dc_precision grade that allows of MP bit stream will be lower than the grade that the 4:2:2P bit stream allows.

This system should provide 4:2:2 to 4:2:0 converter and normal code converter of a kind of combination, 4:2:2P to the MP code conversion that has format conversion with execution.

This system needing should to avoid the complete decoding of bit stream that will process during format conversion.

The invention provides a system with above-mentioned and other advantage.

Summary of the invention

The present invention is relevant with the form of converting digital video bit stream, and for example being used for the special-purpose quality bit stream translation of 4:2:2P precommpression is MP distribution quality bit stream.

The forward transposition circulation that will have the first form is changed to the method for the rear conversion bit stream with second form, comprise step: this forward of decompressing at least in part transposition stream is to recover the chroma data in the pixel domain wherein, change the quantization matrix data that the bit stream recovery is associated with brightness data in the past, and again compress the data corresponding to the chroma data that recovers. Again compression comprises the data re-quantization corresponding to the chroma data that recovers according to the luminance quantization matrix that recovers, so that rear conversion bit stream to be provided.

Current conversion bit stream comprises the in-line coding image, and the chroma data that recovers has the first chroma format corresponding to the first form of forward transposition stream, the method comprises further step: change the motion vector that the bit stream recovery is associated with brightness data in the past, use the brightness movement vector of recovery with the first motion compensation process of the recovery chroma data of execution in-line coding image, the first motion compensation process is then used the first chroma format, after the first motion compensation process, filter chroma data, the chroma data that has corresponding to the second chroma format of the second form of rear conversion bit stream is provided, and carry out the second motion compensation process of the chroma data with second chroma format with the brightness movement vector that recovers, with the data that provide again compression step to use.

With regard to the second chroma format, a chrominance block is provided for per at least two chrominance block (namely being two piece and pieces that are used for 4:2:0 that are used for 4:2:2) that adopt the first chroma format.

The data corresponding to the chroma data with second chroma format that provided by filter step are converted into transform domain from the pixel field, then quantize, then inverse quantization and the inverse transformation data to provide the second motion compensation process to use.

The method can comprise changes the further step that bit stream recovers the first quantified precision grade in the past, and if the first quantified precision grade can allow accuracy class greater than the maximum of rear conversion bit stream the second form: (i) reduce the first quantified precision grade; (ii) change bit stream in the past recover DC luminance transformation data; And (iii) will be corresponding to the data re-quantization of the DC luminance transformation data of recovering according to the quantified precision grade that reduces.

In addition, current conversion bit stream comprises the image of in-line coding, and the chroma data that recovers is when having the first chroma format corresponding to the first form of forward transposition stream, and the method comprises further step; Filter and recover chroma data so that the chroma data of the residue with second chroma format to be provided, this second chroma format is corresponding to the second form of rear conversion bit stream, change the motion vector that the bit stream recovery is associated with brightness data in the past, and the motion compensation process of using the brightness movement vector executing data that recovers, these data correspond to the residue chroma data of in-line coding image, and data are offered again compression step. At this, motion compensation process is used the second chroma format, so only need a motion compensator. The mistake of this importing is usually also uncertain, because the colourity mistake is wanted difficult detect many of specific luminance mistake in the image of gained.

The first form is MPEG4:2:2 profile form, and the second form is MPEG main outline form.

The piece pattern of the coding of forward transposition stream is changed bit stream after can be used for through modification.

Moreover the forward transposition is flowed and is changed afterwards bit stream and can use identical macroblock encoding type.

Further, the re-quantization step can be in response to the speed control signal of conversion bit stream bit rate after setting.

The luminance quantization matrix that recovers can be through revising the code efficiency with conversion bit stream after promoting.

Also provide a kind of device of correspondence at this.

Brief description of drawings

Fig. 1 explanation is converted to 4:2:2P according to the present invention the first method of MP bit stream.

Fig. 2 explanation is converted to 4:2:2P according to the present invention the second method of MP bit stream.

Fig. 3 explanation is according to the chrominance block bit stream translation of the present invention from 4:2:2P 4:2:2 bit stream to the MP4:2:0 bit stream.

Fig. 4 explanation is according to the simplification chrominance block bit stream translation of the present invention from 4:2:2P 4:2:2 bit stream to MP 4:2:0 bit stream.

Fig. 5 explanation is according to the rough chrominance block bit stream translation of the present invention from 4:2:2P 4:2:2 bit stream to MP 4:2:0 bit stream.

Fig. 6 explanation is according to the chrominance block bit stream translation of the present invention from 4:2:2P 4:2:0 bit stream to MP 4:2:0 bit stream.

Fig. 7 explanation is according to the present invention's chrominance block bit stream translation from 4:2:2P 4:2:2 or 4:2:0 bit stream to MP 4:2:0 bit stream when the intra_dc_precision=11.

Fig. 8 explanation is according to the combination of 4:2:2 of the present invention to 4:2:0 converter and code converter.

Detailed description of the present invention

The present invention is relevant with the method that with the special-purpose quality bit stream translation of compression before the 4:2:2P is MP distribution quality bit stream.

In general, the MPEG-2 code converter be a kind of be the instrument of another MPEG-2 bit stream of new speed with front compression MPEG-2 bit stream translation, be 4:2:2 to the 4:2:0 converter that the code converter of MP@ML (MP@HL) bit stream can be the 4:2:2P of cascade (@ML or@HL) or MP code converter and has speed control with MPEG-2 4:2:2P@ML (or 4:2:2P@HL) bit stream translation. The following code change over condition is significant especially: 4:2:2P@ML to MP@ML, 4:2:2P@HL to MP@ML and 4:2:2P@HL to MP@HL. Code conversion from HL to ML is very general, namely is to become the SDTV form from the HDTV form. But the code conversion from ML to HL is usually not meaningful, because can cause high bit rate still also can't improve image quality. If will obtain higher resolution from the ML bit stream, then can be with bitstream decoding, and come magnified image with the post processing interpolation method.

As illustrated in fig. 1 and 2, have two kinds of diverse ways and can realize that 4:2:2P is to the code conversion of MP.

In figure, the element of numeral number corresponds to each other.

Fig. 1 explanation is according to the first method of 4:2:2P to MP bit stream of the present invention code conversion, at this when input during precompressed 4:2:2P bit stream, then can partly decode 4:2:2 bit stream and reconfigure to provide the MP bit stream with the result of 4:2:2 to 4:2:0 bit stream translation device 110 produces the new MP compression position flow of new speed with traditional 4:2:0 code converter 120. Rate control feature element (function) 130 can be controlled the bit stream rate that code converter 120 is exported.

Fig. 2 explanation is converted to 4:2:2P according to the present invention the second method of MP bit stream, at this when output during precompressed 4:2:2P bit stream, 4:2:2P code converter 210 and 4:2:2 to a 4:2:0 bit stream translation device 220 phase cascades, the MP compression position flow of the new speed of generation under the control of rate control feature element 230.

With regard to speed control, please pay special attention to the dominant bit quantity in each macro block because in the 4:2:0 chroma data macro block dominant bit quantity on be limited to 4,608, but for 4; The macro block of 2:2P 4:2:2 chroma format is then unrestricted. The present invention is placed on 4:2:2 to 4:2:0 bit stream translation device 110,220 with focus.

4:2:2P@ML form is the extension of MP@ML form in many aspects, and form 1 is interior has summed up crucial ML feature, and same 4:2:2P@HL form is the extension of MP@HL form in many aspects, and form 2 is interior has summed up crucial HL feature.

	4：2：2P@ML	MP@ML
			Chroma format	4:2:2 or 4:2:0	4：2：0
Profile and recovery identification	10000101	1001000
			Repeat first	Be limited the ML such as MP@, have two extra restrictions for the image that surpasses 512 scan lines: if frame per second is 25Hz, then the B image should be unable to repeat first.	Form 8-7 in the MPEG-2 image
The internal DC precision	8,9,10 or 11	8,9 or 10
			The upper limit of sampling density:	720 sampling/scan lines, 608 scan lines/frame, 30 frame/second (608 scan lines of 512 scan lines of 30Hz/frame, 25Hz/frame	720 sampling/scan lines, 576 scan lines/frame, 30 frame/seconds
The upper limit of brightness sampling rate:	11,059,200 pixel/seconds	10,368,000 pixel/second
			The upper limit of bit rate:	50MB/ second	15MB/ second
Maximum vbv buffer size	9,437,184	1,835,008
			Quantization table	Divide other brightness and chromaticity quantization table	Brightness and colourity are same table
The bit quantity of maximum in the macro block	4:2:2 is unrestricted, and 4:2:0 then is 4,608	4,608

Table 2-4:2:2P@HL and MP@HL

	4：2：2P@HL	MP@HL
			Chroma format	4:2:2 or 4:2:0	4：2：0
Profile and grade identification	10000010	1000100
			Repeat first	Be limited the HL such as MP@, have two extra restrictions for the image that surpasses 1152 scan lines: if frame per second is 25Hz, then the B image should be unable to repeat first.	Form 8-7 in the MPEG-2 image
The internal DC precision	8,9,10 or 11	8,9 or 10
			The upper limit of sampling density:	1920 sampling/scan lines, 1152 scan lines/frame, 60 frame/seconds	1920 sampling/scan lines, 1152 scan lines/frame, 60 frame/seconds
The upper limit of brightness sampling rate:	62,668,800 pixel/seconds	62,668,800 pixel/seconds
			The upper limit of bit rate:	300MB/ second	80MB/ second
Maximum vbv buffer size	47,185,920	9,781,248
			Quantization table	Divide other brightness and chromaticity quantization table	Brightness and colourity are same table
The bit quantity of maximum in the macro block	Unrestrictedly	4,608

Do not change the coded macroblocks type in the 4:2:2P bit stream, the difference between form 1 and 2 interior pointed 4:2:2P and the MP can cause may changing of form 3 interior listed grammers.

Note that many difference characteristics of the macro block (mb) type indication macro block that the term macroblock_type of VLC coding gives comprise:

Whether there is quant_scale_code (macroblock_quant) in the bit stream,

Whether use moves forwards or backwards compensation (being respectively macroblock_motion-forward and macroblock_motion_backward), whether has coded_block_pattern (macrobleock_pattern) in the bit stream,

Macro block is (macroblock_intra) of in-line coding whether, the spatial_temporal_weight_classes that whether has spatial_temporal_weight_code (spatial_temporal_weight_code_flag) in the bit stream and allow.

The grammer change that form 3-is possible

The element of grammer change	Grammer change action
		A	In Sequnce_header (), reset vbv_buffer_size_vlaue=1,835,008@ML (9,781,248@HL) and reset bit_rate_value in Sequence_extenion (), reset profile_ard_level_indication=01001000@ML (01000100@HL).
In Picture_header (), recomputate and reset vbv_delay.
	B		In Sequence_extension (), reset chroma_format=" 4:2:0 "
C		In Picture_coding_extension (), reset intra_dc_precision=8,9 or 10 (able to programme) setting chroma_420_type=progressive_frame;
	D		In Quant_matrix_extension (), reset load_chroma_intra_quantiser_matrix=0 and remove chroma_intra_quantiser_matrix[64]
E		In Quant_matrix_extension (), reset load_chroma_non_intra_quantiser_matrix=0 and remove chroma_non_intra_quantiser_matrix[64]
	F		Change the encoding block pattern of each macro block.

Can further recognize form 3 interior disclosed grammer change actions with reference to " Sequence header (sequence header) ", " Sequence extension (sequence extension) ", " Picture Header (visual head) ", " Pictute coding extension (picture coding expansion) " and " Quant matrix extension (quantitatively matrix-expand) " that Moving Picture Experts Group-2 is announced.

If note that original intra_dc_precision=11 position, so in order to be reset to 8,9 or 10 (seeing also form 1 and 2) with MP is compatible mutually.

MPEG parameter intra_dc_precision is 2 integers, its be defined have binary numeral 00,01,10 or 11, to specify respectively 8,9,10 or 11 accuracy. Parameter intra_dc_precision points out the value intra_dc_mult for multiplexer, is used for the inverse quantization of the DC coefficient of in-line coding piece. In other words, intra_dc_mult=8,4,2 or 1 is respectively applied to 00,01,10 or 11 intra_dc_precision value.

In general for inverse quantization, the two-dimensional array QF[v of coefficient] [u] be reversed and quantize the DCT coefficient rebuild to produce, and this processes and is multiplied by in essence quantiser step size. Have two mechanisms and can revise quantiser step size, in other words be exactly to change step-length in one with a weighting matrix, and the usage ratio factor, so only need consume some positions just can revise step-length (with the coding of whole new weighting matrix by comparison).

In the bit stream of wish conversion, use quantiser_scale_code, unsigned nonzero integer (scope from 1 to 31 is pointed out quantization scaling factor) and weighting matrix W[w] [v] [u], can carry out suitable inverse quantization at conversion coefficient and calculate. Coefficient F as a result " can produce F ' [v] [u] after [v] [u] is saturated, then can carry out the control operation that mismatches and draw last reconstruction DCT coefficient F[v] [u].

The 4:2:0 data use two kinds of weighting matrixs-to be used for intra-macroblock (brightness and colourity), and another uses for non-intra-macroblock (brightness and colourity). For the 4:2:2 data, altogether use four kinds of matrixes (inner brightness, inner colourity, non-inner brightness and non-inner colourity), so brightness can be used different matrixes with chroma data. Each matrix has the default values collection, just can override this set of values and utilize the download user to define matrix.

For weighting matrix W[w] for [v] [u], w gets 0 to 3 the employed matrix of numeric representation. The rule of the selection of form 4 blanket management w.

The selection of form 4-w

	4：2：0		4：2：2
						Brightness	Colourity	Brightness	Colourity
Internal block (maeroblock_intra=1)	0	0	0	2
					Non-internal block (macroblock_intra=0)	1	1	1	3

For internal block, quantizing dct transform coefficient QF[0] the two-dimensional array DC numerical value of [0] can process in the mode different with AC numerical value. DC numerical value can inverse quantization producing dct transform DC coefficient arrays numerical value F ' ' [0] [0], according to relation:

F’’[0][0]＝intra_dc_mult×QF[0][0]。

Moreover, can recomputate vbv_delay from vbv_delay (n)=Bd (n)/R, wherein Bd (n) is decoded picture n decoder (VBV) buffering area spill-over degree before, and R is bit rate. In general, vbv_delay and vbv_buffer_size reset and require to quantize a feedback control loop, avoid overflow or the not enough stream of VB buffering area and actual speed rate buffering area.

The chroma data of re-quantization can determine the new encoding block pattern (CBP) of a MB chrominance block in the MP bit stream. CBP indicates the encoding block quantity of each macro block, namely is that 4:2:2 color form has eight encoding blocks, and perhaps 4:2:0 color form has six encoding blocks. For example for 4:2:2 to 4:2:0 conversion, grammer change F may involve and change CBP pointing out to exist two chrominance block rather than four, the quantity of luminance block then do not change (namely being four pieces).

If carry out the conversion of 4:2:2P to MP, then need to analyze following crucial syntactic element from the 4:2:2P bit stream:

a＝(intra_dc_precision＝11)?1：0；

b＝(chroma_format＝“4：2：2”)?1：0；

C=(load_chroma_intra_quantiser_matrix=1) 1:0; And

d＝(1oad_chroma_non_intra_quantiser_matrix＝1)?1：0。

Above statement uses the false code syntax of C language sample, for example:

" a=(intra_dc_precision=11) 1:0 " if expression " intra_dc_precision=11 " is true, " a " designated value " 1 " then, otherwise " a " designated value " 0 "

As above mentioned, intra_dc_precision must have 8,9 or 10 numerical value for MP, if therefore intra_dc_precison=11 (decade 11 or 1110) then must reset.

Proprietary term cbroma_format is two integers, the brightness form of the bit stream of indication wish conversion. In other words, chroma_format=01 (binary system) indication 4:2:0 form, and chroma_format=10 indication 4:2:2 form. Remember that the 4:2:2P bit stream can hold 4:2:2 color form (b=1) and 4:2:0 color form (b=0). 4; The 4:2:0 form of 2:2P also is important situation. This and ML are different, even both have identical resolution, because with 4:2:2P, brightness all can have different quantization matrixes with colourity. In comparison, with ML, brightness is all used identical quantization matrix with colourity. Experiment shows to encode with the brightness that separates and chrominance matrix can provide quality better image.

Proprietary term load_chroma_intra_quantiser_matrix is a bit flag, is set as " 1 " if chroma_intra_quantiser_matrix follows then, if be set as " 0 ", then employed numerical value there is no change. If chroma_formt is " 4:2:0 " (namely being 10), then sign can peek value " 0 ".

Proprietary term chroma_intra_quantiser_matrix is the integer inventory of unsigning for 8 of 64 non-zeros that quantize inner chromaticity transformation value.

Proprietary term load_chroma_non_intra_quantiser_matrix is a flag, is set as " 1 " if chroma_non_intra_quantiser_matrix follows then, if be set as " 0 ", then employed numerical value there is no change. If chroma_format is " 4:2:0 " (namely being 01), then flag can peek value " 0 ".

Proprietary term chroma_non_intra_quantiser_matrix is 8 of 64 non-zeros that are used for quantizing inner chroma conversion value symbolism integer inventory not.

16 kinds of possible situations in the form 5 definition conversion process, for example: for situation=0, abcd=0000 (namely being a=0, b=0, c=0 and d=0). For the option of using the Q matrix, it determines 4:2:2P to MP code conversion/conversion by the bit stream that detect to receive separative Q matrix for colourity whether. If so, then replace chrominance matrix with luminance matrix.

The situation that the change of form 5-grammer and texture transformation are processed

Situation:	A bcd：	Grammer change (seeing also form 3)	Conversion process	Option
					0：	0000	A	Vbv_delay and vbv_size change
1：	0001	A、E	Fig. 6	Use chroma_intra_quantiser_matrix
					2：	0010	A、D	Fig. 6	Use chroma_non_intra_quantiser_matrix
3：	0011	A、D、E	Fig. 6
					4：	0100	A、B、F	Fig. 3-5
5：	0101	A、B、E、F	Fig. 3-5	Use chroma_intra_quantiser_matrix
					6：	0110	A、B、D、F	Fig. 3-5	Use chroma_non_intra_quantiser_matrix
7：	0111	A、B、D、E、F	Fig. 3-5
					8：	1000	A、C	Fig. 6,7	Change intra_dc_precision
9：	1001	A、C、E	Fig. 6,7	Change intra_dc_precision uses chroma_intra_quantiser_matrix
					10：	1010	A、C、D	Fig. 6,7	Change intra_dc_precision uses chroma_non_intra_quantiser_matrix
11：	1011	A、C、D、 E	Fig. 6,7	Change intra_dc_precision
					12：	1100	A、B、C、F	Fig. 3-5,7	Change intra_dc_precision
13：	1101	A、B、C、E、F	Fig. 3-5,7	Change intra_dc_precision uses chroma_non_intra_quantiser_matrix
					14：	1110	A、B、C、D、F	Fig. 3-5,7	Change intra_dc-precision uses chroma_non_intra-quantiser_matrix
15：	1111	A、B、C、D、 E、F	Fig. 1,2,4,5,7	Change intra_dc_precision

In processing according to the texture transformation of format converter of the present invention, if intra_dc_precision ≠ 11, then only have luminance block to need recompile, and do not need to carry out locomotion evaluation, because the motion vector of having been determined by luminance block is used to the recompile chrominance block.

For the situation of intra_dc_precision=11 position in the 4:2:2P bit stream, the re-quantization of internal DC T DC luminance elements is required for conversion process (re-quantization that also has the colourity element). If simplify transfer process, the MP bit stream can be reused motion vector, it is 8,9 or 10 because intra_dc_precidion is resettable, so between the accuracy of the motion vector that quantization and the reservation of saving internal DC T DC element are reused, will draw on the strong points to offset the weaknesses, therefore the method for adjusting intra_dc_precision can be set default intra_dc_precision=10 (threshold limit value of 4:2:0MP), but also can be rewritten as 8 or 9 according to user's wish.

For higher accuracy, just need more position during transmission. Moreover both full-pixel and half-pix motion vector all can use once again. Because half-pix MV can originate from the image of reconstruction at least, and the change of intra_dc_precision can produce mistake (namely be skew), so this change is down to minimum possible 4:2:2P to the MP code conversion mistake that reduces during format conversion.

If re-quantization is processed and is employed, and will reduce the quantity of bits of coded, then can use intra_dc_precision=8 in format conversion.

If colourity Q matrix (chroma_intra_quantiser_matrix and/or chroma_non_intra_quantiser_matrix) is employed in the 4:2:2P bit stream, then brightness Q matrix (luma_intra_quantiser_matrix and/or luma_non_intra_quantiser_matrix) can be employed in the re-quantization of chrominance block according to the present invention.

When quantization matrix (Q matrix) was provided at format converter, they can be encoded in bit stream with a kind of scanning sequency, and converted the two-dimentional weighting matrix W[w that uses to as top distribution in inverse quantizer] u[] [v].

Fig. 3 explanation is according to the chrominance block bit stream translation of the present invention from the 4:2:2P bit stream to the MP bit stream. Fig. 3 is provided for two (possibility) chrominance block in the 4:2:2P bit stream (being forward transposition stream) are converted to the complete block diagram of the converter of a chrominance block in the MP bit stream (namely being from rear conversion bit stream). Converter 300 receives the 4:2:2P bit stream in VLD function element 305, and this function element provides the MV data to be respectively applied to the first and second chrominance motion compensation function element, colourity MC (1) 320 and colourity MC (2).

About the inside of receive data or the information of middle model state can offer switch 325,327 and 370.

With regard to the internal schema data, switch 325 will start to transmit a null value " 0 " signal to adder 330, switch 327 can start to transmit a null value " 0 " signal to adder 345, and switch 370 will start to transmit a null value " 0 " signal to adder 367. With regard to the middle model data, switch 325 can start to be passed to adder 330 with reference to image data from colourity MC (1) function element 320, and switch 327 can start to be passed to adder 345 with reference to image data from colourity MC (2) function element 325.

Colourity MC (1) function element 320 is the motion compensation units for the 4:2:2 chroma data, and colourity MC (2) function element 325 is the motion compensation units for the 4:2:0 chroma data, according to the present invention, use the brightness MV data motion compensation of the 4:2:2P colourity 4:2:2P data on colourity MC (1) function element 320 (and also be used for) in the motion compensation meeting of the 4:2:0 colourity MP data on colourity MC (2) function element 325, this can be avoided 4:2:0 colourity MP data to need other locomotion evaluation to process, and is included in the reference frame and searches.

Comprise quantization transform coefficient data from present image (being frame) from the data of VLD function element 305 output, these data can provide to inverse quantizer Q-11310, and provide to IDCT function element 315. The pixel domain data can offer adder 330 as a result, and the output of adder 330 can offer slicing (clip) function element 335, (namely being the data of scope [0,255] between slicing minimum and the greatest measure when needed.

The data of slicing can offer wave filter 340, and it comprises the sampling filter vertically downward based on the vertical filter of field and 2:1. For example 16,0,79,130,79,0 ,-the vertical filter tap (tap) of 16} is used in top, 1,7,7, the filter tap of 1} then is used for field, the end, the as a result clipped wave of filtering becomes scope [0,255].

Please note that the 320 meeting modifications of colourity MC (1) function element are received from the MV of VL305, use the 2:1 of level (X) component of these MV downsampled, similarly, colourity MC (2) function element 325 can be revised the MV that receives from VLD305, uses the 2:1 of vertical (y) component of these MV downsampled.

The data of slicing also offer colourity MC (1) function element 320.

Please note that some function element unit can be embedded in other unit, for example: quantize the differential coding of DCT DC coefficient and the part that decoding can consider to treat as respectively VLD unit 395 and VLD unit 305. Inverse quantization Q1-1 on the unit 310 comprises with scope [2048 ,+2047] and carries out saturated processing for each coefficient from inverse quantization arithmetic. IDCT unit 315 comprises with scope [256,255] and carries out saturated processing for each reciprocal transformation numerical value. The MC unit, the memory that colourity MC (1) 320 and colourity MC (2) 325 comprise for reference picture.

CBP function element 315 can be used to change the CBP of MP bit stream of setting out, and brightness Q matrix functions element 390 is a kind of factor of revising CBP, other factors then are the grades that increases re-quantization, and only have pixel number at original (bit stream of reception) two Cb of MB or a field piece in the Cr piece.

Current image data with filtering of 4:2:0 color form can offer adder 345, can deduct from the null value signal of colourity MC (2) function element or with reference to image data at this. On DCT function element 350, data from adder 345 (comprise the present image 4:2:0 pixel data for the in-line coding image, or comprise correspond to for the present image of intermediate code image and difference 4:2:0 pixel data with reference to the image difference) can be transformed into the DCT coefficient, and at re-quantization device (Q₁) quantize on 355.

At this shown re-quantization grade Q₁Can be corresponding to inverse quantization grade Q_-1, but re-quantization is recovered to be adjusted into according to the suitable control signal from the rate control feature element different grade (Q₂), the code conversion with bit stream that reception is provided wherein can provide the MP bit stream with the speed different with the 4:2:2P bit stream.

The realization of relevant complete code converter sees also Fig. 8.

Quantized data can offer VLC function element 395 to obtain the MP bit stream of transmission or other processing usefulness, and for example: code converter can link with format converter 300.

Quantized data will be processed in inverse quantizer 360 and IDCT function element 365, then the present image pixel data of recovering can offer adder 367, can be according to switch 370 and null value signal (being used for the internal schema data) or from the reference image data (being used for the middle model data) of colourity MC (2) function element 325 totalling mutually at this, switch 370 is in response to offering its inside/middle model information of (for example from VLD305).

Slicing function element 375 is understood slicings from the output of adder 367 and is offered colourity MC (2) function element 325.

In converter 300, dotted line functional unit and path are (namely being in some situation) with good conditionsi, for example: if the user be set with require or

intra_dc_precision＝11 ₁₀, particularly, QDC unit 380 can be used to carry out the re-quantization of DC chromaticity coefficent.

QDC during intra_dc_precision=8 (8)=DC//8,

QDC during intra_dc_precision=9 (9)=DC//4 and

QDC during intra_dc_precision=10 (10)=DC//2. " DC " points out the numerical value of non-quantized DC DCT coefficient, and " // " points out to be rounded up near the integer partitioning of integer, and rounding up, not give up be zero.

Words if necessary can as what distribute before, provide function element 382 to set new intra_dc_precision grade (for example 8-10 position).

Colourity Q matrix functions element 385 recovers the chromaticity quantization matrix for inverse quantizer 310. Brightness Q matrix functions element 390 in re-quantization 4:2:0MP chromaticity coefficent for re-quantization device 355. Randomly, brightness Q matrix can be in order to encode former thereby to make amendment, and namely by the obvious mode of those skilled in the art, the numerical value of adjusting in the matrix improves code efficiency. Work as intra_dc_precision=11₁₀The time, re-quantization device 355 also can re-quantization DC luminance factor.

If in the bit stream that receives, there is not quantization matrix to exist, then can use default value (for example default MPEG quantization matrix), default matrix for example can be stored in the local converter 300.

For drawing on the strong points to offset the weaknesses between balance cost and the usefulness, in ensuing Figure 4 and 5, provide between bit stream translation of colourity of two kinds of simplification.

Fig. 4 explanation becomes the 4:2:2P bit stream translation according to the present invention bit stream translation of simplification colourity of MP bit stream. In converter 400, only use a motion compensation units, be exactly colourity MC (2) function element 325, the converter 300 of this and Fig. 3 can be saved 2/3 frame buffer memory by comparison. 340 in wave filter in Fig. 4 is used in the chrominance block of rebuilding from IDCT function element 315, and can not use slicing in filtering.

But, although converter 400 can significantly reduce complexity, this simplification has also imported mistake. Its three kinds of error sources are respectively:

1, the removal of colourity MC (1) and 4:2:2 chroma reference image appearance quantized by new luminance quantization device matrix in (a) 4:2:0 form and (b).

2, the position of change wave filter and slicing function element piece, and

3, the integer arithmetic of filtering processing.

Because whole processing only involves colourity, so the mistake that imports is very little, because in image, more can not perceive the colourity mistake.

Except above-mentioned approximation, quantize and mistake that slicing produces if can ignore, then bit stream translation processing of colourity of providing of the converter 400 of Fig. 4 converter 500 of just being similar to Fig. 5 provides.

Fig. 5 explanation becomes the 4:2:2P bit stream translation according to the present invention bit stream translation of approximate colourity of MP bit stream. In converter 500, motion compensation units (being exactly colourity MC (3) function element 510) is applied to difference (or residue) image. By IDCT function element 315, wave filter 340 and DCT function element 350 are combined into single unit 520, converter 500 can reduce complexity. But this class formation can cause color offset.

With regard to the 4:2:2P bit stream situation with 4:2:0 color form, shown in converter can further be simplified in Fig. 6.

The 4:2:2P bit stream translation that Fig. 6 explanation will have a 4:2:0 color form according to the present invention becomes bit stream translation of colourity of MP bit stream. For converter 600, the low cost enforcement that can realize being similar to by removing three broken broken line functional unit IDCT315, slicing function element 335 and DCT function element 350.

For luminance block, the conversion that only needs may be the DC accuracy change 382 that shows in Fig. 7.

Fig. 7 explanation is worked as intra_dc_precision=11 according to the present invention₁₀The time 4:2:2P 4:2:2 or 4:2:0 bit stream translation become the brightness of MP 4:2:0 bit stream-piece bit stream translation. Converter 700 comprises function element 382, is used for exceeding the numerical value that limiting when or input to set new intra_dc_precision according to the user at numerical value.

The form 5 interior situations of having summed up many texture transformation processing.

Fig. 8 explanation is according to the combination of 4:2:2 to 4:2:0 converter of the present invention and code converter. 4:2:2 to 4:2:0 converter 400 in Fig. 4 can be combined with normal code converter, becomes 4:2:2P to a MP code converter 800. In code converter/converter 800, re-quantization function element (Q₂) 855 provide Q₁Difference quantification gradation in addition namely is in response to the speed control signal, and corresponding inverse quantization function element Q is provided₂ ^-1860. The rate control feature element can produce the speed control signal with traditional approach, the bit rate of using to reach desired MP bit stream.

Function element 810,830,850 and 867 corresponds to respectively function element 310,330,350 and 367, and function element 855 ', 860 ' and 365 ' corresponds to respectively function element 855,860 and 365.

Also provide brightness MC function element 820 so that the motion compensation of brightness data to be provided in the bit stream of reception at this. Note that MV can be re-used to avoid the needs of locomotion evaluation.

Therefore, can understand the system that the invention provides a kind of color form for the converting digital video bit stream, native system is taken into account admissible front and rear conversion bit stream format (comprising the quantizer accuracy resumption), reach brightness and chroma data and whether have minute other quantization matrix, or share a shared quantization matrix. In certain specific embodiments, the MPEG-2 4:2:2P bit stream (being forward transposition stream) with 4:2:2 or 4:2:0 color form is converted into the MP flow filament (conversion bit stream namely) with 4:2:0 color form.

By using the luminance quantization matrix with the chroma data re-quantization, and reuse to carry out the brightness movement vector of the motion compensation of chroma data, can realize code efficiency.

Can realize further efficient by the position that changes a pixel mini filter and slicing merit element.

Forward transposition stream to the 4:2:2P4:2:0 form also can be simplified.

When the quantification accuracy of current conversion bit stream can't be compatible mutually with the quantification accuracy of bit stream after the rear conversion, then can expose effective conversion of luminance block.

Also expose the code conversion format converter at this, different bit rates of conversion bit stream after being used for realizing.

Although the present invention explains with many certain specific embodiments, but those skilled in the art's understanding can be carried out many adaptability revisions and modification under the prerequisite that does not deviate from the spirit and scope of the present invention that defined by claim.

Claims (26)

1, a kind of forward transposition circulation that will have one first form changes the method for the rear conversion bit stream with one second form into, comprises step:

To major general's forward transposition stream partial decompressing in a pixel domain, to recover the chroma data in it;

Flow the quantization matrix data that recovery is associated with brightness data from this forward transposition; And recompression is with should be in the data of the chroma data that recovers;

This recompression comprises according to the luminance quantization matrix that recovers, and will correspond to the data re-quantization of the chroma data of recovery, changes bit stream after this to be provided.

2, method according to claim 1, wherein this forward transposition stream comprises interior coded image, and the chroma data of this recoverys has one and correspond to replace the first chroma format of the first form of flowing of this forward, further comprises step:

Change the motion vector that the bit stream recovery is associated with brightness data in the past;

Carry out the first motion compensation process for the chroma data of the recovery of interior coded image with the brightness movement vector that recovers;

This first motion compensation process is used this first chroma format;

After this first motion compensation process, filter chroma data, so that the chroma data of the second chroma format with second form that corresponds to this rear conversion bit stream to be provided; And

Carry out the second motion compensation process of the chroma data with second chroma format with the brightness movement vector that recovers, recompress step so that these data to be provided to this.

3, method as claimed in claim 2, wherein:

Should the second chroma format, adopt the chrominance block of these the first chroma formats that one chrominance block is provided for per at least two.

4, method according to claim 2, wherein:

The data that the chroma data with second chroma format that is provided by this filtration step is provided are converted into transform domain from pixel domain, then quantize, and inverse quantization then, then reciprocal transformation thinks that this second motion compensation process provides data.

5, the method for claim 1 further comprises step:

Changing bit stream recovers one first quantified precision grade in the past; And

If this first quantified precision grade allows accuracy class greater than the maximum of the second form of rear conversion bit stream;

(i) reduce this first quantified precision grade,

(ii) recover DC luminance transformation data from this forward transposition stream, and

(iii) according to the quantified precision grade that reduces, the data re-quantization of the DC luminance transformation data of recovery will be corresponded to.

6, method according to claim 1, wherein this forward transposition stream comprises interior coded image, and the chroma data of this recoverys has one and correspond to replace the first chroma format of the first form of flowing of this forward, further comprises step:

Filter the chroma data of recovery so that the residue chroma data to be provided, these data have the second chroma format of the second form that corresponds to this rear conversion bit stream;

Change the motion vector that the bit stream recovery is associated with brightness data in the past; And

Carry out the motion compensation process of the data of the residue chroma data that corresponds to interior coded image with the brightness movement vector that recovers, think that the recompression step provides this data;

Wherein this motion compensation process is used this second chroma format.

7, method according to claim 6, wherein:

Should the second chroma format, adopt the chrominance block of these the first chroma formats that one chrominance block is provided for per at least two.

8, method according to claim 1, wherein:

This first form comprises a 4:2:2 color form, and this second form comprises a 4:2:0 color form.

9, method according to claim 1, wherein:

This first form comprises a MPEG4:2:2 profile form, and this second form comprises a MPEG main outline form.

10, method according to claim 1, wherein:

One encoding block pattern of forward transposition stream is modified in the rear conversion bit stream.

11, method according to claim 1, wherein:

This forward transposition is flowed and should be used identical macroblock encoding type by rear conversion bit stream.

12, method according to claim 1, wherein:

This quantization step is in response to being used for setting the rear speed control signal of changing the bit rate of bit stream.

13, method according to claim 1, wherein:

The luminance quantization matrix of this recovery is modified to improve the code efficiency of this rear conversion bit stream.

14, a kind of forward transposition circulation that will have one first form changes the device of the rear conversion bit stream with one second form into, comprises:

To major general's forward transposition stream partial decompressing to recover the device of the chroma data in pixel domain its;

Change the device that bit stream recovers the quantization matrix data that are associated with brightness data in the past;

Recompression comprises according to the luminance quantization matrix that recovers corresponding to the device of the data of the chroma data that recovers, and will correspond to the data re-quantization of the chroma data of recovery, so that the device of this rear conversion bit stream to be provided.

15, device according to claim 14, wherein this forward transposition stream comprises interior coded image, and the chroma data of this recoverys has one and correspond to replace the first chroma format of the first form of flowing of this forward, further comprises:

Change the device that bit stream recovers the motion vector that is associated with brightness data in the past;

The brightness movement vector that use to recover is carried out the device for the first motion compensation process of the chroma data of the recovery of interior coded image;

This first motion compensation process is used this first chroma format;

One filters chroma data after this first motion compensation process, correspond to this afterwards wave filter of the chroma data of the second chroma format of the second form of conversion bit stream to provide to have; And

Use the brightness movement vector that recovers, have the second motion compensation process of the chroma data of the second chroma format with execution, so that the device of these data to this recompression device to be provided.

16, device according to claim 15, wherein:

Should the second chroma format, for per at least two chrominance block that adopt this first chroma format provide a chrominance block.

17, according to claim 15 device, wherein:

The data that the chroma data with second chroma format that is provided by this wave filter is provided are converted into transform domain from pixel domain, then quantize, and inverse quantization then, then reciprocal transformation is to provide data to this second motion compensation process.

18, according to claim 14 device further comprises:

Change the device that bit stream recovers one first quantified precision grade in the past; And

Be used for following device: (i) reduce this first quantified precision grade,

(ii) change bit stream in the past recover DC luminance transformation data, and

(iii) according to the quantified precision grade that reduces, the data re-quantization of the DC luminance transformation data of recovery will be corresponded to, if this first quantified precision grade is greater than the maximum permission accuracy class of the second form of rear conversion bit stream.

19, according to claim 14 device, wherein this forward transposition stream comprises interior coded image, and the chroma data of this recoverys has one and correspond to replace the first chroma format of the first form of flowing of this forward, further comprises:

One chroma data that is used for filtering recovery is to provide the wave filter of residue chroma data, and wherein this residue chroma data has the second chroma format of the second form that corresponds to this rear conversion bit stream;

Change the device that bit stream recovers the motion vector that is associated with brightness data in the past; And

The brightness movement vector that use to recover corresponds to data motion compensation process for the residue chroma data of interior coded image with execution, these data are offered the device of this recompression device;

Wherein this motion compensation process is used this second chroma format.

20, device according to claim 19, wherein:

Should the second chroma format, for per at least two chrominance block that adopt this first chroma format provide a chrominance block.

21, device according to claim 14, wherein:

This first form comprises a 4:2:2 color form, and this second form comprises a 4:2:0 color form.

22, device according to claim 14, wherein:

This first form comprises a MPEG 4:2:2 profile form, and this second form comprises a MPEG main outline form.

23, device according to claim 14, wherein:

One encoding block pattern of forward transposition stream is modified in the rear conversion bit stream.

24, device according to claim 14, wherein:

This forward transposition is flowed and should be used identical macroblock encoding type by rear conversion bit stream.

25, device according to claim 14, wherein:

This re-quantization device is in response to being used for setting the rear speed control signal of changing a bit rate of bit stream.

26, device according to claim 14, wherein:

The luminance quantization matrix of this recovery is modified to improve the code efficiency of this rear conversion bit stream.

Images