CN1910921A - Picture coding and decoding method, apparatus, and program thereof - Google Patents

Abstract

The picture decoding method according to the present invention is a decoding method for decoding coded pictures by inverse quantization and inverse orthogonal transformation, in which a quantization matrix which defines a scaling ratio of a quantization step for each component is multiplied by a multiplier, which is a coefficient for frequency transformation or a quantization step, and also, a result of the multiplication is multiplied by a quantized value, as a process of inverse quantization.

Description

Method for encoding images and equipment, picture decoding method and equipment and program thereof

The cross reference of related application

The application requires the rights and interests of following U.S. Provisional Application: the No.60/540 that on January 30th, 2004 submitted to, 636; The No.60/551 that on March 9th, 2004 submitted to, 690; The No.60/552 that on March 12nd, 2004 submitted to, 907; And the No.60/561 of submission on April 12nd, 2004,351, their content is this whole being introduced into as a reference.

Technical field

The present invention relates to a kind of method for encoding images of the image of compression movement effectively, the picture decoding method that is correctly decoded this image of compression movement and image encoding apparatus, image decoding apparatus and program thereof.

Background technology

Recently, along with the arrival of the multimedia era of processing audio, video and pixel value synthetically, the existing information medium, promptly newspaper, magazine, TV, broadcast receiver and phone and other device so as to conveying a message to people have been listed in the multimedia scope.Generally, multimedia refers to a kind of representation: not only character, and graphic symbol, audio frequency, especially image etc. are relative to each other.Yet for above-mentioned existing information medium are included in the multimedia scope, obviously, prerequisite is to represent this information with digital form.

Yet, when estimate above-mentioned digital form information medium each in comprised amount of information the time, the amount of information of every character needs the 1-2 byte, and audio frequency need surpass per second 64 kilobits (telephony quality), and when considering moving image, it need surpass per second 100 megabits (television reception quality at present).Therefore, the above-mentioned information medium information of handling such magnanimity same as before with digital form is unpractical.For example, visual telephone drops into actual use by Integrated Service Digital Network, its transfer rate be 64 kilobits to 1.5 megabits, yet, can not be directly be transmitted in the image of catching on the video screen or take by television camera by ISDN.

Therefore, this needs compressionism, for example under the visual telephone situation, uses and abides by H.261 and the H.263 video compression technology of standard of being carried out International standardization by standardization department of international telecommunication union telecommunication (ITU-T).Compressionism according to abideing by Moving Picture Experts Group-1 can be stored in image information and audio-frequency information on the common music CD (CD).

At this, Motion Picture Experts Group (MPEG) is a kind of international standard that is used for the compression movement picture signal, MPEG-1 a kind ofly is compressed to the standard of per second 1.5 megabits (bit) downwards with vision signal, just the downward approximate compression of information included in the TV signal is arrived one of percentage.The aimed quality of Moving Picture Experts Group-1 is a mean quality, so that realize the transfer rate about main 1.5 megabits, therefore, satisfying more, high quality graphic requires to come the transmitting moving picture signal with 2 megabits to the transfer rate of 15 megabits for the standardized MPEG-2 of purpose realizes the television broadcasting quality.

Under current environment, be responsible for the standardized working group of MPEG-1/MPEG-2 (ISO/IEC/JTC1/SC29/WG11) further standardization MPEG-4, MPEG-4 has realized being better than the compression ratio of the compression ratio that MPEG-1/MPEG-2 realizes, permission is operated based on the coding/decoding of each object, and has realized the new function that multimedia era is required.At first, in the MPEG-4 standardisation process, target is the standardization low rate encoding, yet current goal expands to more general coding, comprises the high bit rate coding that is used for horizontally interlaced image etc.And, by ITU-T and International Standards Organization/International Electrotechnical Commissio (ISO/IEC) common that formulate, as the MPEG-4 AVC (advanced video coding) and the ITU standardization well afoot H.264 of coding method of future generation, they have higher compression ratio.From in August, 2002, coding method of future generation comes forth with the name of committee draft (CD).

In moving image encoding, normally carry out the amount of information compression by eliminating redundancy on the room and time direction.Therefore, purpose is to reduce inter frame image (inter-picture) predictive coding of time redundancy and estimates motion, and with reference to forward direction and back to image one by one piece produce predicted picture, then to the predicted picture that obtained with currently want the difference between the image encoded to encode.At this, " image " is the term of the image of expression on the screen, and when being used for progressive scanning picture interval scale frame, and when be used for horizontally interlaced image its representative frame or field.At this, horizontally interlaced image is that wherein single frames is by two images of forming, and each field all has different capture times.For to the horizontally interlaced image Code And Decode, three kinds of modes of handling single frames are possible: as frame; As two; And, depend on the piece in the frame as frame/field structure.

Be called as " I image " at the image that does not have to be performed I picture (intra-picture) predictive coding under the situation of reference picture.The image of carrying out the inter frame image predictive coding with reference to single image is called as " P image ".The image of carrying out the inter frame image predictive coding with reference to two width of cloth images by the while is called as " B image ".The B image can refer to by display order from forward direction or the back to image optional two width of cloth images.Can be each piece designated reference image (being reference picture) as basic coding/decoding unit.Will be by calling first reference picture to a reference picture of will be in coding stream describing earlier, and reference picture that will the back is described in coding stream called second reference picture, distinguish this reference picture.Notice that as the condition of these types of image being carried out Code And Decode, the image that is used for reference must be encoded and decode.

The motion compensation interframe image forecasting encoding is used for to P image or B image encoding.The coding that utilizes the motion compensation interframe image prediction is a kind of coding method of using motion compensation in the inter frame image predictive coding.With only different according to the execution of the pixel value in reference picture forecast method, motion compensation is a kind of amount of exercise (hereinafter referred to as " motion vector ") that can pass through each part in the estimated image, and further contemplate this amount of exercise and carry out prediction, improve precision of prediction and reduce the technology of data volume.For example, might be by estimating the current motion vector of wanting image encoded, to the predicted value that obtains by the amount skew that only makes each motion vector and currently want the prediction residual between the image encoded to encode, carry out motion compensation then, reduce data volume thus.In this technology, also with coding form record or transmission motion vector, because when decoding, need motion vector information.

Come estimated motion vector based on each macro block.More particularly, should be in advance at the current fixedly macro block of wanting in the image encoded, so that come estimated motion vector by the most similar reference block of seeking this fixedly macro block in the region of search in reference picture.

Fig. 1 shows bit stream sample data structure.As shown in Figure 1, bit stream has following hierarchy.Bit stream (stream) is made up of a plurality of image sets (GOP).By utilizing GOP as the basic coding unit, might editing moving image and carry out random access.Each GOP is made up of multiple image, and every width of cloth image is I image, P image or B image.Every width of cloth image is further formed by a plurality of.Each sheet as slab region in every width of cloth image consists of a plurality of macro blocks.And each stream, GOP, image and sheet all comprise the synchronizing signal (sync) that is used to indicate each unit terminal point and as the head (head) of the common data block in this unit.

Be not in as the bit stream of stream sequence but under the transmission data conditions as the grouping of data cell (unit ofpiecemeal data) piecemeal, can transmit head respectively and remove partial data.In this case, head and data division should not incorporated in the identical bit stream, as shown in Figure 1.Yet, under the situation of grouping,, just in different grouping, transmit them though must not sequentially transmit head and corresponding data division.Therefore, even under head and data division are not merged in situation in the identical bit stream, as applicable with reference to the same concept of figure 1 described bit stream yet.

In general, the human visual system is characterised in that with high fdrequency component and compares that it is to the sensitivity of image medium and low frequency component.In addition, because the energy of picture signal medium and low frequency component is greater than the high fdrequency component energy, so come the carries out image coding by the order from the low frequency component to the high fdrequency component.As a result, low frequency component is encoded required figure place greater than the required figure place of high fdrequency component coding.

In view of above some, when the conversion coefficient to each frequency of obtaining by orthogonal transform quantizes, compare with low frequency component, existing coding method is used for high fdrequency component with bigger quantization step.Conventional coding method might take place under the situation of negligible degradation at the image subjective quality in this technology, realizes the increase greatly of compression ratio.

Because depend on picture signal, use a kind of technology by convention by each frequency component quantization step of width of cloth image modification with the high fdrequency component quantization step of low frequency component quantization step contrast.Quantization matrix (being also referred to as " weighting matrix ") is used to derive the quantization step of each frequency component.Fig. 2 shows the example of quantization matrix.In the figure, upper left component is a DC component, and is horizontal high fdrequency component to right component, and component is the vertical high frequency component downwards.Quantization matrix among Fig. 2 is also indicated, on duty becoming when big more, and it is big more that quantization step becomes.Usually, might use different quantization matrixes for every width of cloth image.The value of indicating each frequency component quantization step size is fixed-length code (FLC).Notice that each component of quantization matrix and the value of each quantization step are approximated to ratio each other usually, but needn't defend tenaciously this relation, as long as clearly define the correspondence between them.

Fig. 3 shows, carry out by normal image encoding device or image decoding apparatus, as the flow chart of the re-quantization that proposed among MPEG-2 and the MPEG-4.

As shown in the figure, normal image encoding device or image decoding apparatus obtain weighting matrix Wi, and j and quantization parameter QP (S11 and S12) calculate quantization step QStep, and obtain quantized value (i.e. the coefficient of frequency of Liang Huaing) fi, j (S14).Then, image encoding apparatus is by calculating fi, and j * QStep * Wi, j derive re-quantization value (S15-S17).

In the performed quantification treatment of image encoding apparatus, the coefficient of frequency that obtains as the result of orthogonal transform be multiply by from calculating QStep * Wi, j and the inverse of the value that produces.

Yet problem is that conventional quantification and re-quantization are handled and forced many amounts of calculation, because need to carry out a large amount of divisions and multiplication in processing.

Summary of the invention

The purpose of this invention is to provide a kind of be used to reduce quantize and re-quantization calculates picture decoding method, method for encoding images, image decoding apparatus, image encoding apparatus and the program of required workload.

In order to realize above purpose, method for encoding images of the present invention is that a kind of re-quantization of carrying out by block-by-block and inverse orthogonal transformation come picture decoding method that coded image is decoded.This method comprises the following step of handling as re-quantization: quantization matrix and multiplier are multiplied each other, and described quantization matrix is indicated the pantograph ratio of the quantization step of each frequency component, and described multiplier is the coefficient of orthogonal transform or quantization step; And will multiply each other by the sum of products quantized value that described multiplication produces.

Described multiplier can be relevant with employed normalization factor in the inverse orthogonal transformation processing.

According to above structure, can reduce and calculate required workload, because in inverse orthogonal transformation is handled, do not need to carry out the required multiplication of coefficient of frequency.That is to say, be the required multiplication of derivation quantization step by precomputation, might reduce the required multiplication of inverse orthogonal transformation, so that do not increase the required workload of quantification calculating.

Can carry out multiplication between quantization matrix and the multiplier to the coded data of each predetermined unit, and can block-by-block carry out the multiplication between the sum of products quantized value, the coded data of predetermined unit comprises encoding block, and this product is that encoding block is common.

The product that is produced by the multiplication between quantization matrix and the multiplier can be stored in the memory, and the multiplication between this sum of products quantized value can relate to reference to storage.

The coded data of predetermined unit can be the data corresponding to image.

According to above structure,, operand might be reduced so that further reduce amount of calculation: based on the multiplication of every width of cloth image by processing being divided into following two parts; And based on the multiplication of each piece.

Have identical structure according to method for encoding images of the present invention, image decoding apparatus and image encoding apparatus, program and semiconductor device, and acquisition and above-mentioned the same effect.

Description of drawings

Described by following the present invention who considers together with the accompanying drawing of the explanation specific embodiment of the invention, these and other purpose of the present invention, advantage and feature will become apparent.In the accompanying drawings:

Fig. 1 shows the sample data structure of bit stream;

Fig. 2 shows the quantization matrix example;

Fig. 3 shows, carry out by the normal image encoding device, as the flow chart of the re-quantization that proposed among MPEG-2 and the MPEG-4;

Fig. 4 shows the structured flowchart according to the image encoding apparatus of first embodiment of the invention;

The orthogonal transform that Fig. 5 shows the block structure under the situation of the macro block brightness piece being carried out 16 * 16 I picture predictive codings and will carry out piece;

The orthogonal transform that Fig. 6 shows the block structure under the situation of the macro block brightness piece being carried out 4 * 4 I picture predictive codings or 4 * 4 inter frame image predictive codings and will carry out piece;

Fig. 7 shows the structure of macro block chrominance block and the orthogonal transform that will carry out piece;

Fig. 8 shows employed equation in the Hadamard conversion;

Fig. 9 A shows employed equation among the integer precision DCT;

Fig. 9 B shows employed equation in the integer precision inverse DCT;

Figure 10 A-10D shows the example of coded sequence in the quantization matrix respectively;

Figure 11 A shows the weighted components array in the quantization matrix, wherein defines each weighted components for orthogonal transform;

Figure 11 B and 11C show respectively how being placed in the head by each data that is obtained that each component in the quantization matrix is encoded;

Figure 12 shows the input-output characteristic curve of quantification;

Figure 13 shows with quantizing the quantization step characteristic curve that parameter becomes;

Figure 14 shows with quantizing the signal-to-noise characteristic curve that parameter becomes;

Figure 15 A to 15C is used to illustrate normalization and re-quantization processing;

Figure 16 A shows the block diagram of quantifying unit first example structure;

Figure 16 B shows the block diagram of inverse quantization unit first example structure;

Figure 17 A shows the block diagram of quantifying unit second example structure that is suitable for weighting matrix;

Figure 17 B shows the block diagram of inverse quantization unit second example structure of using weighting matrix;

Figure 18 A shows the block diagram of quantifying unit the 3rd example structure of using weighting matrix;

Figure 18 B shows the block diagram of inverse quantization unit the 3rd example structure of using weighting matrix;

Figure 19 shows, based on quantization parameter QP to each (i, j) component is carried out the re-quantization process chart under the situation that each quantization step comprise the frequency translation multiplication calculates;

Figure 20 shows, the quantization flow figure under the situation of carrying out each quantization step calculating that comprises the frequency translation multiplication in advance based on quantization parameter QP;

Figure 21 shows the quantization flow figure under the situation of carrying out each quantization step calculating that comprises the frequency translation multiplication as required based on quantization parameter QP;

Figure 22 shows the image decoding apparatus structured flowchart;

Figure 23 A-23C is the diagram of stored program storage medium;

Figure 24 shows contents providing system and always disposes block diagram;

Figure 25 shows the cellular object lesson that uses the image coding and decoding method;

Figure 26 shows the cell phone block diagram;

Figure 27 shows the digit broadcasting system example;

Figure 28-31 shows according to second execution mode, based on the derive example of quantization matrix of 8 * 8 weighting matrixs;

Figure 32-35 shows based on the derive example of quantization matrix of 4 * 4 weighting matrixs;

Figure 36 shows the inverse quantization unit block diagram according to the 3rd execution mode;

Figure 37 shows the example of weighting matrix;

Figure 38 shows re-quantization and handles;

Figure 39 and 40 shows re-quantization and handles;

Figure 41 and 42 shows the example of table respectively; And

Figure 43 shows the re-quantization that 4 * 4 colourity DC pieces are carried out.

Embodiment

(first execution mode)

Fig. 4 shows the structured flowchart according to the image encoding apparatus of first embodiment of the invention.

Image encoding apparatus 1 is a kind of like this equipment, and its output is by carrying out compressed encoding to received image signal Vin, then encoded image signal is converted to the coding image signal Str that obtains as the bit stream of variable-length code (VLC).This image encoding apparatus 1 comprises motion estimation unit ME, motion compensation units MC, subtracter Sub, orthogonal transform unit T, quantifying unit Q, inverse quantization unit IQ, inverse orthogonal transformation unit IT, adder Add, video memory PicMem, switch SW and variable length coding unit VLC.

Picture signal Vin is imported into subtracter Sub and motion estimation unit ME.Every width of cloth image among the subtracter Sub calculating input image signal Vin and the residual image between every width of cloth predicted picture, and the residual image that is calculated outputed to orthogonal transform unit T.

Orthogonal transform unit T carries out orthogonal transform to residual image, so that it is transformed to orthogonal transform coefficient or coefficient of frequency, and they is outputed to quantifying unit Q.

Quantifying unit Q utilizes the quantization step of deriving with reference to the quantization matrix WM of input from the outside, comes the coefficient of frequency from each piece of orthogonal transform unit T input is quantized, and resulting quantized value Qcoef is outputed to variable length coding unit VLC.

Inverse quantization unit IQ utilizes reference quantization matrix W M and the quantization step derived, comes quantized value Qcoef is carried out re-quantization, so that they are transformed to coefficient of frequency, and they is outputed to inverse orthogonal transformation unit IT.Inverse quantization unit IQ according to this execution mode carries out re-quantization in two steps: the first step is, make the quantization matrix of each quantization step pantograph ratio (scaling ratio) of each frequency component of indication multiply by a multiplier, and multiplication result is stored in the memory as orthogonal transform coefficient or quantization step; Second step was to make the result who stores in the memory multiply by each quantized value.The first step is based on that every width of cloth image operates, and each piece operates and second step is based on.The multiplier that is used for frequency translation comprises the inverse orthogonal transformation normalization factor.In this case, the multiplication result of storing in the memory is the value by making each quantization step and inverse orthogonal transformation normalization factor multiply each other and obtain.

Inverse orthogonal transformation unit IT carries out the frequency inverse conversion to coefficient of frequency, so that they are transformed into residual image, and this residual image is outputed to adder Add.Adder Add is with every width of cloth residual image with from every width of cloth predicted picture addition of motion compensation units MC output, so that obtain decoded picture.Should store under the situation of this decoded picture in indication, actuating switch SW, thus decoded picture is stored among the video memory PicMem.

Be transfused to based on each macro block and detect in the decoded picture that the motion estimation unit ME of picture signal Vin stores in video memory PicMem and the most similar image area of picture signal Vin of input, and determine the motion vector MV of this image area of indication position.To carrying out estimation of motion vectors by further cutting apart each piece that macro block obtains.

Motion compensation units MC utilizes detected motion vector in above the processing, takes out only image area in the decoded picture that comes to store from video memory PicMem, as predicted picture.

Variable length coding unit VLC is to each execution variable length code of quantization matrix WM, quantized value Qcoef and motion vector MV, so that obtain bit stream Str.

Fig. 5 and 6 shows according to MPEG-4AVC respectively, passes through the orthogonal transform that orthogonal transform unit T carries out.For by 16 * 16 luminance macroblock of forming pixel, to based on the situation of the I picture predictive coding of 16 * 16 macro blocks and the situation of other type coding, differently carry out orthogonal transform and cut apart by means of the piece of orthogonal transform.

Fig. 5 shows and will carry out under the situation of 16 * 16 I picture predictive codings the structure of macro block brightness piece and orthogonal transform to luminance block.In this case, orthogonal transform unit T carries out orthogonal transform as following (1) to (4).(1) luminance block with 16 * 16 pixels is divided into 16 4 * 4 block of pixels.(2) to each 4 * 4 execution of producing by cutting apart orthogonal transform based on integer precision 4 * 4 discrete cosine transforms (DCT).At this, integer precision DCT does not keep the characteristic identical with DCT, because value is rounded off, but it still can be as the conversion that is similar to DCT.(3) produce 4 * 4DC piece of forming by the direct current in each orthogonal transform block (DC) component.(4) 4 * 4DC piece is carried out the Hadamard conversion.The Hadamard conversion that correctly is called as " discrete Hadamard conversion (DHT) " is a kind of the simple orthorhombic conversion of carrying out addition and subtraction.

Fig. 6 shows at the coding that is different from 16 * 16 I picture predictive codings, under the situation as 4 * 4 I picture predictive codings and 4 * 4 inter frame image predictive codings, and the structure of luminance block and the orthogonal transform that will carry out luminance block.In this case, orthogonal transform unit T carries out orthogonal transform as following (1) and (2).(1) luminance block with 16 * 16 pixels is divided into 16 4 * 4 block of pixels.(2) to each 4 * 4 execution of producing by cutting apart orthogonal transform based on integer precision DCT.

Fig. 7 shows the structure of macro block chrominance block and the orthogonal transform that will carry out chrominance block.In this case, orthogonal transform unit T carries out orthogonal transform as following (1) to (4).(1) chrominance macroblock of being made up of 8 * 8 pixels is divided into 44 * 4 block of pixels.(2) to each 4 * 4 execution of producing by cutting apart orthogonal transform based on integer precision 4 * 4DCT.(3) produce 2 * 2 pixel DC pieces of forming by the DC component in each orthogonal transform block.(4) each 2 * 2DC piece is carried out the Hadamard conversion.

Thereby, orthogonal transform unit T will to the DC piece use as one of simple orthorhombic conversion, can be only by carrying out the Hadamard conversion that addition and subtraction be realized.

Fig. 8 shows equation and the oscillogram of using in the Hadamard conversion, each width of cloth oscillogram is represented corresponding equation.In the drawings, " h0 "-" h3 " represents 4 input signals, and " H0 "-" H3 " represents to be performed the component of Hadamard conversion respectively." H0 " is the DC component that is performed the Hadamard conversion, and " H3 " is the high fdrequency component that is performed the Hadamard conversion.The inverse transformation of Hadamard conversion is the Hadamard conversion.That is to say, can obtain " h0 "-" h3 " by once more " H0 "-" H3 " being carried out the Hadamard conversion.

Note, can utilize following equation to obtain the Hadamard conversion that 2 * 2DC luminance block is carried out.That is, the every row in 2 * 2DC piece and every row are used once following equation.

$\mathrm{<figure-callout\; id="0"\; label="H"\; filenames="A20058000303500563.png,A20058000303500661.png"\; state="\{\{state\}\}">H</figure-callout>}0=(\mathrm{<figure-callout\; id="0"\; label="h"\; filenames="A20058000303500563.png,A20058000303500661.png"\; state="\{\{state\}\}">h</figure-callout>}0+h1)/\sqrt{2}$

$\mathrm{<figure-callout\; id="1"\; label="H"\; filenames="A20058000303500451.png,A20058000303500541.png"\; state="\{\{state\}\}">H</figure-callout>}1=(h0-h1)/\sqrt{2}$

Fig. 9 A shows equation and the oscillogram that orthogonal transform unit T uses for integer precision DCT, and every width of cloth oscillogram is all represented corresponding equation." d0 "-" d3 " represents 4 input signals, and " D0 "-" D3 " expression is performed the component of integer precision DCT.In integer precision 4 * 4DCT, the every row in 4 * 4 block of pixels and every row are used 4 input DCT shown in Fig. 9 A one time.

" D0 " is the DC component, and " D3 " is high fdrequency component.Between frequency component H1 and D1, the difference shown in the oscillogram of integer precision DCT and Fig. 8 A between the oscillogram of Hadamard conversion is conspicuous.That is to say that H1 compares with frequency component, frequency component D1 (current component of low-limit frequency) represents more smoothly (promptly gradually becoming minimum value from maximum).

Fig. 9 B shows the equation that orthogonal transform unit T uses for the integer precision inverse DCT.In the drawings, need actual operation so that utilize frequency component D1 and D3 to obtain D1 ' and D3 '.In this embodiment, make the quantization step of re-quantization in handling multiply by some value (first step-length of being got by inverse quantization unit IQ as mentioned above) in advance, purpose is to avoid the actual operation in the integer precision inverse DCT.Thereby, the multiplication decreased number carried out for the integer precision inverse DCT, and calculate required workload and also reduced.As a result, in re-quantization was handled, the quantization step of odd number number of times frequency component equaled the quantization step of even number number of times frequency component Doubly or 8/5 times.

Figure 10 A-10D shows the example of coded sequence in the quantization matrix respectively.This is used for coding or decoding quantization matrix in proper order, and quantization matrix is by in fact will the order of operated component rearranging in quantification and re-quantization process.In image encoding in the performed orthogonal transform, 4 * 4 pixels and 8 * 8 pixels are two kinds of unit of the most frequent use.Figure 10 A and 10C show the example that uses 4 * 4 pixel cells, and Figure 10 B and 10D show the example that uses 8 * 8 pixel cells.Beginning from low frequency component to the coding that high fdrequency component is carried out, the compression efficiency height shown in Figure 10 A and 10B, but in some cases, can use the coding by the level order, shown in Figure 10 C and 10D.

Figure 11 A-11C shows quantization matrix, and the data structure of the quantization matrix (weighting matrix) that is encoded when forming stream (streaming).In the drawings, the head of " GOP " or " image " that " head " expression is shown in Figure 1, or equivalent information.Figure 11 A shows the frequency component array in the quantization matrix.Component in " Wi, j " expression quantization matrix in row " i " and the row " j ".How the coded data that Figure 11 B and 11C show each component in the quantization matrix respectively is placed in the example in the head.The bit stream of " WeightingMatrix (weighting matrix) " expression by quantization matrix is encoded and obtained.Figure 11 B shows the stream of quantization matrix being encoded and obtaining by by the order shown in Figure 10 B, and Figure 11 C shows the stream of quantization matrix being encoded and obtaining by by the order shown in Figure 10 D.Note the coding variable-length code (VLC) of " Wi, j " represented position in " Wi, j " expression quantization matrix in the stream shown in Figure 11 B and the 11C.

Figure 12 shows the input-output characteristic by the performed quantification (or re-quantization) of quantifying unit Q and inverse quantization unit IQ.Quantification means, and the result of cutting apart orthogonal transform frequency component (value that will quantize among Figure 12) by quantization step is rounded off, so that frequency component is become integer.The integer that rounds off is called as quantized value.On the contrary, the value that quantize is become again to frequency component be called as " re-quantization ".By changing the size of quantization step, might increase or reduce the bit quantity that produces by coding.Thereby, by changing the size of quantization step, might keep identical compression ratio (being the sign indicating number amount of time per unit).

Figure 13 shows the characteristic curve of quantization step to quantization parameter.Quantization parameter is used to derive quantization step, and is conditioned, so that allow the sign indicating number amount corresponding to bit rate.Inverse quantization unit IQ and quantifying unit Q utilize the quantization parameter that will encode to derive quantization step, because directly quantization step is not encoded.In example shown in Figure 13, when quantification parameter QP increased by 6, quantization step doubled.As a result, under the situation that quantization parameter QP changes, quantization parameter QP become with noise (SN) than proportional, and the variable quantity of SN ratio maintains par, as shown in figure 14, and irrelevant with the value of quantization parameter QP.

Notice that when the value of quantization parameter indication from 0 to 51 change, the maximum indication of quantization step equals the value of 256 times of minimum values.

Figure 15 A to 15C is used to illustrate normalization and re-quantization.Handle in a kind of simplification re-quantization processing is below described

The method that is used for the normalized multiplication of orthogonal transform shown in Fig. 9 B.Shown in Fig. 9 B, component D1 and D3 are positioned at N position (N is an odd number).The component that all is positioned at the N position in the horizontal and vertical directions be multiply by twice

This means component and multiply by 8/5.Based on this, for the normalization of in re-quantization is handled, carrying out in advance, should multiply by quantization step in advance, as following (a) to (c).

(a) all be positioned in the horizontal and vertical directions under the situation of M (M is an even number) position at component, do not carry out the normalization multiplication.

(b) be positioned on level or the vertical direction under the situation of N (N is an odd number) position at component, component be multiply by

(c) all be positioned in the horizontal and vertical directions under the situation of N (N is an odd number) position at component, make component multiply by 8/5.

Now, given $\mathrm{\&beta;}=\mathrm{\&alpha;}\&times;\sqrt{8}/\sqrt{5},$ γ=α * 8/5, (a) become to (c) below (A) to (C).

(A) all be positioned in the horizontal and vertical directions under the situation of M (M is an even number) position at component, make component multiply by α.

(B) be positioned on level or the vertical direction under the situation of N (N is an odd number) position at component, making component multiply by β.

(C) all be positioned in the horizontal and vertical directions under the situation of N (N is an odd number) position at component, make component multiply by γ.

Utilize this simple rule, might in re-quantization is handled, realize normalization together.In inverse DCT and quantification treatment, special normalization multiplication becomes unnecessary.

When quantification parameter QP increased by 6, quantization step doubled.Therefore, represent quantization parameter QP that calculating obtained by comprising the normalization multiplication and the relation between the quantization step in order to following equation.

Quantization step=(quantization step of QP%6) * (2 ^(QP/6))

Based on this, shown in Figure 15 B, by make " quantization step of QP%6 " to shift left " QP/6 " position, can easily obtain to relate to normalized quantization step, as with the corresponding quantization step of any quantization parameter QP (quantization step that promptly relates to the normalization multiplication).This can keep 18 quantization steps altogether by only being respectively α, β and γ pairing " quantization step of QP%6 ", realizes, shown in Figure 15 C.In this embodiment, the ratio of the quantization parameter size of the quantization parameter of luminance signal size and carrier chrominance signal will be based on changing piecewise.Because color the degeneration of (especially red) visually specific luminance degenerate obviously, so preferably the quantization parameter QP of carrier chrominance signal less than the quantization parameter of luminance signal.

Obviously, owing to can represent quantization step by making quantization step displacement QP/6 position, so can represent to quantize and re-quantization with the combination of multiplication and shifting function.Figure 16 A and 16B show the block diagram of first example structure of quantifying unit Q and inverse quantization unit IQ.In first example, under the situation of not using weighting matrix, only use multiplication and shift operation.Quantifying unit Q1 comprises multiplication unit Q11 and dextroposition device Q12.Multiplication unit Q11 multiply by Q1 with orthogonal transform coefficient." Q1 " is the multiple of the inverse of quantization step (quantization step of QP%6).Quantization step performed division in being generally used for quantizing, however the inverse of quantization step is calculated in advance so that multiply each other, because division arithmetic is more complicated than multiplying.Inverse quantization unit IQ1 comprises multiplication unit IQ11 and left shifter IQ12.

Quantifying unit Q1 operation is as follows.Dextroposition device Q12 makes 1 of the performed multiplication result right shift S of multiplication unit Q11.That is to say that dextroposition device Q12 makes the performed multiplication result of multiplication unit Q11 divided by 2 ^S1S1 value and QP/6 change pro rata.Inverse quantization unit IQ1 operation is as follows.Multiplication unit IQ11 makes orthogonal transform coefficient multiply by Q2.The quantization step of " Q2 " value and QP%6 changes pro rata.Left shifter IQ12 makes the performed multiplication result of multiplication unit IQ11 to the S1 position of shifting left.That is to say that left shifter IQ12 makes the performed multiplication result of multiplication unit IQ11 multiply by 2 ^S2" S2 " value and QP/6 change pro rata.

At this, " S1 " and " S2 " is the value of all fixing for all coefficient of frequencies, they each all become with quantizing parameter QP." Q1 " and " Q2 " has the value that depends on quantization parameter QP and coefficient of frequency position.In this case, require to satisfy by Q1 * (2 ^-S1) * Q2 * (2 ^S2The relation that)=1 is represented.In this case, Q1 * Q2 causes binary index, and S2-S1 obtained be fixed value.

Figure 17 A and 17B show, the block diagram of quantifying unit Q and inverse quantization unit IQ second example structure under the situation of using weighting matrix.Quantifying unit Q2 comprises: multiplication unit Q21, and this multiplication unit Q21 makes coefficient of frequency multiply by Q1; Multiplication unit Q22, this multiplication unit Q22 make the performed multiplication result of multiplication unit Q21 multiply by Qa; Dextroposition device Q23, this dextroposition device Q23 make the performed multiplication result right shift S1 position of multiplication unit Q22; And dextroposition device Q24, this dextroposition device Q24 makes the performed shift result right shift Sa position of dextroposition device Q23.Inverse quantization unit IQ2 comprises: multiplication unit IQ21, and this multiplication unit IQ21 makes the sampling frequency coefficient multiply by Q2; Left shifter Q22, this left shifter Q22 makes multiplication result to the S2 position of shifting left; Multiplication unit IQ23, this multiplication unit IQ23 make the performed shift result of left shifter Q22 multiply by Qb; And dextroposition device IQ24, this dextroposition device IQ24 makes the performed shift result right shift Sb position of left shifter Q22.

At this, " Qa " and " Sa " corresponding to the weighted components Wi that specifies in the weighting matrix, j, and can set up following relation: Qa * 2 ^-Sa* Qb * 2 ^-Sb=1.

Figure 18 A and 18B show the block diagram of quantifying unit Q and inverse quantization unit IQ the 3rd example structure, and wherein the structure shown in Figure 17 A and the 17B has been simplified, so that carry out multiplication and shifting function together.Quantifying unit Q3 among the figure comprises: multiplication unit Q31, and this multiplication unit Q31 makes orthogonal transform coefficient multiply by Q1a; And dextroposition device Q32, this dextroposition device Q32 makes performed multiplication result right shift " S1+Sa " position of multiplication unit Q31.Re-quantization matrix IQ3 comprises: multiplication unit IQ31, and this multiplication unit IQ31 makes the orthogonal transform coefficient of quantification multiply by Q2a; And dextroposition device IQ32, this dextroposition device IQ32 makes performed multiplication result right shift " Sb-S2 " position of multiplication unit IQ31.Multiplication unit IQ31 is by carrying out a multiplication with Q1a, come shown in the execution graph 17A respectively with twice multiplication of Q1 and Qa.That is, utilize following equation: Q1b=Q1 * Qb finishes execution.Dextroposition device Q32 is by carrying out once displacement, carries out respectively twice dextroposition of S1 position and Sa position to the right.Multiplication unit IQ31 is by carrying out a multiplication with Q2b, carries out respectively twice multiplication with Q2 and Qb.That is, utilize following equation: Q2b=Q2 * Qb, finish execution.Dextroposition device IQ32 carries out to S2 position and the right shift Sb position of shifting left by carrying out once displacement.

Figure 19 shows, in that (i, j) component is carried out the re-quantization flow chart under the situation that the quantization step comprise the frequency translation multiplication calculates to each based on quantization parameter QP.For example, by inverse quantization unit IQ3 (or IQ2), carry out re-quantization based on piece.

At first, inverse quantization unit IQ3 obtains weighting matrix { Wi, j} and quantization parameter QP (S31, S32).Then, inverse quantization unit IQ3 derives from quantization parameter QP further that { Q2i, j} and S2 (S33) as by comprising the quantization step that calculating obtained of frequency translation multiplication, and obtain quantized value (coefficient of frequency of quantification) { fi, j} (S34).At this, { Q2i, the obtained quantization step of j} as QP%6.Utilize QP/6 to obtain S2b.

Then, inverse quantization unit IQ3 carries out quantification to each coefficient of frequency in the piece in circulation 1 (S35-S40).That is, { Wi, j} derive { Qbi, j} and Sb (S36), and by making Qbi, j and Q2i, j multiply each other and obtain point scale (level scale) LSi, j (S37) to inverse quantization unit IQ3 according to weighting matrix.Inverse quantization unit IQ3 further obtains the figure place S2b (S38) that will be shifted by deduct S2 from Sb, so that displacement simultaneously, and by making quantized value fi, j and point scale (level scale) LSi, j multiplies each other and calculates the re-quantization value, makes multiplication result to the S2b position (S40) of shifting left then.

Thereby, can carry out re-quantization by following straightforward procedure:, utilize quantization parameter QP to calculate quantization step simultaneously to carrying out re-quantization by the quantization step that calculating obtained that comprises the frequency translation multiplication.

Figure 20 shows, and is utilizing quantization parameter QP to calculate quantification treatment flow chart under the situation of the quantization step that calculating obtained by comprising the frequency translation multiplication in advance.For example, carry out re-quantization by inverse quantization unit IQ3 (or IQ2).Difference between Figure 20 and Figure 19 is, based on every width of cloth image all is each all quantization step LSi that obtain by the calculating that comprises the frequency translation multiplication, j is stored in conduct table (S43c in the circulation 1) in the memory, and reads LSi from table, j (S49a in the circulation 2).Other operation is almost the same with Figure 19, thereby omits description of them.According to the re-quantization of this execution mode, in circulation 1, all be all quantization step LSi that obtain by the calculating that comprises the frequency translation multiplication each, j is stored in conduct table in the memory, and it is suitable for calculating the re-quantization value at high speed based on piece.

Figure 21 shows, the quantification treatment flow chart under the situation of calculating the quantization step that calculating obtained by comprising the frequency translation multiplication as required based on quantization parameter QP.For example, carry out re-quantization by inverse quantization unit IQ3 (or IQ2).Difference between Figure 19 and Figure 21 is, be that the table in the memory distributes free zone (free area) when initialization, and by comprising the quantization step LSi that calculating obtained of frequency translation multiplication, whether j is stored in (S56) in the table.Judging quantization step LSi, under j and the not stored situation of the figure place S2b that will be shifted, calculate LSi, j and S2b (S57a-S57c), and the LSi that is obtained, j and S2b are added to table and go up (S57d), and judging quantization step LSi, under the situation that j has been stored, from table, read LSi, j and S2b (S57e).According to the re-quantization of this execution mode, compare with Figure 20, amount of calculation might be reduced to the desired minimum of computation amount of piece subsequently of common use quantization parameter QP, although for first, amount of calculation becomes with component more or less.

Figure 22 shows the structured flowchart according to the image decoding apparatus of this execution mode.In the drawings, identical Reference numeral be endowed according to the image encoding apparatus shown in Fig. 4 block diagram in the unit operated of the included the same mode in unit, and omit explanation to them.Inverse quantization unit IQ and inverse orthogonal transformation unit IT have illustrated these operations there with shown in Figure 4 the same.

In addition, by the program that is used to realize the dynamic image encoding/decoding method described in each of above execution mode is recorded in storage medium such as floppy disk etc., can in stand alone computer system, easily carry out the processing shown in the above execution mode.

Figure 23 A, 23B and 23C utilize such as institute's program stored in the storage medium of floppy disk in the computer system, realize the diagram of the dynamic image encoding/decoding method described in the above execution mode.

Figure 23 B shows integral outer appearance, its cross-sectional structure and the floppy disk self of floppy disk, and Figure 23 A shows the physical format as the floppy disk of storage medium main body.Floppy disk FD is accommodated among the shell F, and a plurality of magnetic track Tr are formed on the surface of dish from outside to inside with one heart, and each magnetic track is divided into 16 sector Se on the angular direction.Thereby dynamic image encoding method and dynamic image decoding method are recorded in floppy disk FD as program and go up in the district that distributes for it.

Figure 23 C shows and is used for program is recorded in the configuration that floppy disk FD goes up and reproduces the program on the floppy disk FD.When program being recorded in floppy disk FD when going up, computer system Cs writes moving image encoding and coding/decoding method as program by floppy disk FDD.When the program on utilizing floppy disk is come in computer system tectonic movement image coding and decoding method, from the floppy disk read routine, send it to computer system then by floppy disk FDD.

Be to have carried out above explanation under the supposition of floppy disk at storage medium, but also can utilize CD to carry out same treatment.In addition, storage medium is not limited to floppy disk and CD, and can use can logging program other any medium, as integrated circuit (IC) card and read-only memory (ROM) cassette tape.

The system of below describing the application of moving image encoding illustrated in the above-mentioned execution mode and coding/decoding method and using this moving image encoding and coding/decoding method.

Figure 24 shows and is used to realize that content transmits the overall arrangement block diagram of the contents providing system ex100 of service.Be used to provide the district of communication service to be divided into sub-district, and be arranged in sub-district separately as cell site (cell site) ex107～ex110 of fixed radio station with expectation size.

This contents providing system ex100 for example is connected to various device by internet ex101, Internet service provider ex102, telephone network ex104 and cell site ex107～ex110, as computer ex111, PDA(Personal Digital Assistant) ex112, camera ex113, cell phone ex114 with have the cell phone ex115 of camera.

Yet contents providing system ex100 is not limited to configuration shown in Figure 24, and can be connected to the combination of any equipment.And each equipment can not be directly connected to telephone network ex104 by cell site ex107～ex110.

Camera ex113 is a kind of equipment that can capture video, as digital video camera.Cell phone ex114 can be the cell phone of following arbitrary system: individual digital communication (PDC) system, code division multiple access (CDMA) system, Wideband Code Division Multiple Access (WCDMA) (W-CDMA) system or global system for mobile communications (GMS), personal handyphone system (PHS) etc.

Stream transmission server ex103 is connected to camera ex113 by telephone network ex104 and cell site ex109, according to the coded data from the user, utilizes camera ex113 to realize live distribution etc.Any one of the server of camera ex113 and transmission data etc. can be encoded to data.The motion image data of being taken by camera ex116 can be sent to stream transmission server ex103 by computer ex111.In this case, camera ex116 or computer ex111 can encode to motion image data.Large scale integrated circuit (LSI) ex117 and camera ex116 included among the computer ex111 carry out encoding process.Can be integrated into any kind storage medium (as read-only optical disc (CD-ROM), floppy disk and hard disk) that can read by computer ex111 etc. to the software that is used for the Code And Decode moving image.In addition, the cell phone with camera ex115 can send motion image data.This motion image data is by LSI coded data included among the cell phone ex115.

Contents providing system ex100 with the same method shown in above-mentioned execution mode, the user is utilized the content (as the live video of music) of shootings such as camera ex113, camera ex116 encode, and they are sent to stream transmission server ex103, in case client requests is transmitted server ex103 as a stream and just with stream mode content-data is sent to the client simultaneously.The client comprises computer ex111, PDAex112, camera ex113, the cell phone ex114 etc. that can decode to above-mentioned coded data.Thereby in contents providing system ex100, the client can receive and reproduce coded data, and further can receive, decodes and reproduce data in real time, so that realize personal broadcaster.

When each equipment in this system is carried out coding or decoding, can use motion picture encoding apparatus shown in the above-mentioned execution mode and moving picture decoding equipment.

To describe cell phone, as an example of this equipment.

Figure 25 shows and uses the moving image encoding illustrated in the above-mentioned execution mode and the cell phone ex115 of coding/decoding method.Cell phone ex115 has: antenna ex201 is used for communicating by letter with cell site ex110 by radio wave; Can taking moving and the camera unit ex203 of still image, as the charge-coupled device (CCD) camera; Display unit ex202 as LCD, is used to show the data of taking by camera unit ex203 or receive by antenna ex201, as the image of decoding etc.; The main unit that comprises one group of operation keys ex204; Voice-output unit ex208 as loud speaker, is used to export voice; Voice-input unit ex205 as microphone, is used to import voice; Storage medium ex207 is used for record coding or decoded data, as motion or static image data, the e-mail data that is received and motion or the static image data by camera; And receptacle unit ex206, be used for storage medium ex207 is appended to cell phone ex115.Storage medium ex207 is a kind of being placed on such as the quick storage element in the plastic casing of secure digital storage card (SD card), and this quick storage element is a kind of Electrically Erasable Read Only Memory as the electric erasable rewritable nonvolatile memory (EEPROM).

Next, will cell phone ex115 be described with reference to Figure 26.In cell phone ex115, be designed to control the main control unit ex311 of each unit of the main body that comprises display unit ex202 and operation keys ex204 comprehensively, be interconnected to power circuit unit ex310, operation Input Control Element ex304, image encoding unit ex312, camera interface unit ex303, LCD (LCD) control unit ex302, picture decoding unit ex309, multiplex/demultiplex unit ex308, read/write cell ex307, modem circuit unit ex306 and Audio Processing Unit ex305 by synchronous bus ex313.

When operation call establishment end (call-end) key or power key by the user, then power circuit unit ex310 powers to each unit from battery pack, so that the digital cellular telephone ex115 activation with camera is ready state.

In cell phone ex115, Audio Processing Unit ex305 is under the control of the main control unit ex311 that comprises central processing unit (CPU), read-only memory (ROM) and random asccess memory (RAM), the voice signal that receives by voice-input unit ex205 under conversation modes is converted to digital voice data, modem circuit unit ex306 carries out spread-spectrum to digital voice data and handles, and telecommunication circuit unit ex301 is to conversion of data actual figure mould and frequency inverted, so that send data by antenna ex201.Equally, in cell phone ex115, telecommunication circuit unit ex301 amplifies the data that receive by antenna ex201 under conversation modes, and to data execution frequency inverted and analog-to-digital conversion, modem circuit unit ex306 carries out contrary spread-spectrum to data and handles, and Audio Processing Unit ex305 becomes the analog voice data with data transaction, so that by voice-output unit ex208 output analog voice data.

In addition, when send Email under data communication mode, the email text data of the operation keys ex204 input by operating main body is sent to main control unit ex311 by operation Input Control Element ex304.In main control unit ex311, text data is carried out spread-spectrum is handled and after telecommunication circuit unit ex301 carried out digital-to-analogue conversion and frequency inverted to text data, data were sent to cell site ex110 by antenna ex201 at modem circuit unit ex306.

When under data communication mode, sending view data, be provided for image encoding unit ex312 by camera interface unit ex303 by camera unit ex203 shot image data.When not sending view data, also might pass through camera interface unit ex303 and LCD control unit ex302, the view data captured camera unit ex203 directly is presented on the display unit ex202.

Comprise the coding method that motion picture encoding apparatus adopted of ex312 utilization in image encoding unit shown in above-mentioned execution mode of motion picture encoding apparatus as described in the present invention, come the view data that camera unit ex203 is provided is compressed and encoded, so that convert thereof into coded image data, and send it to multiplex/demultiplex unit ex308.At this moment, during utilizing that camera unit ex203 takes, cell phone ex115 sends to multiplex/demultiplex unit ex308 to the voice of receiving by voice-input unit ex205 by Audio Processing Unit ex305, as digital voice data.

Multiplex/demultiplex unit ex308 utilizes preordering method, the speech data that coded image data that image encoding unit ex312 is provided and Audio Processing Unit ex305 are provided carries out multiplexing, modem circuit unit ex306 carries out the spread-spectrum processing to the multiplex data that obtains as multiplexing result then, last telecommunication circuit unit ex301 is to conversion of data actual figure mould and frequency translation, so that send by antenna ex201.

About under data communication mode, receiving the motion pictures files data be linked on the webpage etc., modem circuit unit ex306 carries out contrary spread-spectrum to the data of receiving from cell site ex110 by antenna ex201 and handles, and sends the multiplex data of handling as contrary spread-spectrum that the result obtained.

For the multiplex data of receiving by antenna ex201 is decoded, multiplex/demultiplex unit ex308 is the multiplex data demultiplexing audio data stream of image encoded data flow and coding, and by synchronous bus ex313 coded image data is offered picture decoding unit ex309 respectively, speech data is offered Audio Processing Unit ex305.

Then, comprise the coding method pairing coding method of ex309 utilization in picture decoding unit shown in above-mentioned execution mode of moving picture decoding equipment as described in the present invention, the image encoded data flow is decoded, to produce the motion image data that reproduces, and these data are offered display unit ex202 by LCD control unit ex302, be shown thereby for example be linked to view data included in the motion pictures files of web page.Simultaneously, Audio Processing Unit ex305 converts speech data to the analog voice data, and these data are offered voice-output unit ex208, thereby it is reproduced for example to be linked to speech data included in the motion pictures files of web page.

The invention is not restricted to said system, because ground or satellite digital broadcasting are in the news recently, and the motion picture encoding apparatus described in the above-mentioned execution mode or moving picture decoding equipment can be integrated at least in as shown in figure 27 the digit broadcasting system.More particularly, encode video information stream is sent to broadcasting satellite ex410 by radio wave from broadcasting station ex409, perhaps communicates by letter with broadcasting satellite ex410.In case receive this encode video information stream, broadcasting satellite ex410 broadcasts with regard to emitting radio wave.Then, the family expenses antenna ex406 with satellite broadcasting receiving function receives radio wave, and television set (receiver) ex401 or set-top box (STB) ex407 decode so that reproduce to coding stream.Can be at the moving picture decoding equipment of realizing among the reproducer ex403 shown in above-mentioned execution mode, so that read and decode storage medium ex402 such as CD and digital versatile disc (DVD) go up the encoding stream of record.In this case, the motion image signal of reproduction is displayed on the monitor ex404.Also can imagine being connected to the cable ex405 that is used for cable TV or being used for satellite and/or the STB ex407 of the antenna ex406 of ground broadcasting, realize moving picture decoding equipment, so that they are reproduced on the monitor ex408 of television set ex401.Also moving picture decoding equipment can be incorporated in the television set, rather than in the set-top box.Equally, the automobile ex412 with antenna ex411 can be from satellite ex410 or cell site ex107 received signal, so that reset moving image on the set auto-navigation system ex413 in display device such as automobile ex412.

In addition, the motion picture encoding apparatus shown in above-mentioned execution mode can be encoded to picture signal, and they are recorded on the storage medium.As concrete example, can quote and be used for the register ex420 of picture signal record on DVD dish ex421 as the DVD register, and is used for the dish register of picture signal record on hard disk.Can be recorded in them on the SD card ex422.When register ex420 comprises as the described moving picture decoding equipment of above-mentioned execution mode, can reproduce the picture signal that DVD dish ex421 or SD card ex422 go up record, so that they are presented on the monitor ex408.

For the structure of auto-navigation system ex413, can imagine the structure that does not have camera unit ex203, camera interface unit ex303 and image encoding unit ex312 in the parts shown in Figure 26.The same applies to computer ex111, television set (receiver) ex401 etc.

In addition,, it is contemplated that three types realization: with transmission/receiving terminal that encoder realizes, the only transmission terminal that realizes with encoder, and the receiving terminal of only realizing with decoder for terminal such as cell phone ex114.

Notice that each functional block shown in Fig. 4,16A, 16B, 17A, 17B, 18A, the 18B and 22 in the block diagram can be implemented as the LSI as integrated device electronics.Can come integrated this LSI (for example can be integrated into the functional block except that memory in the single chip) with the form of one or more chips.At this, get LSI as an example, yet, depend on integrated level, it can be called as " IC ", " system LSI ", " super LSI " and " ultra LSI ".

The integrated circuit integrated approach is not limited to LSI, can realize it with industrial siding or general processor.After making LSI, can utilize programmable field programmable gate array (FPGA), maybe can reconfigure the re-configurable processor that connects and be provided with for the circuit unit among the LSI.

In addition, the arrival along with replace the integrated circuit integrated technology of LSI owing to semiconductor technology or the another kind of Progress in technique that is different from semiconductor technology can utilize emerging technology to implement the integrated of functional block.Can quote biotechnology applications one of as an example.

In functional block, the unit that only is useful on the storage data can not carry out under the integrated situation with chip form, is configured to the storage medium 115 described in this execution mode separately.

Note, can pass through processor or program, realize the major part in the flow chart shown in the functional block shown in Fig. 4 and 22 or Figure 19 to 21.

As mentioned above, might in arbitrary the said equipment or system, use method for encoding images and the picture decoding method that proposes in the above execution mode.Therefore, might obtain the effect described in the above-mentioned execution mode.

(second execution mode)

Second embodiment of the invention is below described.

＜based on the no division quantization scheme of integer 〉

In order to reduce computational complexity, wish to utilize the quantification of q matrix only to use multiplication and shifting function.In other words, in fact should avoid the big divide operations of amount of calculation.The non-homogeneous high effective quantization method of the no division based on integer that is proposed generally can be applicable to the piece transform and quantization of virtually any size in the video coding system.

In Figure 16 A and 16B, the quantification and de-quantization (de-quantization) operation of the conversion coefficient under the situation of only using multiplication and shifting function have been described.For given QP, the value of S1 and S2 is all fixed for all coefficients, and the value of Q1 and Q2 depends on that QP value and coefficient positions are (referring to Ref.1:ISO/IEC MPEG﹠amp; The joint video team of ITU-T VCEG (JVT), " Draft of ITU-T Recommendation and Final DraftInternational Standard of Joint Video Specification (the ITU-T suggestion and the final draft draft internation standard of joint video standard) (ITU-T Rec.H.264|ISO/IEC14496-10AVC) ", JVT-G050r1.doc, Geneva, Switzerland, May 2003).Notice that this is not a weight quantization mechanism, and,, but do not have identical norm (norm) (seeing Ref.1) because the row of integer transform is a quadrature in video encoding standard nothing but the normalization problem.

The q matrix is integrated into that no division quantizes and de-quantization at first can be considered to be in these two processing introducing another to multiplication and displacement.In Figure 17 A and 17B, this is illustrated.Notice that the value of Qa and Qb depends on the q matrix entries of appointment and the Sa value of appointment.

In order to reduce computational complexity, can only utilize multiplication and shifting function to be similar to divide operations.For the q matrix entries Mq of appointment and given (or encoder all agree) Sa value, the value of integer Qa is defined as (1＜＜(Sa+Bn))/Mq.Correspondingly, the value of integer Qb be defined as Mq＜＜(Sa-Bn).For example, can be set at 4 to Bn.Obviously, in this case, utilize all quantifications that all equal 16 q matrix will the same with the uniform quantization in the video encoding standard (seeing Ref.1).Note Sa＞Bn, and common Sa＞=8.This integer calculations precision that keeps certain level that is designed for is because Qa and Qb are integers.Consider this design that utilizes the quantification of q matrix, can prove that all operations and storage access can calculate with 16.Note, if data are scaled before storage/displacement, and then intermediate object program allows above 16.Bigger Sa value allows better integer quantisation computational accuracy, yet, should come binding occurrence about the dynamic range of intermediate object program with respect to hardware designs.

The single multiplication of＜de-quantization and normalization inverse transformation 〉

According to Figure 17 A and 17B, directly way is to make up multiplication and shifting function in view of the above, reduces computational complexity thus.Figure 18 A and 18B show the further integrated level of utilizing the q matrix quantization.Specifically, integer Q1a is defined as Q1* (1＜＜(Sa+Bn))/Mq, and integer Q2a equal Q2*Mq＜＜(Sa-Bn).Because Q1 and Q2 obtain from look-up table, so the integrated of q matrix causes the look-up table in the implementation to upgrade basically.

At initial phase, can be easily change table according to the q matrix of appointment.Notice that the value of integer S2 is defined as QP/6, and it is always less than 8, because QP must be in the scope of [0,51] (seeing Ref.1) in video encoding standard.Importantly, it all is differentiable for the different value (for example typically being [1,255]) of each q matrix entries that being integrated with of being advised helps make the Q1a value.From the viewpoint of encoder design, this is a desirable specific character, because it allows according to quantizing to carry out the more flexibility of fine tuning.

＜new look-up table the example of deriving based on the q matrix 〉

We obtain the quantitative formula of following general type:

Quantize:

C _ij＝sign(x _ij)*(abs(x _ij)*Q _q(QP％6，i，j)+(1＜＜n)*f)＞＞n

De-quantization:

y _ij＝sign(c _ij)*((abs(c _ij)*Q _d(QP％6，i，j))＞＞m)

At this, x _Ij, c _Ij, y _IjThe coefficient of representing original, quantification and de-quantization respectively.Q _qAnd Q _dIt is respectively the matrix that in quantification and de-quantization, uses.Consider the normalization of integer transform, Q _qAnd Q _dDifferent.Note, under the situation of uniform quantization, Q _qAnd Q _dTo be smooth matrix (flatmatrix).If x is for just, then sign (x) equals 1, if x=0, then sign (x) equals 0, if perhaps x is for negative, then sign (x) equals-1.Abs (x) is the absolute value of x.The value of m and n depends on QP, piece transform size and Sa value.The f value depends on the block encoding type usually; For example, it equals 3/8 for Intra-coded blocks, and it equals 1/6 for inter-coded block.

＜new look-up table/the matrix related〉with 8 * 8q matrix

Consider Bn=4, Sa=8, and as Figure 30 and 31 derivation matrix Q _qAnd Q _dSituation.For 8 * 8 integer transforms of going quadrature and having identical norm, it is simple utilizing q matrix derivation look-up table.Adopt 8 * 8q matrix shown in Figure 28.

Based on 8 * 8 integer transforms, quantization zooming factor (scaling factor) table is conciliate in the quantification that obtains respectively as shown in figure 29.

Under the situation of QP=20, the scale factor that is used to quantize (as multiplier) will equal 1979, and the scale factor that is used for de-quantization will equal 19.Notice that under the situation of uniform quantization, these values all are identical for all conversion coefficients.When using the q matrix, scale factor has become the matrix shown in Figure 30 and 31 in fact respectively.When specifying the q matrix, can carry out initialization to each scale factor.

＜new look-up table/the matrix related〉with 4 * 4q matrix

Consider Bn=4, Sa=8, and shown in Figure 34 and 35 derivation matrix Q _qAnd Q _dSituation.4 * 4 integer transforms (referring to Ref.1) for going quadrature but having different norms utilize q matrix derivation look-up table need consider the position of coefficient in the matrix.Adopt 4 * 4q matrix shown in Figure 32.Determine to quantize to conciliate quantization zooming factor table (considering the normalization of integer transform) as shown in figure 33.

Under the situation of QP=20, matrix will be respectively shown in Figure 34 and 35 to be used to quantize scale factor (as multiplier) with de-quantization.

When specifying 4 * 4q matrix, each row of matrix w that lists among Figure 33 and v comes these matrixes of initialization relatively.

＜handle the DC piece respectively to quantize

Under the situation of using the Hadamard conversion, the quantification of the proposed q of utilization matrix is not suitable for second level conversion, i.e. the DC component transformation of brightness and colourity.This is some reasons.In encoder, quantize immediately following after the conversion, and in decoder, de-quantization is immediately following after the inverse transformation rather than before.This is in order to keep possible dynamic range (integer calculations precision) during inverse transformation.As a result, quantize to be actually in different territories with de-quantization.This is not a problem for uniform quantization, but will cause the misalignment of the scale factor of coefficient in the process of using weighting q matrix.In addition, Hadamard conversion itself needn't keep with the first order in the same DCT (or approximate integer transform) characteristic.Weight quantization in the Hadamard transform domain may not be similarly meaningful.

This innovation expands to the weight quantization scheme of using the q matrix with the uniform quantization scheme, keeps integer operation simultaneously, and complexity is remained to Min..As important example, this solution allows under the situation that insignificant complexity increases and the Min. sentence structure changes, during the quantization matrix solution integration is realized to current video coded system (referring to Ref.1).

(the 3rd execution mode)

Third embodiment of the invention is below described.

In video coding system, define one group of quantization matrix acquiescently, make decoder implement quantization scheme and matrix.When the bit stream of coding uses the quantification weighting scheme, they will be used for decoding.The user can define themselves quantization matrix, and they are sent to decoder.This innovation will stipulate how quantization matrix is sent to decoder.

The conversion that this innovation also is defined in the various coding modes selections is selected.

In quantizing weighting scheme, we describe following principal character:

1. quantize the standard of weighting scheme fully based on the no division of integer, only need the operation of 16 bit memories at the decoder place, compare with the uniform quantization scheme, this does not introduce the increase of complexity.

2. proposed non-uniform quantizing scheme is based on 8 * 8 conversion to brightness, because we think that this is the conversion of protecting image texture better, image texture is one of most important content element that influences the high quality graphic subjective impression.

The non-uniform quantizing weighting is applied to the residual error in infra-frame prediction and the inter prediction.This innovation provides one group of new coding tools, and the uniform quantization scheme is expanded to the weight quantization scheme of using the q matrix, keeps integer operation simultaneously, and complexity is remained to Min..This makes video coding become especially effective in high-quality and the application of high bit rate coding.

The professional expansion curve of AVC (fidelity scope curve) (Professional ExtensionProfiles) (Fidelity Range Profiles)) is intended to the high-definition picture coding is comprised high definition (HD) image.Equally, in HD graphical representation, recognize high visual vivid degree to a great extent.When using AVC specialty expansion curve (Professional Extension Profiles), wish naturally to allow the outstanding code efficiency of AVC to be of immediate benefit to visual quality to greatest extent.In order to improve the HD subjective quality, we advise a kind of quantification weighting scheme that enables for the non-uniform quantizing weighting of block conversion coefficient at this.Owing to following reason, we think that the non-uniform quantizing instrument is critical:

1. improve the visual vivid degree of coded image.

2. non-uniform quantizing makes to carry out pro rata quantize to regulate with human vision sensitivity becomes possibility, and this has improved the code efficiency of image fidelity.

3. the flexible selection of control final image quality is provided, and this is that high-quality content generates the strong request of industrial institute.

Quantification weighting scheme in this proposal comprises following principal character:

1. quantize the standard of weighting scheme fully based on the no division of integer, only need the operation of 16 bit memories at the decoder place, compare with the uniform quantization scheme, this does not introduce the increase of complexity.

2. proposed non-uniform quantizing scheme is based on 8 * 8 conversion to brightness, because we think that this is the conversion of protecting image texture better, image texture is one of most important content element that influences the high quality graphic subjective impression.

3. the non-uniform quantizing weighting is applied to the residual error in infra-frame prediction and the inter prediction.

We in various HD display devices simulation and watch the subjective quality that all demonstrates all video sequences to improve and the raising greatly of many test videos, comprise some JVT sequence and movie contents that our obtain from motion picture studio.

The suggested range of＜8 * 8 transform and quantization weighting matrixs 〉

All show good code efficiency according to many JVT submissions of submitting to JVT in the past, we comprise 8 * 8 conversion with suggestion.The more important thing is, demonstrated the subjective quality that obtains by using 8 * 8 conversion the protection of better pictures texture is provided.Because before we have many submissions and in the past AVC committee draft institute based on, 8 * 8 conversion that formulation is perfect are (referring to Ref.2:S.Gordon, D.Marpe, T.Wiegand, " Simplified use of 8 * 8transform (simplification of 8 * 8 conversion is used); " ISO/IEC JTC1/SC29/WG11 and ITU-TQ6/SG16, document JVT-J029, December 2003..), so we use this conversion in our suggestion.Yet if use other 8 * 8 integer transform matrixes to select, we do not think that the result has than big-difference.

＜8 * 8 brightness infra-frame predictions 〉

At this, except that in the existing model frame 16 * 16, the intra-frame 4 * 4, advise a kind of new macro block mode mb_type I_8 * 8 for brightness 8 * 8 infra-frame predictions.8 * 8 predictive modes in 9 kinds of frames are arranged.In Ref.2, they are stipulated.Low-pass filtering is applied to reference pixel, so that improve predictive validity.Stipulated filtering in same ABT 8 * 8 infra-frame predictions part in Ref.2.

＜colourity infra-frame prediction 〉

Depend on chroma, should use different quantification weightings.For 4:2:0 and 4:2:2 form, use 4 * 4 to quantize weighting, wherein define quantization scheme below.For the 4:4:4 form, the transform and quantization scheme identical to the colourity sample application.

＜8 * 8 inter predictions 〉

8 * 8 conversion are used for all 16 * 16,16 * 8 and 8 * 16 P and B macro block (mb) type.In addition, for the P sheet, 8 * 8 conversion are used for any 8 * 8 sub-macro blocks that sub_mb_type equals P_L0_8 * 8, perhaps for the B sheet, are used for B_Direct_8 * 8, B_L0_8 * 8, B_L1_8 * 8 or B_Bi_8 * 8.

The syntax elements of＜quantification weighting matrix 〉

The additional syntax elements of quantification weighting matrix defines user-defined weighting matrix at the beginning of being included in bit stream.Quote the quantification weighting matrix by the matrix identifier (ID) that image parameter is concentrated.

＜quantification weighting matrix 〉

Quantize weighting matrix just in time before the inverse transformation in decoder, be applied to quantization step.The weighting at each coefficient index place can be different, so that uniform quantization to be provided.Weighting is to append on the QP that defines in the sentence structure, thereby applied quantification is actually the combination (Figure 36) that quantizes weighted sum QP.Weighting matrix can reduce and increase the quantification amount with respect to QP.

In the AVC standard, by the multiplication of de-quantization convergent-divergent and displacement subsequently, carry out de-quantization, wherein calculate de-quantization by QP mould 6.Each de-quantization is operated once multiplication of integers.When using the quantification weighting matrix, definition quantizes similarly.

In quantizing weighting matrix, for weight being maintained from greater than 1 to less than 1 scope that changes, the value that quantizes weighting matrix is actually the integer value that rounds off of actual weighting on duty 16.For example, quantization weight 1.2 is corresponding to quantizing weighting matrix value 19.Figure 37 shows the example that quantizes weighting matrix.

The quantification weighting of＜8 * 8 brightness 〉

For 8 * 8 brightness, d _IjThe expression quantized transform coefficients.(i, j) expression quantizes weighting matrix to W.So we obtain de-quantization operation as shown in figure 38.

The quantification weighting of＜4 * 4 conversion 〉

For 4:2:0 and 4:2:2 chroma format, with chromaticity transformation to 4 * 4DCT territory.4 * 4 quantize weighting matrix only is applied to ac coefficient.For each 8 * 8 chrominance block, form 2 * 2DC piece, and further use wherein coefficient by the conversion of uniform quantization.This is the same with current standard.

Ref.3:ITU-T suggestion H.264 with ISO/IEC international standard 14496-10AVC, document JVT-J010d7, the colourity QP derivation among the October 2003 keeps the same at this.Yet, be defined as follows for the de-quantization of 2 * 2 colourity DC part,

DcC _Ij=(f _Ij* M (QP _C%6,0,0)＜＜QP _C/ 6-5 is for QP _C/ 6 〉=5 (5)

DcC _Ij=(f _Ij* M (QP _C%6,0,0)+1＜＜(4-QP _C/ 6))＞＞5-QP _C/ 6, for QP _C/ 6＜5 (6)

Wherein

M(QP _C％6，0，0)＝W(0，0)＝W(0，0)*LevelScale(QP _C％6，0，0) (7)

Weighting matrix only is applied to the AC portion of quantization parameter.Similarly, define de-quantization, except the 8.5.8 among the Ref.3 shown in Figure 39 and 40 defines the LevelScale function like that as the equation among Figure 38 (1) and (2).

The remainder of decode procedure is the same with standard among the Ref.3.Under 4:4:4 chroma samples situation, come each 8 * 8 chrominance block of transform and quantization with the mode identical with brightness.

Under the coding mode on the scene, the field macro block of macro block centering uses the same quantization matrix group.Under B_Direct_8 under the coded system on the scene * 8 pattern situations, when motion search block mode during less than 8 * 8, we are ready that also suggestion allows to use 4 * 4 to quantize weightings to the residual error of 4 * 4 transform blocks.

In use quantizing weighting scheme, must be noted that design must be careful especially when quantizing weighting matrix combining with QP.Should guaranteeing to quantize weighting, not expand the position of any coefficient long.What wish most is a certain balance of keeping between the matrix coefficient, though this is the responsibility of encoder.

＜based on quantizing the weighting matrix de-quantization convergent-divergent table of deriving 〉

When user-defined quantification weighting matrix was sent to decoder, decoder need come construction solution quantization zooming table according to quantizing weighted sum QP%6.Each QP%6 is corresponding to a convergent-divergent table.Can multiply each other by making each of even de-quantization in the current AVC standard and convergent-divergent table and quantizing weighting matrix, come computational chart.Its example is that the de-quantization table based on the conversion of being advised among the Ref.2 shown in Figure 41 is derived.If we obtain having the quantization matrix of value shown in Figure 41, then in addition to shifting left 4 times.

De-quantization coefficient table during QP%6=0 will be as shown in figure 42.

＜complexity 〉

In case produced quantization zooming table M ( QP% 6,0,0) according to quantizing weighting matrix and QP/6, then compare with current uniform quantization, introducing a quantization matrix does not just have additional complexity.Per 8 * 8 quantize that multiplication of integers is arranged 64 times, and per 4 * 4 quantization matrixes have multiplication of integers 16 times, produce convergent-divergent table M ( QP% 6,0,0).Per 8 * 8 quantization matrixes have the total storage capacity of maximum 768 bytes (64*2*6), and per 4 * 4 quantization matrixes have the total storage capacity of maximum 192 bytes.

The bit stream sentence structure of＜quantification weighting matrix 〉

Encoder should be able to select whether to use the quantification weighting.For allowing to do like this, the concentrated use_weighting_matrix sign that is provided with of the sequential parameter that quantizes weighting matrix need used.

As in MPEG-2, except that the matrix group of acquiescence, the user can define the quantification weighting matrix.Can utilize pic_parameter_set that the quantification weighting matrix that the user provides is loaded in the decoder.Subsequently, the quantification weighting matrix that is loaded can be quoted by other pic_parameter_set.The matrix that is loaded will be used for producing the de-quantization table that will be stored in decoder.The matrix that is loaded is numbered with ID.Weighting matrix can be kept at (can be used for the sheet in the present frame), perhaps long preservation in the memory in the present image.With the weighting matrix in the memory that only is stored in the present image is local weighted matrix.Can come they and long-term matrix are distinguished by weighting matrix ID=0-1.4 * 4 can have identical ID with the weighting matrix of 8 * 4 sizes.

For the situation that does not have user-defined weighting matrix, can utilize the quantification weighting matrix of one group of acquiescence.Quantification weighting matrix of this group acquiescence is known to the decoder.For pre-defined all the quantization zooming tables of this group default weighting matrix, therefore do not need precomputation.This group acquiescence quantizes weighting matrix and comprises two luminance quantization matrixes (infra-frame prediction and inter prediction) and two chromaticity quantization matrixes (4 * 4).

Can quote weighting matrix with identification number subsequently.We advise that for each picture parameter set, every interframe and every infra-frame prediction mode have a luminance quantization matrix at most.Similarly, every interframe and every infra-frame prediction mode have one 4 * 4 quantization matrix at most.

＜sentence structure 〉

Below describe the pic_parameter_set_rbsp advised, it has the embedding item of the quantization matrix that is used to define 8 * 8 or 4 * 4 inter predictions, infra-frame prediction macro block.

pic_parameter_set_rbsp(){

…

new_quantization_matrices_defined

if(new_quantization_matrices_defined)

def_quant_weighting_matrix

…

intra_quant_mat8_update

if(intra_quant_mat8_update)

quant_mat8_id

inter_quant_mat8_update

if(inter_quant_mat8_update)

quant_mat8_id

intra_quant_mat4_update

if(intra_quant_mat4_update)

quant_mat4_id

inter_quant_mat4_update

if(inter_quant_mat4_update)

quant_mat4_id

…

}

def_quant_weighting_matrix(){

load_quant_mat8

if(load_quant_mat8){

num_quant_mat8

for(k=0;k＜num_quant_mat8;k++){

quant_mat8_id

for(i=0;i＜8;++i)

for(j=0;j＜8;++j)

quant_mat8[i][j]

}

}

load_quant_mat4

if(load_quant_mat4)

num_quant_mat4

for(k=0;k＜num_quant_mat4;k++){

quant_mat4_id

for(i=0;i＜4;++i)

for(j=0;j＜4;++j)

quant_mat4[i][j]

}

}

}

(the 4th execution mode)

Four embodiment of the invention is below described.

＜for the convergent-divergent and the conversion process of Intra_16 * 16 (frame in 16 * 16) macro block (mb) type brightness DC conversion coefficient 〉

Notice that following formula can be used for the operation of de-quantization convergent-divergent function.

If QP ' _YMore than or equal to 36, then scaled results will be derived as

dcY _ij＝(f _ij*LevelScale _{4×4L，intra}(QP’ _Y％6，0，0))＜＜(QP’ _Y/6-6)，i，j＝0...3

Otherwise (QP ' _YLess than 36), scaled results will be derived as

$\mathrm{dc}{Y}_{\mathrm{ij}}=(f_{\mathrm{ij}}*{\mathrm{<figure-callout\; id="4"\; label="LevelScale"\; filenames="A20058000303500661.png,A20058000303500713.png"\; state="\{\{state\}\}">LevelScale</figure-callout>}}_{4\&times;4C,\mathrm{intra}}({{\mathrm{QP}}^{\&prime;}}_{\mathrm{<figure-callout\; id="6"\; label="Y\; \%"\; filenames="A20058000303500541.png,A20058000303500563.png"\; state="\{\{state\}\}">\; <figure-callout\; id="0"\; label="Y\; \%"\; filenames="A20058000303500563.png,A20058000303500661.png"\; state="\{\{state\}\}">Y</figure-callout>\; </figure-callout>}}\%\mathrm{6,0,0})+2^{5-{{\mathrm{QP}}^{\&prime;}}_Y/6})>>(6-{{\mathrm{QP}}^{\&prime;}}_Y/6),$ i，j＝0...3

＜for the convergent-divergent and the conversion process of 4:2:0 chroma format colourity DC conversion coefficient 〉

Notice that following formula can be used for the operation of de-quantization convergent-divergent function.

If QP ' C is more than or equal to 30, then scaled results will be derived as

dcC _ij＝(f _ij*LevelScale _{4×4C，Intra}(QP’ _C％6，0，0))＜＜(QP’ _C/6-5)，i，j＝0，1

Otherwise (QP ' _CLess than 30), scaled results will be derived as

$\mathrm{dc}{C}_{\mathrm{ij}}=\left((f_{\mathrm{ij}}*{\mathrm{<figure-callout\; id="4"\; label="LevelScale"\; filenames="A20058000303500661.png,A20058000303500713.png"\; state="\{\{state\}\}">LevelScale</figure-callout>}}_{4\&times;4C,\mathrm{intra}}({{\mathrm{QP}}^{\&prime;}}_{\mathrm{<figure-callout\; id="6"\; label="C\; \%"\; filenames="A20058000303500541.png,A20058000303500563.png"\; state="\{\{state\}\}">\; <figure-callout\; id="0"\; label="C\; \%"\; filenames="A20058000303500563.png,A20058000303500661.png"\; state="\{\{state\}\}">C</figure-callout>\; </figure-callout>}}\%\mathrm{6,0,0})+2^{4-{{\mathrm{QP}}^{\&prime;}}_C/6}\right)>>(5-{{\mathrm{QP}}^{\&prime;}}_C/6),$

i，j＝0，1

＜for the convergent-divergent and the conversion process of 4:2:2 chroma format colourity DC conversion coefficient 〉

The input of this processing is the conversion coefficient level value of the colourity DC conversion coefficient of a chromatic component of macro block, as having Elements C _Ij2 * 4 array C, wherein i and j form two-dimentional frequency indices.

The output of this processing is as having element dcC _Ij8 convergent-divergent DC values of 2 * 4 array dcC.

By stipulate the inverse transformation of 2 * 4 colourity DC conversion coefficients to get off:

Bit stream should not comprise any element f that causes f _IjSurpass [2 ¹⁵, 2 ¹⁵-1] data of integer value scope.

Variable QP ' _{C, DC}=QP ' _C+ 3

After inverse transformation, it is as follows to carry out convergent-divergent.

If QP ' _{C, DC}More than or equal to 36, then scaled results will be derived as

dcC _ij＝(f*LevelScale _{4×4C，Intra}(QP’ _C，DC％6，0，0，0))＜＜(QP’ _C，DC/6-6)，i＝0...3，j＝0，1

Otherwise (QP ' _{C, DC}Less than 36), scaled results will be derived as

$\mathrm{dc}{C}_{\mathrm{ij}}=(f*{\mathrm{<figure-callout\; id="4"\; label="LevelScale"\; filenames="A20058000303500661.png,A20058000303500713.png"\; state="\{\{state\}\}">LevelScale</figure-callout>}}_{4\&times;4C,\mathrm{Intra}}({{\mathrm{QP}}^{\&prime;}}_{C,\mathrm{<figure-callout\; id="6"\; label="DC\; \%"\; filenames="A20058000303500541.png,A20058000303500563.png"\; state="\{\{state\}\}">\; <figure-callout\; id="0"\; label="DC\; \%"\; filenames="A20058000303500563.png,A20058000303500661.png"\; state="\{\{state\}\}">\; <figure-callout\; id="0"\; label="DC\; \%"\; filenames="A20058000303500563.png,A20058000303500661.png"\; state="\{\{state\}\}">DC</figure-callout>\; </figure-callout>\; </figure-callout>}}\%\mathrm{6,0,0,0})+2^{5-{{\mathrm{QP}}^{\&prime;}}_{C,\mathrm{DC}}/6})>>(6-{{\mathrm{QP}}^{\&prime;}}_{C,\mathrm{DC}}/6),$

i＝0...3，j＝0，1

Bit stream should not comprise any element dcC that causes dcC _IjSurpass [2 ¹⁵, 2 ¹⁵-1] data of integer value scope.

＜for the convergent-divergent and the conversion process of 4:4:4 chroma format colourity DC conversion coefficient 〉

The input of this processing is the conversion coefficient level value of the colourity DC conversion coefficient of a chromatic component of macro block, as having Elements C _Ij4 * 4 array C, wherein i and j form two-dimentional frequency indices.

The output of this processing is as having element dcC _Ij16 convergent-divergent DC values of 4 * 4 array dcC.

Stipulate the inverse transformation of 4 * 4 colourity DC conversion coefficients as shown in figure 43.

Bit stream should not comprise any element f that causes f _IjSurpass [2 ¹⁵, 2 ¹⁵-1] data of integer value scope.

After inverse transformation, it is as follows to carry out convergent-divergent.

If QP ' _CMore than or equal to 36, then scaled results will be derived as

dcC _ij＝(f*LevelScale _{4×4C，Intra}(QP’ _C％6，0，0，0))＜＜(QP’ _C/6-6)，i＝0...3，j＝0，1

Otherwise (QP ' _CLess than 36), scaled results will be derived as

$\mathrm{dc}{C}_{\mathrm{ij}}=(f*{\mathrm{<figure-callout\; id="4"\; label="LevelScale"\; filenames="A20058000303500661.png,A20058000303500713.png"\; state="\{\{state\}\}">LevelScale</figure-callout>}}_{4\&times;4C,\mathrm{Intra}}({{\mathrm{QP}}^{\&prime;}}_{\mathrm{<figure-callout\; id="6"\; label="C\; \%"\; filenames="A20058000303500541.png,A20058000303500563.png"\; state="\{\{state\}\}">\; <figure-callout\; id="0"\; label="C\; \%"\; filenames="A20058000303500563.png,A20058000303500661.png"\; state="\{\{state\}\}">\; <figure-callout\; id="0"\; label="C\; \%"\; filenames="A20058000303500563.png,A20058000303500661.png"\; state="\{\{state\}\}">C</figure-callout>\; </figure-callout>\; </figure-callout>}}\%\mathrm{6,0,0,0})+2^{5-{{\mathrm{QP}}^{\&prime;}}_{C,\mathrm{DC}}/6})>>(6-{{\mathrm{QP}}^{\&prime;}}_C/6),$

i＝0...3，j＝0，1

Bit stream should not comprise any element dcC that causes dcC _IjSurpass [2 ¹⁵, 2 ¹⁵-1] data of integer value scope.

＜for the convergent-divergent and the conversion process of 8 * 8 of residual errors 〉

The input of this processing is to have element c _Ij8 * 8 array c, it is the array about luminance component 8 * 8 residual blocks.

The output of this process is as having element r _IjThe residual samples value of 8 * 8 array r.

Function LevelScale64 derives as follows:

If-macroblock prediction pattern is an intra prediction mode, and be input as 8 * 8 brightness residual, then LevelScale64 ()=LevelScale _{8 * 8, Intra}().

If-macroblock prediction pattern is an inter-frame forecast mode, and be input as 8 * 8 brightness residual pieces, then LevelScale64 ()=LevelScale _{8 * 8, Inter}().

8 * 8 block conversion coefficient level c _IjConvergent-divergent carry out as follows.

If-QP _YMore than or equal to 36,8 * 8 block conversion coefficient level c then _IjConvergent-divergent will be performed as

d _ij＝(c _ij*LevelScale64(QP _Y％6，i，j))＜＜(QP _Y/6-6)，i，j＝0...7

-otherwise (QP _YLess than 36), 8 * 8 block conversion coefficient level c _IjConvergent-divergent will be performed as

$d_{\mathrm{ij}}=(c_{\mathrm{ij}}*\mathrm{LevelScale}64({\mathrm{<figure-callout\; id="6"\; label="QP\; Y\; \%"\; filenames="A20058000303500541.png,A20058000303500563.png"\; state="\{\{state\}\}">QP</figure-callout>}}_Y\%6,i,j)+2^{(5-{\mathrm{QP}}_Y/6)})>>(6-{\mathrm{QP}}_Y/6),$ i，j＝0...7

Above formula reflected for merging from the required additional dextroposition of the zoom factor of q matrix because de-quantization/quantification weighting be defined as w (i, j)=QuantizationMatrix (i, j)/16.QuantizationMatrix (i, j) be attached in the LevelScale function after, the 4 gt positions that we need add, with the reflection divided by 16.

Conversion process will be equivalent to following mode with a kind of on mathematics, the scale transformation coefficient block is converted to the output sample piece.

At first, utilize following one dimension inversion to bring each (level) row scale transformation coefficient of conversion.

-by the one group of median e that derives to get off _Ij

e _i0＝d _i0+d _i4，i＝0...7

e _i1＝-d _i3+d _i5-d _i7-(d _i7＞＞1)，i＝0...7

e _i2＝d _i0-d _i4，i＝0...7

e _i3＝d _i1+d _i7-d _i3-(d _i3＞＞1)，i＝0...7

e _i4＝(d _i2＞＞1)-d _i6，i＝0...7

e _i5＝-d _i1+d _i7+d _i5+(d _i5＞＞1)，i＝0...7

e _i6＝d _i2+(d _i6＞＞1)，i＝0...7

e _i7＝d _i3+d _i5+d _i1-(d _i1＞＞1)，i＝0...7

-from median e _IjCalculating second group of intermediate object program fij is

f _i0＝e _i0+e _i6，i＝0...7

f _i1＝e _i1+(e _i7＞＞2)，i＝0...7

f _i2＝e _i2+e _i4，i＝0...7

f _i3＝e _i3+(e _i5＞＞2)，i＝0...7

f _i4＝e _i2-e _i4，i＝0...7

f _i5＝(e _i3＞＞2)-ei5，i＝0...7

f _i6＝e _i0-e _i6，i＝0...7

f _i7＝e _i7-(e _i1＞＞2)，i＝0...7

Then, from these medians f _IjComputational transformation is g as a result _IjFor

g _i0＝f _i0+f _i7，i＝0...7

g _i1＝f _i2+f _i5，i＝0...7

g _i2＝f _i4+f _i3，i＝0...7

g _i3＝f _i6-f _i1，i＝0...7

g _i4＝f _i6-f _i1，i＝0...7

g _i5＝f _i4-f _i3，i＝0...7

g _i6＝f _i2-f _i5，i＝0...7

g _i7＝f _i0-f _i7，i＝0...7

Then, utilize following identical one dimension inversion to bring each (vertically) row of conversion gained matrix.

-from horizontal transformation value g _IjCalculate one group of median h _IjFor

h _i0＝g _i0+g _i4，i＝0...7

h _i1＝-g _i3+g _i5-g _i7-(g _i7＞＞1)，i＝0...7

h _i2＝g _i0-d _i4，i＝0...7

h _i3＝g _i1+g _i7-g _i3-(g _i3＞＞1)，i＝0...7

h _i4＝(g _i2＞＞1)-g _i6，i＝0...7

h _i5＝-g _i1+g _i7+g _i5+(g _i5＞＞1)，i＝0...7

h _i6＝g _i2+(g _i6＞＞1)，i＝0...7

h _i7＝g _i3+g _i5+g _i1+(g _i1＞＞1)，i＝0...7

-from median h _IjCalculating second group of intermediate object program kij is

k _i0＝h _i0+h _i6，i＝0...7

k _i1＝h _i1+(h _i7＞＞2)，i＝0...7

k _i2＝h _i2+h _i4，i＝0...7

k _i3＝h _i3+(h _i5＞＞2)，i＝0...7

k _i4＝h _i2-h _i4，i＝0...7

k _i5＝(h _i3＞＞2)-h _i5，i＝0...7

k _i6＝h _i0-h _i6，i＝0...7

k _i7＝h _i7-(h _i1＞＞2)，i＝0...7

-from median k _IjComputational transformation is m as a result _IiFor

m _i0＝k _i0+k _i7，i＝0...7

m _i1＝k _i2+k _i5，i＝0...7

m _i2＝k _i4+k _i3，i＝0...7

m _i3＝k _i6+k _i1，i＝0...7

m _i4＝k _i6-k _i1，i＝0...7

m _i5＝k _i4-k _i3，i＝0...7

m _i6＝k _i2-k _i5，i＝0...7

m _i7＝k _i0-k _i7，i＝0...7

Carrying out the vertical inverse transformation with one dimension of one dimension level with after producing the conversion array of samples, the residual sample value of final structure will be derived as

r _i7＝(m _ij+2 ⁵)＞＞6，i，j＝0...7

By the present invention of such description, obviously can change embodiments of the present invention with multiple mode.This variation should not be considered to break away from the spirit and scope of the present invention, and for those skilled in the art, obviously all this changes all should be included in the scope of following claim.

Industrial applicibility

The present invention is applicable to the encoding device to Image Coding or decoding, and also is applicable to distribution The webserver of moving image, the network terminal of receiving moving pictures, so can record and The digital camera of playback moving image, the cell phone that is equipped with camera, DVD logger/ Player, PDA and personal computer.

Claims (16)

1. re-quantization of carrying out by block-by-block and inverse orthogonal transformation come picture decoding method that coded image is decoded, and described method comprises the following step of handling as re-quantization:

Quantization matrix and multiplier are multiplied each other, and described quantization matrix defines the pantograph ratio of the quantization step of each component, and described multiplier is the coefficient of orthogonal transform or quantization step; And

To multiply each other by the sum of products quantized value that described multiplication produces.

2. picture decoding method according to claim 1,

Wherein the coded data of each predetermined unit is carried out multiplication between quantization matrix and the multiplier, and

Block-by-block is carried out the multiplication between the described quantized value of described sum of products, and the coded data of described predetermined unit comprises encoding block, and described product is that described encoding block is common.

3. picture decoding method according to claim 2,

Wherein the described product that is produced by the multiplication between described quantization matrix and the described multiplier is stored in the memory, and

Multiplication between the described quantized value of described sum of products relates to the described memory of visit.

4. picture decoding method according to claim 2,

Wherein the coded data of predetermined unit is the data corresponding to image.

5. picture decoding method according to claim 2,

Wherein said multiplier is relevant with employed normalization factor in the inverse orthogonal transformation processing.

6. an orthogonal transform and a quantification of carrying out by block-by-block carried out the image encoded coding method to image, and described method comprises following step as quantification treatment:

The inverse and the multiplier of the pantograph ratio in the quantization matrix are multiplied each other, and described quantization matrix defines the pantograph ratio of the quantization step of each component, and described multiplier is the coefficient of orthogonal transform or quantization step; And

The value that will will be quantized by the described sum of products that described multiplication produces multiplies each other.

7. method for encoding images according to claim 6,

Wherein carry out the inverse of described pantograph ratio and the multiplication between the described multiplier based on predetermined unit, and

Block-by-block is carried out the multiplication between the described value that will quantize of described sum of products, and described predetermined unit comprises piece, and described product be described common.

8. method for encoding images according to claim 7,

Wherein the product that is produced by the multiplication between the reciprocal and described multiplier of described pantograph ratio is stored in the memory, and

Multiplication between the described value that will quantize of described sum of products relates to the described memory of visit.

9. method for encoding images according to claim 7,

Wherein said predetermined unit is corresponding to image.

10. method for encoding images according to claim 7,

Wherein said multiplier is relevant with employed normalization factor in the inverse orthogonal transformation processing.

11. re-quantization of carrying out by block-by-block and inverse orthogonal transformation come image decoding apparatus that coded image is decoded, described equipment comprises:

First and second multiplication units, described first and second multiplication units operation is used to carry out re-quantization,

Wherein said first multiplication unit multiplies each other quantization matrix and multiplier, and described quantization matrix defines the pantograph ratio of the quantization step of each component, and described multiplier is the coefficient of orthogonal transform or quantization step, and

The sum of products quantized value that described second multiplication unit will be produced by the performed multiplication of described first multiplication unit multiplies each other.

12. an orthogonal transform and a quantification of carrying out by block-by-block carried out the image encoded encoding device to image, described equipment comprises:

First and second multiplication units, described first and second multiplication units operation is used for carrying out quantification,

Wherein said first multiplication unit multiplies each other the inverse and the multiplier of the pantograph ratio in the quantization matrix, and described quantization matrix is indicated the pantograph ratio of the quantization step of each component, and described multiplier is the coefficient of orthogonal transform or quantization step, and

The value that the sum of products that described second multiplication unit will be produced by the performed multiplication of described first multiplication unit will quantize multiplies each other.

13. re-quantization of carrying out by block-by-block and inverse orthogonal transformation come program that coded image is decoded, described program makes computer carry out the following step of handling as re-quantization:

Quantization matrix and multiplier are multiplied each other, and described quantization matrix defines the pantograph ratio of the quantization step of each component, and described multiplier is the coefficient of orthogonal transform or quantization step; And

To multiply each other by the sum of products quantized value that described multiplication produces.

14. an orthogonal transform of carrying out by block-by-block and quantizing comes program that image is encoded, described program makes computer carry out following step as quantification treatment:

The inverse and the multiplier of the pantograph ratio in the quantization matrix are multiplied each other, and described quantization matrix defines the pantograph ratio of the quantization step of each component, and described multiplier is the coefficient of orthogonal transform or quantization step; And

The value that will will be quantized by the sum of products that described multiplication produces multiplies each other.

15. re-quantization of carrying out by block-by-block and inverse orthogonal transformation come semiconductor device that coded image is decoded, described device comprises:

First and second multiplication units, described first and second multiplication units operation is used to carry out re-quantization,

Wherein said first multiplication unit multiplies each other quantization matrix and multiplier, and described quantization matrix defines the pantograph ratio of the quantization step of each component, and described multiplier is the coefficient of orthogonal transform or quantization step, and

Described second multiplication unit will be multiplied each other by the sum of products quantized value that described multiplication produces.

16. an orthogonal transform of carrying out by block-by-block and quantizing comes semiconductor device that image is encoded, described device comprises:

Described first and second multiplication units, described first and second multiplication units operation is used for carrying out quantification,

Wherein said first multiplication unit multiplies each other the inverse and the multiplier of the pantograph ratio in the quantization matrix, and described quantization matrix defines the pantograph ratio of the quantization step of each component, and described multiplier is the coefficient of orthogonal transform or quantization step, and

The value that the sum of products that described second multiplication unit will be produced by the performed multiplication of described first multiplication unit will quantize multiplies each other.

Images