CN100455022C - Image encoding method and image decoding method - Google Patents

Abstract

An image encoding apparatus 400 performing pixel interpolation for creating a prediction image selects one of a pixel interpolation unit A 403 and a pixel interpolation unit B 404 according to a value of the picture type signal PicType. When the picture type signal PicType has a value ''0'' indicating a B picture, the pixel interpolation unit A 403 is selected. The pixel interpolation unit A is realized by a pixel filter having the number of filter taps ''4'' for calculating the pixel value of 1/2 pixels of the prediction image. When the picture type signal PicType has a value ''1'' indicating a frame other than B picture, the pixel interpolation unit B 404 is selected. The pixel interpolation unit B is realized by a pixel filter having the number of filter taps ''8''.

Description

The coding method of image and the interpretation method of image

The present invention's dividing an application that be the denomination of invention submitted on August 12nd, 2002 for No. the 02802938.0th, the Chinese patent application of " coding method of image and the interpretation method of image ".

Technical field

The present invention relates to the dynamic of predicted picture, with the picture coding device of image signal compression coding, image decoder, method for encoding images, image decoding method and record are in order to use their program recording medium of software implementation.

Background technology

In recent years, be accompanied by the development of multimedia application, usually all media informations such as image, sound, text unified to handle.At this moment, by with whole medium digitlizations, can handle medium uniformly.But, because digitized image has huge data volume, so, for store, the compressionism of transmitted image is absolutely necessary.On the other hand, in order to use the view data of compression mutually, the standardization of compress technique is very important.Standard criterion as Image Compression, ITU-T (international electrical communication alliance electrical communication standardization department) is arranged H.261, H.263, the MPEG of ISO (the International Organization for Stand) (MovingPicture Experts Gruop)-1, MPEG-2, MPEG-4 etc.

Fig. 1 is the diagram that is illustrated in the notion of the motion compensation in the dynamic image.Wherein, with reference in the picture signal Ref by in the picture signal Img of photo-object Car and input by photo-object CurCar as same by photo-object.In addition, with the dotted line in the picture signal Img express by photo-object, expression with reference in the picture signal Ref by the position of photo-object.Under the situation of picture coding device with the coding of the block of pixels CurBlk in the picture signal Img, as prediction image signal Pred, be with reference to identical in the picture signal Ref by the image of photo-object CurCar, if use the pixel of the predicted picture piece PreBlk that is illustrated in the image that is equivalent to same position in the image of photo-object CurCar, forecasting efficiency improves.Promptly, if with reference to the identical image in the picture signal Ref by photo-object CurCar move to picture signal Img in by the identical position of photo-object CurCar, obtain the words of the difference of pixel value, the absolute value of the difference of pixel value diminishes, the amplitude of data volume dwindles, and compresses easily.The location of pixels information necessary that will be mapped to block of pixels CueBlk from the location of pixels of predicted picture piece PreBlk is referred to as kinematic parameter signal MotionParam.As device kinematic parameter signal MotionParam, for example, at MPEG-1,2,4, H.261, H.263 in, use the parallel mobile motion vector of expressing piece.

Fig. 2 is the block diagram of structure of the picture coding device 100 of expression prior art.Picture coding device 100 has difference engine 101, image encoding portion 102, variable-length encoding part 103, image decoding portion 104, adder 105, video memory 106, block of pixels acquisition unit 107, switch 108, switch 109, pixel interpolation portion 110, motion deduction portion 111 and pixel interpolation use judging part 112.At first, picture coding device 100 is from outside received image signal Img.Secondly, difference engine 101 outputs are as the difference image signal Res of the difference of the pixel value of the prediction image signal Pred that obtains from the picture signal Img of outside input with from reference picture signal Ref.Picture signal Pred is the frame of having encoded in advance, is decoded at decoding part 104, according to different situations, utilizes pixel interpolation portion 110 to carry out interpolation, is cut into the image of piece according to the motion vector from motion deduction portion 111.Image encoding portion 102 exports difference image code signal CodedRes with difference image signal Res coding.The occasion of in picture, encoding, owing to the motion compensation of not carrying out between picture, so, think that the pixel value of predicted picture is " 0 ".Variable-length encoding part 103 outputs to the outside of picture coding device 100 with difference image code signal CodedRes and kinematic parameter signal MotionParam Variable Length Code as a code signal Bitstream.Image decoding portion 104, for as motion prediction with reference to image, with difference image code signal CodedRes decoding, output decoding difference image signal ReconRes.Adder 105 will be deciphered the pixel value addition of difference image signal ReconRes and prediction image signal Pred, export as decoded signal Recon.The decoding picture signal is stored in the video memory 106, uses as the reference image when later frame is encoded.Video memory 106 will be from the carrying out of adder 105 output some being kept of frame of coding as prediction usefulness with reference to picture signal Ref.

In the frame that becomes reference frame of block of pixels acquisition unit 107 from remain on video memory 106, extract block of pixels Blk, output to switch 108 according to motion vector from motion deduction portion 111.Switch 108 judges that according to the interpolation of using judging part 112 from pixel interpolation signal UsePolator switches between terminal " 1 " and " 2 ".Terminal " 1 " is connected on the terminal " 1 " of switch 109, and terminal " 2 " is connected in the pixel interpolation portion 110.Pixel interpolation portion 110 when the amount of movement of the piece of being represented by motion vector comprises than the little unit amount of movement of integer pixel unit, generates the pixel value of the position corresponding with it, outputs on the terminal " 2 " of switch 109.Switch 109 judges that according to the interpolation of using judging part 112 from pixel interpolation signal UsePolator is connected terminal " 1 " with terminal " 2 " with switching.Motion deduction portion 111 obtains kinematic parameter signal MotionParam from by the picture signal Img of outside input with reference to picture signal Ref.Pixel interpolation uses judging part 112, according to by kinematic parameter signal MotionParam judging when using with reference to picture signal Ref generation forecast picture signal Pred whether carry out pixel interpolation.

That is, according to shown in Figure 1 by the motion of photo-object, when the motion that utilizes the unit littler than integer pixel unit is predicted, the prediction effect height, usually, in the calculating of the motion circular image pixel value that is accompanied by the unit littler, use pixel interpolation than integer pixel unit.This pixel interpolation utilizes linear filter to carry out filtering by the pixel value to the reference image and carries out.If increase this linear filter tap number, the filter that can realize having good frequency characteristic, prediction effect height and treating capacity increase.On the other hand, when the tap number of filter after a little while, filter frequency characteristic variation, prediction effect reduces, processing is dwindled.

Pixel interpolation uses judging part 112, judges whether carry out pixel interpolation by the generation of kinematic parameter signal MotionParam predicted picture the time.Specifically, pixel interpolation uses judging part 112, when kinematic parameter signal MotionParam comprises the amount of movement of the unit littler than integer pixel unit, is judged as the use pixel interpolation, and output valve is that the pixel interpolation of " 1 " uses control signal UsePolator.When kinematic parameter signal MotionParam represents the motion of integer pixel unit, be not judged as and should use pixel interpolation, output valve is that the pixel interpolation of " 0 " uses control signal UsePolator.Using control signal UsePolator at pixel interpolation is the occasion of " 0 ", switches to terminal " 1 " side before switch 108 and the switch 109, and when pixel interpolation used control signal UsePolator to be " 1 ", switch 108 and switch 109 switched to terminal " 2 " side.When switch 108 and switch 109 are connected to terminal " 2 " and go up, use pixel interpolation portion 110, block of pixels Blk is carried out pixel interpolation, become circular image signal Pred.If switch 108 and switch 109 become " 0 ", do not carry out pixel interpolation, remain untouched and use block of pixels Blk to use as prediction image signal Pred.

Fig. 3 is the block diagram of structure of the image decoder 200 of expression prior art.At first, image decoder 200 is from outside incoming coded signal Bitstream.Secondly, utilize variable length decoding part 201, code signal Bitstream is carried out variable length decoding, be separated into difference image code signal CodedRes and kinematic parameter signal MotionParam.Image decoding portion 202 is with difference image code signal CodedRes decoding, as decoding difference image signal ReconRes output.Adder 203 is with prediction image signal Pred and decoding difference image signal ReconRes addition, as decoding picture signal Recon output.In addition, with deciphering the several of picture signal Recon, be stored in the video memory 204 as with reference to image Ref.Block of pixels acquisition unit 207 represents that the position of kinematic parameter signal MotionParam obtains pixel set (but sometimes for the interpolation processing, obtaining the zone bigger than the prediction piece of reality) in reference picture signal Ref.

Pixel interpolation uses judging part 212, judges in the obtaining of predicted picture whether should use pixel interpolation by kinematic parameter signal MotionParam.For example, as described in MPEG1-1,2,4, under the situation of the motion vector that walking abreast of remarked pixel piece moved, can utilize motion vector whether to be eliminated, judge whether use pixel interpolation by integer.When pixel interpolation uses judging part 212 to be judged as to use pixel interpolation, output valve is that the pixel interpolation of " 1 " uses control signal UsePolator, when being judged as should not use pixel interpolation the time, output valve is that the pixel interpolation of " 0 " uses control signal UsePolator.When pixel interpolation used control signal UsePolator to be " 0 ", switch 208 and switch 209 switched to terminal " 1 " side, and when pixel interpolation used control signal UsePolator to be " 1 ", switch 208 and switch 209 switched to terminal " 2 " side.If switch 208 and switch 209 become " 2 ", use the pixel interpolation that is undertaken by pixel interpolation portion 209, block of pixels Blk is carried out pixel interpolation, as prediction image signal Pred.If switch 208 and switch 209 become terminal " 1 ", do not carry out pixel interpolation, remain untouched and utilize block of pixels Blk as prediction image signal Pred.

But, the personal digital assistant) etc. at pocket telephone and PDA (Personal Digital Assistant: in the portable set, in order to be that power supply can use for a long time with the battery, suppress power consumption, the low arithmetic unit of disposal ability can only be used, the little pixel interpolation method of handling can only be used sometimes.On the other hand, according to different images,,, also wish to use the high pixel interpolation method of prediction effect sometimes although treating capacity is big in order to realize high code efficiency.If the coded system of dynamic image can adapt to these requirements neatly, be useful for enlarging its scope of application.

Summary of the invention

In view of above-mentioned problem, the objective of the invention is, provide a kind of and can select the method for encoding images and the image decoding method of different pixel interpolation methods according to encoded image signal.

For achieving the above object, method for encoding images of the present invention, utilization is encoded to input picture by the predicted picture that the reference image generates, it is characterized in that, in the generative process of above-mentioned predicted picture, use following pixel interpolation method:, obtain pixel value to above-mentioned above-mentioned decimal position pixel with reference to image by pixel interpolation and according to the pixel value of a plurality of integer position pixels of the periphery that is positioned at the decimal position pixel; This method for encoding images comprises the steps: to select the selection step of a pixel interpolation method utilizing N pixel to generate in the filtration treatment of an inter polated pixel from the numerical value of N pixel interpolation method different, that have a plurality of above-mentioned filtration treatment; The pixel interpolation method that use is selected is according to the above-mentioned predicted picture generation step that generates above-mentioned predicted picture with reference to image; Utilize the coding step of above-mentioned predicted picture with above-mentioned input picture coding.

In addition, for achieving the above object, in another method for encoding images of the present invention, it is characterized in that, each frame is carried out the selection of above-mentioned pixel interpolation method, the pixel interpolation sorting signal of determining the selected above-mentioned pixel interpolation method that goes out is configured in the frame title as code signal.

In addition, for achieving the above object, in a method for encoding images more of the present invention, it is characterized in that, utilize N pixel to generate the above-mentioned filtration treatment of an inter polated pixel, is that tap number is the calculation process of the filter of N.

And, for achieving the above object, in image decoding method of the present invention, utilization is deciphered code signal by the predicted picture that the reference image generates, it is characterized in that, in the generation of above-mentioned predicted picture, use following pixel interpolation method:, obtain pixel value to above-mentioned above-mentioned decimal position pixel with reference to image by pixel interpolation and according to the pixel value of a plurality of integer position pixels of the periphery that is positioned at the decimal position pixel; This image decoding method comprises the steps: to select the selection step of a pixel interpolation method utilizing N pixel to generate in the filtration treatment of an inter polated pixel from the numerical value of N pixel interpolation method different, that have a plurality of above-mentioned filtration treatment; The pixel interpolation method that use is selected is according to the above-mentioned predicted picture generation step that generates above-mentioned predicted picture with reference to image; Utilize the decoding step of above-mentioned predicted picture with above-mentioned code signal decoding.

In addition, for achieving the above object, in another image decoding method of the present invention, it is characterized in that, each frame is selected above-mentioned pixel interpolation method, and the selection of above-mentioned pixel interpolation method is to select according to the pixel interpolation sorting signal that is configured in as code signal in the frame title.

In addition, for achieving the above object, in another image decoding method of the present invention, it is characterized in that, utilize N pixel to generate the above-mentioned filtration treatment of an inter polated pixel, is that tap number is the calculation process of the filter of N.

Description of drawings

Fig. 1 is the diagram that is illustrated in the notion of motion compensation in the dynamic image.

Fig. 2 is the block diagram of structure of the picture coding device of expression prior art.

Fig. 3 is the block diagram of structure of the image decoder of expression prior art.

Fig. 4 is the block diagram of the structure of expression picture coding device of the present invention.

Fig. 5 is the block diagram of the structure of expression image decoder of the present invention.

Fig. 6 (a) is illustrated in the 1/2 pixel filter, calculates from the diagram of existing pixel at an example of the computational methods of the pixel value of the pixel of 1/2 locations of pixels of i axial direction skew.Fig. 6 (b) is illustrated in the 1/2 pixel filter, calculates from the diagram of existing pixel at an example of the computational methods of the pixel value of the pixel of 1/2 locations of pixels of j axial direction skew.

Fig. 7 (a) is expression expression for the diagram of the relation between the method for the kind of the image of each frame of dynamic image and pixel interpolation.Fig. 7 (b), on represent the flow chart of the selecting sequence of the interpolating method in picture coding device of the present invention and the image decoder.

Fig. 8 is the block diagram of expression with the structure of the picture coding device of frame unit's switching pixel interpolation method.

Fig. 9 is the flow chart of selecting sequence of the interpolating method of presentation video code device.

Figure 10 (a) is the diagram of the bitstream structure of expression code signal Bitstream of the present invention.Figure 10 (b) is to be the diagram of the bitstream structure of the code signal Bitstream under the situation of the unit method of switching pixel interpolation with the frame.

Figure 11 is the block diagram of structure of the other image decoder of this form of implementation of expression.

Figure 12 is the diagram that expression remains on an example of the interpolation classification in the pixel interpolation classification transformation component shown in Figure 11.

Figure 13 is the block diagram according to the structure of the picture coding device of form of implementation 2 of the present invention.

Figure 14 is that expression will be as the block diagram of the structure of the image decoder of the code signal Bitstream3 decoding of the output of picture coding device shown in Figure 13.

Figure 15 utilizes storage from above-mentioned form of implementation 1 to the method for encoding images of form of implementation 3 or the floppy disk of image decoding method, the key diagram of situation about being implemented by computer.Figure 15 (a) is the example of expression as the physical format of the floppy disk of recording medium.Figure 15 (b) is outward appearance, the cross-section structure of expression during from the top view of floppy disk, and floppy disk, and Figure 15 (c) is illustrated in the structure that the record regenerating that carries out said procedure on the floppy disk FD is used.

Figure 16 is that expression realizes that the content distribution sends the block diagram of the total of the contents providing system of serving.

Figure 17 is the diagram of an example of the outward appearance of expression portable phone.

Figure 18 is the block diagram of expression portable phone.

Figure 19 is that the described encoding process of above-mentioned form of implementation or the equipment of decoding processing and the diagram of using the system of this equipment are carried out in the explanation use.

Embodiment

Below, with reference to description of drawings concrete form of implementation of the present invention.

(form of implementation 1)

Fig. 4 is the block diagram of the structure of expression picture coding device 400 of the present invention.In addition,, give identical reference number, omit its explanation for each structure member in the picture coding device 100 of prior art shown in Figure 2 and structure member and the signal of finishing same action.

Picture coding device 400, be according to image category signal PicType from the outside input, optionally carry out the picture coding device of the different pixel interpolation of precision, comprise: difference engine 101, image encoding portion 102, variable-length encoding part 103, image decoding portion 104, adder 105, video memory 106, block of pixels acquisition unit 107, switch 108, switch 109, switch 401, switch 402, the A403 of pixel interpolation portion, the B404 of pixel interpolation portion, motion deduction portion 111 and pixel interpolation use judging part 112.Picture coding device 400 is imported the picture signal Img that comprises image category signal PicType from the outside.On switch 401 and switch 402, as image category signal PicType, for example usually input is under the situation with reference to " 1 " of other frame, expression B image not, and switch 401 and switch 402 switch to terminal " 1 " side, utilize the A403 of pixel interpolation portion to carry out pixel interpolation.That is, when switch 401 and switch 402 are connected respectively to terminal " 1 " and go up,, adopt the pixel interpolation that is undertaken by the A403 of pixel interpolation portion for block of pixels signal Blk.At the A403 of pixel interpolation portion, utilize filter taps few, for example tap number is the interpolating method of the simplification of " 4 ".As image category signal PicType, for example, in input expression during with reference to " 2 " of the image of other frame, promptly, when " 1 " of input expression B image in addition value, switch 401 and switch 402 switch to terminal " 2 " side, for block of pixels signal Blk, adopt the pixel interpolation that carries out with the B404 of pixel interpolation portion.At the B404 of pixel interpolation portion, utilize the filter taps number many, for example tap number is the high interpolating method of precision of " 8 ".Like this, be carried out the block of pixels Blk of pixel interpolation,, be imported into difference engine 101 as prediction image signal Pred.

In addition, because two image generation forecast images of B image while reference, so, in each image of reference, be necessary to carry out pixel interpolation.Thereby, and only to compare with reference to the P image of an image, the operand of pixel interpolation becomes 2 times, so, in the B image, use the interpolating method of simplifying, in each image, can be effective with the operand smoothing of necessity.Thereby in the B image with other frame reference, it also is useful using the interpolating method of simplifying.

In addition, the value " 2 " of image category signal PicType, " 1 ", " 0 ", be for convenience of description and the definition value, so long as can distinguish the value of the interpolating method of a plurality of pixels, can be any value.By representing the image category signal PicType notice image decoder of employed pixel interpolation method, in image decoder, can use the pixel interpolation method identical with the pixel interpolation portion that in picture coding device, uses.

Fig. 5 is the block diagram of the structure of expression image decoder 500 of the present invention.In addition, in the figure,,, omit the device explanation so give identical label owing to carried out explanation with image decoder 200 same structure members shown in Figure 3.Image decoder 500 is image decoders that the different pixel interpolation method encoded signals Bitstream that each image category is used is deciphered, and comprising: variable length decoding part 210, image decoding portion 202, adder 203, video memory 204, block of pixels acquisition unit 207, switch 208, switch 209, pixel interpolation uses judging part 212, switch 501, switch 502, A503 of pixel interpolation portion and the B504 of pixel interpolation portion.In image decoder 500, variable-length encoding part 505, will be from the code signal Bitstream variable length decoding of outside input, image category signal PicType and difference image code signal CodedRes and kinematic parameter MotionParam are separated from variable length decoded signal Bitstream, respectively image category signal PicType is input to switch 501 and switch 502, kinematic parameter signal MotionParam is input to pixel interpolation uses judging part 212 and block of pixels acquisition unit 207, CodedRes is input to image decoding portion 202 with the difference image code signal.When as image category effect picType, for example, when not being input on switch 501 and 502 with reference to " 1 " of the B image of other frame expression, switch 501 and switch 502 switch to terminal " 1 " side, utilize the A503 of pixel interpolation portion to carry out pixel interpolation.That is, when switch 501 and switch 502 are connected respectively to terminal " 1 " and go up,, adopt the pixel interpolation that is undertaken by the A503 of pixel interpolation portion to block of pixels signal Blk.Adopt the filter taps number few at the A503 of pixel interpolation portion, for example the simplification interpolating method of tap number " 4 ".As image category signal PicType, for example, when input is represented with reference to " 2 " of the P image of other frame, promptly, during " 1 " of input expression B image in addition value, switch 501 and switch 502 switch to terminal " 2 " side, to block of pixels signal Blk, adopt and carry out institute's pixel interpolation by the B504 of pixel interpolation portion.At the B504 of pixel interpolation portion, adopt the filter taps number many, for example tap number is the high interpolating method of precision of " 8 ".

In addition, because the B image is simultaneously with reference to two image generation forecast images, so, be necessary to carry out pixel interpolation at each image of institute's reference.Thereby, and only to compare with reference to the P image of an image, the pixel interpolation operand becomes 2 times, so, in the B image, use the interpolating method of simplifying, can be effective with operand smoothing necessary in each image.Thereby in the B image with reference to other frame, it is useful using the interpolating method of simplifying.

In addition, in the occasion of all having used the pixel interpolation bit stream of same pixel interpolation filter to decipher to P, B image with picture coding device, in image decoder 500, can use and only simplify the B image.In this case, because the pixel interpolation filter of B image is different with the pixel interpolation filter that uses in picture coding device, so, image quality aggravation takes place in the B image, but because the B image is by the lacking of other image reference, so, compare with the situation that image quality aggravation takes place in the P image, to later image, image quality aggravation is propagated less.

And then, in image decoder 500, do not install under the filter situation identical, can replace the pixel interpolation filter of the immediate tap number below the tap number of the pixel interpolation filter that use adopts in picture coding device with the pixel interpolation filter that uses in the picture coding device.In this case, owing to use and the different pixel interpolation filter of employed pixel interpolation filter in picture coding device, so, image quality aggravation can take place, but can proceed the decoding of bit stream.

Fig. 6 (a) is illustrated in the 1/2 pixel filter, calculates be positioned at from the diagram of existing pixel at an example of the method for the pixel value of the pixel of 1/2 locations of pixels of i direction of principal axis skew.Fig. 6 (b) is illustrated in the 1/2 pixel filter, calculates be positioned at from the diagram of existing pixel at an example of the method for the pixel value of the pixel of 1/2 locations of pixels of j direction of principal axis skew.In Fig. 6 (a) and Fig. 6 (b), the pixel of zero expression integer position, the pixel of * expression decimal position.In addition, in the figure, i, j are integers.(x y) is illustrated in coordinate (x, the pixel value of y) locating to I.1/2 pixel filter is realized by the software and the integrated circuit of the pixel value of the pixel of the non-existent decimal position of in fact pixel of the calculated for pixel values usefulness * expression at the pixel place of the integer position of utilizing zero expression.In Fig. 6 (a), (i-0.5, j) (i-0.5, the situation in the time of j) describes the pixel value I of the pixel of position to being positioned at coordinate.For example, tap number N (N is the even number natural number) when " 2 ", to coordinate (i-0.5, j), utilize two pixels of the axial both sides of i adjacency pixel value (I (and i-1, j) and pixel value I (i, j).Pixel value I (i-0.5, j), utilize as the long-pending of the axial pixel value of i and formula 1, be expressed as follows.

$I(i-0.5,j)=\underset{k=0}{\overset{N}{\mathrm{\Σ}}}{a}_k\·I(i-\mathrm{trunc}(N/2)+k,j)...$ (formula 1)

In formula 1, a _kThe expression filter coefficient, trunc (n) represents with respect to the n fractions omitted.Like this, in formula 1, by selecting appropriate filters coefficient a _k, as 1/2 pixel (i-0.5, (i-0.5 j), is to obtain as the mean value of N pixel value in the pixel of its both sides to the pixel value I of locational pixel j).In addition, for example, when tap number N is " 4 ", to (i-0.5, locational pixel j), remove the pixel value I (i-1 of two pixels of the adjacency of i direction of principal axis both sides, j) and pixel value I (i, j) outside, further utilize the pixel value I (i-2 of the axial both sides of i pixel adjacent, j) and pixel value I (i+1, j).Similarly, be increased to " 6 ", when " 8 ",, can obtain at an easy rate by these values being updated to the N in the formula 1 at tap number N.

In Fig. 6 (b), (i, j-0.5) (i, situation j-0.5) describes the pixel value I of the pixel of position at coordinate to obtaining.That is, predicted picture moves the situation of fractional pixel at the j of reference image direction of principal axis.For example, at tap number M (M is the even number natural number) when " 2 ", to coordinate (i, the j-0.5) pixel of position, utilize the axial both sides of j adjacency two pixel value I (i, j-1) and pixel value I (i, j).This pixel value II (i, j-0.5), be used as the long-pending of the axial pixel value of j and formula 2, be expressed as follows.

$I(i,j-0.5)=\underset{m=0}{\overset{N}{\mathrm{\Σ}}}{a}_m\·I(i,j-\mathrm{trunc}(M/2)+m)...$ (formula 2)

In formula 2, am represents filter coefficient.In this case, be increased in tap number " 4 ", " 6 " when " 8 ", by being worth the M of substitution formula 2, can obtain coordinate (i, the pixel value of the pixel of position j-0.5).

In addition, do not illustrate among the figure, at predicted picture when the i of reference image direction of principal axis, j direction of principal axis move 1/2 pixel respectively, at coordinate (i-0.5, j-0.5) locate predicted picture pixel value I (i-0.5, j-0.5), be used as the long-pending of i direction of principal axis and the axial pixel value of j and formula 3 be expressed as follows.

$I(i-0.5,j-0.5)=\underset{m=0}{\overset{M}{\mathrm{\Σ}}}\underset{k=0}{\overset{N}{\mathrm{\Σ}}}{a}_m\·a_k\·I(i-\mathrm{trunc}(N/2)+k,j-\mathrm{trunc}(M/2)+m)...$

(formula 3)

As can be seen from above formula, realize that pixel filter filter taps (tap) number of pixel interpolation A of portion and the B of pixel interpolation portion is many more, precision of prediction is high more, and conversely, the calculation process amount increases, and the processing load of picture coding device increases the weight of.

As mentioned above, between the pixel interpolation portion of picture coding device 400 and image decoder 500, can use the pixel interpolation portion of a plurality of different estimated performances, treating capacity.Use the pixel interpolation portion of different estimated performances, treating capacity that following advantage is arranged.For purposes of simplicity of explanation, the A of pixel interpolation portion lacks than the B of pixel interpolation portion treating capacity, and the B of pixel interpolation portion is than the A of pixel interpolation portion forecasting efficiency height.As the image decoder that the code signal of picture coding device output of the present invention is deciphered, consider only to be equipped with the image decoder of the A of pixel interpolation portion and to be equipped with the A of pixel interpolation portion and two kinds of picture coding devices of the image decoder of the B of pixel interpolation portion.

The treating capacity that preceding a kind of picture coding device is suitable for requirement is few, the equipment that disposal ability is low.A kind of image decoder in back is suitable for the big equipment for the treatment of capacity.A kind of image decoder in back can have upper interchangeability to preceding a kind of image decoder with using the code signal of any one the pixel interpolation portion among pixel interpolation A of portion and the pixel interpolation B to decipher.As described above, by according to image decoder, select pixel interpolation portion with suitable estimated performance, treating capacity, coded system can be applied to kind equipment widely.

In addition, except that the code signal that is used to generate the disposal ability that is adapted to image decoder, can switch pixel interpolation portion according to the disposal ability of picture coding device.For example, under the size of coded image and frame lag behind big situation, require the treating capacity of whole encoding process big.Thereby, lag behind under the situation below the certain value in image encoded size and frame, use the pixel interpolation B of portion, lag behind under the situation more than the certain value in image encoded size and frame, the instructions for use disposal ability is low, that is, handle the little A of pixel interpolation portion of load, can not improve the treating capacity that requires whole encoding process.

In addition, using the occasion of carrying out the time-sharing system image encoding of a plurality of processing simultaneously, because the treating capacity that the influence of other processing spends on the image encoding might dynamically change.Therefore, the treating capacity on spending in image encoding can be used the many B of pixel interpolation portion for the treatment of capacity when certain value is above, when certain value is following, can use the few A of pixel interpolation portion for the treatment of capacity in the treating capacity on spending in image encoding.

In addition, also can be equipped with a plurality of image pixel interpolation portions that are suitable for special properties, be that unit switches pixel interpolation portion with the frame according to image institute character.For example, under the important situation of edge information such as literal, use the pixel interpolation portion of the retention excellence at edge.If can carry out the switching of a plurality of pixel interpolation portion, owing to can select to be suitable for most the pixel interpolation method of image property, so, can further improve forecasting efficiency.

And then, as B image etc. simultaneously with reference under the situation of two image generation forecast images, be necessary to carry out pixel interpolation in reference to image at each, compare with a P image of an image of reference, the operand of pixel interpolation becomes 2 times, so, if only in the B image, use the interpolating method of simplifying, because operand smoothing that can each image is necessary, so, can realize the software of real-time action easily.

Fig. 7 (a) is the relation diagram of the method for the image category of expression each frame of expressing dynamic image and pixel interpolation.Be illustrated in the flow chart of order of the selection interpolating method of picture coding device 400 of the present invention and image decoder 500 on Fig. 7 (b).Shown in Fig. 7 (a), will represent that each frame is the I image, or the image category signal PicType of B image or P image, give picture coding device 400 from the outside.At the I image, owing to carry out coding in the picture, so the pixel value of predicted picture is " 0 ".Thereby pixel interpolation uses judging part 112 that switch 108 and switch 109 are switched to terminal " 1 ", does not carry out pixel interpolation.In addition, at the B image, according to the value of image category signal PicType, switch 401 and switch 402 switch to terminal " 1 ", use the pixel interpolation A403 of portion to carry out easy pixel interpolation A.In addition, at the P image, according to image category signal PicType, switch 401 and switch 402 switch to terminal " 2 ", use the pixel interpolation B402 of portion to carry out high-precision pixel interpolation B.

That is,,, carry out the selection shown in the flow chart of Fig. 7 (b) and handle by value diverter switch 401 and switch 402 according to image category signal PicType at picture coding device 400.Switch 401 and switch 402 judge whether the value of the image category signal PicType of input goes up the value (S701) of expression B image, if the value of expression B image, by being connected respectively on the terminal " 1 ", select the interpolating method A (S702) that is undertaken by the A403 of pixel interpolation portion.In addition, not under the situation of value of expression B image at the value table of image category effect PicType, by being connected respectively on the terminal " 2 ", select the interpolating method B (S703) that is undertaken by the B404 of pixel interpolation portion.In picture coding device 400, carry out the processing of above-mentioned steps S701 to S703 repeatedly for the frame of the picture signal Img of each input.

As mentioned above, adopt picture coding device 400, because for originally in order to carry out the bigger B image of processing duty ratio that image encoding is handled, can select to handle the littler pixel interpolation portion of load, so, even in the lower picture coding device of disposal ability, also can carry out pixel interpolation.In addition, for being not easy,, can dwindle owing to select of the influence of the lower pixel interpolation portion of precision to other frame owing to select the lower pixel interpolation portion of precision by the B image of other frame reference.And then, usually, owing to select pixel interpolation portion according to the image category signal PicType that is included in the picture signal, so, in code signal Bitstream, needn't comprise which pixel interpolation portion of expression corresponding to the information of which frame, can corresponding minimizing in the treating capacity of variable-length encoding part, the amount that is reduced is with respect to this part information.In addition, for the less P image of the processing duty ratio of original picture coding device, handle load and become big, but owing to can select the higher pixel interpolation portion of precision of prediction, so,, also can carry out the higher pixel interpolation of precision of prediction even in the lower picture coding device of disposal ability.In addition, owing to, can carry out the higher pixel interpolation of precision of prediction for P image with reference to other frame, so, can prevent the deterioration of image quality to greatest extent.

Fig. 8 is that expression is the block diagram of the structure of the unit picture coding device 800 that switches the pixel interpolation method with the frame.In Fig. 8, structure member and the signal identical with the picture coding device 400 of the picture coding device 100 of Fig. 2 and Fig. 4 are given identical label, omit its explanation.Picture coding device 800 is equipped with: difference engine 101, image encoding portion 102, image decoding portion 104, adder 105, video memory 106, block of pixels acquisition unit 107, switch 108, switch 109, motion deduction portion 111, pixel interpolation uses judging part 112, the A403 of pixel interpolation portion, the B404 of pixel interpolation portion, pixel interpolation switching position judging part 801, switch 802, variable-length encoding part 803, switch 804 and switch 805.Pixel interpolation switching position judging part 801 is at the switching unit's (frame that detects A403 of pixel interpolation portion and the B404 of pixel interpolation portion from the picture signal Img of input, sheet, miniature, piece etc.) under the situation, by output pixel interpolation switch-over control signal SetPolatorType " 1 ", switch 802 is connected (conducting state).

Here, for example, with the frame of picture signal Img switching unit, diverter switch 802 as the pixel interpolation method.The effect of switch 802 is, in beginning place that becomes each frame that switches unit, only in extremely short time interval, make pixel interpolation sorting signal PolatorType and switch 804 and 805 conductings, and in the interval At All Other Times of aforementioned switching unit, blocking pixel interpolation sorting signal is to the input of PolatorType and switch 804 and switch 805.This is in order to prevent that A403 of pixel interpolation portion and the B404 of pixel interpolation portion switch at the coding that switches unit midway.Switch 802 with the beginning of each frame as switching timing, when being connected by pixel interpolation switch-over control signal SetPolatorType " 1 ", in the regular hour, closed terminal will be from the pixel class signal PolatorType actuating switch 804 and the switch 805 of outside input.This pixel interpolation sorting signal PolatorType, be with the residual volume of the data in the not shown transmission buffer in the picture coding device 800 etc. as standard test, according to the processing load of picture coding device 800 or according to the decoding ability of predetermined picture code translator, be used to the signal of selecting the pixel interpolation classification to use from the outside input.When through aforementioned certain hour, switch 802 is opened terminal, becomes disconnection, when importing pixel interpolation control signal SetPolatorType " 1 " once more, becomes connection.In addition, when switch 804 and switch 805, when importing pixel interpolation sorting signal PolatorType from the outside through switch 802, till the pixel interpolation sorting signal PolatorType of input different value, remain connected to by the state on the splicing ear of the value representation of this pixel interpolation sorting signal PolatorType always.

For example, in the beginning of some frames, when the value of the pixel class signal PolatorType of input was " 1 " in the short time of switch 802 conductings, switch 804 and switch 805 were connected respectively on the terminal " 1 ", and remain on this state.Then, when the beginning at next frame, when the pixel interpolation sorting signal PolatorType that imports in the short time of switch 802 conductings was " 0 ", switch 804 and switch 805 were connected respectively on the terminal " 2 ", and remain on this state.Whereby, cause the switching of only carrying out the pixel interpolation method, can prevent to switch the pixel interpolation method midway at the coding of frame in the beginning of frame.

In the interpolation that utilizes switching judging, have the different a plurality of pixel interpolation portion of precision of prediction, at each frame, undertaken by from a plurality of pixel interpolation methods, in the scope that does not surmount treating capacity, selecting the filter of forecasting efficiency the best.In addition, as other pixel interpolation that uses switching judging, in the middle of each frame cataloged procedure, under the situation that the disposal ability of picture coding device becomes not enough, can switch to few pixel interpolation method of handling at next frame.Consequently, utilize pixel interpolation sorting signal PolatorType to select new pixel interpolation portion.

Fig. 9 is the flow chart that is illustrated in the selecting sequence of interpolating method in the picture coding device 800.Specifically, in picture coding device 800, value according to pixel interpolation switch-over control signal SetPolator connects switch 802, by during connecting switch 802, according to value diverter switch 804 and switch 805, carry out the selection shown in the flow chart of Fig. 9 and handle from the pixel interpolation sorting signal PolatorType of outside input.Picture coding device 800 detects picture signal Img (S901) in the middle of the frame title of representing the beginning of each frame etc., from pixel interpolation switching position judging part 801, for example output pixel interpolation switch-over control signal SetPolatorType " 1 " connects (S902) with switch 802.Picture coding device 800, judgement switch 802 connect during the value of pixel interpolation sorting signal PolatorType of input whether be " 1 " (S903), if the words of " 1 ", switch 804 and switch 805 are connected respectively on the terminal " 1 ", select to utilize the interpolating method A (S904) of the A403 of pixel interpolation portion.If pixel interpolation sorting signal PolatorType is not " 1 ", switch 804 and switch 805 are connected respectively on the terminal " 2 ", select to utilize the interpolating method B (S905) of the B404 of pixel interpolation portion.Picture coding device 800 repeats from the processing of above-mentioned steps S901 to S905 each frame of picture signal Img of input.

And then, picture coding device 800 is in variable-length encoding part 803, the pixel interpolation classification is being switched under the situation of unit as frame, for each frame as the code signal Bitstream of the output of picture coding device 800, for example, the value of record output pixel interpolation sorting signal PolatorType on each frame title of code signal Bitstream.Figure 10 (a) is the diagram of the bitstream structure of expression code signal Bitstream of the present invention.Figure 10 (b) is that expression is the diagram of the bitstream structure of the code signal Bitstream of unit when switching the pixel interpolation method with the frame.The feature of code signal of the present invention is that PolatorType is included in the code signal Bitstream with the pixel interpolation sorting signal.By this bit stream,,, can use and the identical pixel interpolation method of when encoding, using of pixel interpolation portion by research pixel interpolation sorting signal at code translator with code signal Bitstream decoding of the present invention.

In the code signal Bitstream shown in Figure 10 (a), in the title 1001 that gives whole code signal Bitstream (for example oblique line portion), record and narrate the value of pixel interpolation sorting signal PolatorType of interpolating method that expression is used for the pixel interpolation of each frame.In addition, in the code signal Bitstream shown in Figure 10 (b), in being located at the frame title 1002 of each frame (for example oblique line portion), record and narrate the value of pixel interpolation sorting signal PolatorType of interpolating method that expression is used for the pixel interpolation of this frame.Like this, by pixel interpolation sorting signal PolatorType being stored in as the title 1001 of the beginning part of code signal Bitstream and becoming in frame title 1022 grades of beginning of random access point, in picture coding device, by from title 1001 or random access point incoming coded signal Bitstream, before the coding of described frame, can specify the pixel interpolation classification of this frame.

In addition, here, illustrated to be the situation that unit switches pixel interpolation portion with the frame, but not only be unit with the frame, also can be to switch pixel interpolation portion less than the image-region position unit of frame (also can be the above zone of pixel of set, for example the sheet of MPEG, miniature, piece etc.).For example, with sheet when switching unit, can record and narrate on the title that is added on the whole code signal Bitstream corresponding to the value of the pixel interpolation sorting signal PolatorType of each sheet, also can be aggregated on each frame and record and narrate on the frame title.In addition, also can on sheet title, record and narrate the value of the pixel interpolation sorting signal of each sheet corresponding to each sheet.In addition, with under miniature or the situation of piece, the value of the pixel interpolation sorting signal of each miniature or each piece can be aggregated on each sheet, record and narrate on the sheet title as the switching unit of pixel interpolation method.

As described above, be input to the value of the pixel interpolation sorting signal PolatorType in the picture coding device of the present invention by setting, make and to select pixel interpolation portion, can make the code signal that the disposal ability with the image decoder of the code signal of the picture coding device output of the present invention of regenerating adapts according to the disposal ability of image decoder.In addition, can select to be adapted to the picture interpolation portion of the disposal ability of picture coding device.

In addition, in existing image encoding mode, the position according to prediction image signal is arranged, switch to the coded system of the pixel interpolation method of the pixel value that generates this position.For example, generate the occasion of the pixel value of 1/2 location of pixels, the pixel interpolation method of selecting 1/2 location of pixels to use when generating the pixel value of 1/4 location of pixels, is selected the coded system of 1/4 location of pixels with the pixel interpolation method.But, these coded systems, for required location of pixels, have to select one of them pixel interpolation method, different therewith, in coded system of the present invention, owing to be equipped with a plurality of pixel interpolation portion for the calculating of the pixel value of the same position of prediction image signal, to the calculating of the pixel value of same location of pixels, then can freely select a plurality of pixel interpolation methods.In image encoding mode of the present invention,, the said method that switches a plurality of pixel interpolation methods can be made up mutually for different location of pixels.In this case, the different pixel value of precision of prediction of a plurality of pixel interpolation portion calculate to(for) same location of pixels also is equipped with when being equipped with each pixel value of a plurality of pixel interpolation portion calculate to(for) different location of pixels.

In addition, in above-mentioned each form of implementation, the picture coding device that is equipped with two pixel interpolation portions is illustrated, but also can be equipped with pixel interpolation portion more than three.In this case, one of select to use in the pixel interpolation more than three, the pixel interpolation sorting signal that also can comprise the classification of the employed pixel interpolation of expression portion carries out encoded signals.

Figure 11 is the block diagram of structure of the other image decoder 1100 of this form of implementation of expression.In Figure 11, owing to carried out explanation for structure member and the signal finished with picture coding device shown in Figure 5 500 same action, so, give identical label, omit its explanation.Image decoder 1100 comprises: image decoding portion 202, adder 203, video memory 204, block of pixels acquisition unit 207, switch 208, switch 209, pixel interpolation uses judging part 212, the A503 of pixel interpolation portion, the B504 of pixel interpolation portion, variable length decoding part 1101, pixel interpolation classification transformation component 1102, switch 1103 and switch 1104.Code signal Bitstream2 by picture coding device shown in Figure 8 800 outputs is input in the image decoder 1100.That is, the last record of code signal Bitstream2 that is input in the image decoder 1100 has pixel interpolation sorting signal PolatorType1.Being characterized as of image decoder 1100, under the pixel interpolation portion that represents with the pixel interpolation sorting signal PolatorType1 among the code signal Bitstream2 is not provided in situation in the image decoder 1100, one of use in the pixel interpolation portion that image decoder 1100 is equipped with to substitute.

1100 of image decoders are equipped with the value " 1 " of remarked pixel interpolation sorting signal PolatorType and two kinds of pixel interpolation portions of " 2 ", that is, only be equipped with A503 of pixel interpolation portion (filter taps is counted N=4) and the B504 of pixel interpolation portion (filter taps is counted N=8).In image decoder 1100, variable length decoding part 1101 is separated into the decoding of code signal Bitstream2 variable length difference image code signal CodedRes and kinematic parameter and goes out MotionParam and pixel interpolation sorting signal PolatorType1.Pixel interpolation classification transformation component 1102, expression is utilized pixel interpolation sorting signal PolatorType1 appointment pixel interpolation classification, described each the pixel interpolation portion of statement characteristic parameter, have or not installation, ready-made interpolation classification table remains on inside.Pixel class transformation component 1102, whether the pixel interpolation portion with the value representation of pixel interpolation sorting signal PolatorType1 is installed in the image decoder 1100 according to described interpolation classification table judgement.

Figure 12 is the diagram of an example in the interpolation classification table of representing to remain in the pixel interpolation classification transformation component shown in Figure 11 1,102 1200.As shown in the figure, in interpolation classification table 1200, recorded and narrated with the installation of each pixel interpolation portion of the value representation of pixel interpolation sorting signal PolatorType1 have or not 1201, the filter taps number (N) 1203 of the characteristic of the value 1201 of pixel interpolation sorting signal PolatorType1 and each pixel interpolation portion of representing with the value 1 of pixel interpolation sorting signal PolatorType1.Pixel interpolation classification transformation component 1102, in the value of pixel interpolation sorting signal PolatorType1 is that the value that the C of pixel interpolation portion (filter taps is counted N=16) is not installed in the presentation video code translator 1100 for example is under the situation of " 3 ", in the pixel interpolation portion of installing, select to record and narrate the B504 of pixel interpolation portion (filter taps is counted N=8) that in pixel interpolation classification table 1200, records and narrates the most approaching C of pixel interpolation portion that represents by pixel interpolation sorting signal PolatorType1 of institute's filter taps number (N) 1203, with pixel interpolation sorting signal PolatorType1 value, for example be transformed into the value of " 2 ", export as pixel interpolation sorting signal PolatorType2.Undeniable, by this conversion process, in image decoder 1100, because the different pixel interpolation portion of pixel interpolation portion that uses when utilizing with coding, so can cause the deterioration of image quality, still, handle owing to can carry out the decoding of difference image code signal CodedRes and kinematic parameter signal MotionParam, so, can preview (prediction) reflection of decoding.During as pixel interpolation sorting signal PolatorType2 input " 1 ", switch 1103 and switch 1104 switch to terminal " 1 " side, and Recon uses the pixel interpolation A503 of portion to the decoding picture signal.As pixel interpolation sorting signal PolatorType2 input " 2 " time, switch 1103 and switch 1104 switch to terminal " 2 " side, and Recon uses the pixel interpolation B504 of portion to the decoding picture signal.

As described above, according to image decoder 1100 of the present invention, even at incoming coded signal Bitstream, this code signal Bitstream comprises that expression is not installed under the situation of pixel interpolation sorting signal PolatorType2 of the pixel interpolation portion in the image decoder 1100, the effect of deciphering for the code signal Bitstream that can not have obstacle ground with input.In addition, be under one the situation in the pixel interpolation portion that image decoder 1100 is had, by using this pixel interpolation portion forcibly, can be with the code signal Bitstream decoding of input.The pixel interpolation portion that is equipped with more than three in addition, in the image decoder 110 of this form of implementation, is equipped with two pixel interpolation portions, even but also can carry out same processing.In addition, shown in enforcement form 1, with the frame unit, perhaps little than frame image-region is the occasion that unit switches the pixel interpolation classification, image decoder 1100 switches pixel interpolation portion in the moment of pixel interpolation classification change.In addition, here, showing the characteristic of a plurality of pixel interpolation portion with the filter taps numerical table, but might not be the tap number of filter, also can be other parameter.And then, knowing that image decoder 1100 all is equipped with under the situation of the pixel interpolation filter with possibility of using, can not have the pixel interpolation classification transformation component 1102 of image decoder 1100 yet in picture coding device.In this case, pixel interpolation sorting signal PolatorType1 is remained untouched use as pixel interpolation sorting signal PolatorType2.

(form of implementation 2)

Figure 13 is the block diagram of expression according to the structure of the picture coding device 1300 of form of implementation 2 of the present invention.In addition, for Fig. 2, Fig. 4 and picture coding device 100 shown in Figure 8, each structure member moves equally in picture coding device 400 and the picture coding device 800 structure member and signal are given identical reference number, omit its explanation.Picture coding device 1300, the frame that becomes coded object be not by the situation of the frame of other frame reference under, use the littler pixel interpolation portion for the treatment of capacity in the pixel interpolation, in other frame, be used as under the situation of reference frame use, use the big pixel interpolation portion for the treatment of capacity, comprising: difference engine 101, image encoding portion 102, image decoding portion 104, adder 105, video memory 106, block of pixels acquisition unit 107, switch 108, switch 109, pixel is used judging part 112, switch 1301, switch 1302, variable-length encoding part 1303, cavity 1304 and motion deduction portion 1305.With presentation code to picture frame afterwards whether as reference frame use with reference to index signal AvairableRef, be input in the motion deduction portion 1305 from the outside.With reference to index signal AvairableRef, be according to the operator of picture coding device 1300 setting with not shown input parts such as keyboard input, signal from the outside input, on duty when being " 0 ", the frame of coded object shown in the expression does not use as reference frame, on duty when being " 1 ", the frame of coded object shown in the expression uses as reference frame afterwards.

Therewith accordingly, switch 1301 and switch 1302 are when the value of reference index signal AvairableRef is " 0 ", promptly, for not by the frame of the coded object of other frame reference, be connected respectively in the action " 1 ", select the littler A403 of pixel interpolation portion for the treatment of capacity.In addition, when reference index signal AvairableRef is " 1 ", that is, the frame for the coded object that is used as the reference frame use afterwards is connected respectively on the terminal " 2 ", yet selects the big B404 of pixel interpolation portion of forecasting efficiency high treating capacity.In addition, when reference index signal AvairableRef was " 0 ", switch 1304 was connected, and described frame coding difference image signal ReconRes portion is stored in the video memory 106.Otherwise when reference index signal AvairableRef was " 1 ", switch 1304 became connection, and described frame coding differential signal ReconRes is stored in the video memory 106.Variable-length encoding part 1303, for each frame as code signal Bitstream3 output, that have chip architecture shown in Figure 10 of picture coding device 1300, record output is with reference to the value of index signal AvairableRef on the title of each frame of code signal Bitstream3.

The block diagram of the structure of the image decoder 1400 that the code signal Bitstream3 as the output of picture coding device 1300 that Figure 14 is that expression will be shown in Figure 13 deciphers.On this figure, for with the image decoder 200 shown in Fig. 3, Fig. 5 and Figure 11, the structure member and the signal of each structure member same action of image decoder 500 and image decoder 1100 owing to carried out explanation, omit its explanation so give identical reference number.Image decoder 1400, be to comprise the aforementioned image decoder of deciphering with reference to the code signal Bitstream3 of index signal AvairableRef, comprise: image decoding portion 202, adder 203, video memory 204, block of pixels acquisition unit 207, switch 208, switch 209, pixel interpolation uses judging part 212, variable length decoding part 1401, switch 1402, switch 1403 and switch 1404.Variable length decoding part 1401 with the code signal Bitstream3 variable length decoding of input, is separated into difference image code signal CodedRef/ kinematic parameter MotionParam and with reference to index signal AvairableRef.Be separated with reference to index signal AvairableRef, be imported into switch 1402 and switch 1403 and switch 1404.When the value of reference index signal AvairableRef is " 0 ", that is, the code signal Recon that does not use described frame is during as reference frame, and switch 1404 disconnects.Thereby described frame coding picture signal Recon is not stored in the video memory 204.Otherwise when the value with reference to index signal AvairableRef is " 1 ", that is, during as reference frame, switch 1404 is connected with the code signal Recon of described frame.Thereby described frame coding picture signal Recon is stored in the video memory 204.In addition, when the value of reference index signal AvairableRef is " 0 ", that is, the code signal Recon that does not use described frame is during as reference frame, switch 1402 and switch 1403 are connected respectively on the terminal " 1 ", select the few A503 of pixel interpolation portion of pixel interpolation treating capacity.Otherwise, when the value of reference index signal AvairableRef is " 1 ", be connected respectively on the terminal " 2 " the many but high B504 of pixel interpolation portion of forecasting efficiency of the treating capacity of selecting pixel interpolation.

As mentioned above, in the picture coding device 1300 and image decoder 1400 of this form of implementation, when by other frame reference do in the coded object frame to use the few pixel interpolation method for the treatment of capacity amount the time, the deterioration of the low image quality that causes of the precision of prediction of the pixel interpolation method that treating capacity is few is also influential to the image quality of other frame, but, because only to not used the few pixel interpolation method for the treatment of capacity by the frame of the coded object of other frame reference, so, because the deterioration of the image quality of the frame of the low coded object that causes of precision of prediction can not impact other frame, can alleviate the load of the processing of the pixel interpolation that is used for picture coding device 1300 and image decoder.Thereby as picture coding device 1300 and code translator, the deterioration that precision of prediction can be caused remains on bottom line, and can reduce the processing load in the image encoding processing significantly.

In addition, in the image decoder 1400 of this form of implementation, switch 1404 is set, not not being stored on the video memory 204 by the decoding picture signal Recon of the frame of other frame reference, but, in other image decoder of the present invention that had illustrated, also can be equipped with switch 1404.

In addition, in above-mentioned form of implementation 1 and form of implementation 2, use the situation of pixel interpolation A403 of portion and the B404 of pixel interpolation portion to be illustrated to utilizing switch to switch, but the present invention is not limited thereto, also A403 of pixel interpolation portion and the B404 of pixel interpolation portion can be replaced as a pixel interpolation portion of moving in a plurality of patterns.This pixel interpolation portion, for example be according to precision of prediction or processing load as purpose, a pixel filter according to a plurality of operation methods or a plurality of operational formula are carried out computing determines a pattern (operation method or operational formula) according to the parameter that gives from the outside.

(form of implementation 3)

And then, be recorded on the recording mediums such as floppy disk by realizing the program that the method for encoding images installation drawing shown in the above-mentioned form of implementation looks like the structure of the method for translating, can independently implement the processing shown in above-mentioned each form of implementation simply in the computer system.

Figure 15 utilizes the method for encoding images of storage from above-mentioned form of implementation 1 to form of implementation 3 or the floppy disk of the program of image decoding method, utilizes the key diagram of the situation that computer system implements.

Figure 15 (b) expression outward appearance, cross-section structure and the floppy disk during from the top view of floppy disk, Figure 15 (a) are expressions as the example of the physical format of the floppy disk of recording medium main body.Be loaded in the floppy disk FD in the shell F, on the surface of this dish, from the inwardly all a plurality of magnetic track Tr that form concentric circles of periphery, each magnetic track is divided into 16 sector Se along angle direction.Thereby in the floppy disk of storage said procedure, in the zone on being allocated in above-mentioned floppy disk FD, record is as the method for encoding images of said procedure.

In addition, Figure 15 (c) expression is used for carrying out the structure of the record regenerating of said procedure on floppy disk FD.Under situation about said procedure being recorded on the floppy disk FD, will write from computer system Cs as the image volume method or the image decoding method of said procedure via floppy disk FD.In addition, utilize the program in the floppy disk that above-mentioned method for encoding images is constructed under the situation in the computer system, utilize floppy disk read routine from the floppy disk, send computer system to.

In addition, in the superincumbent explanation, the situation as recording medium use floppy disk is illustrated, but utilizes CD can carry out too.In addition, simple medium are not limited to these, utilize IC-card, and the device that ROM box etc. can logging program can be implemented equally.

Figure 16 to Figure 19 is the equipment that encoding process shown in the above-mentioned form of implementation or decoding processing are carried out in explanation, and the key diagram that utilizes the system of these equipment.

Figure 16 is an integrally-built block diagram of realizing the contents providing system ex100 of content distribution transmission.The Region Segmentation that communication service is provided is become required size, in each unit, be provided as the base station ex107～ex110 of fixed radio station respectively.This contents providing system ex100, for example, on the Internet ex101 via ISP ex102 and telephone network ex104, be connected to computer ex111, PDA (personal digital assistant: the personal digital assistant) ex112, video camera ex113 is on the portable phone ex114.But contents providing system ex100 is not limited to combination shown in Figure 16, can carry out any being connected.In addition, also can be not via base station ex107～ex110 as fixed radio station, and be directly connected on the telephone network ex104.

Video camera ex113 is the equipment that digital TV video camera etc. can be produced animation.Portable phone can be PDC (Personal Digital Communications) mode in addition, CDMA (Code Division Multiple Accses) mode, W-CDMA (Wideband-CodDivision Multiple Accses) mode, or the pocket telephone of GSM (Global System for MobileCommunication) mode, perhaps PHS (Personal HandyphoneSystem) etc. all can.

In addition, data flow server ex103 connects by base station ex109, telephone network ex104 from video camera ex113, can utilize the live broadcast etc. of the data of the process encoding process that video camera ex113 sends according to the user.The encoding process of data of photography can be carried out with video camera ex113, also can carry out with the server that carries out data transfer process etc.In addition, the animation data with camera 116 productions can be sent to data flow server ex103 via computer ex111.The last digital camera of camera ex116 etc. can be produced the equipment of rest image, dynamic image.In this case, the coding of dynamic image data can carry out with camera ex116, also can be undertaken by computer ex111.In addition, encoding process is handled in the LSIex117 that computer ex111 and camera ex116 have.In addition, the software of image encoding, decoding usefulness can be encased in and can read in any storage medium (CD-ROM, floppy disk, hard disk etc.) as recording medium with computer ex111 etc.And then, also can transmit dynamic image data with the portable phone ex115 that has camera.At this moment dynamic image data is the data of carrying out encoding process with the LSI that portable phone ex115 is had.

Figure 17 is the diagram of the example of expression portable phone ex115.Portable phone ex115 has: and transmit between the base station ex110 and accept the antenna ex210 that radio wave is used; Ccd video cameras etc. can be produced the image pickup part ex203 of image, rest image; The images that demonstration is accepted by the image of image pickup part ex203 production, by antenna ex201 etc. are through the display part ex202 such as LCD of the data of decoding; The main part ex204 that constitutes by the operation keys group; The audio output unit ex208 such as loud speaker that are used for output sound; The sound input part ex205 such as microphone that are used for sound import; Preserve the coded data such as data, animation data or Still image data of mail of data, acceptance of the animation of production or rest image or the storage medium ex207 that decoding data is used; Be used for storage medium ex207 is installed to slot portion ex206 in the portable phone ex115.Storage medium ex207 is the storage medium of a kind of flash memory element among the EEPROM (Electrically Erasable andProgrammable Read Only Memory) of the nonvolatile memory that can write with the mode of electricity and wipe in the plastic casings such as SD card.

In this contents providing system ex100, the content that the user is utilized production such as video camera ex113, camera ex116 (for example, the images of the music live telecast of production etc.) the same encoding process of carrying out with above-mentioned form of implementation is sent to data flow server ex103, on the other hand, data flow server ex103 sends the data of foregoing with distribution of flows to the client that requirement is arranged.As the client, the computer ex111 that the data that above-mentioned encoding process is crossed can be deciphered is arranged, PDAex112, video camera ex113, portable phone ex114 etc.Like this, contents providing system ex100 is that a kind of client can accept regeneration and carried out coded data and then by accept decoding, regeneration in real time in the customers' place, can realize the system that the individual sends.

Below, with Figure 18 portable phone ex115 is further described.Figure 18 is the block diagram of the structure of expression portable phone ex115.Portable phone ex115, master control part ex311 with respect to blanket control display part ex202 and each one of main part ex204, the ex310 of its electric power loop portion, operation input control part ex304, the ex312 of image encoding portion, the ex303 of camera interface portion, LCD (Liquid Crystal Display) control part ex302, the ex309 of image decoding portion, multiplexing separated part ex308, the ex307 of record regenerating portion, modulation loop ex306 of portion and the ex305 of acoustic processing portion interconnect via synchronous bus ex313.The ex310 of electric power loop portion, when the conversation of the EO by the user and power key were connected, by supplying electric power by battery to each one, but the digital mobile phone ex115 that will have a video camera started to operate condition.Portable phone ex115 is according to CPU, the control of the master control part ex311 that ROM and RAM etc. constitute, when the sound call mode, to utilize the ex306 of acoustic processing portion to be transformed into digital audio data with the voice signal of sound input part ex205 input, the ex306 of portion carries out spread processing with it with the modulation loop, transmission accept that the ex301 of loop portion carries out that digitaltoanalogconversion is handled and frequency conversion process after, send via antenna ex201.In addition, portable phone ex115, when the sound call mode, to amplify by the acknowledge(ment) signal that antenna ex201 accepts, carrying out frequency conversion process and analog-to-digital conversion handles, carry out the frequency spectrum inverse expansion at the modulation loop ex306 of portion and handle, utilize the ex205 of acoustic processing portion to be transformed into analoging sound signal after, via audio output unit 208 with its output.And then, when the digital communication pattern, in the occasion of send Email, the text data of the Email of the operation of the operation keys by main part ex204 input, ex304 delivers to master control part ex311 via operation input control part.Master control part ex311 carries out spread processing to text data at the modulation loop ex306 of portion, transmission accept that the ex301 of loop portion carries out that digitaltoanalogconversion is handled and frequency conversion process after, send to base station ex110 via antenna ex201.

When data communication mode, under the situation that sends view data, will offer antenna encoding section ex312 via the ex303 of camera interface portion by the view data of image pickup part ex203 production.In addition, under the situation that does not send view data, also the view data of image pickup part ex203 production directly can be presented on the display part ex202 via ex303 of camera interface portion and LCD control part ex302.

The ex312 of image encoding portion by utilizing the described coding method of above-mentioned form of implementation, will be transformed into coded image data from the Data Compression of image pickup part ex203 supply, be sent to multiplexing separated part ex308.In addition, at this moment, pocket telephone ex115, will be when making a video recording with image pickup part ex203 the sound of sound input part ex205 collection sound, be sent to multiplexing separated part ex308 via the ex305 of acoustic processing portion as the voice data of numeral.

Multiplexing separated part ex308, mode with regulation will be from the coded image data of the ex312 of image encoding portion supply and multiplexingization of voice data of supplying from the ex305 of acoustic processing portion, to carry out spread processing at the modulation loop ex306 of portion as the multiplexingization data that its result obtained, transmission accept that the ex301 of loop portion carries out that digitaltoanalogconversion is handled and frequency conversion process after, send via antenna ex201.

When data communication mode, under the data conditions of the dynamic image file of accepting to link with homepage etc., to carry out reverse spread processing from the acknowledge(ment) signal that base station ex110 accepts at modulation loop ex306 via antenna ex201, will be sent to multiplexing separated part ex308 by the multiplexingization data that its result obtains.

In addition, for the multiplexingization data decoding that will accept via antenna ex201, multiplexing separated part ex308 is by the separation multiplexing data, be divided into coded image data and voice data, via synchronous bus ex313 described coded image data is offered the ex309 of image decoding portion, simultaneously, described voice data is offered the ex305 of acoustic processing portion.

Secondly, the ex309 of image decoding portion deciphers coded image data by the interpretation method of utilizing the described coding method correspondence of above-mentioned form of implementation, generate the regeneration dynamic image data, whereby, for example, show the dynamic image that is included in the dynamic image file that links with homepage.Meanwhile, the ex305 of acoustic processing portion is transformed into voice data after the analoging sound signal, provides it to audio output unit ex208, and whereby, for example, regeneration is included in the voice data in the dynamic image file that links with homepage.

In addition, be not limited to the example of said system, recently, utilizing satellite, surface wave to carry out digital broadcasting is becoming the topic that people discuss, as shown in figure 19, the coding method at least or the interpretation method of above-mentioned form of implementation can be encased in digital broadcasting with in the system.Specifically, the bit stream in that broadcasting station ex409 carries out image coding sends communication or broadcasting satellite ex410 to via radio wave.Receive its broadcasting satellite ex410, emission broadcasting radio wave, accept this radio wave with the antenna ex406 of family, utilize the bitstream interpretation of devices such as television receiver ex401 or set-top box ex407 coding with satellite broadcasting accepting device, and with its regeneration.In addition, can at playback record on, the described interpretation method of above-mentioned form of implementation be installed as the coding stream on the storage medium ex402 of recording medium, the regenerating unit ex403 that deciphers.In this case, the signal of video signal of regeneration is presented on the monitor ex404.In addition, also can code translator be installed in the set-top box ex407 on the antenna ex406 of cable ex405 or satellite/terrestrial ripple broadcasting being connected to cable TV, with it with monitor ex408 regeneration.At this moment, can code device be encased in the set-top box yet, but be encased in the television set.In addition, also can accept from satellite ex410 or the signal that comes from base station ex107 by car ex412 with antenna ex411, with dynamic image regeneration on the display unit of car steering automated navigation system ex413 that car ex412 had etc.

In addition, the structure of car steering automated navigation system ex413 for example, can be considered to adopt in the structure shown in Figure 180, structure except that image pickup part ex203 and the ex303 of camera interface portion also can consider to adopt computer ex303 and television ex401 etc.In addition, terminals such as above-mentioned portable phone ex114 except that the terminal that the transmission with encoder and decoder is accepted, also can consider to adopt three kinds of installation forms of transmission terminal that encoder only is installed and the receiving terminal that decoder only is installed.

Like this, by the described coding method of this specification, interpretation method are installed, can realize the described any device of this form of implementation, system.

Industrial utilizability

As mentioned above, according to picture coding device of the present invention and image decoder, as joining Put picture coding device and image decoder on the telephone set that transmits image, and, do For being configured in picture coding device and the image decoder in the car steering automated navigation system, Useful. In addition, as carrying out method for encoding images of the present invention and image decoding method Program and the recording medium that records described program are useful. And then, utilize this as record The recording medium of the code signal that the picture coding device of invention generates is useful.

Claims (3)

1. a method for encoding images utilizes the predicted picture that is generated by the reference image that input picture is encoded, it is characterized in that,

In the generative process of above-mentioned predicted picture, use following pixel interpolation method:, obtain pixel value to above-mentioned above-mentioned decimal position pixel with reference to image by pixel interpolation and according to the pixel value of a plurality of integer position pixels of the periphery that is positioned at the decimal position pixel;

This method for encoding images comprises the steps:

Utilizing N pixel to generate in the filtration treatment of an inter polated pixel, from the numerical value of N pixel interpolation method different, that have a plurality of above-mentioned filtration treatment, select the selection step of a pixel interpolation method;

The pixel interpolation method that use is selected is according to the above-mentioned predicted picture generation step that generates above-mentioned predicted picture with reference to image;

Utilize the coding step of above-mentioned predicted picture with above-mentioned input picture coding.

2. method for encoding images as claimed in claim 1 is characterized in that, each frame is carried out the selection of above-mentioned pixel interpolation method, and the pixel interpolation sorting signal of determining the selected above-mentioned pixel interpolation method that goes out is configured in the frame title as code signal.

3. method for encoding images as claimed in claim 1 is characterized in that, utilizes N pixel to generate the above-mentioned filtration treatment of an inter polated pixel, is that tap number is the calculation process of the filter of N.

Images