JP4503959B2 - Image coding method - Google Patents

Description

  The present invention relates to a moving picture coding method, and more particularly to a motion compensated inter-picture predictive coding process for coding a picture to be coded using one or more pictures among pictures temporally different from the picture to be coded. It is.

  In recent years, with the development of multimedia applications, it has become common to handle all media information such as images, sounds, and texts in a unified manner. At this time, by digitalizing all the media, it becomes possible to handle the media in a unified manner. However, since a digitized image has an enormous amount of data, image information compression technology is indispensable for storage and transmission. On the other hand, in order to interoperate compressed image data, standardization of compression technology is also important. Image compression technology standards include ITU-T (International Telecommunication Union Telecommunication Standardization Sector) H.261, H.263, ISO / IEC (International Standardization Organization International Electrotechnical Commission) MPEG (Moving Picture Experts Group). ) -1, MPEG-2, MPEG-4, etc. There is also H.264 (MPEG-4AVC) currently being standardized by JVT (JointVideoTeam), which is a congruence of ITU-T and MPEG.

  In general, in encoding of moving images, the amount of information is compressed by reducing redundancy in the time direction and the spatial direction. Therefore, in inter-picture predictive coding for the purpose of reducing temporal redundancy, motion is detected and a predicted image is created in units of blocks with reference to the forward or backward picture, and the resulting predicted image and the encoded image are encoded. Encoding is performed on the difference value from the target picture. Here, a picture is a term representing a single screen. In a progressive image, it means a frame, and in an interlaced image, it means a frame or a field. Here, an interlaced image is an image in which one frame is composed of two fields having different times. In interlaced image encoding and decoding processing, one frame may be processed as a frame, processed as two fields, or processed as a frame structure or a field structure for each block in the frame. it can.

  A picture that does not have a reference picture and performs intra prediction coding is called an I picture. A picture that performs inter-frame predictive coding with reference to only one reference picture is called a P picture. A picture that can be subjected to inter-picture prediction coding with reference to two reference pictures at the same time is called a B picture. The B picture can refer to two pictures as an arbitrary combination of display times from the front or the rear. A reference picture (reference picture) can be specified for each macroblock that is a basic unit of encoding. The reference picture described earlier in the encoded bitstream is the first reference picture, The one described is distinguished as the second reference picture. However, as a condition for encoding these pictures, the picture to be referenced needs to be already encoded.

  Motion compensation inter-picture prediction coding is used for coding a P picture or a B picture. The motion compensation inter-picture prediction encoding is an encoding method in which motion compensation is applied to inter-picture prediction encoding. Motion compensation is not simply predicting from the pixel value of the reference frame, but detecting the amount of motion of each part in the picture (hereinafter referred to as a motion vector) and performing prediction in consideration of the amount of motion. This improves the prediction accuracy and reduces the amount of data. For example, the amount of data is reduced by detecting the motion vector of the encoding target picture and encoding the prediction residual between the prediction value shifted by the motion vector and the encoding target picture. In the case of this method, since motion vector information is required at the time of decoding, the motion vector is also encoded and recorded or transmitted.

  The motion vector is detected in units of macroblocks. Specifically, the macroblock on the encoding target picture side is fixed, the macroblock on the reference picture side is moved within the search range, and is most similar to the reference block. The motion vector is detected by finding the position of the reference block.

  FIG. 13 is a block diagram showing a configuration of a conventional image encoding apparatus 1000. The image coding apparatus 1000 includes a difference unit 1001, an image coding unit 1002, a variable length coding unit 1003, an image decoding unit 1004, an adder 1005, an image memory 1006, a picture memory 1007, a motion compensation coding unit 1008, a motion A vector detection unit 1009, a prediction direction determination unit 1010, and a motion vector storage unit 1011 are provided. Note that the encoding process is performed in units called 16 × 16 pixel macroblocks, and the size of the motion compensation block is H.264, which is a currently drafted standard. H.264 selects the appropriate block size for each macroblock from 7 types of motion compensation block sizes: 4x4, 4x8, 8x4, 8x8, 8x16, 16x8, 16x16 And used for encoding.

  The picture memory 1007 stores image data Img representing a moving image input in units of pictures in display time order. The differentiator 1001 calculates a difference between the image data Img read from the picture memory 1007 and the predicted image data Pred input from the motion compensation encoding unit 1008, and generates predicted difference image data Res. The image encoding unit 1002 performs encoding processing such as frequency conversion and quantization on the input prediction difference image data Res to generate difference image encoded data CodedRes. In the case of intra-frame coding, since motion compensation between screens is not performed, the value of the predicted image data Pred is considered to be “0”.

  The motion vector detection unit 1009 uses the reference image data Ref, which is the decoded image data that has been encoded and stored in the image memory 1006, as a reference picture, and indicates the position predicted to be optimal in the search region within the picture. The motion vector MotionVector and the reference picture number RefIdx are detected and output. The motion vector MotionVector detected at this time is one for one reference image data Ref.

  FIG. 14 is a diagram for explaining the processing of the motion vector detection unit 1009 in more detail. Here, as an example, a case will be described where a motion vector at the time of a B picture is detected with a motion compensation block in the encoding target macroblock (16 × 16 pixels) having a size of 16 × 16 pixels.

  The motion vector detection unit 1009 stores the pixels of the macroblock MB0 to be encoded from the picture memory 1007 in which the picture to be encoded is stored, and the reference pictures RefIdx1 and RefIdx2 from the image memory 1006 in which the reference picture is stored. Pixels P3 and P5 in the search area are acquired. The position of the encoding target macroblock in the reference pictures RefIdx1 and RefIdx2 is MB0a. First, a position most similar to the pixel of the encoding target macroblock MB0 is detected in the search area P3 (assuming that it is detected as MB1). At this time, the detected similar position is represented by the motion vector MV1 as the movement distance from the position MB0a of the encoding target macroblock. Similarly, a similar position is also detected in the search area P5 (assuming that the position of MB2 is detected), and this position is represented as a motion vector MV2.

  The motion vectors MV1 and MV2 detected by the motion vector detection unit and the referenced picture number RefIdx (RefIdx1, RefIdx2, etc.) are output to the prediction direction determination unit 1010.

  The prediction direction determination unit 1010 determines the prediction direction Dir (forward unidirectional (Fwd), backward unidirectional (Bwd), bidirectional (Bid), etc.) of the encoding target block, and the determined direction Dir and reference picture number RefIdx and the motion vector MoitonVector are input to the motion compensation encoding unit 1008. The motion compensation encoding unit 1008 generates predicted image data Pred based on this input. The reference picture number RefIdx and the motion vector MotionVector used in the prediction direction Dir determined here are stored in the motion vector storage unit 1011.

  In the motion compensation encoding unit 1008, when the motion vector indicates a pixel position of a decimal number such as 1/2 pixel or 1/4 pixel, a low pass filter or the like is used. Interpolate and generate pixel values at decimal pixel positions such as 4 pixels. The variable length coding unit 1003 performs variable length coding or the like on the input differential image coded data CodedRes and the motion parameter MotionParam obtained by the motion vector detection unit 1009, and further outputs from the motion compensation coding unit 1008. The encoded data Bitstream is generated by adding the prediction direction Dir and the reference picture number RefIdx to which the block to be referenced belongs. Here, the motion parameter MotionParam is a motion vector difference obtained from a motion vector predicted from a motion vector MotionVector and a motion vector of an already encoded picture (see the motion vector storage unit 1011).

  The image decoding unit 1004 performs decoding processing such as inverse quantization and inverse frequency conversion on the input encoded data difference image encoded data CodedRes to generate decoded difference image data ReconRes. The adder 1005 adds the decoded differential image data output from the image decoding unit 1004 and the predicted image data Pred input from the motion compensation encoding unit 1008 to generate decoded image data Recon. The image memory 1006 stores the generated decoded image data Recon.

  Depending on the movement of the subject, the prediction effect may be high when the prediction is performed in a unit smaller than the integer pixel unit. In general, pixel interpolation is used to calculate a pixel value of a predicted image with a unit movement smaller than an integer pixel unit. This pixel interpolation is performed by filtering the pixel value of the reference image with a linear filter (low-pass filter). If the number of taps of this linear filter is increased, a filter having good frequency characteristics can be realized, and the prediction effect is enhanced, but the processing amount is increased. On the other hand, when the number of taps of the filter is small, the frequency characteristic of the filter is deteriorated and the prediction effect is lowered, but the processing amount is reduced.

  H. is a draft standard currently being developed. In H.264, motion compensation in units of up to 1/4 pixel is permitted (up to 1/2 pixel in MPEG-4 Simple Profile), and a 6-tap filter is adopted as a linear filter pixel interpolation method. Yes. A pixel interpolation method using this 6-tap filter will be described with reference to FIG.

FIG. 2 is a diagram for explaining a pixel interpolation method of luminance components in H.264. F1 to F36 represent pixel values at integer pixel positions, H1 to H7 represent pixel values at 1/2 pixel positions, and Q1 and Q2 represent pixel values at 1/4 pixel positions. For example, when the pixel value at the ½ pixel position H1 is obtained, the pixel value is predicted and generated by calculation using integer pixel values (F1 to F6) of the surrounding six pixels as shown in Expression A (the same applies to H2 to H6). Can ask for). When the pixel value of H7 is obtained, the pixel value is predicted and generated by calculation using the pixel values (H1 to H6) at the ½ pixel positions of the surrounding six pixels as shown in Expression B. Further, when obtaining the pixel values at the quarter pixel positions Q1 and Q2, the pixel values are generated by calculation using the neighboring half pixel values as in Expression C and Expression D. At this time, the constant α in the expressions A and B is a value based on a rounding coefficient. (See Non-Patent Document 1)
"Draft ITU-T Recommendation and Final Draft International Standard of Joint Video Specification", Joint Video Team (JVT) of ISO / IEC MPEG & ITU-T VCEG, JVT-G050r1, 27 May 2003

  However, depending on the movement of the subject, the motion compensation up to 1/4 pixel position has a small prediction error, that is, it cannot be encoded efficiently, and the image quality deteriorates when the encoded stream is decoded. is there.

  The present invention has been made to solve such a problem, and an object of the present invention is to provide an image coding method with high coding efficiency by highly accurate motion compensation.

In order to achieve the above object, an image encoding method according to the present invention determines two motion vectors with decimal pixel precision for a block to be encoded, and determines a pixel value of a reference block indicated by the two motion vectors. An image encoding method that encodes a difference value between an average value and a pixel value of the encoding target block together with the two motion vectors, and performs motion detection with integer pixel accuracy on a reference picture. A motion vector determination in which a motion vector is detected with a pixel accuracy of 1 / 2n and a motion vector is determined around a motion detection step and a motion reference position detected in the motion detection step. Step, the two motion vectors determined in the motion vector determination step, and the reference block indicated by the two motion vectors for the encoding target block. And having a coding step of coding a difference value between Tsu value obtained by averaging the pixel values of the click and the pixel value of the coding target block.

Here, in the image coding method, the average of the pixel values of the reference block indicated by the two motion vectors is a ½ average obtained by dividing the sum of the pixel values of the reference block indicated by the two motion vectors by 2. Is preferred.

The present invention can be realized not only as such an image encoding method but also as an image encoding device. That is, for the encoding target block, two motion vectors having a pixel precision of 1 / 2n are determined, and a value obtained by averaging the pixel values of the reference block indicated by the two motion vectors and the pixel of the encoding target block It can also be realized as an image encoding device that encodes a difference value with a value together with the two motion vectors.

Similarly, the present invention can be realized as a program for such an image encoding device. That is, the present invention determines two motion vectors with a pixel precision of 1 / 2n for the encoding target block, averages the pixel values of the reference block indicated by the two motion vectors, and the encoding It can also be realized as a program for causing an image encoding apparatus to realize an image encoding method for encoding a difference value with a pixel value of a target block together with the two motion vectors.

  When encoded by the motion compensated inter-picture predictive encoding method and encoding apparatus according to the present invention, it is possible to perform motion compensation with higher accuracy than conventional methods in encoding an image with fine temporal motion, that is, a prediction error. Therefore, it is possible to perform encoding with less encoding efficiency. That is, higher quality encoding can be performed at the same bit rate.

  Further, when the image encoding method according to the present invention is realized by a program or hardware, the value of 1/2 pixel is temporarily held in a memory in the program or hardware at the time of motion interpolation for generating a decimal pixel. Thus, a plurality of 1/2 pixel value calculations can be omitted, an increase in the amount of calculation can be suppressed, and power consumption can be reduced.

  Furthermore, according to the image coding method and apparatus of the present invention, the amount of computation and the amount of memory required for motion compensation processing can be realized without increasing from the conventional bi-prediction of B pictures.

  Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1)
Hereinafter, a first embodiment of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of an image encoding device 100 using an image encoding method according to the present invention. The image encoding device 100 includes a subtractor 1001, an image encoding unit 1002, a variable length encoding unit 1003, an image decoding unit 1004, an adder 1005, an image memory 1006, a picture memory 1007, a motion compensation encoding unit 1008, a mode. A determination unit 102, a motion vector candidate detection unit 101, and a motion vector storage unit 1011 are provided. The encoding process is performed in units called 16 × 16 pixel macroblocks as in the conventional example. The processing units other than the motion vector candidate detection unit 101 and the mode determination unit 102 included in the image encoding device 100 are the same as the processing units described with reference to FIG. 13 in the background art. Therefore, only the processing of the motion vector candidate detection unit 101 and the mode determination unit 102 will be described in detail here, and the other processing units will be omitted.

  FIG. 2 is a diagram for explaining a motion vector detection method in the motion vector candidate detection unit 101 when the motion compensation block size is 4 × 4. As described in the background art with reference to FIG. 14, it is the same to detect the most similar position (either integer precision or decimal precision) in the search area, but the conventional example determines one similar position. On the other hand, the present invention is characterized by detecting two or more similar positions.

  First, as shown in FIG. 2A, motion detection is performed with integer pixel precision, and the position where the absolute value sum SAD of the difference between the pixel value and the encoding target block MB0 is the smallest from the reference area P3 shown in FIG. (Similar positions) are determined (assuming that MB1 is determined, integer precision pixels included in MB1 are F1 to F16).

  Next, as shown in FIG. 2 (b), the absolute value sum SAD of the difference between the pixel values with the precision of the fractional pixels (1/2, 1/4 pixels) around the position MB1 that is most similar with the integer pixel precision is obtained. The two smallest positions (the pixel position Best with the smallest SAD and the pixel position Best2 with the second smallest SAD) are determined to obtain a motion vector (for example, the 1/2 pixel position indicated by x in FIG. 2B is the Best1). The 1/4 pixel position indicated by + is assumed to be Best2.) The pixel value generation with decimal precision may be the same as or different from the method described with reference to FIG. 15 in the background art. The most similar position determination method here is the absolute value sum SAD of the difference between the pixel value of the encoding target block and the pixel value of the reference region, but other determination methods may be used.

  Motion detection is performed in the same manner in a reference area (for example, P5 in FIG. 13) other than the motion compensation block size 4x4 as shown in FIG. 2 or a reference area P3 (reference picture RefIdx1 in FIG. 14). The motion vector is determined and output to the mode determination unit 102 together with the reference picture number RefIdx. At this time, the output information is assumed to be delivered in the form of a group for each macroblock (16 × 16 pixels) as shown in FIG. For example, when the motion compensation block is 8 × 16, there are two motion compensation blocks in the macro block as shown in FIG. 3B, and each block has a reference picture number RefIdx and a motion vector (Best1 vector is MV1, Best2 vector). All candidates) are stored. 3A shows a case where the unit of the motion compensation block is 16 × 16, and FIG. 3C shows a case where the unit of the motion compensation block is 8 × 8.

The method of determining the most similar positions Best1 and Best2 described with reference to FIG. 2 first determines the most similar position with integer pixel accuracy, and then detects the similar position with decimal precision around that position. Although the two most similar positions are determined, the method for determining the two most similar positions may be other than this. For example, two points that are the most similar among all the integer pixels and decimal pixels (1/2, 1/4, etc.) in the search area may be obtained. Further, the most similar position is determined with integer pixel accuracy, and then the most similar position is determined with the surrounding 1/2 pixel accuracy around the position, and the most similar position 2 with the surrounding 1/4 pixel accuracy is further determined. One may be detected.
≪Determine the partition and direction with the best coding efficiency from the candidates≫
Next, the processing of the mode determination unit 102 will be described in detail. The mode decision unit 102 receives all combinations of the motion vector MotionVector and the corresponding reference picture number RefIdx from the motion vector candidate detection unit 101 in units of macroblocks for each motion compensation block size.

  First, the mode determination unit 102 determines a mode that can be most efficiently encoded from the input information. The mode here refers to a motion compensation block size, a prediction type of each block in that case (forward unidirectional, backward unidirectional, bidirectional, etc.), and a combination of a reference picture and a motion vector. In the case of B pictures, this mode may use motion vectors with one or two different reference picture numbers for one block. In the case of H.264, the motion vector of the same reference picture may be used. Therefore, the mode is determined using a method described below.

  FIG. 4 is a diagram for specifically explaining the mode determination method. In order to simplify the description, the candidate combinations (combination of the reference picture number RefIdx and the motion vector given from the motion detection unit 101) are motion compensation block types of 16x16 and 8x16, and each reference picture number is RefIdx1, Two RefIdx2s are assumed. There are two motion vectors MV1 and MV2 for each reference picture. Here, the motion vectors indicated by MV1 and MV2 are motion vectors representing the positions of Best1 and Best2 determined by the motion vector candidate detection unit 101, respectively. In order to clarify the combination of the reference picture number and the motion vector, FIG. In (a), it is shown in the form of MV1 # Ref1 (MV1 of reference picture number 1).

  First, the evaluation value of each mode candidate 1-5 for every motion compensation block as shown to Fig.4 (a) is derived | led-out. (In the case of motion compensation block type 8x16, the macroblock is divided into left and right parts, so it is expressed as 8x16 # Left and 8x16 # Right) Fig. 4 (b) shows an example of derivation of evaluation values of mode candidates with each block. Shimese. It is determined whether or not the mode candidate for which the evaluation value is to be obtained is a one-way one vector candidate (S301). If it is a one-way one vector candidate, it is encoded with one reference picture pixel to be referred to. A value obtained by adding the value α to the sum of absolute values of pixel value differences from the pixel of the target block is set as an evaluation value (S302). If it is not a one-way vector in S301 (one-way two vectors or one vector in both directions), the process proceeds to S303, an average pixel of two pictures to be referred to is obtained, and this is used as a predicted pixel. Next, the value obtained by adding the value β to the absolute value sum of the pixel value difference between the prediction pixel calculated in S303 and the encoding target block is set as an evaluation value (S304). The values α and β used here are values given in consideration of bits representing motion vectors. The method for obtaining the predicted pixel is not limited to the average pixel of the two pictures to be referred to, and other methods may be used.

  With the above method, an evaluation value is given to each block candidate for each motion compensation block type, and one mode with the smallest evaluation value is selected from the block candidates.

  Next, an evaluation value is calculated for each macroblock, and a motion compensation block type in the macroblock is determined. FIG. 5 shows an example in which an evaluation value is given to each mode candidate of each block of FIG. 4A by the method of FIG. 4B. In the case of this example, the motion compensation block 16x16 has an evaluation value of 480 mode candidate 1 (one-way 1 vector, MV1 # Ref1), and the motion compensation block type 8x16 # Left (the left block when the macroblock is divided). In 232 mode candidate 5 (one vector in each direction, MV1 # Ref1, MV1 # Ref1), and motion compensation block type 8x16 # Right, mode candidate 3 (one-way two vector, MV1 # Ref1, MV2 # Ref1) is Each is chosen. It is determined by comparing the evaluation value obtained here in units of macroblocks in which motion compensation unit the macroblock which is a processing unit is encoded in the most motion compensation unit. When the minimum evaluation value determined in FIG. 5 is compared in units of macroblocks, the evaluation value of the motion compensation unit 16 × 16 is 480, and the evaluation value of the motion compensation unit 8 × 16 is 452 (232 + 210 = 452), and the evaluation value is small. The motion compensation unit is 8x16. With the above procedure, the coding mode is the motion compensation block type 8x16, and the coding of each block is bi-prediction using 8x16 # Left blocks with 1 vector in each direction (MV1 # Ref1, MV1 # Ref2), and 8x16 # Right block is It is determined that encoding is performed by bi-prediction using one-way two vectors (MV1 # Ref1, MV2 # Ref1).

  In this manner, evaluation values are also given to a plurality of other motion compensation block types, and the encoding mode of the encoding target macroblock is determined.

  Here, as the evaluation value, the absolute value sum of the pixel value differences between the encoding target block and the reference block (or the calculated prediction pixel) is used as part of the evaluation value, but the evaluation value is not limited to this. However, any value may be used as long as it determines the mode with the best coding efficiency.

  In accordance with the encoding mode determined by the mode determination unit 102, a flag Flag indicating whether or not unidirectional prediction is performed, a reference picture number, and a motion vector are input to the compensation encoding unit 1008, and the same processing as in the conventional encoding is performed thereafter.

  When encoding is performed by the image encoding apparatus including the motion vector candidate detection unit 101 and the mode determination unit 102 described in the first embodiment, two references that a macroblock refers to when bi-predictive encoding a B picture Considering the possibility that the block becomes the same reference picture, it is possible to select the prediction with the highest coding efficiency regardless of whether the two reference blocks are the same picture or not. That is, when referring to images at two adjacent positions with 1/4 pixel accuracy in the same reference picture, it has substantially the same meaning as that for generating a pixel at 1/8 position, and has a finer motion. An improvement in encoding efficiency can be expected for an image.

  Furthermore, the image coding method and apparatus according to the first embodiment is a conventional bi-directional prediction (one vector from different reference pictures, that is, one vector in both directions) input to a program or hardware that realizes motion compensation coding. It is possible to realize a special increase in memory only by changing.

  Also, in the first embodiment described above, if the encoding target macroblock is a bi-predictive coding block that refers to two, whether or not to permit the two reference images to be the same picture? Switching can be realized by using a simple flag.

  FIG. 6 is a diagram for explaining image encoding processing in a case where switching is performed by using a simple flag as to whether or not two reference images to be referred to are permitted to be the same picture in encoding that performs bi-prediction. FIG. FIG. 6 differs only in the processing and input of the motion vector candidate detection unit 101 in the block diagram of FIG. In FIG. 6, a 1-bit flag BIPflag indicating whether or not two reference images are permitted to be the same picture in bi-prediction is input to the motion vector candidate detection unit 103 from the outside. BIPflag is set to 1 when the reference picture is permitted to be the same picture, and BIPflag is set to 0 when not permitted. When BIPflag is 1, the motion vector candidate detection unit 103 determines two points (Best 1 and Best 2) most similar to the encoding target from the reference area of each reference picture as described above with reference to FIG. If BIPflag is 0, only the most similar point (Best 1) is determined from the reference area. Similarly to the above, the reference picture number RefIdx and the motion vector MV1 are output to the mode determining unit 102 for one point determined in each reference picture. As described with reference to FIGS. 4 and 5, the mode determination unit 102 determines the mode with the best coding efficiency for the input mode candidate. When BIPflag is 0, only one point position is input to the mode determination unit 102 for each reference picture, so that no candidate for referring to two points in the same reference picture is generated as a mode candidate for bi-prediction (two vectors). . As a result, bi-predictive coding that refers to the same picture can be prohibited.

  In this way, by switching whether or not to allow reference to the same picture by a simple flag, for example, when encoding using two reference images, the MPEG- which is not permitted to be the same picture as the two pictures. An image encoding apparatus that realizes both the encoding standard such as 2 and the encoding standard such as H.264 that is permitted to be the same picture can be easily realized. .

  The bit stream of an image encoded by the image encoding method and apparatus described above has a feature that a macroblock in a B picture may refer to two positions of the same reference picture.

(Embodiment 2)
Embodiment 2 of the present invention will be described below. The second embodiment of the present invention relates to the processing of the motion compensation encoding unit 1008 and the mode determining unit 102 shown in FIG. 1 in the image encoding method and the image encoding device shown in the first embodiment. The other processes in FIG. 1 are the same.

  Hereinafter, an embodiment of pixel interpolation processing performed by the motion compensation encoding unit 1008 and the mode determination unit 102 in FIG. 1 will be described. As described with reference to FIG. 15 in the conventional example, the motion compensation encoding unit 1008 performs pixel interpolation in order to perform motion compensation with decimal pixel accuracy. Further, in order to obtain the evaluation value, the mode determination unit 102 needs a pixel of a prediction pixel (reference picture), and at this time, it is necessary to perform pixel interpolation.

  Here, for the mode candidate in the mode determination unit 102 and the prediction mode (optimum mode) (one vector prediction or two vector prediction) input to the motion compensation encoding unit 1008 shown in the first embodiment, the number of reference images Is represented by a binary flag. It is assumed that the number of predicted images is 1, that is, 1 vector prediction is 0, and the number of predicted images is 2, that is, 2 vector prediction is 1.

  FIG. 7 shows a process flow of pixel interpolation performed by the motion compensation encoding unit 1008 and the mode determination unit 102. First, the input prediction mode is checked (S501). If the flag is 1 (2 vectors), the two motion vectors (the two motion vectors are MV1 and MV2 and the horizontal and vertical components are mv1 # x and mv1). #y, mv2 # x, and mv2 # y) and reference picture numbers (MV1 # RefIdx, MV2 # RefIdx) are referred to (S502). The two reference picture numbers are the same (MV1 # RefIdx == MV2 # RefIdx) and the absolute value of the difference between the horizontal and vertical components is smaller than the constant values α and β, respectively, that is, the two reference images are the same picture and overlap If this is the case (S502 is YES), the 1/2 pixel value is temporarily stored in the memory during the pixel interpolation calculation of the first reference image (MV1) (S503). Otherwise, it is not saved (S504). At this time, it is assumed that the values α and β in S502 are 0 or more values determined by the number of reference pixels (number of taps) at the time of pixel interpolation.

  When two images are referenced (bi-predicted) and the two images refer to overlapping pixels in the same reference picture, the 1/2 pixel value calculated and generated during pixel interpolation of the first reference pixel is stored in memory In this case, the calculation related to the second pixel interpolation can be omitted by temporarily storing the data in the memory and referring to the memory during the motion interpolation of the second reference pixel. This is effective in reducing power consumption when the motion compensation encoding unit 1008 and the mode determination unit 102 are realized by hardware.

(Embodiment 3)
Furthermore, by recording a program for realizing the configuration of the image encoding method shown in each of the above embodiments on a storage medium such as a flexible disk, the processing shown in each of the above embodiments is performed. It can be easily implemented in an independent computer system.

FIG. 8 is an explanatory diagram in the case of implementing by a computer system using the flexible disk storing the image coding method of the first embodiment and the second embodiment.
FIG. 8B shows an appearance, a cross-sectional structure, and a flexible disk as seen from the front of the flexible disk, and FIG. 8A shows an example of a physical format of the flexible disk that is a recording medium body. The flexible disk FD is built in the case F, and on the surface of the disk, a plurality of tracks Tr are formed concentrically from the outer periphery toward the inner periphery, and each track is divided into 16 sectors Se in the angular direction. ing. Therefore, in the flexible disk storing the program, an image encoding method as the program is recorded in an area allocated on the flexible disk FD.

  FIG. 8C shows a configuration for recording and reproducing the program on the flexible disk FD. When recording the program on the flexible disk FD, the image encoding method or the image decoding method as the program is written from the computer system Cs via the flexible disk drive. When the image encoding method is built in a computer system by a program in a flexible disk, the program is read from the flexible disk by a flexible disk drive and transferred to the computer system.

  In the above description, a flexible disk is used as the recording medium, but the same can be done using an optical disk. Further, the recording medium is not limited to this, and any recording medium such as an IC card or a ROM cassette capable of recording a program can be similarly implemented.

  Furthermore, the motion compensation method and the image coding method application example shown in the above embodiment and a system using the same will be described.

  FIG. 9 is a block diagram showing an overall configuration of a content supply system ex100 that implements a content distribution service. The communication service providing area is divided into desired sizes, and base stations ex107 to ex110, which are fixed radio stations, are installed in each cell.

  The content supply system ex100 includes, for example, a computer ex111, a PDA (personal digital assistant) ex112, a camera ex113, a mobile phone ex114, a camera via the Internet ex101, the Internet service provider ex102, the telephone network ex104, and the base stations ex107 to ex110. Each device such as the attached mobile phone ex115 is connected.

  However, the content supply system ex100 is not limited to the combination as shown in FIG. 9, and may be connected in any combination. Further, each device may be directly connected to the telephone network ex104 without going through the base stations ex107 to ex110 which are fixed wireless stations.

  The camera ex113 is a device capable of shooting a moving image such as a digital video camera. The mobile phone is a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access) system, or a GSM (Global System for Mobile Communications) system mobile phone, Alternatively, PHS (Personal Handyphone System) or the like may be used.

  In addition, the streaming server ex103 is connected from the camera ex113 through the base station ex109 and the telephone network ex104, and live distribution or the like based on the encoded data transmitted by the user using the camera ex113 becomes possible. The encoded processing of the captured data may be performed by the camera ex113 or may be performed by a server or the like that performs data transmission processing. Further, the moving image data shot by the camera 116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device that can shoot still images and moving images, such as a digital camera. In this case, the encoding of the moving image data may be performed by the camera ex116 or the computer ex111. The encoding process is performed in the LSI ex117 included in the computer ex111 and the camera ex116. Note that image encoding / decoding software may be incorporated into any storage medium (CD-ROM, flexible disk, hard disk, etc.) that is a recording medium readable by the computer ex111 or the like. Furthermore, you may transmit moving image data with the mobile telephone ex115 with a camera. The moving image data at this time is data encoded by the LSI included in the mobile phone ex115.

  In this content supply system ex100, the content (for example, video shot of music live) captured by the user with the camera ex113, camera ex116, etc. is encoded and transmitted to the streaming server ex103 as in the above embodiment. On the other hand, the streaming server ex103 distributes the content data to the requested client. Examples of the client include a computer ex111, a PDA ex112, a camera ex113, a mobile phone ex114, and the like that can decode the encoded data. In this way, the content supply system ex100 can receive and reproduce the encoded data at the client, and also realize personal broadcasting by receiving, decoding, and reproducing in real time at the client. It is a system that becomes possible.

What is necessary is just to use the image coding apparatus shown in said each embodiment for the encoding of each apparatus which comprises this system.
A mobile phone will be described as an example.
FIG. 10 is a diagram illustrating the mobile phone ex115 using the motion compensation method, the image encoding method, and the image decoding method described in the above embodiment. The cellular phone ex115 includes an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110, a camera such as a CCD camera, a camera unit ex203 capable of taking a still image, a video shot by the camera unit ex203, and an antenna ex201. A display unit ex202 such as a liquid crystal display that displays data obtained by decoding received video and the like, a main body unit composed of a group of operation keys ex204, an audio output unit ex208 such as a speaker for outputting audio, and a voice input To store encoded data or decoded data such as a voice input unit ex205 such as a microphone, captured video or still image data, received mail data, video data or still image data, etc. Recording medium ex207, and slot portion ex20 for enabling the recording medium ex207 to be attached to the mobile phone ex115 The has. The recording medium ex207 stores a flash memory element which is a kind of EEPROM (Electrically Erasable and Programmable Read Only Memory) which is a nonvolatile memory that can be electrically rewritten and erased in a plastic case such as an SD card.

  Further, the cellular phone ex115 will be described with reference to FIG. The cellular phone ex115 controls the power supply circuit ex310, the operation input control unit ex304, and the image coding for the main control unit ex311 which is configured to control the respective units of the main body unit including the display unit ex202 and the operation key ex204. Unit ex312, camera interface unit ex303, LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, demultiplexing unit ex308, recording / reproducing unit ex307, modulation / demodulation circuit unit ex306, and audio processing unit ex305 via a synchronization bus ex313 Are connected to each other.

  When the end call and power key are turned on by a user operation, the power supply circuit ex310 activates the camera-equipped digital mobile phone ex115 by supplying power from the battery pack to each unit. .

  The mobile phone ex115 converts the voice signal collected by the voice input unit ex205 in the voice call mode into digital voice data by the voice processing unit ex305 based on the control of the main control unit ex311 including a CPU, a ROM, a RAM, and the like. The modulation / demodulation circuit unit ex306 performs spread spectrum processing, and the transmission / reception circuit unit ex301 performs digital analog conversion processing and frequency conversion processing, and then transmits the result via the antenna ex201. In addition, the cellular phone ex115 amplifies the received signal received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog-digital conversion processing, performs spectrum despreading processing by the modulation / demodulation circuit unit ex306, and analog audio by the voice processing unit ex305. After conversion into a signal, this is output via the audio output unit ex208.

  Further, when an e-mail is transmitted in the data communication mode, text data of the e-mail input by operating the operation key ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spread spectrum processing on the text data in the modulation / demodulation circuit unit ex306, performs digital analog conversion processing and frequency conversion processing in the transmission / reception circuit unit ex301, and then transmits the text data to the base station ex110 via the antenna ex201.

  When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image encoding unit ex312 via the camera interface unit ex303. When image data is not transmitted, the image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.

  The image encoding unit ex312 has a configuration including the image encoding device described in the present invention, and an encoding method using the image data supplied from the camera unit ex203 in the image encoding device described in the above embodiment. The encoded image data is converted into encoded image data by compression encoding, and sent to the demultiplexing unit ex308. At the same time, the cellular phone ex115 sends the sound collected by the audio input unit ex205 during imaging by the camera unit ex203 to the demultiplexing unit ex308 as digital audio data via the audio processing unit ex305.

  The demultiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the audio data supplied from the audio processing unit ex305 by a predetermined method, and the multiplexed data obtained as a result is a modulation / demodulation circuit unit A spectrum spread process is performed at ex306, a digital-analog conversion process and a frequency conversion process are performed at the transmission / reception circuit unit ex301, and then transmitted through the antenna ex201.

  When receiving data of a moving image file linked to a home page or the like in the data communication mode, the received signal received from the base station ex110 via the antenna ex201 is subjected to spectrum despreading processing by the modulation / demodulation circuit unit ex306, and the resulting multiplexing is obtained. Is sent to the demultiplexing unit ex308.

  In addition, in order to decode the multiplexed data received via the antenna ex201, the demultiplexing unit ex308 separates the multiplexed data to generate an encoded bitstream of image data and an encoded bitstream of audio data. The encoded image data is supplied to the image decoding unit ex309 via the synchronization bus ex313, and the audio data is supplied to the audio processing unit ex305.

  Next, the image decoding unit ex309 generates reproduction moving image data by decoding the encoded bit stream of the image data, and supplies this to the display unit ex202 via the LCD control unit ex302. The moving image data included in the moving image file linked to the home page is displayed. At the same time, the audio processing unit ex305 converts the audio data into an analog audio signal, and then supplies the analog audio signal to the audio output unit ex208. Thus, for example, the audio data included in the moving image file linked to the home page is reproduced. The

  Note that the present invention is not limited to the above system, and recently, digital broadcasting using satellites and terrestrial waves has become a hot topic. As shown in FIG. 12, the image coding apparatus of the above embodiment is also incorporated into a digital broadcasting system. Can do. Specifically, in the broadcasting station ex409, the encoded bit stream of the video information is transmitted to the communication or broadcasting satellite ex410 via radio waves. Receiving this, the broadcasting satellite ex410 transmits a radio wave for broadcasting, and receives the radio wave with a home antenna ex406 having a satellite broadcasting receiving facility, such as a television (receiver) ex401 or a set top box (STB) ex407. The device decodes the encoded bit stream and reproduces it. Also, the encoded bit stream recorded on the storage medium ex402 such as a CD or DVD as a recording medium is read and decoded by the reproduction apparatus ex403.

  In this case, the reproduced video signal is displayed on the monitor ex404. Further, a configuration in which an image decoding device is mounted in a set-top box ex407 connected to a cable ex405 for cable television or an antenna ex406 for satellite / terrestrial broadcasting, and this is reproduced on the monitor ex408 of the television is also conceivable. At this time, the image decoding apparatus may be incorporated in the television instead of the set top box. It is also possible to receive a signal from the satellite ex410 or the base station ex107 by the car ex412 having the antenna ex411 and reproduce a moving image on a display device such as the car navigation ex413 that the car ex412 has.

  Further, the image signal can be encoded by the image encoding device shown in the above embodiment and recorded on a recording medium. As a specific example, there is a recorder ex420 such as a DVD recorder that records an image signal on a DVD disk ex421 or a disk recorder that records on a hard disk. Further, it can be recorded on the SD card ex422. If the recorder ex420 includes the image decoding device described in the above embodiment, the image signal recorded on the DVD disc ex421 or the SD card ex422 can be reproduced and displayed on the monitor ex408.

  For example, the configuration of the car navigation ex413 may be a configuration excluding the camera unit ex203, the camera interface unit ex303, and the image encoding unit ex312 in the configuration illustrated in FIG. 11, and the same applies to the computer ex111 and the television (receiver). ) Ex401 can also be considered.

  In addition to the transmission / reception type terminal having both the encoder and the decoder, the terminal such as the mobile phone ex114 has three implementation formats: a transmitting terminal having only an encoder and a receiving terminal having only a decoder. Can be considered.

  As described above, the motion compensation method and the image encoding method described in the above embodiment can be used in any of the devices and systems described above, and by doing so, the effects described in the above embodiment can be obtained. Can do.

  The image coding method and the image coding apparatus according to the present invention can perform coding of an image with high coding efficiency, which is a field that handles moving images, for example, content distribution, cellular phone, digital broadcasting, etc. It can be applied to other uses.

It is a block diagram which shows the image coding apparatus concerning Embodiment 1 of this invention. It is an image figure of the pixel explaining the process of the motion vector candidate detection part of FIG. It is a figure for demonstrating the motion compensation block type and information output from the motion vector candidate detection part of FIG. It is the table | surface and flowchart for demonstrating the process of the mode determination part of FIG. 1 is a table and a flow diagram for explaining the processing of the mode determination unit. It is a block diagram which shows the image coding apparatus concerning Embodiment 1 of this invention. It is a flowchart explaining the process of the motion compensation encoding part concerning Embodiment 2 of this invention. It is explanatory drawing in the case of implementing with a computer system using the flexible disk which stored the image coding method or the image decoding method of the said Embodiment 1 to Embodiment 2. It is a block diagram which shows the whole structure of the content supply system which implement | achieves a content delivery service. It is a figure which shows the mobile telephone using the motion compensation method, the image coding method, and the image decoding method. It is a block diagram which shows the structure of a mobile telephone. It is a block diagram which shows the whole structure of the system for digital broadcasting. It is a block diagram which shows the structure of the conventional image coding apparatus. It is a figure for demonstrating the process of the motion vector detection part in the conventional image coding. It is a figure for demonstrating the pixel interpolation method of the decimal precision pixel at the time of the motion compensation in the conventional image coding.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image coding apparatus 101 Motion vector candidate detection part 102 Mode determination part 103 Motion vector candidate detection part 1000 Image coding apparatus 1001 Differentiator 1002 Image coding part 1003 Variable length coding part 1004 Image decoding part 1005 Adder 1006 Image Memory 1007 Picture memory 1008 Motion compensation encoding unit 1009 Motion vector detection unit 1010 Prediction direction determination unit 1011 Motion vector storage unit
Img image data
Res difference image data
CodedRes Difference image encoded data
Bitstream encoded data
Recon Decoded image data
Ref reference image data
MotionParam motion parameters
Mod encoding mode
MotionVector motion vector
RefIdx reference picture number
Dir prediction mode

Claims (4)

Two motion vectors with decimal pixel precision are determined for the encoding target block, and a difference value between a value obtained by averaging the pixel values of the reference block indicated by the two motion vectors and the pixel value of the encoding target block is calculated. An image encoding method for encoding together with the two motion vectors,
A motion detection step of performing motion detection with integer pixel precision on the reference picture;
A motion vector determination step of detecting two motion vectors by detecting a motion vector with a pixel accuracy of 1 / 2n around the reference position of the motion detected in the motion detection step;
For the encoding target block, two motion vectors determined in the motion vector determination step, a value obtained by averaging pixel values of reference blocks indicated by the two motion vectors, and a pixel value of the encoding target block An image encoding method comprising: an encoding step for encoding a difference value between the two.   2. The image coding method according to claim 1, wherein the average of the pixel values of the reference block indicated by the two motion vectors is 1 / which is obtained by dividing the sum of the pixel values of the reference block indicated by the two motion vectors by 2. An image encoding method characterized by being an average of two. Two motion vectors with decimal pixel precision are determined for the encoding target block, and a difference value between a pixel value of a reference block indicated by the two motion vectors and a pixel value of the encoding target block is determined. An image encoding device for encoding together with the two motion vectors,
Motion detection means for performing motion detection with integer pixel accuracy on a reference picture;
Motion vector determining means for detecting a motion vector with a pixel accuracy of 1 / 2n and determining two motion vectors around a reference position of the motion detected by the motion detecting means;
The difference between the two motion vectors determined by the motion vector determination means for the encoding target block, the average value of the pixel values of the reference block indicated by the two motion vectors, and the pixel value of the encoding target block An image encoding device having encoding means for encoding a value. Two motion vectors with decimal pixel precision are determined for the encoding target block, and a difference value between a pixel value of a reference block indicated by the two motion vectors and a pixel value of the encoding target block is determined. , A program for causing an image encoding apparatus to implement an image encoding method for encoding together with the two motion vectors,
A motion detection step in which the image encoding device performs motion detection with integer pixel accuracy on a reference picture;
A motion in which the image encoding device detects a motion vector with a pixel accuracy of 1 / 2n and determines two motion vectors around the reference position of the motion detected in the motion detection step. A vector determination step;
The image encoding device averages the two motion vectors determined in the motion vector determination step, the pixel values of the reference block indicated by the two motion vectors, and the code for the encoding target block. A program for realizing an encoding step for encoding a difference value with a pixel value of a conversion target block.

Images