JP2015088954A - Image encoder and image encoding program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image encoder and an image encoding program, with which encoding efficiency by bidirectional prediction is improved.SOLUTION: A storage section stores a reference image obtained by decoding encoded data of an encoded input image for each frame. A prediction section performs intra-frame prediction, forward prediction or bidirectional prediction by using the reference image stored in the storage section and generates a predicted image block corresponding to an input image block being a part of the input image. When a code amount control section is to perform bidirectional prediction with a decoded image of the predicted image block generated by bidirectional prediction by the prediction section as the reference image, it determines a code amount on the predicted image block to be predicted according to data on motion interpolation of the reference image whose motion interpolation is predicted and decoded in the bidirectional prediction that is previously performed.

Description

  The present invention relates to an image encoding device and an image encoding program.

With the development of digital technology, development of electronic devices capable of processing high frame rate image signals is progressing. In general, the amount of data increases as the image signal has a higher frame rate. When transmitting such an image signal, the amount of data is often reduced by compression encoding. In image coding systems used for broadcasting and communication, the amount of data is often compressed by removing the redundancy of information in the time direction by performing motion compensation prediction processing. This realizes highly efficient image coding at a low bit rate.
A frame to be encoded is classified into an intra frame (I picture) or an inter frame depending on whether or not motion compensation prediction processing is performed. An intra frame is a frame that is encoded by performing intra-frame prediction (intra prediction). An inter frame is a frame that is encoded by performing inter-frame prediction (inter prediction). Also, depending on the type (number of frames) of the reference image frame used when performing the motion compensation prediction process, the frame is classified into a forward prediction frame (P picture) or a bidirectional prediction frame (B picture). A P picture is a frame that is encoded using forward prediction processing. A B picture is a frame that is encoded by performing bidirectional prediction processing using two or more reference images. When a prediction process is performed on a B picture, a reference image related to a frame of an I picture, a P picture, or both may be used.

  On the other hand, in the image coding methods described in Non-Patent Documents 1 and 2, a B picture that has already been subjected to motion compensation prediction and decoded in the prediction process, that is, a frame (current frame) to be processed at that time. In addition, it is allowed to predict the motion of other B pictures using a past or future B picture as a reference image. The image encoding methods described in Non-Patent Documents 1 and 2 are also called MPEG (Moving Picture Experts Group) -4 AVC (Advanced Video Coding) and HEVC (High Efficiency Video Coding), respectively. In the image coding described in Non-Patent Documents 1 and 2, the code amount assigned to each frame may differ depending on the type of frame. For example, it is conceivable that the amount of code assigned to a B picture that is not used for prediction is made smaller than the amount of code assigned to another type of frame.

Recommendation ITU-T H.D. 264, (04/2013), “Advanced video coding for generic audiovisual services”, International Telecommunication Union, April 2013. Recommendation ITU-T H.D. 265, (04/2013), “High efficiency video coding”, International Telecommunication Union, April 2013.

  However, the allocation of the code amount depending on the frame type is not defined by the encoding standard, and is left to the implementation of the encoding apparatus. If the code amount assignment is fixed, the encoding process is always performed with the assigned code amount regardless of the characteristics of the image to be processed. For this reason, there is a possibility that the allocated code amount is insufficient and the image quality is deteriorated, or that the allocated code amount is excessive and the encoding efficiency is lowered.

  The present invention has been made in view of the above points, and provides an image encoding device and an image encoding program that improve the encoding efficiency by bidirectional prediction.

  SUMMARY An advantage of some aspects of the invention is that a storage unit that stores, for each frame, a reference image obtained by decoding encoded data of an encoded input image, and the storage A prediction unit that performs intra-frame prediction, forward prediction, or bidirectional prediction using the reference image stored in the unit, and generates a predicted image block corresponding to an input image block that is a part of the input image; When the prediction unit further performs bidirectional prediction using the decoded image of the predicted image block generated by performing bidirectional prediction as a reference image, the code amount related to the predicted image block is motion-compensated prediction by the bidirectional prediction already performed. And a code amount control unit determined based on data relating to motion compensation of the decoded reference image.

Another aspect of the present invention is the image coding apparatus, wherein the code amount control unit uses a motion vector as the data related to the motion compensation.
Another aspect of the present invention is the image coding device, wherein the code amount control unit uses a motion prediction mode as the data related to the motion compensation.

  In another aspect of the present invention, the computer uses a reference image stored in a storage unit that stores, for each frame, a reference image obtained by decoding encoded data of an encoded input image. A prediction unit that performs bidirectional prediction to generate a predicted image block corresponding to an input image block that is a part of the input image, and a decoding of the predicted image block that has already been generated by bidirectional prediction by the prediction unit When bi-directional prediction is performed using an image as a reference image, the code amount for the predicted image block is determined based on the motion compensation prediction performed in the bi-directional prediction already performed and the data related to the motion compensation of the decoded reference image. And a quantity control unit.

  According to the present invention, the encoding efficiency by bidirectional prediction is improved.

It is a schematic block diagram which shows the structure of the image coding apparatus which concerns on embodiment of this invention. It is a figure which shows the example of a reference image and a prediction image. It is a conceptual diagram which shows an example of a GOP structure. It is a conceptual diagram which shows an example of the hierarchy B structure. It is a conceptual diagram which shows the other example of the hierarchy B structure. It is a table | surface which shows an example of the quantization corresponding | compatible data which concern on this embodiment. It is a flowchart which shows the process which the code amount control part which concerns on this embodiment determines a code amount. It is a schematic block diagram which shows the structure of the image decoding apparatus which concerns on this embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
First, the configuration of the image encoding device 1 according to the present embodiment will be described.
FIG. 1 is a schematic block diagram illustrating a configuration of an image encoding device 1 according to the present embodiment.
The image encoding apparatus 1 includes a frame buffer 101, a subtracting unit 102, an orthogonal transform unit 103, a quantization unit 104, a lossless encoding unit 105, a storage buffer 106, an inverse quantization unit 107, an inverse orthogonal transform unit 108, and an addition unit 109. , A deblocking filter 110, a reference memory 111, an intra prediction unit 112, an inter prediction unit 113, a mode determination unit 114, and a code amount control unit 115.

  An input image signal is input from the outside of the image encoding device 1 to the frame buffer 101, and the input image signal is temporarily stored for each frame. The frame buffer 101 may rearrange the order of the frames in accordance with a predetermined GOP (Group of Picture) structure. The GOP structure is an arrangement of an I picture, a P picture, and a B picture having a predetermined number of frames, and starts with an I picture in decoding order. The frame buffer 101 specifies an image signal of a frame (also referred to as a current frame or a current frame) that is currently processed from the stored input image signal. The frame buffer 101 reads an input image block signal indicating an image of a block (current block or current block) that is currently processed as a part of the image signal of the specified frame. The order in which the frame buffer 101 selects the current block may be the same as the raster scan order, for example. The frame buffer 101 outputs the read input image block signal to the subtraction unit 102, the intra prediction unit 112, and the inter prediction unit 113.

  The subtraction unit 102 subtracts the prediction image block signal input from the intra prediction unit 112 or the inter prediction unit 113 from the input image block signal input from the frame buffer 101 to generate a difference image block signal. The subtraction unit 102 outputs the generated difference image block signal to the orthogonal transformation unit 103. When the current frame is an intra frame, the prediction image block signal is input from the intra prediction unit 112 to the subtraction unit 102. If the current frame is an inter frame, the predicted image block signal is input from the inter prediction unit 113 to the subtraction unit 102. The subtracting unit 102 subtracts the signal value of the corresponding pixel among the signal values forming the reference image block signal from the signal value for each pixel forming the input image block signal.

  The orthogonal transform unit 103 performs orthogonal transform on the signal value indicated by the difference image block signal input from the subtraction unit 102 to generate transform coefficient data indicating the transform coefficient. The orthogonal transform unit 103 performs, for example, discrete cosine transform (DCT), Karhunen-Loeve transform (KLT), and the like as orthogonal transform.

  The quantization unit 104 quantizes the transform coefficient indicated by the transform coefficient data input from the orthogonal transform unit 103. The quantization unit 104 quantizes the transform coefficient by division by a value determined by the quantization parameter determined by the code amount control unit 115. The quantization width may be a predetermined fixed value. The quantization unit 104 outputs quantized transform coefficient data indicating the quantized transform coefficient (quantized transform coefficient) to the lossless encoding unit 105 and the inverse quantization unit 107.

  The lossless encoding unit 105 receives the quantized transform coefficient data from the quantization unit 104. The mode data is input from the mode determination unit 114 to the lossless encoding unit 105. The mode data is data indicating a prediction mode when predicting a predicted image block signal using a reference image block signal. Depending on the prediction mode indicated by the input mode data, the type of data to be encoded and the presence or absence thereof differ.

  For example, when the mode data indicates a motion compensation mode without data division, the motion vector data of the current block is input from the mode determination unit 114 to the lossless encoding unit 105. When the mode data indicates a motion compensation mode with division, the lossless encoding unit 105 receives motion vector data for each sub-block obtained by further dividing the current block from the mode determination unit 114. When one collective block (collective block) is a coding unit (Coding Unit) in which encoding is performed, both the collective block and the sub-block can be a prediction unit (Prediction Unit) in which prediction is performed. The relationship between CU and PU will be described in the description of the inter prediction unit 113.

When the mode data indicates the intra prediction mode in the inter frame, the quantized transform coefficient data is input to the lossless encoding unit 105 from the quantization unit 104, but the reference image data and motion vector data are not input. When the mode data indicates the skip mode, quantized transform coefficient data, reference image data, and motion vector data other than the mode data indicating the skip mode are not input to the lossless encoding unit 105.
The lossless encoding unit 105 integrates input data, and performs lossless encoding on the integrated data (integrated data), for example, variable length encoding, arithmetic encoding, and the like to generate encoded data. The lossless encoding unit 105 temporarily stores the generated encoded data in the accumulation buffer 106.

  The lossless encoding unit 105 may generate integrated data by integrating the quantization parameter determined for each block by the code amount control unit 115 with other input data (for example, quantized transform coefficient data). . In addition, the lossless encoding unit 105 integrates a differential quantization parameter, which is a difference between the quantization parameter related to the current block and the quantization parameter related to the adjacent block, instead of the quantization parameter to generate integrated data. May be. When the quantization parameter is almost constant between the blocks, the difference quantization parameter is 0 continuously between the blocks. Therefore, the amount of codes can be reduced by lossless encoding.

  Note that the lossless encoding unit 105 includes an encoded data storage unit that stores a motion vector related to an input image block that has already been encoded, and selects a motion vector having the smallest distance from the input motion vector as a prediction vector. (Vector prediction) may be performed. In that case, the lossless encoding unit 105 calculates a difference vector of the prediction vector selected from the input motion vector. The lossless encoding unit 105 may generate integrated data by integrating the calculated difference vector and a vector index indicating the selected prediction vector instead of the input motion vector data.

  The accumulation buffer 106 stores the encoded data input from the lossless encoding unit 105. The accumulation buffer 106 outputs the stored encoded data to the outside of the image encoding device 1, for example, the image decoding device 2.

The inverse quantization unit 107 inversely quantizes the quantized transform coefficient indicated by the quantized transform coefficient data input from the quantizing unit 104, so that any one of the ranges corresponding to the quantization parameter determined by the code amount control unit 115 is obtained. Restored transform coefficient data indicating the restored transform coefficient that takes these values is generated. The inverse quantization unit 107 outputs the restored transform coefficient data to the inverse orthogonal transform unit 108.
The inverse orthogonal transform unit 108 performs inverse orthogonal transform on the restored transform coefficient indicated by the restored transform coefficient data input from the inverse quantization unit 107 to generate a restored difference image block signal. The inverse orthogonal transform used by the inverse orthogonal transform unit 108 is often an inverse operation of the orthogonal transform performed by the orthogonal transform unit 103. The inverse orthogonal transform unit 108 outputs the generated restored difference image block signal to the addition unit 109.

  The addition unit 109 adds the restored difference image block signal input from the inverse orthogonal transform unit 108 and the predicted image block signal input from the intra prediction unit 112 or the inter prediction unit 113 to generate a restored image block signal. Here, the adding unit 109 adds the signal value for each pixel forming the restored difference image block signal and the signal value of the corresponding pixel among the signal values forming the predicted image block signal. The adder 109 outputs the generated restored image block signal to the deblocking filter 110.

The deblocking filter 110 performs filtering processing on the restored image block signal input from the adder 109 to remove a block distortion component. The deblocking filter 110 stores the restored image block signal from which the block distortion component has been removed in the reference memory 111 as a reference image block signal.
The reference memory 111 stores the reference image block signal input from the deblocking filter 110 in a storage area corresponding to the reference image block for each frame. Thereby, the reference image signal indicating the reference image is stored for each frame.

  When the current frame is an intra frame (I picture), the intra prediction unit 112 performs intra prediction using the image signal related to the current frame among the reference image signals already stored in the reference memory 111 and performs the current prediction. A prediction image block signal related to the block is generated. The intra prediction unit 112 performs intra prediction in each of a plurality of predetermined prediction modes. The prediction directions are different between the plurality of prediction modes. The intra prediction unit 112 calculates a cost value, which will be described later, and associates mode data determined thereby to output to the mode determination unit 114. The intra prediction unit 112 outputs the generated predicted image block signal to the subtraction unit 102 and the addition unit 109.

When the current frame is an inter frame (P picture, B picture), the inter prediction unit 113 mainly performs inter prediction using the reference image signal already stored in the reference memory 111, and relates to the current block. A prediction image block signal is generated. Note that the inter prediction unit 113 may perform intra prediction on the inter-frame image signal to generate a predicted image block signal, and the mode determination unit 114 may determine whether to adopt the intra prediction mode.
The frame of the reference image signal used for inter prediction is determined by which frame is the current frame in the frame order corresponding to the GOP structure. For example, when the current frame is a P picture, the inter prediction unit 113 performs forward prediction using a reference image of one frame (for example, I picture) at a time (past) before the current frame. There are many cases. When the current frame is a B picture, a reference image of one past frame (for example, I picture or B picture) and one frame (for example, a P picture or B picture) after the current frame (future) Bidirectional prediction is often performed using a reference picture of (picture). The relationship between the frame order, the frame related to the reference image signal used in the prediction process, and the frame related to the predicted image signal generated in the prediction process is indicated by a GOP structure. An example of the GOP structure will be described later.

  Here, the inter prediction unit 113 performs the prediction process in any of a plurality of predetermined prediction modes. The prediction mode includes, for example, a motion compensation prediction mode and a skip mode. The motion compensation prediction mode is a prediction mode for performing the above-described inter prediction. That is, the motion compensation prediction mode is a prediction mode in which a predicted image block is predicted by detecting a motion of an image between frames in a target block using a reference image. In the motion compensation prediction mode, there are a case where there is no division in which prediction processing is performed on a batch block in which input image blocks are batched as an encoding unit, and a case where there is division in which prediction processing is performed for each more sub-block. is there. In the case of a collective block, the input image block, that is, the PU of a block having the same size as the CU, and in the case of a sub-block that is subdivided, is a divided PU. The size of the PU is, for example, horizontal 16 pixels × vertical 16 pixels, horizontal 8 pixels × vertical 16 pixels, horizontal 16 pixels × vertical 8 pixels, horizontal 8 pixels × vertical 8 pixels, and the like. . The skip mode is a mode in which encoded data such as a motion vector or a difference vector for a motion vector or a quantized transform coefficient is generated for the current block, and information on adjacent encoded blocks is copied without being transmitted.

  In the motion compensation prediction mode without division, the inter prediction unit 113 generates a predicted image block signal for each PU having the same size as the CU. The inter prediction unit 113 selects a block that most closely approximates the input image block input from the frame buffer 101 among the predicted image blocks indicated by the generated predicted image block signal. The inter prediction unit 113 uses, for example, RD (Rate-Distortion) cost as a cost value indicating the degree of approximation to be selected (RD optimization). The RD cost is a value obtained by weighting and adding, for example, an information amount (code amount) when encoding a difference vector to a distortion amount of an input image block with respect to a predicted image block. The difference vector is a difference between a motion vector and a prediction vector generated by performing prediction processing on the motion vector. This difference vector is an object to be encoded. In addition, the inter prediction unit 113 may use a sum of absolute differences (SAD), a sum of square differences (SSD), or the like as a cost value.

  The inter prediction unit 113 relates to, for example, a predicted image block signal indicating the selected predicted image block, a reference frame number indicating a frame of the reference image signal referred to when the predicted image block signal is generated, and the selected predicted image block. The motion vector data indicating the motion vector, the calculated cost value, and the mode data indicating the motion compensation prediction mode without division are associated with each other and output to the mode determination unit 114.

In the motion compensated prediction mode with division, the inter prediction unit 113 generates a predicted image block signal for each divided PU. The inter prediction unit 113 calculates a cost value based on the difference between the prediction image block indicated by the prediction image block signal for each PU and the portion of the input image block corresponding to each PU, for example, RD optimization Thus, the predicted image block signal that is closest to each PU can be selected. The inter prediction unit 113 adds the cost values calculated for the predicted image block signal selected for each PU within the current block to which the PU belongs.
The inter prediction unit 113 selects a predicted image block signal selected for each PU, a reference frame number indicating a frame of the referenced reference image signal, motion vector data indicating the selected motion vector, a calculated cost value, and motion compensated prediction with division. The mode data indicating the mode is associated and output to the mode determination unit 114.

  In the intra prediction mode, the inter prediction unit 113 performs intra prediction on the current block to generate a predicted image block signal. The inter prediction unit 113 calculates a cost value indicating the degree of approximation between the predicted image block signal and the input image block signal. The inter prediction unit 113 associates the predicted image block signal generated for the current block, the calculated cost value, and the mode data indicating the intra prediction mode, and outputs them to the mode determination unit 114.

  In the skip mode, the inter prediction unit 113 calculates a representative value, for example, a median vector of motion vectors related to a processed block that is adjacent to the current block (median prediction). When the processing is performed in the order of raster scanning, the processed blocks are, for example, four blocks adjacent to the upper left, right above, upper right, and left of the current block, respectively. The inter prediction unit 113 uses, as a reference image block signal, a reference image block signal stored at a position compensated with a shift amount indicated by the calculated median vector from the current block in the reference image of the frame corresponding to the current frame. Read from 111. The inter prediction unit 113 calculates a cost value indicating the degree of approximation between the predicted image block signal and the input image block signal. The inter prediction unit 113 associates the calculated cost value with the mode data indicating the skip mode, and outputs them to the mode determination unit 114.

  The mode determination unit 114 receives at least a cost value and mode data in association with each other from the inter prediction unit 113. For example, when the mode data indicates a motion compensation mode without division, a predicted image block signal, reference image data, and motion vector data for each PU having the same size as the CU are also input in association with each other. When the mode data indicates a motion compensation mode with division, a predicted image block signal, reference image data, and motion vector data for each divided PU are also input in association with each other. When the mode data indicates the intra prediction mode, a predicted image block signal is also input in association with it.

  The mode determination unit 114 selects mode data indicating a prediction mode corresponding to the cost value at which the input cost value is the minimum among the plurality of prediction modes. The mode determination unit 114 outputs the selected mode data to the lossless encoding unit 105. For example, when the selected mode indicates a motion compensation mode without data division, the mode determination unit 114 outputs a prediction image block signal for each PU having the same size as the CU to the subtraction unit 102 and the addition unit 109, and The motion vector data for each PU with the same size is output to the lossless encoding unit 105. When the selected mode data indicates a motion compensation mode with division, the mode determination unit 114 integrates the divided prediction image block signals for each PU in the CU, and outputs them to the subtraction unit 102 and the addition unit 109. The motion vector data for each PU is output to the lossless encoding unit 105. When the selected mode data indicates the intra prediction mode, the mode determination unit 114 outputs the predicted image block signal to the subtraction unit 102 and the addition unit 109.

  The code amount control unit 115 controls the code amount of the quantized transform coefficient data generated by the quantization unit 104 for each block according to the frame type of the input image. As described above, the transform coefficient indicated by the quantized transform coefficient data is obtained by subtracting the difference image block signal obtained by subtracting the prediction image block signal generated by the intra prediction unit 112 or the inter prediction unit 113 from the input image block signal. It is obtained by orthogonal transformation at 103.

  By the way, when the current frame is a B picture, the inter prediction unit 113 may generate a predicted image block signal by using a B picture that has already been generated by bi-directional prediction as a reference image signal. In this case, the code amount control unit 115 may determine the code amount of the quantized transform coefficient data related to the current block based on the data related to motion compensation performed in the already performed bidirectional prediction. The code amount control unit 115 may be set with a predetermined fixed code amount depending on the type of frame (for example, I picture, P picture). A code amount setting example and a control example will be described later.

Next, examples of reference images and predicted images will be described.
FIG. 2 is a diagram illustrating an example of a reference image and a predicted image.
FIGS. 2A-1, 2A-2, and 3A-3 illustrate a reference image of a past frame, an image of a current frame, and a reference image of a future frame, respectively. These images include a barrier b and a vehicle c, respectively. FIGS. 2A-1 to 2A-3 show that the vehicle c moves behind the barrier b from the left to the right as time elapses.

  FIG. 2 (p-2) shows a forward prediction image at the same time as (a-2) generated by performing forward prediction based only on the reference image shown in FIG. 2 (a-1). The region shown on the right side of the barrier b in FIG. 2 (p-2) is the (occlusion) portion hidden behind the barrier b in FIG. 2 (a-1) and moves. (Uncovered background) This blank area indicates that it cannot be predicted only with reference images of past frames, and therefore, a predicted image related to this area. Since it is necessary to further encode the signal value of the difference image block between the block and the input image block, the code amount cannot be reduced.

  FIG. 2 (b-2) shows a bi-directional prediction image generated by performing bi-directional prediction using the reference image shown in FIG. 2 (a-3) in addition to the reference image shown in FIG. 2 (a-1). Show. In FIG. 2B-2, the blank area generated in FIG. 2P-2 does not appear, and approximates the current frame image shown in FIG. 2A-2. That is, the signal value of the difference image block can be sufficiently reduced from the signal value of the input image block, and the code amount can be reduced. However, since the reference image signal of a future frame is used in bidirectional prediction, it is required to store the reference image signal used in the processing and to change the order of the frames. This causes a delay.

Next, an example of the GOP structure will be described.
FIG. 3 is a conceptual diagram showing an example of the GOP structure.
The GOP structure shown in FIG. 3 is also used in conventional image coding schemes such as MPEG-4 AVC and HEVC. In FIG. 3, the horizontal direction indicates the time, and the right side indicates the newer time than the left side.
A vertically long rectangle indicates an image of each frame, and a symbol is shown at the approximate center of each rectangle. Each of the symbols consists of two characters. Among the two characters, “I” on the left side indicates the type of frame (I picture, etc.), and “0” on the right side indicates the frame order (0th etc.) in the GOP. ). The starting point and the ending point of the arrows connecting the squares indicate the frame related to the reference image used for the prediction process and the frame related to the predicted image generated by the prediction process. For example, the predicted image of the frame P4 is generated by performing forward prediction using the reference image of the frame I0. The predicted images of the frames B1, B2, and B3 are all generated by performing bi-directional prediction using the reference images of the frames I0 and P4. In this GOP, the B picture is not a reference image.

  On the other hand, the GOP structure used in the image coding apparatus 1 according to the present embodiment is a structure in which a certain B picture is used as a reference image and a predicted image of another B picture is generated in bidirectional prediction. This structure is called a hierarchy B structure and includes an upper hierarchy and a lower hierarchy. The upper layer is a layer that performs bidirectional prediction using an I picture, a P picture, or both as reference images, and generates a B picture as a predicted image. The lower layer is a layer in which the B picture generated in the upper layer can be used as a reference image, and bidirectional prediction is thereby performed to predict another B picture. A B picture that is not used as a reference image is called a non-reference picture. In the following description, a non-reference B picture generated in the lower hierarchy is distinguished from a B picture generated in a general B picture or an upper hierarchy by expressing it as a b picture.

FIG. 4 is a conceptual diagram illustrating an example of a hierarchical B structure.
In the example illustrated in FIG. 4, the predicted image of the frame B2 in the upper layer is generated by performing bi-directional prediction using the reference images of the frames I0 and P4 as in the example illustrated in FIG. In the lower hierarchy, the predicted image of the frame b1 is generated by performing bi-directional prediction using the reference images of the frames I0 and B2. The predicted image of the frame b3 is generated by performing bi-directional prediction using the reference images of the frame B2 and the frame P4. Since the predicted images of the frames b1 and b3 are each predicted using the reference image related to the temporally adjacent frame, only the frames that are further apart in time (for example, the frames I0 and P4) as in the example illustrated in FIG. The image quality is improved as compared with the case of using. Therefore, since the signal value of the residual image becomes small, the code amount can be reduced. Note that the frames b1 and b3 are non-reference pictures that are not used for bidirectional prediction. That is, bi-directional prediction of the lower hierarchy is not performed using these reference images. Therefore, the image quality of the frames b1 and b3 does not affect the image quality of the predicted image generated in the other frames.

  The reason why the other B picture is used as the reference image as in the lower hierarchy shown in FIG. 4 is that the prediction accuracy is improved in the relatively new image coding method (new method) compared to the old method. Even if it is used, there is a background that deterioration of the image quality of the predicted image is suppressed. The new system is, for example, MPEG-4 AVC, HEVC, etc., and the old system is, for example, MPEG-2, etc. Further, in the new method, the image quality of the predicted image of the frame relating to the I picture is also improved compared to the old method, and therefore the image quality of the predicted image of the P picture or B picture generated using the reference image based on this predicted image. There is also a background that has improved.

FIG. 5 is a conceptual diagram showing another example of the hierarchical B structure.
In FIG. 5, the horizontal axis indicates the time, and the lower level indicates the upper level and the upper level indicates the lower level.
In the example shown in FIG. 5, the upper layer has frames I0, B1, P2, B3, and P4 at regular time intervals, and the lower layer has frames b1, b2, b3, and b4 at regular time intervals. The times of the frames b1, b2, b3, and b4 are the middle of the frames I0 and B1, the middle of the frames B1 and P2, and the middle of the frames P2, B3, B3, and P4, respectively.

In this example, the predicted images of the frames P2 and P4 in the upper layer are generated by performing forward prediction using the frame I0 as a reference image. The predicted image of the frame B1 is generated by performing bi-directional prediction using the frames I0 and P2 as reference images. The predicted image of the frame B3 is generated by performing bi-directional prediction using the frames P2 and P4 as reference images.
In the lower layer, the predicted image of the frame b1 is generated by performing bi-directional prediction using the frames I0 and B1 as reference images. The predicted image of the frame b2 is generated by performing bi-directional prediction using the frames B1 and P2 as reference images. The predicted image of the frame b3 is generated by performing bi-directional prediction using the frames P2 and B3 as reference images. The predicted image of the frame b4 is generated by performing bi-directional prediction using the frames B3 and P4 as reference images.

  Also in this example, since the predicted images of the frames b1, b2, b3, and b4 are predicted using the reference images related to the temporally adjacent frames, for example, frames that are more distant from each other (for example, the frames I0, P2,. The image quality is improved as compared with the case of using only P4). Therefore, the code amount can be reduced. Note that the frames b1, b2, b3, and b4 are non-reference pictures, and therefore do not affect the image quality of predicted images generated in other frames.

Next, the code amount controlled by the code amount control unit 115 will be described.
The code amount control unit 115 uses, for example, a quantization parameter Q _p as an index of the code amount. Q _p is, for example, 1 to 31 (examples of MPEG-2), (example of AVC) from 1 to 51, an integer value in the range such as from 1 to 57 (examples of HEVC). Q _p is the example AVC, the code amount for each 6 increases is an indicator showing that reduced to 1/2. That is, Q _p indicates a range of values that the transform coefficient indicated by the transform coefficient data input to the quantization unit 104 can take after quantization.
The code amount control unit 115 may predetermine Q _p (the code amount is large) smaller than that of other types of frames in the I picture that is an intra frame. Since the I picture is referred to when predicting other frames, the image quality of the entire encoded video can be improved by increasing the image quality of the I picture.

The code amount control unit 115 may predetermine Q _p (the code amount is small) larger than that of the I picture for a P picture that is an inter frame. For example, Q _p given to the P picture is made 1 or 2 larger than Q _p given to the I picture. Since a P picture is generated by performing motion prediction compensation using an I picture, the amount of information generated by encoding tends to be smaller than that of an I picture. Therefore, even if the code amount is reduced, the influence on the image quality is limited.
The code amount control unit 115 may predetermine Q _p (the code amount is small) larger than that of the P picture for the B picture for which bi-directional prediction is performed in the upper layer. For example, Q _p given to the B picture is made 2 or 3 larger than Q _p given to the P picture. This is because the prediction residual of a B picture tends to be smaller than that of a P picture.

The code amount control unit 115 may predetermine Q _p (the code amount is small) larger than that of the B picture for b pictures for which bi-directional prediction is performed in the lower layer. For example, Q _p given to the b picture is made 2 or 3 larger than Q _p given to the B picture. This is because when the b picture is arranged so as to be sandwiched between images of frames having sufficient image quality as described above, the b picture can also ensure sufficient image quality. This is because even when the picture quality of the b picture deteriorates, if the frame images before and after the sufficient quality are viewed, it is difficult to detect the deterioration of the image quality of the entire moving image. When the b picture is a non-reference picture, it does not affect the image quality of the predicted image generated in other frames.

Accordingly, the quantization parameter Q _p set in advance in the code amount control unit 115 may be given the magnitude relationship shown in Expression (4) between the frame types.
Q _p (I) <Q _p (P) <Q _p (B) <Q _p (b) (4)
In Expression (4), Q _p (I) and the like indicate a quantization parameter Q _{p and the} like related to the I picture. Q _p (P) is, for example, 2 to 4 larger than Q _p (I). Q _p (B) is, for example, a value 2 to 4 larger than Q _p (P). Q _p (b) is, for example, a value 2 to 4 larger than Q _p (B).
Thereby, the code amount control unit 115 switches the quantization parameter Q _p according to the type of the current frame, and outputs the switched quantization parameter Q _p to the quantization unit 104.
These quantization parameters Q _p (I) and the like are determined so that the code amount of encoded data for each GOP output from the accumulation buffer 106 does not exceed a predetermined transmission capacity.

When the type of the current frame is a b picture, as described above, a predicted image block is generated in the lower layer using a reference image signal that is a B picture subjected to bi-directional prediction in the upper layer. In this case, the code amount control unit 115 uses the motion compensation data used when the quantization parameter Q _p related to the current block is generated by bi-predicting the predicted image block signal of the B picture in the upper layer. You may decide based on. The data related to motion compensation includes, for example, (I) motion vector data indicating a motion vector and (II) mode data indicating a prediction mode. The type of frame related to the reference image signal used in the bidirectional prediction is, for example, a P picture. Therefore, the lossless encoding unit 105 or the code amount control unit 115 may include a storage unit that stores data related to motion compensation for each block. The code amount control unit 115, the frame according to the reference image signal, reads the data relating to motion compensation in the current block from the storage unit, determining the quantization parameter Q _p, based on the read data. However, the code amount control unit 115 may use the above-described predetermined Q _p (b) as the initial value Q _p0 of the quantization parameter.

(I) When using motion vector data The code amount control unit 115 determines a smaller quantization parameter Q _p as the motion vector indicated by the motion vector data is larger. A large size of the motion vector acquired in the upper layer indicates that the motion of the image represented in the reference block is large between frames. In that case, the code amount in the lower layer is increased. On the other hand, a small size of the motion vector acquired in the upper layer indicates that the motion of the image represented in the reference block is small between frames. In that case, the code amount in the lower layer is reduced.

Here, the code amount control unit 115 may determine Q _p using, for example, Expression (5).
Q _p = Q _p0 −INT (α · | mv |) (5)
In the expression (5), INT (...) Represents a realization function that gives an integer obtained by rounding down values after the decimal point of a real value. α is a predetermined positive real number. | Mv | is a real number of 0 or greater than 0 indicating the magnitude of the motion vector. As for α, the value of INT (α · | mv |) is about 0 to 10 and Q _p is in the above-mentioned range, for example, the range of 1 to 31 (example of MPEG-2) in advance. It may be determined by performing a simulation.

Note that when two reference image block signals used for bidirectional prediction are both reference image signals related to a P picture (see FIG. 5, B3), each of a plurality of PUs included in one reference image block Motion vector data may be defined for. In that case, the code amount control unit 115, the magnitude of the representative value of the plurality of motion vectors obtained, for example, the median value, average value, one of the minimum value, by using in determining the quantization parameter Q _p Also good.

(II) When Mode Data is Used The code amount control unit 115 determines a quantization parameter Q _p corresponding to the prediction mode indicated by the mode data acquired in the upper layer. For example, the code amount control unit 115 stores in advance quantization-corresponding data indicating the correspondence relationship between the prediction mode of the upper layer and the quantization parameter, and corresponds to the mode data acquired from the stored quantization-corresponding data. Read the quantization parameter.

FIG. 6 is a table showing an example of quantization-compatible data according to the present embodiment.
The left column and the right column in FIG. 6 indicate the upper layer prediction mode and the lower quantization parameter, respectively. The second row in FIG. 6 indicates that the intra mode and the quantization parameter _Qp1 are associated as the upper layer prediction modes. The third row indicates that the motion compensated prediction mode with division and the quantization parameter _Qp2 are associated as the higher-layer prediction mode. The fourth row indicates that the motion compensation prediction mode without division and the quantization parameter Q _p3 are associated as the upper layer prediction mode. The fifth row indicates that the skip mode and the quantization parameter Q _p4 are associated as the upper layer prediction mode.

Between the quantization parameters Q _p1 , Q _p2 , Q _p3 , Q _p4 and the initial value Q _p0 , for example, the magnitude relationship shown in the equation (6) is given.
_{Q p1 <Q p2 <Q p0} <Q p3 <Q p4 ... (6)
In Expression (6), Q _p1 is a value that is 2 or 3 smaller than the initial value Q _p0 , for example. Q _p2 is, for example, a value that is 1 smaller than the initial value Q _p0 . Q _p3 is, for example, a value that is 1 larger than the initial value Q _p0 . Q _p4 is, for example, a value that is 2 or 3 larger than the initial value Q _p0 .

Here, unlike the other prediction modes, the smallest quantization parameter Q _p1 for the intra mode is determined because only the reference image signal of the current frame is used and the image signal of the other frame is not used. This is because a large amount of code is required.
The _{reason why the} largest quantization parameter Q _p4 is determined for the skip mode is that the motion vector of the adjacent block is used as the motion vector of the current block. This indicates that the image motion is uniform over at least a region larger than the block, or the image motion itself is small. For this reason, it is expected that the image quality of the prediction image indicated by the prediction image block generated using the reference block signal of the adjacent frame is good, and the required code amount is small. In bidirectional prediction, a predicted image block signal of a lower layer frame is generated using reference image block signals of frames before and after the frame. Therefore, when the skip mode is used in the upper layer, the skip mode is also used in the lower layer, so there is a high possibility that the residual image block signal is not encoded.

Between the motion compensated prediction mode without division and the motion compensated prediction mode with division, the quantization parameter _Qp2 related to the motion compensated prediction mode with division is the quantization parameter Q related to the motion compensated prediction mode without division. _It is smaller than _p3 . The motion-compensated prediction mode without division indicates that the image motion uniformity in the current block is high. In this case, the uniformity of image motion within the current block is high even in the lower layer, and it is expected that image quality deterioration will not be noticeable even if the code amount is reduced. On the other hand, the motion-compensated prediction mode with division indicates that the motion of the image in the current block is non-uniform. In this case, the uniformity of the image motion in the current block is low even in the lower layer, and it is expected that the image quality will be significantly deteriorated if the code amount is reduced.

  Thus, the code amount control unit 115 controls the code amount according to the type of frame. Here, the code amount control unit 115 determines the code amount of the encoded data generated by the bidirectional prediction to be smaller than the code amount in the intra prediction or the forward inter prediction. Also, the code amount control unit 115 determines the code amount generated by the bidirectional prediction according to the motion of the image in the already encoded bidirectional prediction. Therefore, a larger amount of code is assigned to a region where image quality is likely to deteriorate, such as a region where the motion of the image is remarkable or a region where the motion is non-uniform, and a smaller amount of code is assigned to a region where the image quality is less likely to deteriorate. As a result, the degradation of the image quality in the decoded image is reduced and the coding efficiency is improved.

In addition, the code amount control unit 115 determines the quantization parameter Q _p based on the motion compensation data for the reference image signal used in the bi-level prediction of the upper layer, so that the motion characteristics of the displayed image can be obtained. Accordingly, the code amount is controlled. That is, since data relating to motion compensation is used when determining the code amount, the processing amount can be increased without increasing the amount of processing. In addition, the code amount is quantitatively determined so that the code amount falls within a predetermined range by associating the data with motion compensation data.

The effect of such control is consistent with the determination of whether or not to perform block division by RD optimization. In the RD optimization, when the quantization parameter Q _p is large (the code amount is small), encoding of flag information indicating division is omitted so that block division is not performed as much as possible, and the code amount (encoding rate) of encoded data is omitted. ) Reduction is given priority. When the quantization parameter Q _p is small (the code amount is large), priority is given to the reduction of the distortion amount by performing block division and coding the flag information indicating the division.

Next, processing in which the code amount control unit 115 according to the present embodiment determines the code amount will be described.
FIG. 7 is a flowchart illustrating processing in which the code amount control unit 115 according to the present embodiment determines the code amount.
(Step S101) The code amount control unit 115 determines whether or not the frame of interest is a b frame. If it is determined that the current frame is the b frame (YES in step S101), the process proceeds to step S102. If it is determined that the current frame is a type of frame other than the b frame (NO in step S101), the process proceeds to step S106.

(Step S102) The code amount control unit 115 identifies the frame of the reference image signal used for bidirectional prediction in the upper layer. Taking the case where the current frame is the frame b1 shown in FIG. 5 as an example, the frame of the reference image signal used for bidirectional prediction in the lower layer is the frame B1. When the predicted image signal related to the frame B1 is generated in the upper layer, the frames of the reference image signal used for bidirectional prediction are the frame I0 and the frame P2. Among them, motion compensation is performed on the frame P2, which is an inter frame. Therefore, in this example, the frame P2 is specified. Thereafter, the process proceeds to step S103.

(Step S <b> 103) The code amount control unit 115 reads data related to motion compensation for the current block in the identified frame from the storage unit included in the self unit or the lossless encoding unit 105. The read data is, for example, motion vector data and mode data. Thereafter, the process proceeds to step S104.
(Step S104), the code amount control unit 115, as mentioned in (I), (II), defines the quantization parameter _{Q p,} based on the read data. Thereafter, the process proceeds to step S105 (step S105). The code amount control unit 115 determines whether there is an unprocessed block in the current frame. If there is an unprocessed block (YES in step S105), the current block is switched to the next block, and the process proceeds to step S102. If there is no unprocessed block (NO in step S105), the process shown in FIG. 7 is terminated.

(Step S106) The code amount control unit 115 sets the quantization parameter Q _p to a value corresponding to the type of the current frame. For example, when the current frame is an I picture, the code amount control unit 115 determines Q _p as Q _p (I), and when the current frame is a P picture, Q _p is set as Q _p (P). If the current frame is a B picture, Q _p is defined as Q _p (B). Thereafter, the process shown in FIG. 7 ends.

Next, the configuration of the image decoding device 2 according to the present embodiment will be described.
FIG. 8 is a schematic block diagram showing the configuration of the image decoding device 2 according to this embodiment.
The image decoding apparatus 2 includes a storage buffer 201, a lossless decoding unit 202, an inverse quantization unit 203, an inverse orthogonal transform unit 204, an addition unit 205, a deblocking filter 206, a frame buffer 207, a reference memory 208, an intra compensation unit 209, and an inter A compensation unit 210 is included.

The accumulation buffer 201 receives encoded data from the outside of the image decoding device 2, for example, the image encoding device 1, and temporarily stores the input encoded data.
The lossless decoding unit 202 reads the encoded data from the accumulation buffer 201, performs lossless decoding on the read encoded data, and generates integrated data. The lossless decoding method used by the lossless decoding unit 202 is a method corresponding to the lossless encoding method used by the lossless encoding unit 105.

  When the generated integrated data includes mode data and quantized transform coefficient data, the lossless decoding unit 202 extracts mode data and quantized transform coefficient data from the integrated data.

  For example, when the mode data indicates a motion compensation mode without data division, the lossless decoding unit 202 extracts quantized transform coefficient data, a reference frame number for each PU, and motion vector data from the integrated data. When the mode data indicates a motion compensation mode with data division, the lossless decoding unit 202 extracts quantized transform coefficient data, a reference frame number for each PU, and motion vector data from the integrated data. When the mode data indicates the intra prediction mode, the lossless decoding unit 202 extracts the quantized transform coefficient data and the intra prediction mode from the integrated data. When the mode data indicates the skip mode, the integrated data does not include quantized transform coefficient data, reference frame numbers, and motion vector data, so that data other than the mode data is not extracted.

The lossless decoding unit 202 outputs the extracted quantized transform coefficient data to the inverse quantization unit 203. The lossless decoding unit 202 outputs the extracted mode data to the intra compensation unit 209 or the inter compensation unit 210. The lossless decoding unit 202 outputs the extracted reference frame number and motion vector data to the inter compensation unit 210.
Note that the lossless decoding unit 202 extracts a quantization parameter from the restored integrated data, and outputs the extracted quantization parameter to the inverse quantization unit 203. If the integrated data includes a differential quantization parameter, add the quantization parameter already calculated for the neighboring block and the differential quantization parameter for the current block to calculate the quantization parameter for the current block. Also good. In that case, the lossless decoding unit 202 outputs the calculated quantization parameter related to the current block to the inverse quantization unit 203.

  When the extracted motion vector data indicates a difference vector, the lossless decoding unit 202 further extracts a vector index from the integrated data. In that case, the lossless decoding unit 202 includes a decoded data storage unit that stores a motion vector for each block that has already been decoded, and reads the motion vector indicated by the extracted vector index from the decoded data storage unit as a prediction vector. The lossless decoding unit 202 adds the read prediction vector and the difference vector, and outputs motion vector data indicating the motion vector to the inter compensation unit 210.

  The inverse quantization unit 203 inversely quantizes the quantized transform coefficient indicated by the quantized transform coefficient data input from the lossless decoding unit 202 to generate restored transform coefficient data indicating the restored transform coefficient. Here, the inverse quantization unit 203 calculates a restored transform coefficient by multiplying the quantized transform coefficient by a value determined by the quantization parameter input from the lossless decoding unit 202, and a restored transform indicating the calculated restored transform coefficient Generate coefficient data. The inverse quantization unit 203 outputs the generated restored transform coefficient data to the inverse orthogonal transform unit 204.

  The inverse orthogonal transform unit 204 performs inverse orthogonal transform on the restored transform coefficient indicated by the restored transform coefficient data input from the inverse quantization unit 203 to generate a restored difference image block signal. The inverse orthogonal transform used by the inverse orthogonal transform unit 204 is, for example, an inverse operation of orthogonal transform performed by the orthogonal transform unit 103 of the image encoding device (FIG. 1). The inverse orthogonal transform unit 204 outputs the generated restored difference image block signal to the addition unit 205.

  The adding unit 205 adds the restored difference image block signal input from the inverse orthogonal transform unit 204 and the compensated image block signal input from the intra compensating unit 209 or the inter compensating unit 210 to generate a restored image block signal. The adding unit 205 outputs the generated restored image block signal to the deblocking filter 206. The adding unit 205 adds the signal value of the corresponding pixel in the predicted image block signal from the signal value for each pixel in the restored difference image block signal.

  The deblocking filter 206 performs filtering processing on the restored image block signal input from the adding unit 205 to remove a block distortion component. The deblocking filter 206 stores the restored image block signal from which the block distortion component is removed in the frame buffer 207, and stores the restored image block signal in the reference memory 208 as a reference image block signal.

The frame buffer 207 stores the restored image block signal input from the deblocking filter 206 in a storage area corresponding to the display area corresponding to the restored image block for each frame. Thereby, the restored image signal for each frame is stored. The frame buffer 207 may rearrange the restored image signals in the order of frames according to a predetermined GOP structure. The frame buffer 207 may output the restored image signal for each frame to the outside of the image decoding device 2, for example, an image display device (display).
The reference memory 208 stores the reference image block signal input from the deblocking filter 206 in a storage area corresponding to the display area corresponding to the reference image block for each frame. Thereby, the reference image signal indicating the reference image is stored for each frame.

  When the current frame is an intra frame (I picture), the intra compensation unit 209 performs compensation processing in a compensation mode corresponding to the intra prediction mode indicated by the mode data input from the lossless decoding unit 202, and relates to the current block. A prediction image block signal is generated. Here, the intra compensation unit 209 performs intra compensation using the reference image signal related to the current frame among the reference image signals stored in the reference memory 208, and generates a compensated image block signal related to the current block. The intra compensation unit 209 outputs the generated compensated image block signal to the addition unit 205.

  When the current frame is an inter frame (P picture, B picture), the inter compensation unit 210 performs a compensation process in a compensation mode corresponding to the prediction mode indicated by the mode data input from the lossless decoding unit 202 to obtain the current block. Such a compensated image block signal is generated. Here, the inter compensation unit 210 uses the reference image signal stored in the reference memory 208. The inter compensation unit 210 generates the generated compensated image block signal.

For example, when the mode data indicates the intra prediction mode, the inter compensation unit 210 performs intra compensation to generate a compensated image block signal.
When the mode data indicates a motion-compensated prediction mode without division, the inter-compensation unit 210 receives a reference frame number and motion vector data for a PU that is a batch block in which the entire current block is batched. The inter compensation unit 210 reads, from the reference memory 208, the reference image block signal of the area specified by the motion vector indicated by the motion vector data from the reference image signal specified by the input reference frame number. The inter compensation unit 210 generates a compensated image block signal using the read reference image block signal.

When the mode data indicates a motion compensated prediction mode with division, the inter compensation unit 210 receives a reference frame number and motion vector data for each PU that is subdivided in the current block. The inter compensation unit 210 reads, from the reference memory 208, the reference image block signal in the region specified by the motion vector indicated by the motion vector data from the reference image signal indicated by the input reference frame number. The inter compensation unit 210 generates a compensation image block signal for each PU using the read reference image block signal, and integrates the generated compensation image block signal between PUs (subblocks in this example) to the current block ( Compensation image block related to (collective block) is generated.
When the mode data indicates the skip mode, the inter compensation unit 210 calculates a representative value (for example, a median value) of a motion vector related to a processed block that is a block adjacent to the current block. The inter compensation unit 210 uses, as a predicted image block signal, a reference image block signal stored in a position compensated with a deviation amount indicated by the calculated representative value from the current block in the reference image of the frame corresponding to the current frame. Read from.

  As described above, the present embodiment includes a storage unit (for example, the reference memory 111) that stores a reference image obtained by decoding encoded data of an encoded input image for each frame. Further, in the present embodiment, a predicted image corresponding to an input image block that is a part of the input image by performing intra-frame prediction, forward prediction, or bidirectional prediction using the reference image stored in the storage unit. A prediction unit (for example, an inter prediction unit 113) that generates a block is provided. In the present embodiment, when the prediction unit further performs bi-directional prediction using the decoded image of the predicted image block generated by performing bi-directional prediction as a reference image, the code amount related to the predicted image block to be predicted is A code amount control unit (for example, a code amount control unit 115) is provided that is determined based on data related to motion compensation prediction and motion compensation of a decoded reference image that has already been performed in bidirectional prediction.

  Thereby, the code amount generated by the bidirectional prediction is determined according to the motion of the image in the past bidirectional prediction. For example, a larger code amount is assigned to an area where image quality is likely to deteriorate, and a smaller code amount is assigned to an area where image quality is less deteriorated. As a result, the degradation of the image quality in the decoded image is reduced and the coding efficiency is improved. In addition, since the data related to motion compensation is used when determining the code amount, the processing amount does not become excessive, and the processing speed can be increased. In addition, the code amount is quantitatively determined so as to be within a predetermined range by associating with data related to motion compensation.

In the above-described embodiment, the case where the layer B structure is a GOP structure of two layers of an upper layer and a lower layer has been described as an example. However, the present invention is not limited to this. In the embodiment described above, the layer B structure may have more than two layers (for example, three layers). In that case, when the type of the current frame is a b picture, a reference image signal of a frame related to a B picture that has been bi-directionally predicted in a higher layer is used in a layer lower than the second layer. Thus, a predicted image block is generated. In this case, the code amount control unit 115 generates the reference image signal in the current block when generating the quantization parameter Q _p related to the current block by bi-predicting the predicted image block signal related to the B picture in the upper layer. May be determined based on the data related to motion compensation performed for. Therefore, the code amount control unit 115 may predetermine a larger quantization parameter (smaller code amount) as an initial value for a b picture for which bi-directional prediction is performed in a lower layer. In the above-described embodiment, if the lowermost b picture is determined in advance as a non-reference picture, a higher-level b picture may not be treated as a non-reference picture.

  Note that the above-described embodiment may be implemented as an image processing system including the image encoding device 1 and the image decoding device 2. The image processing system may include a storage medium that stores encoded data output from the image encoding apparatus 1, and the encoded data read from the storage medium may be input to the image decoding apparatus 2. In addition, the image processing system may include a network that transmits encoded data output from the image encoding device 1, and the image decoding device 2 may be input with encoded data transmitted by the network. The network may be wired or wireless, or may be part or all of a broadcast transmission path that transmits data to a plurality of transmission destinations all at once.

Note that a part of the image encoding device 1 or the image decoding device 2 in the above-described embodiment, for example, the subtraction unit 102, the orthogonal transformation unit 103, the quantization unit 104, the lossless encoding unit 105, the inverse quantization unit 107, the inverse Orthogonal transformation unit 108, addition unit 109, deblocking filter 110, intra prediction unit 112, inter prediction unit 113, mode determination unit 114 and code amount control unit 115, lossless decoding unit 202, inverse quantization unit 203, inverse orthogonal transformation unit 204, the addition unit 205, the deblocking filter 206, the intra compensation unit 209, and the inter compensation unit 210 may be realized by a computer. In that case, the program for realizing the control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by the computer system and executed. Here, the “computer system” is a computer system built in the image encoding device 1 or the image decoding device 2 and includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” is a medium that dynamically holds a program for a short time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, In this case, a volatile memory inside a computer system that serves as a server or a client may be included that holds a program for a certain period of time. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.
Moreover, you may implement | achieve part or all of the image coding apparatus 1 and the image decoding apparatus 2 in embodiment mentioned above as integrated circuits, such as LSI (Large Scale Integration). Each functional block of the image encoding device 1 and the image decoding device 2 may be individually made into a processor, or a part or all of them may be integrated into a processor. Further, the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. In addition, when an integrated circuit technology that replaces LSI appears due to the advancement of semiconductor technology, an integrated circuit based on the technology may be used.

  As described above, the embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the scope of the present invention. It is possible to

DESCRIPTION OF SYMBOLS 1 ... Image coding apparatus, 101 ... Frame buffer, 102 ... Subtraction part, 103 ... Orthogonal transformation part, 104 ... Quantization part, 105 ... Lossless encoding part, 106 ... Accumulation buffer, 107 ... Dequantization part, 108 ... Inverse orthogonal transform unit, 109 ... addition unit, 110 ... deblocking filter,
111 ... Reference memory, 112 ... Intra prediction unit, 113 ... Inter prediction unit,
114... Mode determination unit, 115.
DESCRIPTION OF SYMBOLS 2 ... Image decoding apparatus, 201 ... Accumulation buffer, 202 ... Lossless decoding part, 203 ... Dequantization part, 204 ... Inverse orthogonal transformation part, 205 ... Adder, 206 ... Deblocking filter,
207 ... Frame buffer, 208 ... Reference memory, 209 ... Intra compensation unit,
210: Inter compensation unit

Claims (4)

A storage unit that stores, for each frame, a reference image obtained by decoding encoded data of an encoded input image;
A prediction unit that performs intra-frame prediction, forward prediction, or bidirectional prediction using the reference image stored in the storage unit, and generates a predicted image block corresponding to an input image block that is a part of the input image; ,
When the prediction unit further performs bi-directional prediction using a decoded image of a predicted image block generated by bi-directional prediction as a reference image, the code amount related to the predicted image block is motion compensated by the bi-directional prediction already performed. A code amount control unit determined based on data relating to motion compensation of the predicted and decoded reference image;
An image encoding device comprising:   The image coding apparatus according to claim 1, wherein the code amount control unit uses a motion vector as data related to the motion compensation.   The image coding apparatus according to claim 1, wherein the code amount control unit uses a motion prediction mode indicating a mode of motion compensation as data related to the motion compensation. Computer
Perform intra-frame prediction, forward prediction or bi-directional prediction using a reference image stored in a storage unit that stores, for each frame, a reference image obtained by decoding encoded data of an encoded input image. Prediction means for generating a predicted image block corresponding to an input image block that is a part;
When bi-prediction is further performed using a decoded image of a predicted image block generated by performing bi-prediction in the prediction unit as a reference image, the amount of code related to the predicted image block is motion compensated by bi-prediction already performed. Code amount control means determined based on data related to motion compensation of the predicted and decoded reference image;
A program for causing an image encoding apparatus to function as an image encoding apparatus.

Images