JP4720284B2 - Image compression encoding device - Google Patents

Description

  The present invention relates to an image compression encoding apparatus. In particular, the present invention relates to an encoding apparatus based on a motion prediction DCT (Discrete Cosine Transform) encoding method.

  Conventionally, H.264 has been used as an encoding method for multimedia applications. H.263 or MPEG-4. These encoding methods did not require strict buffer constraints from the application point of view. This is because it is mainly used for applications that are not affected by transmission characteristics such as download distribution or CD-ROM storage, or for applications that do not focus on quality assurance even for Internet streaming that is affected by transmission characteristics. is there. For example, for applications that require low delay, such as videophones, there is no practical problem even if buffer constraints are not strictly defined by allowing fluctuations in the playback interval on the receiving side.

  On the other hand, H.264, which targets a low bit rate (several hundred kbit / s or less) and has high coding efficiency of moving images. There are H.264 encoding schemes. This encoding method is employed in broadcasting for terrestrial digital broadcasts (generally referred to as “one-segment broadcasting”).

H. H.264 has the following features from the viewpoint of encoding control.
(1) Improvement of compression efficiency by improvement of prediction method in intraframe coding mode (2) Improvement of prediction efficiency by adaptive selection of block size (3) Adopting RD optimization or a mode selection method according to it RD optimization is This refers to selecting a macroblock mode that minimizes both the amount of information (Rate) and distortion (Distortion), and performs optimal selection (Optimization) while actually performing provisional encoding.

  Since this encoding method is adopted for “broadcasting”, high quality is required for the encoder, and immediacy and uniqueness of delay are required. Therefore, the encoder needs to strictly comply with the requirements for bit rate and delay.

  H. As a rate control method based on H.264, there is an HRD (Hypothetical Reference Decoder) buffer restriction method based on an information amount allocation method in MPEG-2TM5 (see, for example, Non-Patent Document 1). The HRD buffer corresponds to an MPEG-2 VBV (Video Buffer Verifier).

  As a development of Non-Patent Document 1, there is a system used in a reference encoder (JM: Joint Model, reference software) (see Non-Patent Document 2, for example). JM is used for performance verification or validity confirmation of an encoding method.

  These prior arts presuppose a constant rate in units of GOP (Group Of Pictures), and assign an amount of information to each frame by weighting according to the picture type. Further, rate control in the prior art is performed by a combination of macroblock QP (Quantization Parameter) settings based on a rate-distortion quadratic equation model (see, for example, Non-Patent Document 3). These systems do not take into account the image properties on a frame basis. In addition, the local distortion amount at the time of determining the intra-frame QP is diverted from that of the macro block at the same position in the previous frame, and the reliability is low for an image with a large motion.

  As another method, there is a method of performing quantization control by paying attention to the relationship between the ratio of the zero coefficient and the amount of generated information when determining the QP in the frame (for example, see Non-Patent Document 4). Furthermore, there is also a method of selecting a desired rate by changing the rate distortion coupling coefficient λ in various ways and performing RD optimization (see, for example, Non-Patent Document 5).

  FIG. 1 is a functional configuration diagram of an image compression coding apparatus in the prior art.

  The image compression encoding apparatus 1 receives screen data (in units of frames) and outputs the encoded data. The current frame memory 101 stores one input frame. The motion prediction unit 102 predicts the motion of each macroblock in the current frame with respect to the frame in the local decode memory 107 (previous local decode frame) by a block matching method or the like, and outputs a motion vector. The motion compensation unit 111 applies the previously calculated motion vector for each macroblock to the previous local decoded frame between frames flowing in the time direction, and outputs a prediction screen in consideration of the motion of the subject. The DCT unit 112 further subdivides the macroblock of the current frame in the intra mode into four for the difference between the macroblock of the current frame and the macroblock of the prediction screen subjected to motion compensation in the inter mode. Orthogonal transform is performed for each divided block, and DCT coefficients indicating frequency components are output. The quantization unit 113 digitizes the orthogonally transformed DCT coefficients based on a predetermined quantization parameter. The variable length coding unit 116 converts motion vectors, mode information, and information generated by quantization into codes that are actually transmitted. At this time, based on statistical properties such as the frequency of occurrence of such information, codewords are assigned so that the code length is shortened, and encoding is performed. The buffer unit 117 temporarily buffers the compressed encoded data for transmission.

  The rate control of the quantization unit according to the prior art is to control the quantization parameter of the quantization unit 113 according to the amount of data stored in the buffer unit.

  The inverse quantization unit 114 converts the quantized DCT coefficient data into an original value. The inverse DCT unit 115 restores normal pixel information from the DCT coefficient. The local decoding memory 107 adds the restored pixel information in the intra mode to the motion compensated image information in the inter mode, and temporarily stores the same frame as the frame reproduced on the receiving side. . The inter-frame prediction error calculation unit 103 and the intra-frame prediction error calculation unit 104 optimize the block size at the time of prediction for each of the inter (inter-frame) / intra (intra-frame) mode, and output a prediction error. The mode determination unit 105 selects either the inter mode or the intra mode so that the magnitude of the prediction error is reduced. The mode information memory 106 temporarily stores the selected mode, and switches the switch in that mode.

S.Ma, W.Gao, F.Wu and YanLu, `` Rate control for JVT video coding scheme with HRD considerations '', IEEE ICIP2003, Vol.III, pp.793-796, Sep. 2003. Z.Li, W.Gao and F.Pan, `` Adaptive rate control with HRD consideration '', Joint Video Team of ISO / IEC MPEG and ITU-T VCEG, JVT-H014, May 2003. T.Chiang and Y.Zhang, “A new rate controlscheme using quadratic rate distortion model”, IEEE Trans. Circuits Syst. Video Technol., Vol. 7, pp. 287-331, Apr. 1997. S. Milani, L. Celetto and G.A. Mian, `` A rate control algorithm for the H.264 encoder '', 6th Baiona Workshop on Signal Processing in Communications, pp. 390-396, Sep. 2003. M.M.Mahdi and M.Ghanbar, `` A lagrangian optimized rate control algorithm for the H.264 / AVC encoder '', IEEE ICIP2004, Vol.I, pp.123-126, 2004. G. Sullivan, T. Wiegand, K.P. Lim, `` Joint Model Reference Encoding Methods and Decoding Concealment Methods '', Joint Video Team of ISO / IEC MPEG and ITU-T VCEG, JVT-I049, Sep. 2003. ITU-T Recommendation H.264 + Cor1, `` Advanced video coding for generic audiovisual services '', May 2004.

  In digital terrestrial broadcasting, one channel is divided into 13 segments. Depending on the required bandwidth, 12 segments are used when broadcasting HDTV, or 4 segments x 3 programs are used for standard television, and the remaining 1 segment. Is assigned to mobile broadcasting. That is, a fixed band is allocated.

  All the prior arts aim to allocate a certain amount of information in GOP units with respect to bit rate constraints. The difference between the already-encoded frame generation information amount and the allocated amount is controlled so as to be absorbed by encoding the remaining frames in the GOP. Furthermore, after GOP encoding, the difference between the generated information amount of the GOP and the allocated amount is reflected in the target information amount of the next GOP.

  In such a rate control method, since the amount of generated information is controlled to be uniform in all GOPs, the HRD buffer size is implied to be about the amount of information of a frame in which the amount of generation is maximum in the GOP. Is determined. However, since the control is not clearly conscious of the HRD buffer size and initial delay, a buffer constraint specialized for transmission purposes (video phone use: bidirectional ultra-low delay or live broadcast use: one-way low delay, etc.) is set. Therefore, encoding control is difficult.

  In addition, since image properties are not considered in the allocation of the information amount of frames, distortion increases in a scene with a large amount of generated information, and quantization tends to be performed more finely than necessary in a scene with a small amount. This not only causes a decrease in inter-frame prediction efficiency, but also quality fluctuations cause visual discomfort.

  In this case, not only the image quality deteriorates, but also the video disturbance due to the buffer failure on the receiving side and the motion blur (judder) due to the frame not being displayed at the specified time occur. Must be strictly avoided.

  In addition, in order to consider the image property, if encoding is performed with the generated information amount based on the image property as a target value as it is, encoding that reflects the property of each frame can be performed, but the HRD buffer The situation is not considered. In other words, allocating the amount of information requested by each frame as it is is equivalent to performing encoding with the QP value fixed. This is not a problem when the HRD buffer is infinitely large, but under the buffer constraint, the set buffer condition must not be broken.

  As described above, in any of the above-described prior arts, there is no proposal for encoding control in which the buffer size or delay amount is clearly defined and in consideration of the image characteristics of each frame. Not.

  Therefore, the present invention considers application to terrestrial digital one-segment broadcasting that requires high quality and low delay, and performs image compression that achieves both stabilization by considering buffer constraints and improvement of image quality by reflecting image properties. An object is to provide an encoding device.

An image compression encoding apparatus according to the present invention includes:
A quantization means for performing quantization based on the quantization parameter;
From the time immediately after the first frame encoding of the first GOP, the buffer occupation transition of the virtual reception buffer up to the time immediately before the encoding of the first frame of the second GOP is set to the center of the overflow limit and the underflow limit. A buffer restricting means for outputting a frame target generation information amount corrected for the basic generation information amount required by the frame;
And a quantization parameter control unit that derives a quantization parameter from the macroblock target generation information amount calculated based on the frame target generation information amount and outputs the quantization parameter to the quantization unit.

According to another embodiment of the image compression encoding apparatus of the present invention,
A difference absolute value sum calculating means for calculating a difference absolute value sum SAD which is a prediction error after the inter / intra mode determination for each macroblock;
Buffer constraint means
Frame basic generation information amount estimation means for estimating the frame basic generation information amount based on the difference absolute value sum SAD;
Bits so that the buffer occupancy transition of the virtual reception buffer up to the time point immediately before the first frame encoding of the second GOP is the center of the overflow limit and the underflow limit as viewed immediately after the first frame encoding of the first GOP Information amount correcting means for outputting a scaling coefficient that is a ratio of the generated information amount of the frame to the average generated information amount per frame obtained from the rate;
It is also preferable to have multiplying means for multiplying the estimated basic generation information amount by a scaling coefficient and outputting the frame target generation information amount.

According to another embodiment of the image compression encoding apparatus of the present invention,
The information amount correction means is performed in units of GOP, and when the occupancy amount of the encoded data in the buffer is transitioning low, outputs a small scaling factor, and the occupancy amount of the encoded data transitions high. If so, it is also preferable to operate to output a large value of the scaling factor.

Furthermore, according to another embodiment of the image compression encoding apparatus of the present invention,
The frame basic generated information amount estimation means preferably stores in advance a function f or an estimated value for estimating the generated information amount based on the sum of absolute differences SAD and the fixed quantization parameter QP.

Furthermore, according to another embodiment of the image compression encoding apparatus of the present invention,
The buffer restricting means preferably further includes clip means for clipping the frame target generated information amount output from the multiplying means so as not to reach the buffer overflow limit or under limit.

Furthermore, according to another embodiment of the image compression encoding apparatus of the present invention,
The quantization parameter control means is
Macroblock target generation information amount determining means for outputting a macroblock target generation information amount based on the frame target generation information amount;
From the sum of the information amount immediately before the macroblock in the actual generated information amount, the sum of the information amount immediately before the macroblock in the target generated information amount, and the average value of the quantization parameter immediately before the macroblock, It is also preferable to have quantization parameter calculation means for calculating the quantization parameter of the block and outputting it to the quantization means.

  According to the image compression coding apparatus of the present invention, when applied to terrestrial digital one-segment broadcasting requiring high quality and low delay, stabilization by considering buffer constraints and improvement of image quality by reflecting image properties are achieved. Both can be achieved. In particular, according to the rate control in the prior art, a fixed amount is assigned to each GOP, and the buffer size and the image quality cannot be strictly set according to the application.

  According to the present invention, in consideration of buffer constraints, the concept of a scaling coefficient obtained from the buffer transition state of a GOP is newly introduced, and an information amount to be allocated is obtained by applying to the predicted generated information amount. In addition, regarding reflection of image properties, a prediction error for each frame is calculated in advance, and an optimal amount of generated information is predicted based thereon. The amount of information allocated in this way is a relatively reflecting property of each frame under a certain buffer constraint.

  Hereinafter, the best mode for carrying out the present invention will be described in detail with reference to the drawings.

  FIG. 2 is a functional configuration diagram of the image compression coding apparatus according to the present invention.

Compared with FIG. 1, the image compression encoding apparatus 1 includes a frame memory 121, a SAD (Sum of Absolute Difference) calculation unit 122, a buffer restriction unit 123, and a quantization And a parameter control unit 124. The frame memory 121 inputs an inter-mode or intra-mode prediction error. The SAD calculation unit 122 receives the prediction error, and outputs S _j that is SAD in units of macroblocks, an intra-frame total value ΣS _{j of} S _j , and an average value meanSAD in the number of macroblocks. Hereinafter, i is a frame number and j is an intra-frame macroblock number.

The buffer constraint unit 123 is intended to comply with HRD (Hypothetical Reference Decoder) buffer constraints. Buffer constraint unit 123 inputs the meanSAD output from SAD calculation unit 122 outputs the frame target generated information amount _{T i.} The HRD buffer has a fixed bit rate.

The quantization parameter control unit 124 is an H.264 driver. Based on the H.264 coding characteristics, an information amount allocation for achieving the target generated information amount is aimed. The quantization parameter control unit 124 inputs S _j that is SAD in units of macroblocks, an intra-frame total value ΣS _{j of} S _j , and a frame target generation information amount T _i, and sets a QP (quantization parameter) value. Output. The quantization unit 113 is controlled based on the QP value.

  Below, each function part is demonstrated in detail.

[SAD calculation unit]
The SAD calculation unit 122 inputs a prediction error for each macroblock from the frame memory. Therefore, inter prediction errors are used for inter mode macroblocks, and intra prediction errors are used for intra mode macroblocks. Then, the SAD calculation unit 122 calculates the SAD of the prediction error after the mode determination for each macroblock of the frame.

The present invention pays attention to the correlation between the magnitude of prediction error obtained after inter prediction or intra prediction and the actual amount of generated information, and determines basic information amount allocation at the frame level. The difference absolute value sum SAD is used for evaluating the magnitude of the prediction error. The SAD calculation unit 122 outputs S _j that is SAD in units of macroblocks, an intra-frame total value ΣS _{j of} S _j , and an average value meanSAD in the number of macroblocks.

  H. According to H.264, in order to effectively perform mode determination in intra prediction, in the conventional technique, mode determination is performed while performing coding for each macroblock using adjacent macroblocks that have already been encoded. On the other hand, the SAD calculation unit 122 according to the present invention uses the original image as a reference image for intra mode determination because it is SAD calculation for a prediction error before encoding. At the time of encoding, only intra mode determination is performed again using the local decoded image. At the time of encoding, the motion vector and inter-mode prediction error values used at the time of SAD calculation are used to avoid unnecessary duplication processing.

For each macroblock m for which mode determination has been performed, S _m that is SAD is calculated by the following equation (1), where e _m (p, q) is a prediction error in each pixel.
Formula (1)

In addition, the SAD intraframe average value meanSAD is calculated by the following equation (2). Note that x and y are image sizes.
Formula (2)

[Buffer constraints]
The buffer restriction unit 123 includes a frame basic generation information amount estimation unit 1231, an information amount correction unit 1232, a memory 1233, a multiplication unit 1234, and a clip unit 1235. Frame basic information amount estimation unit 1231 receives the SAD average MEANSAD, and outputs the frame fundamental generated information quantity _{T B} based on the QP. The information amount correction unit 1232 outputs the ratio of the actual generated information amount to the generated information amount per frame obtained from the bit rate as the scaling coefficient C _Si . The multiplier 1234 multiplies the scaling coefficient C _Si by the frame basic generation information amount T _B and outputs a frame target generation information amount T _i . Clip portion 1235, the frame target generated information amount T _i is, in effect, a clip so as not to reach the overflow limit or under limit of the buffer. Below, each function part is demonstrated in detail.

[Basic frame generation information estimation unit]
Frame basic generation information amount estimation section 1231 receives SAD average value meanSAD in the number of macroblocks. and MEANSAD, from each QP value, the function f, and calculates the frame base generated information quantity _{T B.} The basic frame generated information amount is a generated information amount that each frame needs purely before considering the buffer constraint.
Formula (3)
T _B = f (meanSAD, QP)

  FIG. 3 is a graph showing the relationship between SAD and the amount of generated information. The graph of FIG. 3 is obtained by a preliminary experiment.

According to the graph of FIG. 3, the horizontal axis represents the average value per macroblock in the SAD intra-frame total value S _j , and the vertical axis represents the amount of generated information. Plots at each QP value exist on substantially the same curve. It can be understood that this correlation is a result obtained by prior measurement and does not depend on the QP value. This means that the amount of generated information required by the frame can be predicted by the frame SAD.

With respect to these relationships, when the least square approximation is performed on each QP using the following linear polynomial for S, each coefficient is as shown in Table 1. Bits is a frame generation information amount, and S is an average value per macroblock of the frame SAD.
bits = a · S ³ + b · S ² + c · S + d

Here, the range of the target QP is 5 to 50. This is because the conventional encoding method is linear with respect to the QP with respect to the definition of the quantization step size, and the maximum value is 31. H. This is because H.264 defines the linear relationship between QP and the amount of distortion by making it proportional to 2 to the power of (QP / 6), and the maximum value of QP is defined as 51. As a result of the approximation, the third-order or higher coefficient becomes a sufficiently small value, so the following quadratic expression is used.
bits = b · S ² + c · S + d

  FIG. 4 is a graph approximating the results of the graph of FIG.

  The function f in Equation (3) means the correlation between the SAD obtained by FIG. 4 and the amount of generated information. As a result, the amount of information generated in a frame can be predicted after calculating the prediction error. Since the prediction error for each macroblock has already been obtained, the amount of information generated for each macroblock in the frame can be predicted at the same time. Therefore, this prediction error information can also be used when determining each QP value.

[Information correction unit]
The information amount correction unit 1232 outputs the ratio of the actual generated information amount to the generated information amount per frame obtained from the bit rate as the scaling coefficient C _Si .

  FIG. 5 is a transition graph of the HRD buffer in the present invention.

  According to the graph of FIG. 5, the vertical axis represents the buffer occupation amount, and the horizontal axis represents the elapsed time. The current time is after encoding the IDR (Instantaneous Decoding Refresh) frame at the head of the GOP. The IDR frame corresponds to a conventional I frame. In general, when an IDR frame having a large amount of information is encoded, the buffer level of the HRD buffer is greatly lowered, and then the buffer level is restored by a non-IDR frame (P frame or B frame).

  An IDR frame is an H.264 frame that can change the distance between reference frames. H.264 means that there is no reference relationship across IDR frames. In digital broadcasting, it is necessary to start reception decoding from data during transmission, and periodic IDR frame insertion is required.

  Here, a condition that the transition of the HRD buffer should satisfy before completing the encoding of one GOP is considered.

  From the viewpoint of preventing underflow, not only does not cause underflow immediately after encoding each IDR frame, but also the amount of information that may be generated by the IDR frame immediately before encoding the first frame of the next GOP. It is necessary to secure.

  On the other hand, from the viewpoint of preventing overflow, it is only necessary that the buffer level immediately before encoding each frame does not exceed the maximum buffer size. However, in order to efficiently use all of the generated information amount for image encoding without generating unnecessary stuffing bits, it is also necessary to prevent any overflow from occurring until immediately before the next GOP encoding. It becomes.

  This is due to the following reason. In general, in a sequence with a small amount of information, since the amount of information generated in each P frame is smaller than the average, buffer transitions in the P frame section monotonically increase. At this time, since the amount of information generated in the frame in the middle of the GOP is small, if it is determined that an overflow occurs immediately before encoding the next frame, stuffing is performed to avoid this, but the next frame is also It is likely to have similar image properties. In this case, stuffing is required continuously for subsequent frames. This can be avoided by performing a special correction such as sufficiently reducing the quantization parameter used for frame encoding immediately after stuffing has occurred. However, since this is contrary to the intended uniformity of image properties, it is necessary to set it on condition that no overflow occurs until the end of the GOP.

Considering the above points, the underflow limit and overflow limit in the middle of the GOP are defined by B _O (i) and B _U (i) in FIG.

  According to the graph of FIG. 5, it is ideal that the buffer transition occurs in a triangle (points A, C and D are vertices) surrounded by both limits and the GOP boundary line. For this purpose, an information amount is assigned based on the following procedure.

  Immediately after encoding the GOP head frame (IDR), the slope of the buffer transition for each GOP is determined. This corresponds to determining the target generation information amount for the GOP. The maximum information amount of the IDR frame is a value obtained by subtracting a margin (10% of the buffer size) from the current buffer level. The amount of generated information predicted from each frame SAD is determined by taking the inclination into account as the amount of generated frame target information. Based on the SAD for each macroblock, the amount of information in the frame is allocated.

  The buffer state is reflected in the “tilt”, that is, the target generated information amount for each GOP (however, the increase / decrease direction is reversed). That is, when the current buffer state is close to underflow and it is necessary to suppress the amount of generated information, the value is large. On the other hand, if the buffer state is close to overflow, the reverse is true. According to the present invention, by applying the “slope” that should be taken by buffer transition to the generated information amount predicted in advance from the SAD, the target value of the generated information amount at the time of encoding is obtained. It is possible to achieve both reflection and buffer constraints.

  Conventionally, in order to avoid reaching the underflow limit, frame skip (dropping) according to the buffer state may be performed. However, this does not control the amount of information of the frame itself. This means that the HRD buffer position at the time of encoding the next frame is set to a higher position. Therefore, although a preventive effect against reaching the underflow limit can be expected, it is not an essential avoidance. In addition, the jerky feeling due to frame skip may worsen the subjective impression. Therefore, according to the present invention, frame skipping is not performed.

According to the graph of FIG. 5, the following equation (4) is established for the buffer state immediately before the i-th frame encoding in the GOP.
Formula (4)
Bh _i = Bh _i-1 -b _i-1 + BR / FR
Bh _i (bit): HRD buffer level immediately before frame i encoding b _i (bit): amount of information generated by encoding frame i BR (bit / s): encoding bit rate FR (frame / s): encoding frame rate

  Next, the underflow limit and overflow limit are defined as follows.

ｂｅ(bit)：次ＧＯＰ先頭のフレームで許容する発生情報量
Ｎ(frame)：ＧＯＰ内フレーム数 The underflow limit B _U (i) is expressed as the following equation (5) as a straight line connecting the buffer position after encoding the IDR frame at the head of the GOP and the buffer position reserved for the head IDR frame of the next GOP. Defined in
Formula (5)

be (bit): The amount of generated information allowed in the first frame of the next GOP N (frame): Number of frames in the GOP

Ｂ：バッファサイズ The overflow limit B _O (i) is expressed as the following equation (6) as a straight line connecting the buffer position before the second frame encoding from the GOP head and the buffer maximum value before the next GOP head IDR frame encoding. Define as follows.
Formula (6)

B: Buffer size

As shown in FIG. 5, the actual buffer transition is preferably performed in the middle of the rectangular area surrounded by both limits from the viewpoint of preventing underflow / overflow. Two points A (0, Bh ₀ -b ₀ ) and B (1, Bh ₀ -b ₀ + BR / FR) midpoint (1/2, Bh ₀ -b ₀ + BR / 2FR) on the left side of the square and the right side of the square A straight line connecting the midpoints (N, (B + be) / 2) of two points C (N, be) and D (N, B) is defined as B _I (i) as shown in the following formula (7). To do.
Formula (7)

From the viewpoint of preventing buffer failure, it is ideal that the HRD buffer transition is close to the straight line B _I (i). That is, it means that the straight line B _I (i) passes through the middle between the buffer position immediately after encoding frame i and the buffer position immediately before encoding the next frame. At this time, the following equation (8) is established. It is necessary to control the generated information amount b _i of the frame i so as to satisfy the equation (8).
Formula (8)
Bh _i −b _i + BR / 2FR = B _I (i)

Next, the ratio of b _{i to} the generated information amount b _f (= BR / FR) per frame obtained from the bit rate is defined as a scaling coefficient C _Si as shown in the following formula (9).
Formula (9)
C _Si = b _i / b _f

From the equations (8) and (9), the following equation (10) is established.
Formula (10)
C _Si = Bh _i −B _I (i) / (BR / FR) +1/2

The scaling coefficient C _Si calculated in this way is output from the information correction unit 1232.

The memory 1233 receives the scaling coefficient C _Si from the information amount correction unit 1232 and temporarily accumulates it.

[Multiplier]
The multiplier 1234 multiplies the scaling coefficient C _Si by the basic generation information amount T _B and outputs a frame target generation information amount T _i . In order to reflect the amount of generated information required for each frame and to prevent underflow / overflow of the HRD buffer, the scaling coefficient C _Si is applied to the basic generated information amount calculated previously. That is, the target generation information amount T _i is calculated by the following equation (11).
Formula (11)
T _i = T _Bi · C _Si

Thereby, the target generation information amount T _i is controlled as follows. When the HRD buffer is transitioned at a low position, a small scaling coefficient is applied to the basic generated information amount to suppress the generated information amount and avoid buffer underflow. When the HRD buffer is transitioning at a high position, a large value scaling factor is applied to the basic generated information amount to increase the generated information amount and avoid buffer overflow.

Since the scaling coefficient corrects the amount of basic generated information, it is not desirable to update the coefficient value frequently, and it is desirable to take a constant value within a certain range. This is due to the following reason. According to equation (11), the high update frequency of C _Si, and thus to ignore the image property of each frame that have been potentially reflected in T _B. As a result, the correlation between the frame target generated information amount _Ti and the image property is reduced. For this reason, the scaling coefficient is updated for each GOP and applied for each frame. The update is performed after the GOP head IDR frame encoding and immediately before the next P frame encoding, which corresponds to the case where i = 1 in equation (10). In this case, the scaling coefficient indicates the ratio of the information amount allocated to the GOP to the average bit rate.

[Clip part]
Clip portion 1235 is in fact a frame target generated information amount T _i is clipped so as not to reach the overflow limit or under limit of the buffer. According to the equation (11), since the information amount allocation is performed in GOP units in consideration of the buffer state, the failure of the buffer does not occur in principle. However, to strictly avoid this, the following buffer constraint is applied.
Formula (12)
Bh _i -b _i ≧ mgn
Formula (13)
Bh _i −b _i + BR / FR ≦ (B−mgn)
However, mgn is a margin for the upper and lower limits of the buffer and is defined as follows.
Formula (14)
mgn = 0.2 · BR / FR

[Quantization parameter controller]
The quantization parameter control unit 124 includes a macroblock target generation information amount determination unit 1241, a target generation information amount sum unit 1242, an actual generation information amount sum unit 1243, a quantization parameter calculation unit 1244, and a quantization parameter averaging. Part 1245. Macroblock target generated information amount determining unit 1241, the frame target generated information amount _{T i,} and _{S j} is the SAD of each macroblock (prediction errors), and intra-frame sum [sigma] s _j of _{S j,} each macroblock The target generation information amount TB _j is determined. The target generation information amount summation unit 1242 adds the macroblock target generation information amount TB _j and outputs the target generation information amount S _TB . The actual generation information amount summation unit 1243 adds the actual generation information amount BM _j for each macroblock, and outputs the actual generation information amount _SBM . The quantization parameter calculation unit 1244 determines a QP value from the target generation information amount S _TB , the actual generation information amount S _BM, and the average value of the QP values, and outputs the QP value to the quantization unit. The quantization parameter averaging unit 1245 calculates the average of the QP values and outputs the average QP value to the quantization parameter calculation unit 1244.

[Macroblock target generation information amount determination unit]
Macroblock target generated information amount determining unit 1241, a frame target generated information amount T _i, and the prediction error S _j for each macroblock, and a frame in the total value [sigma] s _j of S _j, the target amount of information generated for each macroblock TB _j is determined.

The present invention assigns an information amount so as to reflect the information amount as much as possible with respect to the information amount required by each part for the information amount assignment in units of macroblocks, and minimizes the variation in QP, and The purpose is to keep the image quality uniform. The necessary information amount uses the SAD value S _j output from the SAD calculation unit 122, and the target generated information amount TB _j is determined by the following equation (15).
Formula (15)
TB _j = S _j / ΣS _j · T _i

式（１７）

[Quantization parameter calculation unit]
In order to absorb the difference between the target generated information amount and the actual generated information amount after encoding, the QP value is controlled as follows. The amount of information generated so far and the amount of target generated information are as follows.
m: Macroblock currently being encoded BM _j : Information amount actually generated in macroblock j Equation (16)

Formula (17)

従って、以下の式（１９）が成立する。
式（１９）

これにより、以下の式（２０）が成立する。
式（２０）

The QP value QPm for the macroblock m is set as a value for absorbing this difference. H. According to H.264 (decoder standard, for example, see Non-Patent Document 7), the relationship between the quantization parameter QP value and the quantization step is the following exponential function, and the quantization parameter increases by 6 The step is defined to be doubled.
Formula (18)

Therefore, the following equation (19) is established.
Formula (19)

Thereby, the following formula | equation (20) is materialized.
Formula (20)

The quantization parameter averaging unit 1245 averages the QP values output from the quantization parameter calculation unit 1244. The present invention assigns an information amount so as to reflect the information amount as much as possible with respect to the information amount required by each part for the information amount assignment in units of macroblocks, and minimizes the variation in QP, and The purpose is to keep the image quality uniform. From this point of view, it is considered that the average QP value of the previous frame can be effectively used for mode determination. QP ₀ is a moving average value of past QPs. Since the amount of information is allocated to the frame and the macroblock according to the necessity, the fluctuation of the QP value is minimized, and the past QP average value should be used effectively. Can do.

  Finally, in order to confirm the effectiveness of the present invention, a software simulation experiment was conducted on the assumption that terrestrial digital broadcasting (one segment) broadcasting for mobile receivers was assumed.

  FIG. 6 is a graph showing the transition of SNR for each frame in each image.

  In the graph of FIG. 6, the horizontal axis is the frame number, and the vertical axis is PSNR (dB). It is also clear from the graph of FIG. 6 that the present invention suppresses the quality fluctuation at the GOP boundary as compared with the conventional JM method.

Table 2 shows a comparison of SNR and its standard deviation between the present invention and the JM method. According to Table 1, it can be understood that the average SNR of the present invention is improved as compared with the JM method, and its standard deviation is small.

  Although the average SNR was improved by about 0.3, the SNR fluctuation was suppressed to less than half in the local portion, and the maximum deterioration could be improved by about 2 dB. At the same time, it conforms to buffer constraints.

Table 3 shows a comparison of the average QP and its standard deviation between the present invention and the JM method. According to Table 2, it can be understood that the average QP value of the present invention is less fluctuated than that of the JM method, and its standard deviation is small.

  Thus, by reducing the quality fluctuation, the coding efficiency is also improved, the average QP value is decreased, and as a result, the average SNR is improved.

  In the JM method, when there is a frame with a large amount of generated information such as IDR refresh or scene change, information amount allocation to other frames in the GOP is reduced, resulting in a deterioration in quality. On the other hand, according to the present invention, since the frame target generated information amount is determined by the prior prediction of the generated information amount of each frame, the information amount allocation according to the information amount required by the scene is performed, and the local amount Prevent periodic or periodic quality degradation.

  FIG. 7 is a graph showing HRD buffer transition for each frame.

  In the graph of FIG. 7, the horizontal axis represents the frame number, and the vertical axis represents the occupation amount of the HRD buffer. While the present invention considers image properties, the JM method of the prior art performs encoding control considering only buffer constraints (not considering image properties). Therefore, in the present invention, underflow and overflow do not occur even though the change in the buffer occupancy is large because the image property is taken into consideration as compared with the JM method. In this way, by performing rate control based on scaling in consideration of buffer transitions, it is possible to perform information amount allocation according to image properties and control based on HRD buffer constraints.

  With respect to the calculation processing amount, the present invention performs a prior prediction of the amount of generated information based on the SAD before encoding. However, it uses motion prediction / intra prediction performed in a normal encoding process, and the amount of arithmetic processing between the present invention and the conventional method is almost the same.

  In accordance with the present invention, H.264. With regard to H.264 coding rate control, we focused on the correlation between inter-frame prediction error (SAD) and the amount of generated information, and proposed a rate control method based on prior prediction of the amount of generated information. In the present invention, the amount of generated information predicted before encoding is regarded as the amount of information required by the screen, and the frame target generated information amount is determined. In order to consider the HRD buffer constraint, a scaling coefficient based on a buffer transition tendency is introduced, and an information amount is allocated while maintaining a relative relationship between the generated information amounts required by the frame. As a result, it is possible to achieve both stabilization by considering buffer constraints and improvement of image quality by reflecting image properties.

  According to the above-described various embodiments of the present invention, those skilled in the art can easily make various changes, modifications, and omissions in the technical idea and scope of the present invention. The above description is merely an example, and is not intended to be restrictive. The invention is limited only as defined in the following claims and the equivalents thereto.

It is a functional block diagram of the image compression encoding apparatus in a prior art. It is a functional block diagram of the image compression encoding apparatus in this invention. It is a graph showing the relationship between SAD and the amount of generated information. It is the graph which approximated the result of the graph of FIG. It is a transition graph of the HRD buffer in this invention. It is a graph showing transition of SNR for every frame in each image. It is a graph showing the HRD buffer transition for every flame | frame.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image compression coding apparatus 101 Current frame memory 102 Motion prediction part 103 Inter mode inter frame prediction cost calculation part 104 Intra mode inter frame prediction cost calculation part 105 Mode determination part 106 Mode information memory 107 Local decode memory 111 Motion compensation part 112 DCT (Discrete Cosine Transform) Unit 113 Quantization Unit 114 Inverse Quantization Unit 115 Inverse DCT Unit 116 Variable Length Coding Unit 117 Buffer Unit 121 Frame Memory 122 Difference Absolute Value Sum (SAD) Calculation Means 123 Buffer Restriction Unit 1231 Frame Basic Generation information amount estimation unit 1232 Information amount correction unit 1233 Memory 1234 Multiplication unit 1235 Clip unit 124 Quantization parameter control unit 1241 Macroblock target generation information amount determination unit 1242 Target generation information amount sum unit 12 43 Actually generated information amount summation unit 1244 Quantization parameter calculation unit 1245 Quantization parameter averaging unit

Claims (6)

A quantization means for performing quantization based on the quantization parameter;
From the time immediately after the first frame encoding of the first GOP, the buffer occupation transition of the virtual reception buffer up to the time immediately before the encoding of the first frame of the second GOP is
A straight line connecting the buffer position after encoding the IDR frame at the head of the GOP and the buffer position reserved for the head IDR frame of the next GOP is set as an underflow limit,
And the second frame before encoding buffer position from the GOP head, a straight line connecting the buffer maximum value in the next GOP head IDR frame before encoding the overflow limit, so that the center of the overflow limit and the underflow limits In addition, using the relationship between the prediction error and the amount of generated information calculated by prior measurement, frame target generation is performed by correcting the basic generated information amount required for each frame, which is obtained from the prediction error of the encoded frame. A buffer restriction means for outputting the information amount;
A quantization parameter control unit for deriving a quantization parameter from the macroblock target generation information amount calculated based on the frame target generation information amount and outputting the quantization parameter to the quantization unit; An image compression encoding apparatus. A difference absolute value sum calculating means for calculating a difference absolute value sum SAD which is a prediction error after the inter / intra mode determination for each macroblock;
The buffer restriction means includes
Frame basic generation information amount estimation means for estimating a frame basic generation information amount based on the difference absolute value sum SAD;
Viewed from point immediately after the first frame coding of the first GOP, as the buffer occupancy transition of the virtual reception buffer to the beginning frame encoding just before the time of the second GOP is at the center of the overflow limit and the underflow limits Information amount correcting means for outputting a scaling coefficient that is a ratio of the generated information amount of the frame to the average generated information amount per frame obtained from the bit rate;
2. The image compression encoding apparatus according to claim 1, further comprising multiplication means for multiplying the estimated basic generation information amount by the scaling coefficient and outputting a frame target generation information amount.   The information amount correction means is performed in units of GOP, and outputs a small scaling coefficient when the occupancy amount of the encoded data in the buffer is low, and occupies the encoded data. 4. The image compression coding apparatus according to claim 2, wherein when the amount is transitioning high, the image compression coding apparatus operates so as to output a large scaling coefficient.   3. The frame basic generated information amount estimating means stores in advance a function f or an estimated value for estimating a generated information amount based on the sum of absolute differences SAD and a fixed quantization parameter QP. The image compression encoding apparatus described in 1. The buffer constraint means includes a further comprising a clipping means for clipping to prevent relative to the frame target generated information amount output from the multiplying means, reaches the overflow limit or the underflow limits of the buffer The image compression encoding apparatus according to any one of claims 2 to 4. The quantization parameter control means includes
Macroblock target generation information amount determination means for outputting a macroblock target generation information amount based on the frame target generation information amount;
From the sum of the information amount immediately before the macroblock in the actual generated information amount, the sum of the information amount immediately before the macroblock in the target generated information amount, and the average value of the quantization parameter immediately before the macroblock, 2. The image compression coding apparatus according to claim 1, further comprising: a quantization parameter calculation unit that calculates a quantization parameter of the block and outputs the quantization parameter to the quantization unit.

Images