JP3946804B2 - Image coding control method - Google Patents

Description

【００３２】
である。ここで、Ｇ_jは符号化済みフレームの発生情報量である。符号化直前のバッファ占有量Ｂ_iから、目標情報量Ｔｂ_iは、次式より算出できる。この目標情報量Ｔｂ_iが大きい場合には、符号化時に量子化ステップを小さくして、実際の発生情報量を大きくできるし、又目標情報量Ｔｂ_iが小さい場合には、符号化時に量子化ステップを大きくして、実際の発生情報量を小さくする制御を行う。
【００３３】
Ｔｂ_i＝（Ｒ・Ｔｃ（１−ｉ／Ｎ）＋Ｂ_i＋Ｂ_init−２Ｂ₀）／（Ｎ−（ｉ−１））
符号化直前のバッファ占有量Ｂ_iは、スイッチ４１１がＩｎｔｅｒ側に閉じて符号化データバッファメモリ４０９に接続されているときにメモリ４１４に蓄えられる。目標情報量Ｔｂ_iが、最低目標情報量ＴＧ_min以下である場合は、Ｔｂ_i＝ＴＧ_minとする。
【００３４】
例として、図３にフレームスキップ数の変化を示す。実線は本発明による方法であり、破線は従来方法によるものである。この例のリフレッシュ周期Ｔｃ内の全発生情報量は、本発明による方法と従来方法とも同じである。この例から分かるように、従来方法に比べ、符号化フレームレートは低下しているものの、各符号化フレームの発生情報量を増加させることが出来るため、画質を向上させることが出来る。すなわち、従来ではＮ＝４、本発明でＮ＝３となっており、符号化フレームレートは減少しているが、各フレーム間符号化フレームに対応する発生情報量は本発明の方が大きい。また、従来方法では、フレームレートが不均一でフレームレートの変動がはげしいが、本発明による方法では、フレームレートが均一になり、実時間復号処理が可能になることがわかる。
【００３５】
本実施形態例では、時間間隔Ｔは、リフレッシュ周期など符号化処理の構造に合わせているが、入力される画像や発生情報量、符号化データバッファの状態など、他の要因から動的に決定しても良い。例えば、入力されるフレームのシーンチェンジを検出し、各シーンに合わせて時間間隔Ｔを設定する場合などである。
【００３６】
【発明の効果】
以上説明したように、本発明によれば、符号化フレームレートの変動を抑えることができ、さらに、符号化対象フレームの画質劣化を抑えることができる。これにより、本発明は、特に比較的低レートの映像伝送において、高効率な映像符号化アルゴリズムを提供することができる。
【図面の簡単な説明】
【図１】本発明の一実施形態例の画像符号化制御方法を実施するための構成例を示すブロック図である。
【図２】上記実施形態例の画像符号化制御方法による手順を示すフローチャートである。
【図３】本発明方法と従来方法による符号化フレーム数の比較を示す図である。
【図４】従来技術による入力フレームと符号化対象フレームの関係を示す図である。
【図５】従来技術による入力フレームと符号化対象フレーム（バッファメモリあり）の関係を示す図である。
【図６】従来技術によるフレーム内／フレーム間符号化の例を示す図である。
【符号の説明】
４０１…入力端子
４０２…スイッチ
４０３，４０６，４１１…スイッチ
４０４…フレーム内符号化部
４０５…フレーム間符号化部
４０７…符号化アルゴリズム切替え制御部
４０８…目標情報量算出部
４０９…符号化データバッファメモリ
４１０…フレーム間符号化フレーム数算出部
４１２，４１４…メモリ
４１３…フレームスキップ数算出部
４１５…出力端子[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a coding control method for moving picture coding.
[0002]
[Prior art]
When encoding control is performed at a fixed bit rate, the time until the next encoding target frame is determined from the amount of generated information of the frame encoded immediately before. For example, if the bit rate is R and the amount of generated information in the encoding target frame is G, the time T until the next encoding target frame is
T = G / R
It becomes. Actually, since the input frame is input at a fixed frame rate (29.97 Hz in the case of NTSC), it is quantized by the input frame rate. An example is shown in FIG. In this example, with respect to the amount of information generated in each encoding target frame (Frames A to D), the bit rate R is set as a gradient, and when all information has been transmitted, the next input frame is set as the encoding target frame. Encoding is performed. For example, when the generated information amount of Frame A is Ga and the input frame rate is f [fps], the frame dropping number Sab of frames between Frame A and Frame B can be calculated from the following equation.
[0003]
Sab = (Ga / R) · f
As described above, by determining the frame dropping (hereinafter referred to as frame skip) according to the amount of information generated in the encoding target frame, encoding at a fixed bit rate becomes possible.
[0004]
However, in the above method, frame skip frequently occurs. Therefore, there is a method of absorbing a fluctuation in the amount of generated information and suppressing frame skipping by having a buffer memory for temporarily storing encoded data. However, even in this method, when the occupied amount of the encoded data in the buffer memory becomes high, frame skip occurs. An example is shown in FIG. In this example, even if an encoded frame has a large amount of generated information, no frame skip is generated as long as it is less than the buffer capacity. Conversely, if the buffer occupancy becomes high and the occupancy may exceed the buffer upper limit, a frame skip is generated.
[0005]
Thus, by having a buffer memory, encoding at a fixed bit rate is possible while suppressing frame skipping.
[0006]
[Problems to be solved by the invention]
In the conventional encoding control method using the fixed bit rate, the interval (number of frame skips) to the next encoding target frame is determined based on the amount of information generated in the current encoding target frame and the buffer occupation amount of the encoded data. The That is, after a frame with a large amount of generated information or when the buffer memory occupancy is close to the upper limit, the number of frame skips increases and the encoding frame rate decreases. Conversely, when the amount of generated information is small or the buffer occupancy is small In some cases, the number of frame skips decreases and the encoding frame rate increases.
[0007]
A case where intra-frame coding and inter-frame coding are combined will be described as an example. Intra-frame coding has a lower coding efficiency than inter-frame coding, so the amount of generated information is extremely large. However, the first frame to be encoded, transmission error handling, error propagation prevention, or network It is necessary to use this when the packet is discarded due to the protocol and the packet is not retransmitted or when the video data being constantly transmitted is started to be decoded halfway (intra refresh) as in live relay. An example is shown in FIG. In this example, the first encoding target frame is an intra-frame encoding frame (Intra frame: ● mark), and the subsequent frames are inter-frame encoding frames (Inter frame: ○ mark). For refreshing, intra-frame coding is performed every four frames. As can be seen from this example, the number of skipped frames increases immediately after the Intra frame. In addition, the frame skip is large even after switching from Scene A to Scene B. This is because redundancy was not able to be deleted even if inter-frame coding was used because the correlation between the images of Scene A and Scene B was small. As can be seen from this example, when the amount of generated information in each encoding target frame varies greatly, the variation in the frame rate increases accordingly.
[0008]
Next, consider a case in which the buffer occupancy of encoded data is operating near the buffer upper limit. In the conventional method, the encoding frame rate is controlled to be close to the input frame rate. When the buffer occupancy is close to the upper limit of the buffer, control is performed to reduce the amount of generated information by coarse quantization to prevent overflow of the buffer memory. If the amount of generated information cannot be reduced sufficiently even after coarse quantization, frame skip is generated. For this reason, although the encoding frame rate approaches the input frame rate, each encoding target frame is coarsely quantized, so that the image quality is significantly deteriorated.
[0009]
As described above, in the conventional control method for encoding at a fixed bit rate, when the amount of generated information in each encoding target frame varies greatly, the frame rate variation increases accordingly. When the encoded data buffer occupancy is operating close to the upper limit of the buffer, there is a problem that the reproduction image quality of the encoding target frame is significantly deteriorated.
[0010]
An object of the present invention is to uniformly align encoding frame rates and improve the image quality of encoded frames in moving image encoding.
[0011]
[Means for Solving the Problems]
In order to achieve the above object, according to the present invention, when a moving image sequence is encoded, the input frame rate is constant, and encoding is performed that represents the number of frames to be encoded among a plurality of frames input within a unit time. In an encoding method in which the frame rate can be changed, an encoding target frame is encoded by intraframe encoding at a predetermined frame interval, and after the encoding is completed, an intraframe encoded frame is generated. Calculate the provisional target generated information amount of the inter-frame encoded frame from the information amount and a predetermined constant, and calculate the number of inter-frame encodings to be encoded before the next intra-frame encoded frame. Thus, the method has a process of determining an encoding frame rate.
[0013]
Furthermore, the generated information amount of the intra-frame encoded frame is G ₀ , the bit rate is R, the intra-frame encoding interval is T _c , the initial value of the encoded data buffer is B _init , and immediately before the intra-frame encoding is performed. The encoded frame rate Ft, which is the number of inter-frame encoding frames per unit time, is expressed as (R · T) where B _{0 is} the occupation amount of the encoded data buffer and IP _rate is a predetermined constant. _c + (B _int −B ₀ ) −G ₀ ) / (G ₀ · IP _rate · T _c ).
[0014]
In the conventional method, the encoding frame rate is determined by the amount of generated information and the occupation amount of the encoded data buffer. For this reason, when the variation in the amount of generated information is severe, the variation in the coding frame rate is also large under the influence. Also, in order to make the encoding frame rate close to the input frame rate, if the buffer occupancy of the encoded data is operating near the upper limit of the buffer, the encoding target frame is coarsely quantized and the image quality deteriorates. Was invited.
[0015]
Therefore, in the present invention, the encoding process is performed with the encoding frame rate within a fixed time being substantially fixed, regardless of the amount of information generated in the immediately previous encoding target frame and the amount of encoding data buffer occupied during encoding. . Below, the control method of this invention is shown.
[0016]
First, the target encoding frame rate Ft for the time interval T is calculated. The target encoding frame rate Ft is calculated from the amount of generated information of encoded frames and the amount occupied by the encoded data buffer. The number N of encoding target frames within the time interval T at that time is
N = T / Ft
It becomes. By setting a smaller target encoding frame rate Ft with respect to the input frame rate, it is possible to increase the amount of information that can be allocated to each encoding target frame, leading to an improvement in image quality.
[0017]
Next, the target information amount Tbi (i = 1, 2, 3,... N) of the encoding target frame within the time interval T is calculated. The target information amount Tb i of the encoding target frame is calculated from the following equation.
[0018]
Tb _i = f (R, T, Ft, B ₀ , B _i , B _init )
Here, R is the bit rate, B ₀ is the buffer occupancy immediately before the encoding of the first frame in the time interval T, B _i is the buffer occupancy immediately before the encoding of the encoding target frame, and B _init is the initial buffer. Value.
[0019]
Then, after the time interval T has passed, the encoding frame rate within the next time interval is determined, and the encoding process is performed in the same manner.
[0020]
Within the time interval T, the encoding frame rate Ft is basically constant. However, it is possible to vary the encoding frame rate Ft from the viewpoint of buffer occupancy and image quality.
[0021]
In addition, the time interval T used in the present invention can be processed in advance by using, for example, the refresh interval T _c , but the nature of the input image signal, the amount of generated information, encoded data, etc. It may be dynamically determined from other factors such as the buffer status.
[0022]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this embodiment, intra-frame / inter-frame coding is used, and intra refresh is performed for a certain period Tc. Further, R is a bit rate, B _init is an initial value of the encoded data buffer, and an intra refresh interval Tc (refresh cycle) is used as the time interval. Further, the maximum encoding frame rate F _max and the minimum target information amount TG _min are set.
[0023]
FIG. 1 is a block diagram showing a configuration example for carrying out an encoding control method according to an embodiment of the present invention. The frame input to the input terminal 401 is sent to the switch 403 when the switch 402 is closed. The switch 402 is controlled by the frame skip number calculation unit 413 and is opened and closed according to the encoding frame rate. The switches 403, 406, and 411 are switches that are switched by an encoding algorithm. These switches 403, 406, and 411 are controlled by the encoding algorithm switching control unit 407, and are switched to the Intra side at every intra refresh interval Tc. The encoding target frame is encoded by the intra-frame encoding unit 404 if the switching state of the switches 403 and 406 is the Intra side, and is encoded by the inter-frame encoding unit 405 if the switching state is the Inter side. Stored in the data buffer memory 409 and output from the output terminal 415 at the transmission rate R.
[0024]
Here, a control procedure will be described. FIG. 2 shows a flowchart of this embodiment. Consider a case where the encoding process starts from intra-frame encoding. At this time, the switches 403, 406, and 411 are closed to the Intra side.
[0025]
First, the encoding target frame input to the switch 403 is input to the intraframe encoding unit 404, where intraframe encoding is performed. Then, after the end of intra-frame encoding, an inter-frame encoded frame number calculation unit is calculated from the generated information amount G ₀ of the intra-frame encoded frame sent from the intra-frame encoding unit 404 and a constant IP _rate given in advance. At 410, the number N of interframe encoded frames up to the next intraframe encoded frame is determined.
[0026]
The number N of inter-frame encoded frames is calculated as follows. First, a temporary target generation information amount TG _Inter of an _inter- frame encoded frame is calculated. The provisional target generated information amount TG _Inter is a target information amount when the information amount is evenly allocated to the Inter frame encoded within the intra refresh time interval Tc. The provisional target generation information amount TG _Inter of the _inter- frame encoded frame is calculated from the following equation.
[0027]
TG _Inter = G ₀・ IP _rate
Further, the number N of inter-coded frames is determined from the provisional target generation information amount TG _Inter of the obtained inter-coded frames. For example, in FIG. 6, N = 3 in the method according to the invention.
[0028]
N = (R · Tc + (B _init −B ₀ ) −G ₀ ) / TG _Inter
B ₀ is an occupation amount of the encoded data buffer memory 409 immediately before performing intra-frame encoding, and is stored in the memory 412 when the switch 411 is closed to the Intra side. The initial value of the memory 412 is the buffer initial value B _init . Then, a frame skip number calculation unit 413 calculates an encoding frame rate Ft from the number N of interframe encoded frames, calculates a frame skip number from the encoding frame rate Ft, and calculates 1 / The switch 402 is closed at intervals of Ft. At this time, the encoding frame rate Ft is
Ft = N / Tc
It becomes. When the encoding frame rate Ft is larger than the maximum encoding frame rate _Fmax , Ft = _Fmax is set.
[0029]
After the encoding frame rate is calculated, the inter-frame encoding is performed until the next intra refresh. At this time, the switches 403, 406, and 411 are closed to the Inter side by the encoding algorithm switching control unit 407.
[0030]
The target information amount Tb _i (i = 1, 2, 3,... N) of each inter-frame encoded frame is calculated by the target information amount calculation unit 408 immediately before the encoding of each encoding target frame. The buffer occupancy B _i immediately before the encoding of the i-th inter-frame encoded frame is
[0031]
[Expression 1]

[0032]
It is. Here, G _j is the generated information amount of the encoded frame. From the buffer occupancy B _i immediately before encoding, the target information amount Tb _i can be calculated from the following equation. When the target information amount Tb _i is large, the quantization step can be reduced during encoding to increase the actual generated information amount. When the target information amount Tb _i is small, quantization is performed during encoding. Control is performed to increase the step to reduce the actual amount of generated information.
[0033]
Tb _i = (R · Tc (1−i / N) + B _i + B _init −2B ₀ ) / (N− (i−1))
The buffer occupancy B _i immediately before encoding is stored in the memory 414 when the switch 411 is closed on the Inter side and connected to the encoded data buffer memory 409. When the target information amount Tb _i is equal to or less than the minimum target information amount TG _min , Tb _i = TG _min is set.
[0034]
As an example, FIG. 3 shows a change in the number of frame skips. The solid line is the method according to the present invention, and the broken line is the conventional method. The total amount of generated information within the refresh cycle Tc in this example is the same for both the method according to the present invention and the conventional method. As can be seen from this example, although the encoding frame rate is lower than that of the conventional method, the amount of information generated in each encoded frame can be increased, so that the image quality can be improved. That is, N = 4 in the prior art and N = 3 in the present invention, and the encoding frame rate is reduced, but the amount of generated information corresponding to each interframe encoded frame is larger in the present invention. Further, in the conventional method, the frame rate is non-uniform and the frame rate fluctuates greatly. However, in the method according to the present invention, it is understood that the frame rate is uniform and real-time decoding processing is possible.
[0035]
In this embodiment, the time interval T is matched to the structure of the encoding process such as the refresh cycle, but is dynamically determined from other factors such as the input image, the amount of generated information, and the state of the encoded data buffer. You may do it. For example, a scene change of an input frame is detected, and a time interval T is set according to each scene.
[0036]
【The invention's effect】
As described above, according to the present invention, fluctuations in the encoding frame rate can be suppressed, and further, image quality deterioration of the encoding target frame can be suppressed. As a result, the present invention can provide a highly efficient video encoding algorithm, particularly in video transmission at a relatively low rate.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration example for carrying out an image coding control method according to an embodiment of the present invention.
FIG. 2 is a flowchart showing a procedure according to an image encoding control method of the embodiment.
FIG. 3 is a diagram showing a comparison of the number of encoded frames according to the method of the present invention and the conventional method.
FIG. 4 is a diagram illustrating a relationship between an input frame and an encoding target frame according to a conventional technique.
FIG. 5 is a diagram illustrating a relationship between an input frame and a frame to be encoded (with a buffer memory) according to the related art.
FIG. 6 is a diagram illustrating an example of intra-frame / inter-frame coding according to the prior art.
[Explanation of symbols]
401 ... Input terminal 402 ... Switch 403, 406, 411 ... Switch 404 ... Intraframe encoding unit 405 ... Interframe encoding unit 407 ... Encoding algorithm switching control unit 408 ... Target information amount calculation unit 409 ... Encoded data buffer memory 410: Inter-frame encoded frame number calculation unit 412, 414 ... Memory 413 ... Frame skip number calculation unit 415 ... Output terminal

Claims (2)

An encoding method capable of changing an encoding frame rate representing the number of frames to be encoded among a plurality of frames input within a unit time when an input video rate is constant when encoding a moving image sequence In
Encode the frame to be encoded by intra-frame encoding at a predetermined fixed frame interval, and after the encoding is completed, from the generated information amount of the intra-frame encoded frame and a predetermined constant, inter-frame encoding By calculating the provisional target generation information amount of the frame and calculating the number of inter-frame encodings performed until the next intra-frame encoded frame, the encoding frame rate is determined.
An image coding control method characterized by the above. The amount of information generated in the preceding intraframe encoded frame is G ₀ , the bit rate is R, the intraframe encoding interval is Tc, the initial value of the encoded data buffer is Binit, and the encoded data immediately before the intraframe encoding is performed. If the buffer occupancy is B ₀ and the constant given in advance is IPrate,
The encoding frame rate Ft, which is the number of inter-frame encoding frames to be encoded within a unit time, is determined by (R · Tc + (Binit−B ₀ ) −G ₀ ) / (G ₀ · IPrate · Tc). To
The image encoding control method according to claim 1, wherein:

Images