JP3854472B2 - Repetitive image signal encoding method and apparatus, repetitive image signal encoding program, and recording medium recording the program - Google Patents

Description

【００２０】
という連立方程式に帰着される。最初の３式は
【００２１】
【数２】

【００２２】
となる。ここで
【００２３】
【数３】

【００２４】
とおくと、Ｂフレームについて、式（７）より
Ｔ_B＝λ …（１１）
が成り立つ。
【００２５】
またＰフレームについて、Ｒ_Pが増加するとＰフレームの画品質が上がり、結果としてＰフレームを参照するＢフレームは符号化が楽になりＤ_BとＲ_Bは減少することから、
【００２６】
【数４】

【００２７】
が成り立つ。これらを式（８）に代入し
【００２８】
【数５】

【００２９】
となる。
【００３０】
同様にＩフレームについて、Ｒ_Iが増加するとＩフレームの画品質が上がり、結果としてＩフレームを参照するＰ，Ｂフレームは符号化が楽になりＤ_P，Ｒ_P，Ｄ_B，Ｒ_Bが減少することから、
【００３１】
【数６】

【００３２】
が成り立ち、これらを式（９）に代入し
【００３３】
【数７】

【００３４】
を得る。
【００３５】
各フレームにおいて、Ｄ（Ｒ）のグラフは一般に下に凸であるので、−∂Ｄ／∂Ｒが局所未定乗数Ｔに等しくなる点は、局所評価規準Ｄ＋ＴＲが最小になる点と一致する。従って、Ｔ_I，Ｔ_B，Ｔ_Pを局所未定乗数、λを大域未定乗数とすると、
Ｔ_I＜λ、Ｔ_P＜λおよびＴ_B＝λ …（１６）
を保ちながらＪ＝Ｄ＋λＲを最小化するよう（Ｔ_I，Ｔ_P，Ｔ_B）を探索すればよい。この方法により、（Ｔ_I，Ｔ_P、Ｔ_B）について何の先験的知識もなく最適化サイクルを繰り返すよりも、高速で効率的な最適化が可能になる。
【００３６】
上記は３フレームの最も単純な場合を考えたが、一般の長さを持つビデオクリップをフレーム単位でＭＰＥＧ方式にて最適化する場合へ、この考え方を応用すると、手順は以下のようになる。
【００３７】
１．フレーム数と同じ長さの局所未定乗数表（Ｔ_I1，Ｔ_P1，Ｔ_B1，Ｔ_B2，Ｔ_P2，…）を初期化する。この表においては、最適化サイクルの途中、Ｔ_Ix＜λ、Ｔ_Px＜λ、Ｔ_Bx＝λが常に保たれていなければならない。
【００３８】
２．ビデオクリップをフレーム毎に最適化する。最小符号化単位（ＭＰＥＧの場合マクロブロックと呼ぶ）の各々について、離散的に様々な値を取りうる（ＭＰＥＧの場合３１通り）量子化ステップの全てあるいは一部の値で符号化を試行し、最小の局所評価規準Ｄ＋ＴＲを与える量子化ステップを採用する。Ｄ，Ｒはそれぞれ、該マクロブロックを該量子化ステップで符号化した際の符号量および復号誤差であり、Ｔは該フレームの局所未定乗数である。
【００３９】
３．全フレームの符号化が終了後、大域評価規準Ｊ＝Ｄ＋λＲを最小化すべく局所未定乗数表を更新する。これは例えば、一般に用いられる多次元最適化アルゴリズムに基づき行う。もし収束したと判定されたら終了し、さもなくば２から繰り返す。
【００４０】
局所未定乗数に関する拘束条件（式（１６））の正当性を確かめるために、ある画像について、０．０００３〜０．００１２の範囲内の大域未定乗数λを数種類用い、拘束条件を用いずに最適化を行った結果を図１に示す。ここで第１フレームがＩフレーム、第２，３，５，６，８，９，１１，１２フレームがＢフレーム、第４，７，１０，１３フレームがＰフレームである。式（１６）の通り、Ｂピクチャに対する局所未定乗数が大域未定乗数に等しく、かつＩ，Ｐピクチャに対する局所未定乗数が大域未定乗数より小さくなっていることが確認できる。
【００４１】
なお、ビデオクリップが長大となると、変化しうる局所未定乗数の数も増え、最適化サイクルにおける探索空間が指数的に増大するが、ＭＰＥＧにおいてはＧＯＰ（Ｇｒｏｕｐ ｏｆ Ｐｉｃｔｕｒｅ）構造、すなわちビデオクリップを１枚のＩフレームおよび後続の複数のＢ，Ｐフレームから成るグループ構造を単位とする構造毎の符号化を行う。従って、ＧＯＰ末尾のＢフレーム群（もし、あれば）を除き、ＧＯＰ間は独立であるため、最適化もＧＯＰ単位に独立に行ってよい。
【００４２】
〈第二の発明の原理〉
図１の各グラフを、対応する大域未定乗数で除することにより正規化したグラフを図２に示す。この図においてグラフは互いにほぼ重なりあっている。つまり、最適な局所未定乗数の値は、大域未定乗数と比例関係にあることがわかる。
【００４３】
また、ＤとＲの間には反比例の関係Ｄ∝Ｒ^-1があることが実験的、統計的に示されている（Ｓ．Ｔａｋａｍｕｒａ， Ｈ．Ｗａｔａｎａｂｅ ａｎｄ Ｎ．Ｋｏｂａｙａｓｈｉ：“Ｏｎｅ−ｐａｓｓ ＶＢＲ ａｌｇｏｒｉｔｈｍ ｆｏｒａｎ ＭＰＥＧ−２ ｅｎｃｏｄｅｒ ｃｈｉｐ ＳｕｐｅｒＥＮＣ，”Ｐｉｃｔｕｒｅ Ｃｏｄｉｎｇ Ｓｙｍｐｏｓｉｕｍ Ｊａｐａｎ，ＰＰ．３９−４０，Ｓｅｐ．１９９９．）。従って、∂Ｄ／∂Ｒ∝Ｒ^-2となり、これに∂Ｄ／∂Ｒ＝−λを代入すると、以下のような逆二乗関係が成り立つことが予想される。
【００４４】
λ∝Ｒ^-2 …（１７）
これを確かめるために、数種の画像について、０．０００３〜０．００１２の範囲内の大域未定乗数λを数種類用い、最適化サイクルを行った後のビット量Ｒとλとの関係を図３の点群としてプロットした。また、これらの点群を曲線λ＝ｋＲ^-2により最小二乗あてはめしたものを、同図の曲線として示す。点群に曲線がよくあてはまっていることから、実際にλ−Ｒの逆二乗関係が成り立つことが確認できる。
【００４５】
以上の事実を用いれば、あるλについてひとたび最適な局所未定乗数表が得られれば、任意のλに対しても、この未定乗数表を定数倍することで、最適化サイクルを行うことなしに、最適な局所未定乗数表を得ることができる。具体的に、発生符号量を目標量（Ｒ_tgt）に近づける手順を記述すると、以下のようになる。
【００４６】
１．ある与えられたλに基づき、ビデオクリップを上記第一の発明の最適化方法で符号化する。
【００４７】
２．得られた符号量をＲとする。もしＲがＲ_tgtに十分近ければ終了する。さもなければ次へ進む。
【００４８】
３．式（１７）に基づき、局所未定乗数表の各未定乗数をＲ²／Ｒ_tgt ²倍し、その表を用いて再度符号化を行う。その際、最適化のための繰り返しは不要である。符号化の後は２へ戻る。
【００４９】
実際のビデオクリップについて、第二の発明を用いた最適符号化・符号量制御を行った結果を以下に示す。目標ビットレートは４Ｍｂｉｔ／ｓと設定している。ここの例では１％以内の精度を達成するのには２回の試行で十分であることがわかる。
【００５０】
画像Ａ
１回目＝５．８０Ｍｂｉｔ／ｓ→４．０４Ｍｂｉｔ／ｓ（終了）
画像Ｂ
１回目＝５．１１Ｍｂｉｔ／ｓ→３．８６Ｍｂｉｔ／ｓ
２回目＝３．８６Ｍｂｉｔ／ｓ→４．０１Ｍｂｉｔ／ｓ（終了）
〈第一の発明の実施の形態〉
本発明による第一の発明の実施の形態について図４を参照して説明する。図４は、本実施形態例における処理の流れとともに処理部構成すなわち装置構成を示している。各処理部は、本発明の手段を構成する全部もしくは一部である。
【００５１】
始めに、処理開始後、局所未定乗数表初期化部１０１において局所未定乗数表記憶部１０２の局所未定乗数表（以下、局所未定乗数表を１０２と呼ぶ）を初期化する。この局所未定乗数表１０２においては、最適化サイクルの途中、Ｔ_Ix＜λ，Ｔ_Px＜λ，Ｔ_Bx＝λが常に保たれていなければならない。したがって、局所未定乗数表初期化部１０１は、Ｔ_Ix＜λ，Ｔ_Px＜λ，Ｔ_Bx＝λとなるように局所未定乗数表１０２の初期化を行う。
【００５２】
続いて、フレーム符号化部１２０にて符号化を行う。まず、局所未定乗数取得部１０３において符号化フレームを順に選択後、未定乗数表１０２より該当フレームの局所未定乗数Ｔを取得し、マクロブロック選択部１０４において該当フレームのマクロブロックを順に選択する。次に、量子化ステップ選択部１０５において、取り得る量子化ステップを順に全て発生させる。次に、符号化誤差・符号量取得部１０６において符号化を行った結果生じる誤差Ｄ′および符号量Ｒ′を取得し、局所価値基準算出部１０７において局所評価基準Ｄ′＋ＴＲ′を算出する。次に、最小判定部１０８においてこの局所評価規準が該マクロブロックにおいて最小であるかどうかを判断し、最小であれば量子化ステップ記憶部１０９において該量子化ステップを変数Ｑに記憶する。次に、全量子化判定部１１０にて取り得る量子化ステップを全て試行したと判断されなければ、量子化ステップ選択部１０５へ戻り再度別の量子化ステップを試行する。量子化ステップを全て試行したと判断されれば、マクロブロック符号化部１１１にて、記憶した量子化ステップＱで符号化を行う。次いで全マクロブロック判定部１１２にて該フレームの全マクロブロックについて符号化が完了したかどうかを判定し、未だであればマクロブロック選択部１０４へ戻り、次マクロブロックの処理を行う。完了していれば、全フレーム判定部１１３にて、全フレームの処理が完了したかを判定し、未だであれば局所未定乗数取得部１０３へ戻り次フレームの処理を行う。
【００５３】
全フレームの符号化処理が終了すると、総符号化誤差・総符号量取得部１１４ではビデオシーケンス全体での総符号誤差Ｄ、総符号量Ｒを取得し、大域評価基準算出部１１５にて大域評価基準Ｄ＋λＲを算出する。次に、収束判定部１１６では、この大域評価規準が収束したかどうかを判定し、まだ判定していないと判断すれば１１７の未定乗数表修正部へ進む。ここでは大域未定乗数による大域評価基準を最小化するよう、パラメータ（この場合１０２の局所未定乗数表）を調整するものである。ここで、未定乗数表修正部１１７は、Ｔ_Ix＜λ，Ｔ_Px＜λ，Ｔ_Bx＝λとなるように局所未定乗数表１０２の調整（修正）を行う。その後、局所未定乗数取得部１０３から再度ビデオシーケンス全体での符号化を繰り返す。収束判定部１１６で収束したと判定されれば、処理を終了する。
【００５４】
〈第二の発明の実施の形態〉
本発明による第二の発明の実施の形態について図５を参照して説明する。図５は、本実施形態例における処理の流れとともに処理部構成すなわち装置構成を示している。各処理部は、本発明の手段を構成する全部もしくは一部である。
【００５５】
始めに、開始後、ビデオシーケンス全体をビデオクリップ最適化部２０１にて一度最適化を行う。これは、第一の発明の実施の形態におけるビデオクリップ最適化と同じ処理である。その結果得られた局所未定乗数表を局所未定乗数表記憶部２０２（以下、局所未定乗数表を２０２と呼ぶ）に保存する。
【００５６】
続いて、符号量取得部２０３にてビデオシーケンス全体の符号量Ｒを取得する。次に、Ｒ・Ｒ_tgt比較部２０４においてＲとＲ_tgtを比較し、差が十分に小さいと判定されたら終了する。さもなくば、局所未定乗数表更新部２０５において、局所未定乗数表２０２の各値を全てＲ²／Ｒ_tgt ²倍する。
【００５７】
次いで、２０６のフレーム符号化部（これは第一の発明の実施形態例において示した、フレーム符号化部１２０と同一である）において、符号化を行う。ここで最適化のための繰り返しは不要である（但しマクロブロック単位の、量子化ステップ選択のための繰り返しは行う）。その後、符号量取得部２０３の処理より繰り返す。
【００５８】
なお、図４、図５で示した装置の各処理部の一部もしくは全部での処理、あるいはその処理の流れにおける処理手順をコンピュータのプログラムで構成し、そのプログラムをコンピュータを用いて実行して本発明を実現することができることは言うまでもなく、コンピュータでその処理部を実現するためのプログラム、あるいはコンピュータにその処理手順を実行させるためのプログラムを、そのコンピュータが読み取り可能な記録媒体、例えば、ＦＤ（フロッピーディスク（登録商標））や、ＭＯ、ＲＯＭ、メモリカード、ＣＤ、ＤＶＤ、リムーバブルディスクなどに記録して、保存したり、提供したりすることが可能であり、また、そのプログラムをインターネットや電子メール等によりネットワークを通して配布することが可能である。
【００５９】
【発明の効果】
以上説明したように、本発明によれば、最適化を行う際に変化させる局所未定乗数の、大域未定乗数との大小関係を考慮しつつ局所未定乗数調整・符号化・大域評価規準評価という最適化サイクルが行えるようになり、結果として最適にならないはずの局所未定乗数を用いて符号化を試行してしまったりする、無駄な処理を避けられるようになる。
【００６０】
また、最適化サイクルと符号量制御を両立させる場合、得られた符号量と一回求めた局所未定乗数を元に、目的とする符号量において最適となる局所未定乗数を一撃で生成することができるため、新たに最適化サイクルを繰り返す必要がなくなる。
【図面の簡単な説明】
【図１】様々な大域未定乗数λに対する、最適化後の局所未定乗数表（λ）のグラフ。
【図２】図１のデータを正規化したグラフ。
【図３】大域未定乗数λおよび最適化後のＲをプロットし、かつ逆二乗曲線で最小二乗あてはめを行ったグラフ。
【図４】第一の発明の実施の形態によるビデオクリップ最適化符号化の処理の流れと装置構成を示した図。
【図５】第二の発明の実施の形態による最適化時の符号化レート制御の処理の流れと装置構成を示した図。
【符号の説明】
１０１…局所未定乗数表初期化部
１０２…局所未定乗数表記憶部（局所未定乗数表）
１０３…局所未定乗数取得部
１０４…マクロブロック選択部
１０５…量子化ステップ選択部
１０６…符号化誤差・符号量取得部
１０７…局所価値基準算出部
１０８…最小判定部
１０９…量子化ステップ記憶部
１１０…全量子化判定部
１１１…マクロブロック符号化部
１１２…全マクロブロック判定部
１１３…全フレーム判定部
１１４…総符号化誤差・総符号量取得部
１１５…大域評価基準算出部
１１６…収束判定部
１１７…未定乗数表修正部
１２０…フレーム符号化部
２０１…ビデオクリップ最適化部
２０２…局所未定乗数表記憶部（局所未定乗数表）
２０３…符号量取得部
２０４…Ｒ・Ｒ_tgt比較部
２０５…局所未定乗数表更新部
２０６…フレーム符号化部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a high-efficiency repetitive image signal encoding method and apparatus with reduced processing time.
[0002]
[Prior art]
In a video coding method that allows a plurality of coding trials for the same image sequence to improve coding efficiency in terms of code amount versus noise amount, the evaluation of coding efficiency is based on “Lagrange” established in the 18th century. The undetermined multiplier method is applied.
[0003]
In this method, the total code amount R and the total error D at the time of sequence encoding with respect to an undetermined multiplier λ given from outside (this measure may be the sum of squares of errors or the sum of absolute values of errors depending on the situation). From global evaluation criteria J = R + λD (1)
Is obtained every time encoding is performed, and optimization search (parameter adjustment) is performed while repeating the encoding so that J is minimized.
[0004]
In this parameter adjustment, local undetermined multipliers are determined for each frame or smaller coding unit, and local undetermined multipliers, code amounts, and errors are set to λ ′, R ′, and D ′, respectively. Standard J ′ = R ′ + λ′D ′ (2)
The method of localizing the processing to minimize the global evaluation criterion J and minimizing the global evaluation criterion J is used.
[0005]
[Problems to be solved by the invention]
However, in the above conventional technique, when performing optimization corresponding to a given global undetermined multiplier, the local undetermined multiplier is changed, and the optimization cycle of multiplier adjustment / encoding / J evaluation is repeated to perform optimization. However, when determining this local undetermined multiplier, it is unclear how the value is related to the global undetermined multiplier, and as a result, encoding is attempted using a local undetermined multiplier that should not be optimal. We were processing.
[0006]
In addition, code amount control is required when there is a restriction such that “the total amount of code is within a few bytes as a result of encoding with optimization”. When the Lagrange global evaluation criterion J is minimized, there is no guarantee that the desired code amount is obtained. Therefore, the global undetermined multiplier λ is adjusted again and the optimization cycle is repeated. At that time, the processing time was further increased because new optimization was performed without using the local undetermined multiplier obtained in the previous optimization cycle.
[0007]
A first object of the present invention is to optimize a highly efficient repetitive image signal encoding method and apparatus without making useless encoding trials by determining a value of a local undetermined multiplier based on a reference relationship between encoding units. The purpose is to reduce the amount of calculation for the cycle.
[0008]
A second problem of the present invention is to provide a highly efficient repetitive image signal encoding method and apparatus that uses a known statistical relationship between a code amount and an undetermined multiplier to efficiently code using a local undetermined multiplier obtained once. The amount is close to the target value.
[0009]
[Means for Solving the Problems]
The repetitive image signal encoding method according to the first invention for solving the first problem includes encoding by inter-frame motion compensation, and based on an evaluation of an undetermined multiplier by Lagrange's undetermined multiplier method. In the image signal encoding method for optimizing the entire image sequence by encoding the same image sequence a plurality of times in order to optimize the relationship, a local undetermined multiplier corresponding to each encoding unit is initialized. The image sequence is determined by changing the quantization step for each procedure and the coding unit, and determining the quantization step based on the evaluation of the obtained local undetermined multiplier so that the relationship between the code amount and the noise amount is optimized. The second procedure for encoding the entire image and the global undetermined multiplier corresponding to the entire image sequence are evaluated each time the entire image sequence is encoded. A third procedure that repeats the second procedure by correcting the local undetermined multiplier until the relationship of the noise amount is optimized. In the first procedure, the first procedure is performed during the initialization. In the third procedure, at the time of the correction, the local undetermined multiplier of the coding unit referred to for motion compensation from other coding units is made smaller than the global undetermined multiplier, and is not referenced for motion compensation from other coding units. The global undetermined multiplier and the local undetermined multiplier are determined so that the local undetermined multiplier of the encoding unit is equal to the global undetermined multiplier.
[0010]
In addition, the iterative image signal encoding apparatus according to the first invention performs encoding by inter-frame motion compensation, and optimizes the relationship between the amount of code and the amount of noise based on the evaluation of the undetermined multiplier according to the Lagrange undetermined multiplier method In an image signal encoding apparatus for optimizing an entire image sequence by encoding the same image sequence a plurality of times, first means for initializing a local undetermined multiplier corresponding to each encoding unit, and the encoding A second step of encoding the entire image sequence by changing the quantization step for each unit and determining the quantization step based on the evaluation of the obtained local undetermined multiplier so that the relationship between the code amount and the noise amount is optimized. Each time the entire image sequence is encoded, the global undetermined multiplier corresponding to the entire image sequence is evaluated, and the relationship between the code amount and the noise amount is optimized. And a third means for repeatedly performing the encoding by the second means until the local undetermined multiplier is corrected, and the first means at the time of initialization, the third means At this time, the local undetermined multiplier of the coding unit referred to for motion compensation from the other coding unit is made smaller than the global undetermined multiplier, and the local undetermined multiplier of the coding unit not referenced for motion compensation from the other coding unit is changed. A means for determining a global undetermined multiplier and a local undetermined multiplier so as to be equal to the global undetermined multiplier is provided.
[0011]
The repetitive image signal encoding method according to the second invention for solving the second problem includes a fourth procedure for performing encoding by the repetitive image signal encoding method of the first invention, and the fourth procedure. Compare the generated code amount and the target code amount by the encoding of the procedure of step 5 and terminate the encoding if it is sufficiently close, otherwise encode the procedure of the next procedure, the code amount and the undetermined multiplier The global undetermined multiplier corresponding to the entire image sequence and the local undetermined multiplier corresponding to each coding unit are changed using the known statistical relationship of the second image signal encoding method according to the first invention. And a sixth procedure for returning to the fifth procedure after encoding by the procedure.
[0012]
The repetitive image signal encoding device according to the second invention is a fourth means for encoding by the repetitive image signal encoding device according to the first invention, and a generated code generated by the encoding of the fourth means. If the amount is close enough to the target code amount, the encoding is terminated, and if not, the fifth means for encoding by the sixth means, and the known statistical relationship between the code amount and the undetermined multiplier The global undetermined multiplier corresponding to the entire image sequence and the local undetermined multiplier corresponding to each encoding unit are changed, and encoding is performed by the second means in the repetitive image signal encoding device according to claim 1. And a sixth means for returning to the processing of the fifth means.
[0013]
The processing procedure in the first and second repetitive image signal encoding methods can be a repetitive image signal encoding program to be executed by a computer. It is possible to record on a computer-readable recording medium.
[0014]
In the first invention according to the present invention, an optimization cycle of local undetermined multiplier adjustment, encoding, and global evaluation criterion evaluation is performed while considering the magnitude relationship between the local undetermined multiplier to be changed when performing optimization and the global undetermined multiplier. As a result, it is possible to avoid useless processing such as trying to encode using a local undetermined multiplier that should not be optimal as a result.
[0015]
In the second invention according to the present invention, when both the optimization cycle and the code amount control are made compatible, based on the obtained code amount and the local undetermined multiplier obtained once, the optimum local code amount is obtained. Generating an undetermined multiplier eliminates the need for a new optimization cycle.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, the principle and embodiments of the present invention will be described in detail with reference to the drawings.
[0017]
<Principle of the first invention>
High-efficiency video codes using inter-frame motion compensation such as MPEG-1 (ISO / IEC11172-2), MPEG-2 (ISO / IEC13818-2), MPEG-4 (ISO / IEC14496-2), etc. that are currently widely used In the coding technology, an independent coding frame (I frame) that is not referred to by others, a frame that is coded with reference to others (P frame), a frame that refers to I and P frames but is not referenced from anywhere (B frame) ) Exists.
[0018]
Here, the encoding of the simplest image sequence in which three frames of I, B, and P frames are arranged in order will be considered. Since I frame is referenced to P and B frames, considering that this picture quality affects their code amount, and because P frame is referenced to B frame, it affects its code amount. The total code amount R is R = R _I + R _P (R _I ) + R _B (R _P (R _I ), R _I ) (3)
It is written. Here, R _I , R _B , and R _P are the code amounts of the I frame, the B frame, and the P frame, respectively. For the same reason, the total error D is D = D _I (R _I ) + D _P (R _P , R _I ) + D _B (R _B , R _P , R _I ) (4)
It can be written as Here, D _I , D _B , and D _P are errors of the I frame, the B frame, and the P frame, respectively. When Lagrange's undetermined multiplier is λ, the evaluation criterion is J = D + λ (R−R _tot ) (5)
It is written. Here, R _tot is a target total code amount. In the Lagrange method, this minimization is
[Expression 1]

[0020]
It is reduced to the simultaneous equations. The first three formulas are [0021]
[Expression 2]

[0022]
It becomes. Where [0023]
[Equation 3]

[0024]
Then, for the B frame, T _B = λ (11) from Equation (7).
Holds.
[0025]
As for P frame, when R _P increases, the image quality of P frame increases, and as a result, B frame referring to P frame becomes easier to encode and D _B and R _B decrease.
[0026]
[Expression 4]

[0027]
Holds. Substituting these into equation (8)
[Equation 5]

[0029]
It becomes.
[0030]
Similarly, with respect to the I frame, when R _I increases, the image quality of the I frame increases, and as a result, the P and B frames that refer to the I frame become easier to encode and D _P , R _P , D _B , and R _B decrease. From that
[0031]
[Formula 6]

[0032]
Substituting these into equation (9)
[Expression 7]

[0034]
Get.
[0035]
Since the graph of D (R) is generally convex downward in each frame, the point where −∂D / ∂R is equal to the local undetermined multiplier T coincides with the point where the local evaluation criterion D + TR is minimized. Therefore, if T _I , T _B and T _P are local undetermined multipliers and λ is a global undetermined multiplier,
T _I <λ, T _P <λ and T _B = λ (16)
(T _I , T _P , T _B ) may be searched so as to minimize J = D + λR while maintaining. This method allows faster and more efficient optimization than repeating the optimization cycle without any prior knowledge of (T _I , T _P , T _B ).
[0036]
The above has considered the simplest case of 3 frames. However, when this idea is applied to the case where a video clip having a general length is optimized by the MPEG method in units of frames, the procedure is as follows.
[0037]
1. A local undetermined multiplier table (T _I1 , T _P1 , T _B1 , T _B2 , T _P2 ,...) Having the same length as the number of frames is initialized. In this table, T _Ix <λ, T _Px <λ, and T _Bx = λ must always be maintained during the optimization cycle.
[0038]
2. Optimize video clips frame by frame. For each minimum coding unit (referred to as a macroblock in the case of MPEG), the coding is attempted with all or some of the quantization steps that can take various values discretely (31 in the case of MPEG), A quantization step that gives the smallest local criterion D + TR is adopted. D and R are a code amount and a decoding error when the macroblock is encoded in the quantization step, and T is a local undetermined multiplier of the frame.
[0039]
3. After all the frames are encoded, the local undetermined multiplier table is updated to minimize the global evaluation criterion J = D + λR. This is done, for example, based on a commonly used multidimensional optimization algorithm. If it is determined that it has converged, the process ends. Otherwise, the process is repeated from 2.
[0040]
In order to confirm the validity of the constraint condition regarding the local undetermined multiplier (equation (16)), several types of global undetermined multipliers λ within the range of 0.0003 to 0.0012 are used for a certain image, and it is optimal without using the constraint condition. The result of the conversion is shown in FIG. Here, the first frame is an I frame, the second, third, fifth, sixth, eighth, ninth, eleventh and twelfth frames are B frames, and the fourth, seventh, tenth and thirteenth frames are P frames. As shown in equation (16), it can be confirmed that the local undetermined multiplier for the B picture is equal to the global undetermined multiplier, and that the local undetermined multiplier for the I and P pictures is smaller than the global undetermined multiplier.
[0041]
When the video clip becomes long, the number of local undetermined multipliers that can be changed increases, and the search space in the optimization cycle increases exponentially. In MPEG, a GOP (Group of Picture) structure, that is, a video clip is 1 Coding for each structure is performed with a group structure consisting of one I frame and a plurality of subsequent B and P frames as a unit. Accordingly, since the GOPs are independent except for the B frame group at the end of the GOP (if any), the optimization may be performed independently for each GOP.
[0042]
<Principle of the second invention>
FIG. 2 shows a graph normalized by dividing each graph of FIG. 1 by the corresponding global undetermined multiplier. In this figure, the graphs almost overlap each other. That is, it can be seen that the optimal local undetermined multiplier value is proportional to the global undetermined multiplier.
[0043]
It has also been experimentally and statistically shown that there is an inversely proportional relationship D∝R ⁻¹ between D and R (S. Takamura, H. Watanabe and N. Kobayashi: “One-pass VBR algorithm”. foran MPEG-2 encoder chip SuperENC, “Picture Coding Symposium Japan, PP. 39-40, Sep. 1999.). Accordingly, ∂D / ∂R∝R ^{-2 is obtained} , and if ∂D / ∂R = −λ is substituted for this, it is expected that the following inverse square relationship is established.
[0044]
λ∝R ^-2 (17)
In order to confirm this, for several types of images, several types of global undetermined multipliers λ within the range of 0.0003 to 0.0012 are used, and the relationship between the bit amount R and λ after performing the optimization cycle is shown in FIG. It was plotted as a point cloud. In addition, the least squares fitted to these point groups by a curve λ = kR ⁻² is shown as a curve in FIG. Since the curve is well fitted to the point group, it can be confirmed that the inverse square relation of λ-R is actually established.
[0045]
Using the above facts, once an optimal local undetermined multiplier table is obtained for a certain λ, by multiplying this undetermined multiplier table by a constant for any λ without performing an optimization cycle, An optimal local undetermined multiplier table can be obtained. Specifically, the procedure for bringing the generated code amount close to the target amount (R _tgt ) is described as follows.
[0046]
1. Based on a given λ, the video clip is encoded by the optimization method of the first invention.
[0047]
2. Let R be the obtained code amount. If R is close enough to R _tgt , end. Otherwise proceed to the next.
[0048]
3. Based on Equation (17), each undetermined multiplier in the local undetermined multiplier table is multiplied by R ² / R _tgt ² and encoding is performed again using that table. At this time, iterative optimization is not necessary. Return to 2 after encoding.
[0049]
The results of performing optimum encoding / code amount control using the second invention for an actual video clip are shown below. The target bit rate is set to 4 Mbit / s. In this example, it can be seen that two trials are sufficient to achieve an accuracy within 1%.
[0050]
Image A
1st time = 5.80 Mbit / s → 4.04 Mbit / s (end)
Image B
1st time = 5.11 Mbit / s → 3.86 Mbit / s
Second time = 3.86 Mbit / s → 4.01 Mbit / s (end)
<Embodiment of the first invention>
An embodiment of the first invention according to the present invention will be described with reference to FIG. FIG. 4 shows a processing unit configuration, that is, a device configuration, along with a processing flow in this embodiment. Each processing unit is all or part of the means of the present invention.
[0051]
First, after the processing starts, the local undetermined multiplier table initialization unit 101 initializes a local undetermined multiplier table in the local undetermined multiplier table storage unit 102 (hereinafter, the local undetermined multiplier table is referred to as 102). In the local undetermined multiplier table 102, T _Ix <λ, T _Px <λ, and T _Bx = λ must always be maintained during the optimization cycle. Therefore, the local undetermined multiplier table initialization unit 101 initializes the local undetermined multiplier table 102 so that T _Ix <λ, T _Px <λ, and T _Bx = λ.
[0052]
Subsequently, the frame encoding unit 120 performs encoding. First, the local undetermined multiplier acquisition unit 103 sequentially selects the encoded frames, then acquires the local undetermined multiplier T of the corresponding frame from the undetermined multiplier table 102, and the macroblock selection unit 104 selects the macroblock of the corresponding frame in order. Next, the quantization step selection unit 105 sequentially generates all possible quantization steps. Next, an error D ′ and a code amount R ′ generated as a result of encoding in the encoding error / code amount acquisition unit 106 are acquired, and a local evaluation criterion D ′ + TR ′ is calculated in the local value criterion calculation unit 107. Next, the minimum determination unit 108 determines whether or not the local evaluation criterion is the minimum in the macroblock. If it is the minimum, the quantization step storage unit 109 stores the quantization step in the variable Q. Next, if it is not determined that all the quantization steps that can be taken by the all quantization determination unit 110 have been tried, the process returns to the quantization step selection unit 105 and tries another quantization step again. If it is determined that all quantization steps have been tried, the macroblock encoding unit 111 performs encoding using the stored quantization step Q. Next, the entire macroblock determining unit 112 determines whether or not the encoding has been completed for all the macroblocks of the frame. If not, the process returns to the macroblock selecting unit 104 to process the next macroblock. If completed, the all-frame determining unit 113 determines whether all the frames have been processed. If not, the process returns to the local undetermined multiplier acquiring unit 103 to process the next frame.
[0053]
When the encoding process for all frames is completed, the total coding error / total code amount acquisition unit 114 acquires the total code error D and the total code amount R in the entire video sequence, and the global evaluation criterion calculation unit 115 performs global evaluation. A reference D + λR is calculated. Next, the convergence determination unit 116 determines whether or not the global evaluation criterion has converged. If it is determined that the global evaluation criterion has not yet been determined, the convergence determination unit 116 proceeds to the undetermined multiplier table correction unit 117. Here, the parameter (in this case, the local undetermined multiplier table of 102) is adjusted so as to minimize the global evaluation criterion based on the global undetermined multiplier. Here, the undetermined multiplier table correction unit 117 adjusts (corrects) the local undetermined multiplier table 102 so that T _Ix <λ, T _Px <λ, and T _Bx = λ. Thereafter, the encoding of the entire video sequence is repeated again from the local undetermined multiplier acquisition unit 103. If the convergence determination unit 116 determines that the convergence has been completed, the process ends.
[0054]
<Embodiment of the second invention>
A second embodiment of the present invention will be described with reference to FIG. FIG. 5 shows a processing unit configuration, that is, a device configuration, along with a processing flow in the present embodiment. Each processing unit is all or part of the means of the present invention.
[0055]
First, after the start, the entire video sequence is optimized once by the video clip optimizing unit 201. This is the same processing as the video clip optimization in the embodiment of the first invention. The local undetermined multiplier table obtained as a result is stored in the local undetermined multiplier table storage unit 202 (hereinafter, the local undetermined multiplier table is referred to as 202).
[0056]
Subsequently, the code amount acquisition unit 203 acquires the code amount R of the entire video sequence. Next, R and R _tgt comparison section 204 compares R and R _tgt and ends if it is determined that the difference is sufficiently small. Otherwise, the local undetermined multiplier table updating unit 205 multiplies each value of the local undetermined multiplier table 202 by R ² / R _tgt ² .
[0057]
Next, encoding is performed in 206 frame encoding units (this is the same as the frame encoding unit 120 shown in the first embodiment). Here, the iteration for optimization is not necessary (however, the iteration for selecting the quantization step is performed in units of macroblocks). Thereafter, the processing is repeated from the code amount acquisition unit 203.
[0058]
In addition, the processing procedure in part or all of the processing units of the apparatus shown in FIGS. 4 and 5 or the processing procedure in the processing flow is configured by a computer program, and the program is executed using the computer. Needless to say, the present invention can be realized by a computer-readable recording medium, such as an FD, for realizing a program for realizing the processing unit by a computer or a program for causing a computer to execute the processing procedure. (Floppy disk (registered trademark)), MO, ROM, memory card, CD, DVD, removable disk, etc., and can be stored and provided. It can be distributed over the network by e-mail etc.
[0059]
【The invention's effect】
As described above, according to the present invention, the local undetermined multiplier that is changed when performing the optimization, the local undetermined multiplier adjustment, coding, and global evaluation criterion evaluation while considering the magnitude relationship with the global undetermined multiplier. As a result, it is possible to avoid useless processing such as trying to encode using a local undetermined multiplier that should not be optimal.
[0060]
Also, when making the optimization cycle and code amount control compatible, it is possible to generate a local undetermined multiplier that is optimal for the target code amount in one blow based on the obtained code amount and the local undetermined multiplier obtained once. This eliminates the need for a new optimization cycle.
[Brief description of the drawings]
FIG. 1 is a graph of a local undetermined multiplier table (λ) after optimization for various global undetermined multipliers λ.
FIG. 2 is a graph obtained by normalizing the data of FIG.
FIG. 3 is a graph in which a global undetermined multiplier λ and R after optimization are plotted and a least-squares fit is performed using an inverse square curve.
FIG. 4 is a diagram showing a processing flow and apparatus configuration of video clip optimization encoding according to the embodiment of the first invention.
FIG. 5 is a diagram showing a processing flow and apparatus configuration of coding rate control at the time of optimization according to the embodiment of the second invention.
[Explanation of symbols]
101 ... Local undetermined multiplier table initialization unit 102 ... Local undetermined multiplier table storage unit (local undetermined multiplier table)
103 ... Local undetermined multiplier acquisition unit 104 ... Macro block selection unit 105 ... Quantization step selection unit 106 ... Encoding error / code amount acquisition unit 107 ... Local value criterion calculation unit 108 ... Minimum determination unit 109 ... Quantization step storage unit 110 ... all quantization determination unit 111 ... macroblock encoding unit 112 ... all macroblock determination unit 113 ... all frame determination unit 114 ... total coding error / total code amount acquisition unit 115 ... global evaluation criterion calculation unit 116 ... convergence determination unit 117: Undetermined multiplier table correction unit 120 ... Frame encoding unit 201 ... Video clip optimization unit 202 ... Local undetermined multiplier table storage unit (local undetermined multiplier table)
203 ... Code amount acquisition unit 204 ... R / R _tgt comparison unit 205 ... Local undetermined multiplier table update unit 206 ... Frame encoding unit

Claims (6)

Encoding is performed by inter-frame motion compensation, and the same image sequence is encoded multiple times in order to optimize the relationship between code amount and noise amount based on the evaluation of the undetermined multiplier by Lagrange's undetermined multiplier method. In an image signal encoding method for performing optimization,
A first procedure for initializing a local undetermined multiplier corresponding to each of the coding units;
The quantization step is changed for each encoding unit, and the entire image sequence is encoded by determining the quantization step so that the relationship between the code amount and the noise amount is optimized based on the evaluation of the acquired local undetermined multiplier. The second step,
The global undetermined multiplier corresponding to the entire image sequence is evaluated each time the entire image sequence is encoded, and the local undetermined multiplier is corrected until the relationship between the code amount and the noise amount is optimized, and the second procedure is performed. A third step of repeatedly performing
In the first procedure, at the time of initialization, and in the third procedure, at the time of the correction, the local undetermined multiplier of the coding unit referred to for motion compensation from other coding units is determined from the global undetermined multiplier. A repetitive image signal characterized by determining a global undetermined multiplier and a local undetermined multiplier so that the local undetermined multiplier of a coding unit that is not referenced from other coding units for motion compensation is equal to the global undetermined multiplier. Encoding method. A fourth procedure for performing encoding by the iterative image signal encoding method according to claim 1;
A fifth procedure for comparing the generated code amount by the encoding of the fourth procedure with the target code amount and ending the encoding if it is sufficiently close, otherwise encoding by the next procedure;
2. The iterative image signal encoding according to claim 1, wherein a global undetermined multiplier corresponding to the entire image sequence and a local undetermined multiplier corresponding to each coding unit are changed using a known statistical relationship between a code amount and an undetermined multiplier. And a sixth procedure for returning to the fifth procedure after encoding by the second procedure in the method. Encoding is performed by inter-frame motion compensation, and the same image sequence is encoded multiple times in order to optimize the relationship between code amount and noise amount based on the evaluation of the undetermined multiplier by Lagrange's undetermined multiplier method. In an image signal encoding device that performs optimization,
A first means for initializing a local undetermined multiplier corresponding to each of the coding units;
The quantization step is changed for each encoding unit, and the entire image sequence is encoded by determining the quantization step so that the relationship between the code amount and the noise amount is optimized based on the evaluation of the acquired local undetermined multiplier. A second means,
Each time the entire image sequence is encoded, the global undetermined multiplier corresponding to the entire image sequence is evaluated, and the local undetermined multiplier is corrected until the relationship between the code amount and the noise amount is optimized, and the second means And the third means is referred to for motion compensation from other coding units during the initialization, the third means during the initialization, and the third means. The global undetermined multiplier and local undetermined multiplier are set so that the local undetermined multiplier of the coding unit is smaller than the global undetermined multiplier, and the local undetermined multiplier of the coding unit that is not referenced for motion compensation from other coding units is equal to the global undetermined multiplier. An iterative image signal encoding device comprising means for determining a multiplier. A fourth means for performing encoding by the repetitive image signal encoding device according to claim 1;
Comparing the generated code amount by the encoding of the fourth means with the target code amount, and if it is sufficiently close, terminate the encoding; otherwise, the fifth means for encoding by the sixth means;
2. The iterative image signal encoding according to claim 1, wherein a global undetermined multiplier corresponding to the entire image sequence and a local undetermined multiplier corresponding to each coding unit are changed using a known statistical relationship between a code amount and an undetermined multiplier. And a sixth means for returning to the processing of the fifth means after encoding by the second means in the apparatus. A repetitive image signal encoding program characterized in that a program for causing a computer to execute the processing procedure in the repetitive image signal encoding method according to claim 1 or 2 is provided. 6. A recording medium for a repetitive image signal encoding program, wherein the repetitive image signal encoding program according to claim 5 is recorded on a computer-readable recording medium.

Images